[17:51] <blackboxsw> sweet cloud-init 17.1.25  hit xenial
[17:51] <blackboxsw> ok blog post rework this afternoon
[20:23] <jdandrea> I see how to set http/s proxy and no_proxy for apt. Is there a way to set proxies for the default environment? Suspecting it's not wise to just blow away /etc/environment (and I can't append).
[20:43] <smoser> jdandrea: there is not.
[20:44] <smoser> you could use boothook to intelligently update /etc/environment (boothooks run every boot)
[20:44] <smoser> but generally speaking its just a pita when you try to set http_proxy globally
[20:44] <smoser> like that
[20:44] <jdandrea> @smoser Ok. I ask because without that then the apt settings aren't as helpful for us on first boot.
[20:44] <smoser> so i really don't recommend it.
[20:44] <jdandrea> @smoser Hmm, ok. I wish this lab wasn't behind a proxy. :/
[20:45] <smoser> so.. why isnt it helpful for you on first boot ?
[20:45] <smoser> basically, static http_proxy and https_proxy fall apart as soon as you try to use an address that isnt proxy-able
[20:45] <jdandrea> We also set no_proxy tho
[20:45] <smoser> yes, but that is vastly insuffiicent
[20:46] <jdandrea> Indeed it is. I wish I had a better option.
[20:46] <smoser> gnu tools read it sanely, and allow you to use wildcard and even cidr entries
[20:46] <jdandrea> In another lab they have proxyless gateways so it's a sinch.
[20:46] <smoser> but python (and other languages) do not.
[20:46] <jdandrea> s/sinch/cinch
[20:46] <smoser> so they're uselss.
[20:46] <smoser> adn then, you do something like launch a container, and it has its own network that wasnt in your http_proxy and you are foobarred again
[20:46] <jdandrea> Aye, and in those use cases that's already taken care of with install. I'm really more concerned with bootstrapping things.
[20:47] <smoser> here is what i do:
[20:47] <smoser>  http://bazaar.launchpad.net/~smoser/+junk/sstack-proxy/view/head:/sstack-proxy
[20:47] <smoser> so bootstrapping ? as in with maas ?
[20:47] <jdandrea> Right, I get that. Presuming I have a solid handle on communicating proxy settings down to various bits, I'm left with trying to get things to a happy state during cloud-init. In this case it's just cloud-init stuffs that are kicked off after an OpenStack nova boot.
[20:47] <jdandrea> Reading...
[20:48] <jdandrea> This particular lab already has a proxy server, thus I suspect something like tinyproxy wouldn't fly (?).
[20:48] <smoser> ah. ok. i see.
[20:49] <jdandrea> Basically I wish they didn't have one, but ... hands tied.
[20:49] <smoser> jdandrea: oh. i run the tiny proxy on the host
[20:49] <smoser> and point http_proxy/https_proxy at *it*
[20:49] <jdandrea> @smoser Ah
[20:50] <smoser> then you dont ever have to change your http_proxy/https_proxy vars
[20:50] <jdandrea> Ohhhh
[20:50] <smoser> and you can just update tinyproxy and restart
[20:50]  * jdandrea gets clue
[20:50] <smoser> and then all things get updated
[20:50] <smoser> the issue is its slower than no proxy
[20:50] <jdandrea> Right, and in this case I'm betting folks won't like that. But hey, if I can get it to work sanely...
[20:51] <jdandrea> So it requires at least one reboot, ya?
[20:51] <jdandrea> (Not necessarily a bad thing.)
[20:51] <smoser> well, a new session after /etc/environmetn is wrtten, yeah.
[20:51] <smoser> and daemons restarted possibly.
[20:51] <jdandrea> Ok
[20:51] <jdandrea> I will study this. Thank you!
[20:51] <smoser> actually , daemons do not read /etc/environment enerally speaking
[20:51] <smoser> you can make similar change to systemd config...
[20:52] <jdandrea> Right. This is there purely so that the one-time scripts that are running at boot time can do their thing, and then they will communicate proxy settings to whatever services need them from that point on.
[20:52] <smoser> i dont think we added an  'environment' config to cloud-init
[20:52] <smoser> but i have wanted to.
[20:52] <jdandrea> I think I can replace /etc/environment but I suspect that is a Very Bad Idea(tm). :D
[20:52] <smoser> ie, cloud-init would read your settings in 'environment:' and update its environ.
[20:52] <jdandrea> mhm mhm that's what I was looking for, like for apt.
[20:53] <smoser> jdandrea: i'm surprised at the amount of peole that write proxy data into e/tc/environment
[20:53] <jdandrea> Well, honestly? I'd rather not do that.
[20:53] <smoser> it seems without a solution like tinyproxy to be copmetely a broken idea to me.
[20:53] <smoser> so yeah, i'd stay out of taht.
[20:53] <jdandrea> But it *looks* to me (and I could be wrong)... like Ubuntu needs to know about proxies early on in cloud-init. I may be wrong.
[20:53] <smoser> what is it that you ahve to do before then ?
[20:53] <smoser> what is "ubuntu" ?
[20:54] <smoser> in this case... what uses ?
[20:54] <jdandrea> Because there are some things it's trying to get and it can't reach some servers. Ubuntu meaning Ubuntu 14.04 or 16.04.
[20:54] <smoser> apt will do the right thing.
[20:54] <jdandrea> Lemme try and make a minimal paste to show...
[20:54] <jdandrea> I could also just be wrong. :)
[20:55] <smoser> my 'sstack-proxy' came from usage of a cloud that was simliarly configured.
[20:55] <smoser> in that you can't get to the interwebs without proxy
[20:55] <smoser> but i only did things that *needed* access after cloud-init was up and hadn run that
[20:56] <jdandrea> @smoser So this is early on in cloud-init. The IPv6 I'd ignore. But the pollinate stuff overall... connecting to entropy: https://hastebin.com/ucaviyajuq.rb
[20:56] <smoser> http://paste.ubuntu.com/25969929/
[20:56] <jdandrea> This is before any user scripts are run, methinks.
[20:56] <smoser> thats my user-data that i run
[20:56] <smoser> for use case of general system i want to go to.
[20:57] <jdandrea> *nod*...
[20:57] <smoser> you can probably ignore the pollinate.
[20:57] <smoser> they shouldnt cause trouble generally.. failure shoudl not cause issue
[20:57] <jdandrea> Ok, then in that case I don't need to touch environment.
[20:57] <jdandrea> Whew.
[20:57] <smoser> timeouts suck though
[20:57] <jdandrea> Yeah. And y'know it may have absolutely nothing to do with proxies. Could also be noisy neighbors.
[20:57] <jdandrea> (Now seeing 1 minute downloads take 45. Um...)
[20:57] <smoser> you can also i think tell cloud-init to disable pollinate. if you dont like its warnings.
[20:58] <jdandrea> TIL: can disable pollinate. Thanks! Good to know
[20:58] <jdandrea> Hopefully we don't need the entropy. XD
[20:59] <jdandrea> Appreciate the insight. I will look at both of these pastes.
[20:59] <smoser> actually i'm wrong. you can't disable pollinate.
[20:59] <smoser> ewll, you can, but not through cloud-init.
[20:59] <smoser> (you could with a bootcmd:
[20:59] <jdandrea> Hehe. But... it's not a total disaster if it doesn't do its thing.
[20:59] <jdandrea> ?
[20:59] <jdandrea> Ah, ok.
[21:00] <smoser> ln -sf /bin/true /usr/bin/pollinate
[21:00] <smoser> jdandrea: its supposed to be seeding random data.
[21:00] <smoser> so it reads some random information off of a remote web service and writes it to /dev/random
[21:00] <smoser> to help with entropy
[21:00] <jdandrea> *nodnod*
[21:00] <smoser> entropy quality in a vm is a complicated topic
[21:00] <jdandrea> But if it can't reach it... "because proxy"... yeah.
[21:01] <jdandrea> Oh yes, I can imagine.
[21:01] <smoser> people suggest that not having enough entropy when your ssh host keys are generated could leave you at risk of attack
[21:02] <jdandrea> Right. Threat models and all that. So... yeah. Something to ponder.
[21:02] <smoser> oh.
[21:02] <smoser> you're runnign 14.04 there
[21:02] <smoser> 16.04 i dont think will spew error to console
[21:03] <jdandrea> Yup. BUT... 16.04 in some cases.
[21:03] <jdandrea> Ok.
[21:03] <smoser> basically your complaint was "fixed"' by siliently failing :)
[21:03] <jdandrea> Bwahahaha!
[21:03]  * jdandrea applauds
[21:03] <smoser> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1554152
[21:03] <ubot5> Launchpad bug 1554152 in pollinate (Ubuntu) "pollinate fails in many circumstances, cloud-init reports that failure, maas reports node failed deployment" [Critical,Fix released]
[21:03] <jdandrea> Let me just turn down that baby monitor... :p
[21:04] <smoser> actually, openstack
[21:04] <smoser> will provid eyou with some blob of random data from the server
[21:04] <smoser> in metadata
[21:04] <smoser> (soo does azure)
[21:04] <smoser> and cloud-init will use that
[21:05] <smoser> so that provides locally to the cloud the value that pollinate would provide.
[21:07] <jdandrea> Ok, then we're good.
[21:07] <jdandrea> But really, beyond pollinate, I can set the proxies within a user script and it's fine.
[21:07] <jdandrea> And it's temporary. After that the service/VNF that comes up does the right thing.
[21:07] <jdandrea> I'm just glad I don't need to touch /etc/environment.
[21:19] <smoser> yeah./bu13
[21:39] <blkadder> Perhaps a silly question but why is touching /etc/environment considered such a bad thing? Reading through https://help.ubuntu.com/community/EnvironmentVariables there is nothing saying “OMG this is super bad don’t touch.”
[22:15] <blackboxsw> smoser: rharper just pushed a cut at sourcing logfile paths from the cloud-init cfg to https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513
[22:15] <rharper> blkadder: I suspect in general, system wide variables can cause problems in places that may be unexpected;  I would think per-application settings are generally preferred; reading the wiki page; it sounds like that's the line on /etc/environment; which is (IMO) sane policy
[22:16] <blkadder> Definitely preferable to do per app but if this indeed was intended to be have a global, persistent effect that would seem to be the place to do it.
[22:17] <blkadder> Totally off-topic for here just found the earluer convo interesting…
[22:17] <blkadder> earlier...
[22:17] <blkadder> Understood about side-effects though.
[22:23] <rharper> blkadder: yeah, not that off topic for system initiazation