[08:51] <ffledgling> Hello, I'm trying to figure out how to get packages to install via cloud init behind a proxy, I haven't been able to find a good way to do so, are there any recommendations? Seems like there's an open bug about it - https://bugs.launchpad.net/cloud-init/+bug/1089405
[09:05] <b0stik> ffledgling, tried with a bash command to add a line for proxy in apt configuration ?
[09:11] <ffledgling> b0stik: That's the last resort, and I'm not sure if the `packages` directive from cloud config is run before or after runcmd
[09:46] <ffledgling> Is there a guaruntee on the order in which things are run by cloud-init when reading from cloud-config
[13:29] <ffledgling> Answer to my previous question seems to be - https://git.launchpad.net/cloud-init/tree/config/cloud.cfg
[14:11] <smoser> ffledgling, i'm guessing this is not ubuntu / debian ?
[14:11] <smoser> you can define proxy for apt.
[14:12] <ffledgling> smoser: nope, fedora
[14:12] <smoser> but for general system proxy there isnt.
[14:12] <smoser> proxies are a regula pita
[14:12] <smoser> generally speaking.
[14:12] <smoser> ie, setting one up "system wide" never actually works.
[14:12] <smoser> thats myexperience.
[14:13] <smoser> that said, i'm open to patches that set environment in cloud-init.
[14:13] <ffledgling> That's not entirely unexpected, some tools don't support the standard proxy variables etc and then there's the lowercase vs uppercase issue
[14:14] <ffledgling> But yeah, it'd still be nice to have the ability to set variables in the env for at least the cloud-init parts
[14:14] <ffledgling> My current work around for this usecase is to use runcmd to add a proxy via a config file, but that relies on the order in which the steps are executed by cloud-init
[14:15] <ffledgling> Which is okay I guess, but fairly fragile if the ordering is changed
[14:15] <smoser> ffledgling, probably runcmd runs after package installation
[14:15] <smoser> so i'd use a bootcmd which is guaranteed to run before package installation
[14:15] <smoser> order is not fragile really. bootcmd will run as early as it can.
[14:16] <smoser> i suspect there is some way specifically to tell yum about package config ?
[14:16] <ffledgling> smoser: I didn't actually find where the package instruction is run, but for ubuntu it seems to be in the 'final' stage?
[14:16] <smoser> er... sorry. about proxy ?
[14:16] <ffledgling> smoser: /etc/yum.conf
[14:16] <smoser> with the move to systemd, package installation started happening at 'final' stage. it used to run at 'config' stage.
[14:17] <smoser> yeah, so waht i'd do is just in a boot cmd, edit/append to that
[14:17] <smoser> bootcmd:
[14:17] <smoser>  - [sh, -c, 'echo http_proxy=http://proxy.host:port >> /etc/yum.conf']
[14:17] <ffledgling> bootcmd + cloud-init-run-once (I think that's what it's called?)
[14:17] <ffledgling> otherwise isn't bootcmd run at every boot?
[14:17] <smoser> ah, yeah. bootcmd do run every time
[14:18] <smoser> so yeah, you coul use cloud-init-per too
[14:18] <smoser> or
[14:18] <smoser> just otherwise idempotently update that file
[14:18] <ffledgling> That might be a little harder
[14:19] <smoser> well,  yeah, ini file
[14:20] <ffledgling> I was trying to figure out if there's a way to just tell cloud-init to take a file I hand it and stick it someplace on the target system
[14:20] <ffledgling> but I didn't find anything like that yet
[14:20] <smoser> well, there is write_files
[14:21] <smoser> http://cloudinit.readthedocs.io/en/latest/topics/examples.html#writing-out-arbitrary-files
[14:21] <smoser> and that runs very early
[14:21] <smoser> right after bootcmd
[14:21] <ffledgling> That might work as well
[14:22] <ffledgling> Doesn't say if it's written everytime or just once, I suspect, just one?
[14:22] <ffledgling> *once
[14:22] <smoser> that will run once per instance
[14:22] <smoser> so yeah, that'd probably work for you.
[14:23] <ffledgling> sounds good, I'll try both probably and see what works best
[14:23] <smoser> configuring yum is preferable in my opinion to configuring system wide, or relying on environment variables.
[14:23] <ffledgling> smoser: well, the problem is I really do need a system wide proxy for my eventual operations
[14:23] <smoser> well, you can write /etc/environment if you want.
[14:23] <ffledgling> Except in my case I'm using ansible to configure the VM after it's up so I can push that problem down there
[14:23] <smoser> my experience with system wide proxies is just they always end up sucking
[14:24] <smoser> for reasons largely of 'no_proxy'
[14:24] <ffledgling> Applications not respecting no_proxy or people not setting it?
[14:24] <smoser> ie, if the proxy is on the other side of the router, and it wont proxy some cidr, then globally setting http_proxy=that.proxy ends up sending all http requests to it, and it can't proxy for your network
[14:25] <smoser> most obvious example is:
[14:25] <smoser>  http_proxy=http://example.com wget http://127.0.0.1:8000/my-test-application
[14:25] <smoser> but the same general thing occurs when the proxy doesn't work for your 10.0.0.0/24 or something.
[14:26] <ffledgling> right, that probably doesn't workt without no_proxy whitelisting your local network
[14:26] <smoser> and no_proxy only works very coarsely (python apps can't take a cidr, but some others doo)
[14:26] <smoser> ie, if you wantto white list 10.0.0.0/24 you end up having to do:
[14:26] <smoser>  no_proxy=10.0.0.1,10.0.0.2,10.0.0.3,.....10.0.0.255
[14:26] <smoser> which quickly becomes unmaintainable.
[14:27] <ffledgling> ah, right, no_proxy is "only" respects domains
[14:27] <ffledgling> s/is//
[14:27] <smoser> the only solution to such a thing that i've come accross is using a local proxy and pointing everything at that, and having its richer syntax do the selection for you.
[14:27] <smoser> ffledgling, well, no_proxy in some cases (i think wget) supports 10.0.0.*
[14:28] <smoser> but it does not in python's implementation and probably many others
[14:28] <smoser> http://bazaar.launchpad.net/~smoser/+junk/sstack-proxy/view/head:/sstack-proxy
[14:28] <smoser> that is what i've done that actually works... but comes at a cost of a middle man
[14:28] <smoser> then, you can change the config of tinyproxy and restart it, and you dont have to deal with getting the environment changed in existing processes.
[14:29] <smoser> but... that is just really neither here nor there... just sort of my experience with setting proxy and coming out hating the whole thing.
[14:29] <ffledgling> Yeah, proxies aren't really great, but i've just come to treat them as a necessary evil
[14:30] <smoser> the above is what i ended up having to do, and does work fairly well.
[14:30] <smoser> when you realize you forgot yourhost.com from the proxy config, you just add it in tinyproxy and restart tinyproxy
[14:30] <smoser> and tinyproxy config supports cidr
[14:30] <smoser> fully
[14:30] <smoser> so 192.168.2.0/24 or 9.0.0.1/32
[14:30] <smoser> and ipv6 and all that...
[14:31] <ffledgling> yep, I've seen the same setup recommended by a lot of people, after a certain point I'm sure it become necessary
[14:32] <ffledgling> smoser: speaking of patches (saw your comment on the bug for env var support as well), I have a doc fix that I sent in a couple of days ago
[14:34] <smoser> where did i comment on that ?
[14:34] <smoser> i've lost it now
[14:34] <smoser> did i comment?
[14:34] <smoser> https://code.launchpad.net/~ffledgling/cloud-init/+git/cloud-init-1/+merge/312917
[14:34]  * ffledgling was looking for the link to it himself, the launchpad interface is new and confusing
[14:34] <ffledgling> smoser: yeah, that one
[14:35] <smoser> ffledgling, so the link you want is https://code.launchpad.net/%7Ecloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews
[14:35] <smoser> then from there you can find things...
[14:35] <smoser> (that is the topic of this channel too)
[14:35] <smoser> also...
[14:36] <smoser> did you delete another one ?
[14:36] <smoser> you dont hve to delete a MP you can just push to it again (even push --overwrite)
[14:36] <smoser> then you dont lose the history of comments and such
[14:36] <ffledgling> I deleted the repo I think
[14:36] <ffledgling> I'd originally created  repo called ~ffledgling/cloud-init/something/cloud-init
[14:37] <ffledgling> I don't think I've deleted an MP specifically
[14:46] <smoser> did i comment ?
[14:46] <smoser> i swear i commented somehwere.
[14:47] <ffledgling> smoser: I don't see it on the MP
[14:47] <ffledgling> I didn't get an email for a comment either, I just checked
[14:48] <ffledgling> Maybe it's saved as a draft or something?
[14:48] <smoser> well, i'llj ust type again
[15:06] <ffledgling> smoser: thanks
[15:06] <ffledgling> re: the patch for env variable support, is there a particular place in the code I should start with if I wanted to implement it?
[15:21] <smoser> ffledgling, i commented there now. sorry. i thought i did that yesterday but failed i guess.
[15:23] <ffledgling> smoser: thanks, I haven't looked at upstart (don't use ubuntu much outside of experiments), I'll take a look and get back to you
[18:32] <smoser> rangerpbzzzz, around ?
[20:21] <rharper> smoser: looking to detect if the running distro supports netplan;  I'm checking 3 things  1) do we have /etc/netplan dir  2) do we have netplan binary in $PATH 3) do we have systemd-networkd .  (3) is harder as it's not in $PATH and it's not a package (it's part of systemd) possibly optional on other distros ...   thoughts on (3) ?   Also, I was thinking this was a cloudinit.net.netplan method but each distro may need t
[20:21] <rharper> o check for support differently so could be part of the distro obj
[20:25] <smoser> why not just check util.which('netplan') ?
[20:26] <smoser> netplan should depend on systemd-networkd if it needs it
[20:30] <rharper> ok
[20:33] <smoser> dont you think ?
[21:05] <rharper> smoser: I think that's reasonable