[08:51] Hello, I'm trying to figure out how to get packages to install via cloud init behind a proxy, I haven't been able to find a good way to do so, are there any recommendations? Seems like there's an open bug about it - https://bugs.launchpad.net/cloud-init/+bug/1089405 [09:05] ffledgling, tried with a bash command to add a line for proxy in apt configuration ? [09:11] b0stik: That's the last resort, and I'm not sure if the `packages` directive from cloud config is run before or after runcmd [09:46] Is there a guaruntee on the order in which things are run by cloud-init when reading from cloud-config === shardy is now known as shardy_lunch === shardy_lunch is now known as shardy [13:29] Answer to my previous question seems to be - https://git.launchpad.net/cloud-init/tree/config/cloud.cfg [14:11] ffledgling, i'm guessing this is not ubuntu / debian ? [14:11] you can define proxy for apt. [14:12] smoser: nope, fedora [14:12] but for general system proxy there isnt. [14:12] proxies are a regula pita [14:12] generally speaking. [14:12] ie, setting one up "system wide" never actually works. [14:12] thats myexperience. [14:13] that said, i'm open to patches that set environment in cloud-init. [14:13] That's not entirely unexpected, some tools don't support the standard proxy variables etc and then there's the lowercase vs uppercase issue [14:14] But yeah, it'd still be nice to have the ability to set variables in the env for at least the cloud-init parts [14:14] My current work around for this usecase is to use runcmd to add a proxy via a config file, but that relies on the order in which the steps are executed by cloud-init [14:15] Which is okay I guess, but fairly fragile if the ordering is changed [14:15] ffledgling, probably runcmd runs after package installation [14:15] so i'd use a bootcmd which is guaranteed to run before package installation [14:15] order is not fragile really. bootcmd will run as early as it can. [14:16] i suspect there is some way specifically to tell yum about package config ? [14:16] smoser: I didn't actually find where the package instruction is run, but for ubuntu it seems to be in the 'final' stage? [14:16] er... sorry. about proxy ? [14:16] smoser: /etc/yum.conf [14:16] with the move to systemd, package installation started happening at 'final' stage. it used to run at 'config' stage. [14:17] yeah, so waht i'd do is just in a boot cmd, edit/append to that [14:17] bootcmd: [14:17] - [sh, -c, 'echo http_proxy=http://proxy.host:port >> /etc/yum.conf'] [14:17] bootcmd + cloud-init-run-once (I think that's what it's called?) [14:17] otherwise isn't bootcmd run at every boot? [14:17] ah, yeah. bootcmd do run every time [14:18] so yeah, you coul use cloud-init-per too [14:18] or [14:18] just otherwise idempotently update that file [14:18] That might be a little harder [14:19] well, yeah, ini file [14:20] I was trying to figure out if there's a way to just tell cloud-init to take a file I hand it and stick it someplace on the target system [14:20] but I didn't find anything like that yet [14:20] well, there is write_files [14:21] http://cloudinit.readthedocs.io/en/latest/topics/examples.html#writing-out-arbitrary-files [14:21] and that runs very early [14:21] right after bootcmd [14:21] That might work as well [14:22] Doesn't say if it's written everytime or just once, I suspect, just one? [14:22] *once [14:22] that will run once per instance [14:22] so yeah, that'd probably work for you. [14:23] sounds good, I'll try both probably and see what works best [14:23] configuring yum is preferable in my opinion to configuring system wide, or relying on environment variables. [14:23] smoser: well, the problem is I really do need a system wide proxy for my eventual operations [14:23] well, you can write /etc/environment if you want. [14:23] Except in my case I'm using ansible to configure the VM after it's up so I can push that problem down there [14:23] my experience with system wide proxies is just they always end up sucking [14:24] for reasons largely of 'no_proxy' [14:24] Applications not respecting no_proxy or people not setting it? [14:24] ie, if the proxy is on the other side of the router, and it wont proxy some cidr, then globally setting http_proxy=that.proxy ends up sending all http requests to it, and it can't proxy for your network [14:25] most obvious example is: [14:25] http_proxy=http://example.com wget http://127.0.0.1:8000/my-test-application [14:25] but the same general thing occurs when the proxy doesn't work for your 10.0.0.0/24 or something. [14:26] right, that probably doesn't workt without no_proxy whitelisting your local network [14:26] and no_proxy only works very coarsely (python apps can't take a cidr, but some others doo) [14:26] ie, if you wantto white list 10.0.0.0/24 you end up having to do: [14:26] no_proxy=10.0.0.1,10.0.0.2,10.0.0.3,.....10.0.0.255 [14:26] which quickly becomes unmaintainable. [14:27] ah, right, no_proxy is "only" respects domains [14:27] s/is// [14:27] the only solution to such a thing that i've come accross is using a local proxy and pointing everything at that, and having its richer syntax do the selection for you. [14:27] ffledgling, well, no_proxy in some cases (i think wget) supports 10.0.0.* [14:28] but it does not in python's implementation and probably many others [14:28] http://bazaar.launchpad.net/~smoser/+junk/sstack-proxy/view/head:/sstack-proxy [14:28] that is what i've done that actually works... but comes at a cost of a middle man [14:28] then, you can change the config of tinyproxy and restart it, and you dont have to deal with getting the environment changed in existing processes. [14:29] but... that is just really neither here nor there... just sort of my experience with setting proxy and coming out hating the whole thing. [14:29] Yeah, proxies aren't really great, but i've just come to treat them as a necessary evil [14:30] the above is what i ended up having to do, and does work fairly well. [14:30] when you realize you forgot yourhost.com from the proxy config, you just add it in tinyproxy and restart tinyproxy [14:30] and tinyproxy config supports cidr [14:30] fully [14:30] so 192.168.2.0/24 or 9.0.0.1/32 [14:30] and ipv6 and all that... [14:31] yep, I've seen the same setup recommended by a lot of people, after a certain point I'm sure it become necessary [14:32] smoser: speaking of patches (saw your comment on the bug for env var support as well), I have a doc fix that I sent in a couple of days ago [14:34] where did i comment on that ? [14:34] i've lost it now [14:34] did i comment? [14:34] https://code.launchpad.net/~ffledgling/cloud-init/+git/cloud-init-1/+merge/312917 [14:34] * ffledgling was looking for the link to it himself, the launchpad interface is new and confusing [14:34] smoser: yeah, that one [14:35] ffledgling, so the link you want is https://code.launchpad.net/%7Ecloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews [14:35] then from there you can find things... [14:35] (that is the topic of this channel too) [14:35] also... [14:36] did you delete another one ? [14:36] you dont hve to delete a MP you can just push to it again (even push --overwrite) [14:36] then you dont lose the history of comments and such [14:36] I deleted the repo I think [14:36] I'd originally created repo called ~ffledgling/cloud-init/something/cloud-init [14:37] I don't think I've deleted an MP specifically [14:46] did i comment ? [14:46] i swear i commented somehwere. [14:47] smoser: I don't see it on the MP [14:47] I didn't get an email for a comment either, I just checked [14:48] Maybe it's saved as a draft or something? [14:48] well, i'llj ust type again [15:06] smoser: thanks [15:06] re: the patch for env variable support, is there a particular place in the code I should start with if I wanted to implement it? [15:21] ffledgling, i commented there now. sorry. i thought i did that yesterday but failed i guess. [15:23] smoser: thanks, I haven't looked at upstart (don't use ubuntu much outside of experiments), I'll take a look and get back to you === shardy is now known as shardy_afk [18:32] rangerpbzzzz, around ? [20:21] smoser: looking to detect if the running distro supports netplan; I'm checking 3 things 1) do we have /etc/netplan dir 2) do we have netplan binary in $PATH 3) do we have systemd-networkd . (3) is harder as it's not in $PATH and it's not a package (it's part of systemd) possibly optional on other distros ... thoughts on (3) ? Also, I was thinking this was a cloudinit.net.netplan method but each distro may need t [20:21] o check for support differently so could be part of the distro obj [20:25] why not just check util.which('netplan') ? [20:26] netplan should depend on systemd-networkd if it needs it [20:30] ok [20:33] dont you think ? [21:05] smoser: I think that's reasonable