| ffledgling | Hello, I'm trying to figure out how to get packages to install via cloud init behind a proxy, I haven't been able to find a good way to do so, are there any recommendations? Seems like there's an open bug about it - https://bugs.launchpad.net/cloud-init/+bug/1089405 | 08:51 |
|---|---|---|
| b0stik | ffledgling, tried with a bash command to add a line for proxy in apt configuration ? | 09:05 |
| ffledgling | b0stik: That's the last resort, and I'm not sure if the `packages` directive from cloud config is run before or after runcmd | 09:11 |
| ffledgling | Is there a guaruntee on the order in which things are run by cloud-init when reading from cloud-config | 09:46 |
| === shardy is now known as shardy_lunch | ||
| === shardy_lunch is now known as shardy | ||
| ffledgling | Answer to my previous question seems to be - https://git.launchpad.net/cloud-init/tree/config/cloud.cfg | 13:29 |
| smoser | ffledgling, i'm guessing this is not ubuntu / debian ? | 14:11 |
| smoser | you can define proxy for apt. | 14:11 |
| ffledgling | smoser: nope, fedora | 14:12 |
| smoser | but for general system proxy there isnt. | 14:12 |
| smoser | proxies are a regula pita | 14:12 |
| smoser | generally speaking. | 14:12 |
| smoser | ie, setting one up "system wide" never actually works. | 14:12 |
| smoser | thats myexperience. | 14:12 |
| smoser | that said, i'm open to patches that set environment in cloud-init. | 14:13 |
| ffledgling | That's not entirely unexpected, some tools don't support the standard proxy variables etc and then there's the lowercase vs uppercase issue | 14:13 |
| ffledgling | But yeah, it'd still be nice to have the ability to set variables in the env for at least the cloud-init parts | 14:14 |
| ffledgling | My current work around for this usecase is to use runcmd to add a proxy via a config file, but that relies on the order in which the steps are executed by cloud-init | 14:14 |
| ffledgling | Which is okay I guess, but fairly fragile if the ordering is changed | 14:15 |
| smoser | ffledgling, probably runcmd runs after package installation | 14:15 |
| smoser | so i'd use a bootcmd which is guaranteed to run before package installation | 14:15 |
| smoser | order is not fragile really. bootcmd will run as early as it can. | 14:15 |
| smoser | i suspect there is some way specifically to tell yum about package config ? | 14:16 |
| ffledgling | smoser: I didn't actually find where the package instruction is run, but for ubuntu it seems to be in the 'final' stage? | 14:16 |
| smoser | er... sorry. about proxy ? | 14:16 |
| ffledgling | smoser: /etc/yum.conf | 14:16 |
| smoser | with the move to systemd, package installation started happening at 'final' stage. it used to run at 'config' stage. | 14:16 |
| smoser | yeah, so waht i'd do is just in a boot cmd, edit/append to that | 14:17 |
| smoser | bootcmd: | 14:17 |
| smoser | - [sh, -c, 'echo http_proxy=http://proxy.host:port >> /etc/yum.conf'] | 14:17 |
| ffledgling | bootcmd + cloud-init-run-once (I think that's what it's called?) | 14:17 |
| ffledgling | otherwise isn't bootcmd run at every boot? | 14:17 |
| smoser | ah, yeah. bootcmd do run every time | 14:17 |
| smoser | so yeah, you coul use cloud-init-per too | 14:18 |
| smoser | or | 14:18 |
| smoser | just otherwise idempotently update that file | 14:18 |
| ffledgling | That might be a little harder | 14:18 |
| smoser | well, yeah, ini file | 14:19 |
| ffledgling | I was trying to figure out if there's a way to just tell cloud-init to take a file I hand it and stick it someplace on the target system | 14:20 |
| ffledgling | but I didn't find anything like that yet | 14:20 |
| smoser | well, there is write_files | 14:20 |
| smoser | http://cloudinit.readthedocs.io/en/latest/topics/examples.html#writing-out-arbitrary-files | 14:21 |
| smoser | and that runs very early | 14:21 |
| smoser | right after bootcmd | 14:21 |
| ffledgling | That might work as well | 14:21 |
| ffledgling | Doesn't say if it's written everytime or just once, I suspect, just one? | 14:22 |
| ffledgling | *once | 14:22 |
| smoser | that will run once per instance | 14:22 |
| smoser | so yeah, that'd probably work for you. | 14:22 |
| ffledgling | sounds good, I'll try both probably and see what works best | 14:23 |
| smoser | configuring yum is preferable in my opinion to configuring system wide, or relying on environment variables. | 14:23 |
| ffledgling | smoser: well, the problem is I really do need a system wide proxy for my eventual operations | 14:23 |
| smoser | well, you can write /etc/environment if you want. | 14:23 |
| ffledgling | Except in my case I'm using ansible to configure the VM after it's up so I can push that problem down there | 14:23 |
| smoser | my experience with system wide proxies is just they always end up sucking | 14:23 |
| smoser | for reasons largely of 'no_proxy' | 14:24 |
| ffledgling | Applications not respecting no_proxy or people not setting it? | 14:24 |
| smoser | ie, if the proxy is on the other side of the router, and it wont proxy some cidr, then globally setting http_proxy=that.proxy ends up sending all http requests to it, and it can't proxy for your network | 14:24 |
| smoser | most obvious example is: | 14:25 |
| smoser | http_proxy=http://example.com wget http://127.0.0.1:8000/my-test-application | 14:25 |
| smoser | but the same general thing occurs when the proxy doesn't work for your 10.0.0.0/24 or something. | 14:25 |
| ffledgling | right, that probably doesn't workt without no_proxy whitelisting your local network | 14:26 |
| smoser | and no_proxy only works very coarsely (python apps can't take a cidr, but some others doo) | 14:26 |
| smoser | ie, if you wantto white list 10.0.0.0/24 you end up having to do: | 14:26 |
| smoser | no_proxy=10.0.0.1,10.0.0.2,10.0.0.3,.....10.0.0.255 | 14:26 |
| smoser | which quickly becomes unmaintainable. | 14:26 |
| ffledgling | ah, right, no_proxy is "only" respects domains | 14:27 |
| ffledgling | s/is// | 14:27 |
| smoser | the only solution to such a thing that i've come accross is using a local proxy and pointing everything at that, and having its richer syntax do the selection for you. | 14:27 |
| smoser | ffledgling, well, no_proxy in some cases (i think wget) supports 10.0.0.* | 14:27 |
| smoser | but it does not in python's implementation and probably many others | 14:28 |
| smoser | http://bazaar.launchpad.net/~smoser/+junk/sstack-proxy/view/head:/sstack-proxy | 14:28 |
| smoser | that is what i've done that actually works... but comes at a cost of a middle man | 14:28 |
| smoser | then, you can change the config of tinyproxy and restart it, and you dont have to deal with getting the environment changed in existing processes. | 14:28 |
| smoser | but... that is just really neither here nor there... just sort of my experience with setting proxy and coming out hating the whole thing. | 14:29 |
| ffledgling | Yeah, proxies aren't really great, but i've just come to treat them as a necessary evil | 14:29 |
| smoser | the above is what i ended up having to do, and does work fairly well. | 14:30 |
| smoser | when you realize you forgot yourhost.com from the proxy config, you just add it in tinyproxy and restart tinyproxy | 14:30 |
| smoser | and tinyproxy config supports cidr | 14:30 |
| smoser | fully | 14:30 |
| smoser | so 192.168.2.0/24 or 9.0.0.1/32 | 14:30 |
| smoser | and ipv6 and all that... | 14:30 |
| ffledgling | yep, I've seen the same setup recommended by a lot of people, after a certain point I'm sure it become necessary | 14:31 |
| ffledgling | smoser: speaking of patches (saw your comment on the bug for env var support as well), I have a doc fix that I sent in a couple of days ago | 14:32 |
| smoser | where did i comment on that ? | 14:34 |
| smoser | i've lost it now | 14:34 |
| smoser | did i comment? | 14:34 |
| smoser | https://code.launchpad.net/~ffledgling/cloud-init/+git/cloud-init-1/+merge/312917 | 14:34 |
| * ffledgling was looking for the link to it himself, the launchpad interface is new and confusing | 14:34 | |
| ffledgling | smoser: yeah, that one | 14:34 |
| smoser | ffledgling, so the link you want is https://code.launchpad.net/%7Ecloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews | 14:35 |
| smoser | then from there you can find things... | 14:35 |
| smoser | (that is the topic of this channel too) | 14:35 |
| smoser | also... | 14:35 |
| smoser | did you delete another one ? | 14:36 |
| smoser | you dont hve to delete a MP you can just push to it again (even push --overwrite) | 14:36 |
| smoser | then you dont lose the history of comments and such | 14:36 |
| ffledgling | I deleted the repo I think | 14:36 |
| ffledgling | I'd originally created repo called ~ffledgling/cloud-init/something/cloud-init | 14:36 |
| ffledgling | I don't think I've deleted an MP specifically | 14:37 |
| smoser | did i comment ? | 14:46 |
| smoser | i swear i commented somehwere. | 14:46 |
| ffledgling | smoser: I don't see it on the MP | 14:47 |
| ffledgling | I didn't get an email for a comment either, I just checked | 14:47 |
| ffledgling | Maybe it's saved as a draft or something? | 14:48 |
| smoser | well, i'llj ust type again | 14:48 |
| ffledgling | smoser: thanks | 15:06 |
| ffledgling | re: the patch for env variable support, is there a particular place in the code I should start with if I wanted to implement it? | 15:06 |
| smoser | ffledgling, i commented there now. sorry. i thought i did that yesterday but failed i guess. | 15:21 |
| ffledgling | smoser: thanks, I haven't looked at upstart (don't use ubuntu much outside of experiments), I'll take a look and get back to you | 15:23 |
| === shardy is now known as shardy_afk | ||
| smoser | rangerpbzzzz, around ? | 18:32 |
| rharper | smoser: looking to detect if the running distro supports netplan; I'm checking 3 things 1) do we have /etc/netplan dir 2) do we have netplan binary in $PATH 3) do we have systemd-networkd . (3) is harder as it's not in $PATH and it's not a package (it's part of systemd) possibly optional on other distros ... thoughts on (3) ? Also, I was thinking this was a cloudinit.net.netplan method but each distro may need t | 20:21 |
| rharper | o check for support differently so could be part of the distro obj | 20:21 |
| smoser | why not just check util.which('netplan') ? | 20:25 |
| smoser | netplan should depend on systemd-networkd if it needs it | 20:26 |
| rharper | ok | 20:30 |
| smoser | dont you think ? | 20:33 |
| rharper | smoser: I think that's reasonable | 21:05 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!