[17:47] <smoser> powersj: c-i is busted for cloud-init . did you look at that ? sorry if i just missed you saying so.
[17:47] <powersj> hmm I wasn't aware till now
[17:48] <powersj> ah it is the same thing as with curtin, our lxd config wasn't handing out network addresses
[17:48] <smoser> ok. fixed ?
[17:48]  * powersj kicks a nightly run
[17:57] <powersj> looks good
[18:28] <dojordan> hey guys, would someone be able to take a quick look at my failed CI run? https://jenkins.ubuntu.com/server/job/cloud-init-ci/696/console It is failing with this: 2018-01-17 02:37:12,902 - tests.cloud_tests - ERROR - stage part: setup func for --deb, install deb encountered error: timeout: after 120s system not started
[18:29] <blackboxsw> hrm peeking
[18:29] <smoser> dojordan: i'll push 'build again'
[18:30] <smoser> dojordan: c-i was busted due to local lxd issue on those sytems
[18:30] <smoser> powersj fixed, so 'Rebuild' is in https://jenkins.ubuntu.com/server/job/cloud-init-ci/699/
[18:30] <smoser> so cross your fingers
[18:30] <blackboxsw> thx smoser
[18:31] <dojordan> thx all
[18:44] <dojordan> @smoser, finally got a pass. Can you take a peek at the changes?
[19:02] <smoser> dojordan: doing so
[19:35] <smoser> dojordan: does it work ?
[19:36] <dojordan> I've confirmed xenial does, still need to run an artful pass
[20:10] <tribaal> Hi folks. I have a bit of a weird issue that I suspect might be cloud-init related, and would love some pointers to investigate:
[20:11] <tribaal> the kubernetes test suite has a test that simulates a network split/problem by killing the main network interface on the host (a VM using ubuntu cloud images, with its networking set by cloud-init).
[20:12] <tribaal> the test fails because the network doesn't come back up. Rebooting the machine however brings the network back up as normal
[20:13] <tribaal> the specific method the test uses to kill the network for 120 seconds is ssh'ing in and doing "nohup sh -c 'sleep 10 && sudo ip link set eth0 down && sleep 120 && sudo ip link set eth0 up' >/dev/null 2>&1 &"
[20:13] <tribaal> rather crude, but it *should* work in theory
[20:15] <tribaal> I tried running that on GCE and AWS and both fail (with ubuntu xenial images at least)
[20:18] <blackboxsw> hrm.... that command also permanently brings down my network on the laptop (which didn't come back)
[20:18] <blackboxsw> and isn't deployed w/ cloud-init
[20:21] <blackboxsw> 123
[20:21] <blackboxsw> n/m I didn't wait long enough
[20:23] <blackboxsw> wonder if it's networkmanager vs networkd related?
[20:24] <smoser> tribaal: i'd suggest you can simplify greatly by using screen or tmux
[20:24] <smoser> rather than  nohup
[20:24] <smoser> and sh -xc
[20:25] <smoser> and sending output to a file (and stder)
[20:25] <smoser> then you can see what happened if you have to go back in
[20:28] <smoser> tribaal: i cannot reproduce such a failure in lxc though.
[20:29] <smoser> actually, i might be able to
[20:29] <smoser> yeah... doing what you're doing is not going to restore any routes
[20:30] <rharper> http://linux-ip.net/html/tools-ip-link.html
[20:31] <rharper> there's a section on side-effects of dropping link
[20:31] <rharper> Now when we down the link layer on eth0, we'll see that there is now no longer a flag UP in the link layer output of ip address. More interesting, though, all of our IP routes to destinations via eth0 are now missing.
[20:31] <rharper> I suspect the routing is busted
[20:31]  * rharper has to head out
[20:31] <rharper> bbiab
[20:56] <jgomo3> Given ruby as a package in the pakages list, and a commad like `gem install bundler`, How to be sure the command would run after the ruby package had been insalled?
[20:56] <smoser> i expect that is guaranteed
[20:56] <smoser> scripts with '#!' or runcmd will run after packages are isntalled
[20:59] <jgomo3> Thank you @smoser.
[21:00] <jgomo3> How can I learn about the order of excecution?
[21:00] <jgomo3> I don't find anything in the documentation... maybe I didn't see well.
[21:11] <smoser> jgomo3: well, not easily. it is in the modules order
[21:12] <smoser> in /etc/cloud/cloud.cfg
[21:12] <jgomo3> @smoser: Oh, I see. TY for the info.
[21:12] <smoser> in cloud_final_modules
[21:12] <smoser>  package-update-upgrade-install
[21:12] <smoser> runs before
[21:13] <smoser> scripts-user
[21:14] <jgomo3> @smoser: When you refer to "cloud_final_modules" are you talking about the sourcecode?
[21:15] <jgomo3> Oh, no, I see now. Thank you.
[21:55] <jgomo3> I'm trying to do `wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -` but in the `apt` -> `sources` section. What I think could work is to define a Source with `keyserver: https://www.postgresql.org/media/keys/` and `keyid: ACCC4CF8`. But seing the example at "additional-apt-configuration" at line 237, where there is an alternative keyserver, and analyzing the MIT keyserver, looks like there 
[21:57] <jgomo3> I'll try it, but I bet I'm going to waste a lot of time trying many combinations of  ways to write the `keyserver` and `keyid` values... So: Is there a way to define the URL of the key we want to add by apt-key somewhere in the `apt` section? Or should I write that command with wget in the bootcmd?
[22:06] <jgomo3> Antoher way I'm thinking is to modify `/etc/ssh/ssh_import_id` to add the postgres key webserver, and use ssh_import_id instead
[22:11] <jgomo3> Is it possible to modify that file (`/etc/ssh/ssh_import_id`) before ssh_import_id would run?
[22:12] <jgomo3> `write_files` would do it.
[22:24] <jgomo3> So, the real question would be: How to add the official Postgres repository via cloud init? -- https://wiki.postgresql.org/wiki/Apt
[22:27] <rharper> jgomo3: it appears that  for now, you'll need to download the key itself and include that in the 'key' field;  this will call apt-key add <contents of the key>
[22:29] <jgomo3> @rharper: That's right. Thank you!
[22:29] <rharper> I wonder why postgresql doesn't publish their key to known servers though
[22:53] <jgomo3> @rharper: They don't have to do it. And as they, there are many repositories who's keys are published not on "keyservers" but in a simple URL. ssh-import-key is aware of this, and allows to configure it with an URL pattern.
[22:53] <rharper> I know not everyone publishes; I'm just wondering why not; what's the downside of not publishing the gpg key you use to sign your packages
[22:55] <rharper> jgomo3: ssh-import-key ?
[22:56] <rharper> ssh-import-id ; interesting
[22:57] <jgomo3> ssh-import-id
[22:59] <jgomo3> No, is not that they don't publish it... they publish it in their own web servers, but not in a keyserver. Maybe Postgres don't trust that launchpad would be eternal, or they think is their responsability to maintain their keys in the Internet.
[23:00] <rharper> I meant publish the gpg key to the gpg keyservers; there are many gpg keyservers not hosted by canonical; you mentioned the MIT one, debian has one, etc;  I was just wondering why someone wouldn't
[23:01] <jgomo3> But, the whole point of a PKI is the distributed trust... so, people should change their minds and at least, publish in many trusted keyservers their keys, not only in one place. But, who am I to say anything :D
[23:01] <rharper> gpg --search-keys ACCC4CF8
[23:01] <rharper> gpg: searching for "ACCC4CF8" from hkp server keys.gnupg.net
[23:01] <rharper> (1)	PostgreSQL Debian Repository
[23:01] <rharper> 	  4096 bit RSA key ACCC4CF8, created: 2011-10-13, expires: 2019-07-02
[23:01] <rharper> looks like it's there
[23:01] <rharper> published
[23:02] <rharper> so maybe you can use keyserver: keys.gnupg.net and keyid: ACCC4CF8
[23:03] <rharper> https://paste.ubuntu.com/26407200/
[23:03] <jgomo3> Oh, that is great! Thank you!
[23:04] <rharper> yeah, maybe that postgresql APT faq can get an update that it publishes the key to gnupg.net
[23:08] <jgomo3> I was thinking exactly in that. I'll try to make it happen.
[23:10] <rharper> cool