[02:15] thx smoser [02:17] artful on its way === otubo1 is now known as otubo === caribou_ is now known as caribou [13:54] Hello ! @smoser & team just so you know, we've deployed cloud-init on our Scaleway infrastructure last week \o/ [13:58] caribou: hey. great. [14:02] I'm backporting my MR to your latest release so our IPv6 config works [14:10] ok. thanks. [15:03] Question : what could cause di_report (in init.cfg) to be empty on Bionic and have {'datasource_list': ['Scaleway', 'None']} on Xenial (same 18.3 package from -proposed) [15:08] matter of fact, on Bionic I have init.cfg['datasource_list'] = ['Scaleway', 'None'] and on Xenial I have init.cfg['di_report'] = {'datasource_list': ['Scaleway', 'None']} [15:10] Maybe I should explain what I'm after : the boot of a Bionic instance with cloud-init has a long (~2min) delay when 'cloud-init init' executes. Last line in the log is `main.py[DEBUG]: no di_report found in config.` [15:12] oh, and it only happens on a 'real reboot' of an instance. Doing clean & init --local & init does not show that delay [15:13] caribou: i'm confused. [15:13] smoser: I am too ;-) [15:13] can you post log with the delay ? [15:13] sure, hold on [15:13] xenial runs in "report only" mode for ds-identify [15:13] bionic runs in "enabled" [15:14] you should not need to specify the datasource_list in either case [15:14] i'm pretty sure.. i think it will identify scaleway correctly [15:14] yeah, I noted the difference in /run/cloud/ds* [15:15] smoser: http://paste.ubuntu.com/p/bkfjyfRQ5g/ [15:16] smoser: sorry wrong log [15:17] di_report might just be a red herring [15:18] di_report should not change behavior at all. but it also should not cause any sort of warning. [15:19] * caribou is just waiting for the reboot to complete [15:20] (I'll run a test with the vanilla cloud-init) [15:23] smoser: http://paste.ubuntu.com/p/bC394TwdQP/, Line 245 [15:32] hrm no di_report in config means no ds_identify ran is that right? [15:34] which makes me think disabled systemd units or something [15:34] ... but that's just me grasping at straws [15:36] blackboxsw: that could be our funky image building process getting in the way [15:36] btw 18.3-0ubuntu1~18.04.1 from proposed does the same thing [15:37] ok, don't spent time on that, I'll run more tests with something that resemble more to an ubuntu image [15:37] well, I mean an ubuntu image built a different way [15:38] blackboxsw: but ds-identity did run, /run/cloud-init/ds-identity.log is there [15:38] caribou: just to be perfectly clear... the goal for us is that you (scalway) does not have to know anything about cloud-init to get it to work right (other than how to install the package). [15:39] smoser: indeed, but so far I've seen a few occurences of some of our 'customization' getting in the way [15:40] & I still need to have some level of knowledge to babysit DataSourceScaleway [15:41] yeah. i undersand that... but the goal is to get rid of those things. [15:42] ok, let me spend some more time on this & I'll let you know what I find [15:44] if you want to let me in to look around... thats fine. [15:50] oh sure hold on [15:52] smoser: are you repushing the artful upload? [15:52] the queue seems stale from last night [15:53] I was going to try pinging in ubuntu-release when ready [16:01] blackboxsw: sorry. i get all those correct. [16:02] blackboxsw: "queue seems stale" ? [16:03] smoser: I meant the timestamp on the upload was 13 hours ago [16:03] and I thought you mentioned you needed to reupload [16:03] https://launchpad.net/ubuntu/artful/+queue?queue_state=1&queue_text=cloud-init [16:03] ah. all of them need re-upload [16:03] because i did not pass '-v' to debuild [16:03] I'll add that to the note for https://github.com/CanonicalLtd/uss-tableflip/pull/1 [16:04] on SRU regression re-uploads [16:04] have we put build-package somwhere ? [16:06] hrm don't see it in the repos... looking around [16:06] https://github.com/cloud-init/qa-scripts/blob/master/scripts/build-package [16:07] hmm should that be in uss-tableflip [16:07] instead of qa-scripts [16:10] blackboxsw: ok. i'll move it and add the feature to add -v based on rmadison [16:10] ok thanks [16:30] blackboxsw: ok. uploaded, and moved build-package and updated build-package to pass '-v' based on rmadison [16:31] excellent [17:09] caribou: i believe your bionic issue is related to bug 1780062 [17:09] bug 1780062 in linux (Ubuntu Bionic) "Cloud-init causes potentially huge boot delays with 4.15 kernels" [High,Fix committed] https://launchpad.net/bugs/1780062 [17:10] the "smoking gun" there is [17:10] Jul 10 10:19:34.562583 cloudinit-bionic systemd[1]: Starting Initial cloud-init job (metadata service crawler)... [17:10] Jul 10 10:21:37.672833 cloudinit-bionic kernel: random: crng init done [17:10] Jul 10 10:21:37.979128 cloudinit-bionic useradd[822]: new group: name=ubuntu, GID=1000 [17:10] in output of 'journalctl -o short-precise' [18:07] blackboxsw: did mgerts have any MP up working off your update_events commit (https://git.launchpad.net/cloud-init/commit/?id=be9ecc12823) ? [18:08] caribou's mp at https://code.launchpad.net/~louis/cloud-init/+git/cloud-init/+merge/347973 would want to use that [18:08] also, and was looking for any work that already existed [18:12] smoser: I pinged mgerts on that during the last week when it landed and he said he'd look at it. https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712 [18:12] I see no updates to his branch though [18:13] but as an example branch that leverages the changes I have one for azure https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/348704 [18:13] thanks [18:13] about line 43 of the visual diff [18:55] larsks: around ? [18:55] Maaaaaaaaybe. [18:55] outside of just continually throuwing stuff at copr [18:55] how can i get a rawhide environment to debug [18:55] https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/build/775457/ [18:57] smoser: do we want to re-upload cloud-init SRU without the additional bug references? slangasek is asking in #ubuntu-release [18:58] Steve Langasek blackboxsw: of course, now we have the situation that the cloud-init upload links to bugs that have no SRU test case: LP: #1780481 LP: #1770712 do you want to add SRU templates to those, or adjust the changelog to not reference them? [18:58] Launchpad bug 1780481 in cloud-init "ubuntu/centos/debian: get_linux_distro has different behavior than platform.dist" [High,Fix committed] https://launchpad.net/bugs/1780481 [18:58] Launchpad bug 1770712 in cloud-init (Ubuntu Bionic) "It would be nice if cloud-init provides full version in logs" [Medium,Fix committed] https://launchpad.net/bugs/1770712 [18:58] smoser: Huh, I'm not sure. I don't think there exists anything like a rawhide image. [18:58] From the build log it looks like /usr/bin/python doesn't exist. Maybe it is only /usr/bin/python3? [18:58] 12:56 S Steve Langasek blackboxsw: ok. Process on https://wiki.ubuntu.com/CloudinitUpdates dsoes say "single process bug, instead of individual bug reports" [18:58] Let me ask around #fedora-cloud. [18:59] larsks: yeah, i assume that is the cause [18:59] but i want to be reasonably confident in a fix that works tehre and elsewhere [18:59] Yup. Let me see what I can find out. [19:10] smoser: https://fedoraproject.org/wiki/Changes/Move_usr_bin_python_into_separate_package [19:11] So if you want /usr/bin/python to exist (and be python2) you can install python-unversioned-command, or you can just explicitly use /usr/bin/python2 or /usr/bin/python3. [19:11] (And have a buildrequire on whichever one you use) [19:20] smoser: my response in #ubuntu-release was "I thought this was an exception to the exception due the the fact that we found a potential regression in the published -proposed version and we needed something to document/validate per a new delta. [19:22] blackboxsw: sorry... reading [19:23] yeah I think the concern was that we were documenting more bugs than just the original SRU process bug (per this sru-test-regression fix) [19:24] blackboxsw: we only added 1 bug [19:24] number [19:24] I don't really feel like this warrants another upload attempt to fix it (stripping all bugs referenced) because we discovered new bugs that may be worth documenting, but it doesn't exactly adhere to the cloudinitupdates exception [19:25] and i had chosen to add that number specificly because it was relevant [19:25] true and pasted comment over in ubuntu-release ... can continue conversation there [19:26] i think we shoudl add a sru template to the version bug (1770712) [21:34] smoser: thanks for the sru template. just finished openstack, gce and azure manual tests. [21:34] pushing results to ubuntu-ru [21:34] pushing results to ubuntu-sru