[10:20] <cpaelzer> once one of you is up please help me to look at bug 1737704
[10:20] <ubot5`> bug 1737704 in cloud-init (Ubuntu) "Cloud-init seems not run on today's bionic images (20171211)" [Undecided,New] https://launchpad.net/bugs/1737704
[13:10] <cpaelzer> I'm not sure when you get online, but around now seems about right so doing a shotgun ping for the bug above
[13:10] <cpaelzer> smoser: rharper: blackboxsw: ^^
[14:13] <telling> Has anyone succesfully managed to make systemd stop using apt at boot? At random my cloud-init scripts will fail because apt is locked
[14:17] <smoser> cpaelzer: i'll take a look
[14:20] <cpaelzer> smoser: thank you, let me know if you need any debug data from me
[15:40] <smoser> rharper: or blackboxsw when you're in if you want to review
[15:40] <smoser>  https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1737704
[15:40] <ubot5`> Launchpad bug 1737704 in cloud-init (Ubuntu) "Cloud-init fails if iso9660 filesystem on non-cdrom path in 20171211 image." [High,In progress]
[15:40] <smoser>  http://paste.ubuntu.com/26170810/
[15:40] <smoser> that'd be good.
[15:40] <smoser> launchpad git is down, so no merge proposal that way right now. but if you can review then we can do that quicker when lp returns
[16:09] <smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/335086
[16:09] <smoser> blackboxsw: ^
[16:19] <smoser> powersj: if you kill https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-integration-nocloud-kvm-x/3/console
[16:19] <smoser> will it collect artifacts ?
[16:19] <smoser> hm..
[16:19] <powersj> smoser: not normally
[16:38] <smoser> powersj: fyi, i just ran this successfully:
[16:38] <smoser>  tox-venv citest python3 -m tests.cloud_tests run  --platform=nocloud-kvm --preserve-data --data-dir=results --verbose --os-name=xenial --repo ppa:cloud-init-dev/daily --test=tests/cloud_tests/testcases/modules/locale.py
[16:38] <smoser> 2017-12-12 16:36:12,825 - tests.cloud_tests - DEBUG - after setup complete, installed cloud-init version is: b'17.1-1723-g05b2308-0ubuntu1+1343~trunk~ubuntu16.04.1'
[16:39] <smoser> (i do have a patch locally to allow it to accept ppa:)
[16:39] <powersj> smoser: there is a --ppa option ;)
[16:40] <powersj> I'll go play with torkoal and see if it is something going on locally
[16:40] <smoser> oh funy. i migth submit a patch to remove it then and just support ppa in --repo
[16:40] <powersj> ok
[16:47] <smoser> blackboxsw: ^ https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/335086 please
[17:00] <smoser> powersj: http://paste.ubuntu.com/26171272/
[17:00] <smoser> that is "launch stuff on clouds" that i have.
[17:00] <smoser> blackboxsw: where do you think that should go? we said qa-scripts, but im' not sure if it fits ther... maybe in a doc/ dir ?
[17:01] <powersj> smoser: thx
[17:01] <blackboxsw> smoser: I was thinking qa-scripts/doc
[17:01] <powersj> I was able to launch xenial nocloud-kvm test with built in cloud-init, so now building the tree and trying that one
[17:01] <blackboxsw> then we can adapt those docs into scripts as we have cycles
[17:11] <torontoyes> can anyone point me to documentation to use cloud-init for deploying windows 10 on bare metal?
[17:12]  * blackboxsw gueses that's going to be https://cloudbase.it/cloudbase-init/ not cloud-init
[17:14] <powersj> smoser: looking at centos failures, I think that was a temporary connectivity/DNS issue and as a result we were unable to update our yum sources. Wasn't able to reproduce locally or on torkoal and the re-run seems to be working
[17:14] <powersj> smoser: now triaging why the nocloud-kvm xenial test is failing with the built deb.
[17:16] <torontoyes> blackboxsw: thanks
[17:21] <powersj> smoser: using the cloud-init-dev/daily ppa with xenial is not going to give you an up to date deb. Looks like Xenial hasn't been built in 5 days
[17:21] <powersj> https://launchpad.net/~cloud-init-dev/+archive/ubuntu/daily
[17:23] <powersj> here is the xenial build failure: https://launchpadlibrarian.net/348862957/buildlog.txt.gz
[17:24] <powersj> ds-identify-behavior-xenial.patch does not apply
[17:40] <powersj> smoser: it looks like your ds-identify fix will unbreak nocloud-kvm tests
[17:41] <powersj> applied it and re-ran locally and tests can run
[17:45] <blackboxsw> smoser I see your ds-identify branch properly fixed the issue https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/335086.
[17:50] <blackboxsw> trivial branch update for droping an unsupported modules cmdline param https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/335094
[17:50] <blackboxsw> dropping even
[17:53] <blackboxsw> ok merging https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/335052 and  https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/335086.  to master
[18:07] <smoser> blackboxsw: https://github.com/cloud-init/qa-scripts/blob/master/doc/launching.md
[18:11] <blackboxsw> good good. almost done w/ a merge-tool for cloud-init.
[18:12] <blackboxsw> wanted to test it on powers' branch
[18:28] <blackboxsw> waiting on tox
[18:32] <smoser> blackboxsw: let me know when you land and i will pull and upload to ubuntu
[18:33] <blackboxsw> smoser: the script I hvae needs work. I'm merging now. and will test on the next branch post this release.
[18:33] <blackboxsw> smoser: do we still want to do a revision bump in cloud-init master
[18:33] <blackboxsw> prior to next upload
[18:33] <blackboxsw> ?
[18:34] <dojordan> @blackboxsw, what else is needed to land my PR?
[18:36] <blackboxsw> dojordan: I think you are good on your branch. in trying to limit deltas I don't think we are landing it for this release. but right afterward. So it should land just after the 17.2 cut. I was able to test and exercise both the markerfile path and non-marker path and watch expected behavior (like camping on an infinite polling loop for IMDS  or and not taking that path on normal installs)
[18:37] <blackboxsw> dojordan: my expectation is that it will be in ubuntu bionic images within the week. Just not at official upstream 17.2 cut.
[18:37] <blackboxsw> it'll be 17.2.X
[18:37] <blackboxsw> dojordan: you also mentioned it's Xenial-only to start right?
[18:38] <blackboxsw> I expect we'll have an Ubuntu SRU into xenial with that shortly into the new year
[18:39] <blackboxsw> even stuff that's landing right now in master isn't going to make it into xenial until we go through our first SRU of the new year.
[18:40] <blackboxsw> and our next SRU to xenial will take an upstream snapshot which would include whatever is landed in master. (post 17.2 cut).
[18:41] <dojordan> Correct, but any Azure image will unblock us until we GA
[18:45] <blackboxsw> dojordan: sorry that went over my head, or didn't jive with my understanding of your branch. Do azure xenial images currently ship with the marker file or ovf PreprovisionedVM content setting?
[18:46] <blackboxsw> dojordan: ohh you mean that markerfile or configuration won't exist in azure images until your team GA's on the IMDS service?
[18:47] <dojordan> no, sorry let me try again. As long as my changes land in an azure image it will help unblock our testing. But when we go GA (middle of next year hopefully), we will start only preprovisioning (using the ovf setting) 16.04 LTS. The purpose of the marker file is incase we occur a VM reboot in the middle of the process
[18:47] <dojordan> @blackboxsw yes
[18:48] <blackboxsw> ok undrestood. thanks for clarification dojordan
[18:48] <dojordan> @blackboxsw, does that make sense?
[18:49] <blackboxsw> ok so from my understanding, your branch not landing today doesn't block your testing currently right? The cut of 17.2 upstream release from ubuntu perspective won't change the speed at which xenial images get your branch because of the following:
[18:50] <blackboxsw> -  ubuntu's policy for cloud-init updates to non-bionic (xenial, zesty, artful) is we take latest snapshot from master.
[18:50] <blackboxsw>  - I'm currently waiting on the last SRU to land per https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1733653
[18:50] <ubot5`> Launchpad bug 1733653 in cloud-init (Ubuntu Artful) "sru cloud-init (17.1-27-geb292c18) update to 17.1.46-g7acc9e86-0ubuntu1" [Medium,Fix committed]
[18:51] <blackboxsw>  which will update xenial ,zesty, artful to pre-17.2 upstream release
[18:53] <blackboxsw>  - If we perform a 17.2 sru to xenial, zesty artful (takes a week or two to test/publish), we can still land new merges into master after that snapshot is taken
[18:53] <blackboxsw>  - anything that lands in master will be available in ubuntu bionic series within a day
[18:53] <dojordan> sorry, what is SRU?
[18:53] <blackboxsw> which would unblock testing if you have the ability to use bionic images
[18:53] <dojordan> great, that works for us
[18:53] <blackboxsw> sorry acronym nightmares
[18:54] <blackboxsw> "sru" = stable release update
[18:54] <blackboxsw> https://wiki.ubuntu.com/StableReleaseUpdates ubuntu stable updates
[18:54] <blackboxsw> specifically for cloud-init we have to do the following  additional verification work: https://wiki.ubuntu.com/CloudinitUpdates
[18:55] <blackboxsw> so that's the only long-pole for publishing to xenial
[18:55] <blackboxsw> and that process we've gotten down from about 4-6 weeks to about 2 weeks as we are trying to perform these SRUs more frequently
[18:58] <robjo> smoser: Sorry, about being thick headed about this, but I still fail to see the point of http://paste.ubuntu.com/26166233/
[18:59] <robjo> I really have no intentions on spending much more time on writing additional tests for the "old network config path"
[19:00] <robjo> I have no problem with the changes, I am just missing the point
[19:01] <dpb1> is it covered at all?
[19:02] <smoser> robjo hm.. well i dont knwo. I'm not bent on it if you're going to drop that code.  But if you're going to do so, then adding a test of it immediately before you do so doesn't help much.
[19:02] <smoser> right ?
[19:02] <smoser> but if you're going to fix it (as one of your MPs) was doing,t hen it makes sense to add a test and make the code more testable.
[19:03] <smoser> that patch is fairly simple code motion.
[19:03] <robjo> yes and no, since the test is already written and covers what was recently broken it does do the trick
[19:04] <smoser> i didntthink it *did* cover what was recently broken
[19:04] <robjo> yes, and I guess the light in my head is not coming on about the advantage of moving the code out of the class
[19:04] <blackboxsw> smoser: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/335099 for ubuntu/devel snapshot update w/ your and powersj changees
[19:04] <smoser> testing via 'distro.apply_network' is just more complexity, which required you to use the filesystemmocking test.  this just seemed easier.
[19:04] <smoser> blackboxsw: oh nice. thanks. i'll pull
[19:07] <robjo> two things broke recently, when I built and released 17.1 in openSUSE w.r.t. networking, 1.) there is a path where network was set to "manual" for start up, that's obviously not useful in a cloud environment and 2.) there was no path from v1 to the translation of the network config
[19:09] <robjo> The latter was addressed in "netV1ToTranslate" branch and if I remember correctly the comment there was "this should be tested"
[19:10] <robjo> Anyway it's sufficiently aged that I do not remember all the details
[19:10] <smoser> :-).
[19:11] <smoser> i think all your mp are generally in reasonable shape, there were just things we wanted to take care of to make them more maintainable.  the state things are in isnt your fault, but we want things to get better.
[19:12] <smoser> blackboxsw: uploaded.
[19:14] <blackboxsw> thx smoser
[19:14] <robjo> I am in support for making things better, from my perspective that means in the not too distant future rip out _write_network() and figure out how to get to the sysconfig renderer
[19:19] <robjo> Is there a way to banish ijw for the rest of the day, may the network trouble on that end will resolve itself tomorrow?
[19:19] <smoser> i dont think so :)
[19:19] <smoser> at least not that i'm aware of
[19:32] <rharper> smoser: well, we've ops, we can kickban
[19:33] <blackboxsw> robjo: what IRC client do you use?
[19:33] <robjo> pidgin
[19:34] <blackboxsw> from xchat I can disable join/departure messages. ahh used to use pidgin, wasn't sure if that was a config option.
[19:34] <smoser> wow. pidgin
[19:34] <blackboxsw> for a moment I was wondering if you were talking about some network software stack ... didn't realize it was the re-join irc messages
[19:35] <smoser> i used pidgin because it was the nicest chat client with IBM/Lotus sametime support.
[19:35] <rharper> yes
[19:35] <rharper> meanwhile
[19:35] <smoser> and i think it allowed you to change your user agent
[19:35] <rharper> sneaky
[19:35] <rharper> cheat the sametime servers
[19:35] <rharper> blamed the clients for crashing the server
[19:35] <smoser> so that you could avoid being shutoff for identifying yourself as using an unapproved client.
[19:35] <smoser> :)
[19:35] <rharper> classic IBM
[19:35] <smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/335100
[19:36] <rharper> +1
[19:37] <rharper> smoser:  I think that upload to bionic that bothered cloud-init hit curtin's vmtest on bionic
[19:37] <rharper> it caused no network, right?
[19:37]  * rharper will kickban next join
[19:38] <smoser> rharper: well, cloud-init disalbes itself.
[19:38] <smoser> but it would only hit that path if you had an iso9660 filesystem on something not /dev/cdrom or /dev/sr0
[19:38] <rharper> right, I think that's how we launch in xkvm
[19:38] <rharper> but lemme check
[19:39] <smoser> so it doesnt seem likely, unless we're misunderstanding. no we seed rom url dont we ?
[19:39] <rharper> for the boot
[19:39] <rharper> after install
[19:39] <rharper> we inject user data for collection
[19:39] <rharper> vi an nocloud seed
[19:39] <rharper> iso
[19:39] <smoser> ah. yeah. then that is it.
[19:39] <smoser> yes.
[19:39] <rharper> -drive file=/var/lib/jenkins/slaves/torkoal/workspace/curtin-vmtest-devel-amd64/output/BionicTestBonding/boot/seed.img,id=disk02,if=none,format=raw,index=2,media=cdrom
[19:40] <rharper> -device virtio-blk,drive=disk02,serial=seed.img
[19:40] <smoser> hm... media=cdromn
[19:40] <robjo> rharper: thanks
[19:40] <rharper> so, I think that trips it
[19:40] <rharper> it's not IDE
[19:40] <rharper> robjo: sure
[19:40] <rharper> it's virtio device, with iso9660 filesystem
[19:40] <rharper> so, not /dev/sr*
[19:40] <smoser> i wondered how virtio cdroms would show up.
[19:40] <rharper> as a virtio device
[19:40] <rharper> there are no virtio cdroms; just virtio block
[19:41] <smoser> they just show as a /dev/vdx
[19:41] <rharper> right
[19:41] <rharper> which I think fails the uploaded check
[19:41] <rharper> right?
[19:41] <rharper> well, before your fix
[19:42] <smoser> hm..
[19:46] <smoser> i'm just thinking. i knew it wasnt a great filter, but we are currently allowing OVF only on a /dev/sr? device or a /dev/sr[0-9]|/dev/hd[a-z]
[19:51] <rharper> we do a blkid query for is9660 filesystem; do we also check sr ?
[19:52] <smoser> i guess if we get a bug for supporting OVF on virtio cdrom then we can just open that up a bit.
[19:52] <smoser> we do blkid query yes. and filter ovf by iso9660 fs based on that.
[19:52] <smoser> but the check for "is this an OVF iso transport" is
[19:52] <smoser>  grep http://<dmtf>url<here> /dev/<path>
[19:53] <rharper> ovf supports just iso9660, not where it comes from
[19:53] <smoser> so in order to avoid that as far as possible will only do that on a cd device
[19:53] <smoser> its not perfect
[19:53] <rharper> hrm
[19:54] <rharper> it seems to me that ovf is less reliable, but things like labels are better; can we order the higher quality "yes" answers first ?
[19:54] <rharper> for example, OVF can only be a yest if we fail to find the cidata/config-2 labels ?
[19:54] <smoser> https://www.dmtf.org/sites/default/files/standards/documents/DSP0243_2.1.1.pdf
[19:54] <rharper> when dealing with iso9660 checks ?
[19:55] <smoser> it prety clearly says "cd-rom device"
[19:55] <rharper> well, vmware fails that
[19:55] <smoser> ?
[19:55] <smoser> they attach a cdrom
[19:56] <smoser> no?
[19:56] <rharper> there is an file in dir support
[19:56] <rharper> we tested this via lxd
[19:56] <smoser> sure. and that will get identified.
[19:56] <smoser> and that is fine.
[19:56] <smoser> we only go down to  looking for a ovf on a cdrom if other measures fall out.
[19:56] <smoser> and we explicitly avoid known lables
[19:57] <rharper> ok, then I'm not sure why you're worried about that gep on sr ?
[19:57] <rharper> grep on sr*
[19:57] <smoser> http://paste.ubuntu.com/26172208/
[19:57] <rharper> oh, I see your question w.r.t virtio cdrom
[19:57] <rharper> it's not really a cdrom
[19:57] <smoser> because if you attach a iso9660 ovf via virtio it will not be found right now.
[19:57] <rharper> yeah
[19:58] <smoser> which is probably "oh well"
[19:58] <rharper> yeah
[19:58] <rharper> well, they can always use a virtio-scsi cdrom device, whihc does show up as an sr0
[19:58] <rharper> instead of a virtio-disk
[19:59] <rharper> not that they can tell
[19:59] <rharper> or know to do that
[19:59] <smoser> ah. ok yeah.
[19:59] <smoser> :)
[19:59] <smoser> yeah, i was looking to do taht.
[20:13] <blackboxsw> smoser: rharper can you guys remind me again what https://launchpad.net/ubuntu/bionic/+queue?queue_state=3&queue_text=cloud-init a "proposed" pocket done means for cloud-init?
[20:13] <blackboxsw>  
[20:13] <blackboxsw> rmadison still shows cloud-init rev ....58 instead of 60. I wonder how long we wait for updates in bionic when we our updates hit the proposed pocket
[20:14] <smoser> blackboxsw: https://launchpad.net/ubuntu/+source/cloud-init
[20:14] <smoser> it is in -proposed, or pending a publisher runwill be
[20:15] <blackboxsw> so is the publisher run only daily, or more frequently
[20:16] <smoser> hourly-ish ?
[20:16] <smoser> something on that order
[20:19] <smoser> blackboxsw: when i was testing resolvconf things adn was impatient
[20:19] <smoser> i did this
[20:19] <smoser>  http://paste.ubuntu.com/26172361/
[20:19] <smoser> oh. /me updates gist. forgot i put it there
[20:20]  * blackboxsw reads that. and nice tracking gist comment :)
[20:20] <blackboxsw> I was curious per a discussion in #ubuntu-release.  I wanted to tell folks around what time they could expect fixes to be seen
[20:22] <blackboxsw> ok SRU looks unblocked
[20:22] <smoser> anyway, what you can do is:
[20:22] <smoser> POCKET=bionic wait-for-package cloud-init 17.1-60-ga30a3bb5-0ubuntu1 && mpg321 ~/Music/22-Andrew_Lloyd_Webber-Joseph_Megamix.mp3
[20:23] <smoser> i'm not exactly sure how often things get copied. i think proobably more frequently for development releases.
[20:29] <blackboxsw> hehe will have to spin up the "Amazing technicolor deamcoat mix" as general policy
[21:13] <blackboxsw> approved https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/335051 with unrelated question about direction for ec2.
[21:15]  * powersj looks
[21:19] <blackboxsw> I'm guessing with EC2, we'd probably also have a ec2.publish_keys method or something in the EC2Instance.start method to ensure we've upoaded the known key to EC2.
[21:22] <powersj> blackboxsw: for ec2 you can take a sneak peak at https://git.launchpad.net/~powersj/cloud-init/commit/?id=2d5c6d156cb4506260cb1abef54daefb7a0ffe05
[21:22] <powersj> specifically tests/cloud_tests/platforms/ec2/platform.py
[21:22] <powersj> and  def _upload_public_key(self, config):
[21:41] <dpb1> powersj burning down the AWS infra
[21:42] <blackboxsw> https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/335053 questions about where we ultimately should be controlling/documenting integration test deps
[21:42] <blackboxsw> thx for the peek powersj looking
[21:43] <powersj> blackboxsw: agreed on tox discussion
[21:43] <powersj> I was a little confused why it even came up given paramiko has been in there for a while. All I thought I did was move it from one file to another
[21:44] <blackboxsw> powersj: maybe we need to extend tools/read-dependencies to also handle these one-off integration test deps... as I'd really like to see make ci-deps-ubuntu work for everything (not just unit test deps)
[21:57] <smoser> blackboxsw: i'm interested in thoguhts on https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/335108
[21:59] <blackboxsw> checking it out smoser
[22:01] <blackboxsw> hrm I thought oauth-key was based on the user client connecting to maas. accessing a maas instance to peek at the metadata now
[22:02] <smoser> blackboxsw: well, maas sends to curtin the credentials for this node. and those get written into /etc/cloud/cloud.cfg.d
[22:03] <smoser> and then cloud-init uses those to talk to maas.
[22:03] <blackboxsw> ohh these are cloud-init's keys to talk back to maas. ok so they'd be unique per node
[22:04] <smoser> well, they are unique per install
[22:04] <smoser> after https://bugs.launchpad.net/maas/+bug/1507586
[22:04] <ubot5`> Launchpad bug 1507586 in MAAS "previous owner of node can use oauth creds to retrieve current owner's user-data" [Critical,Fix released]
[22:05] <blackboxsw> who is that smoser guy, he sure finds a lot of bugs
[22:05] <smoser> that bug was really just a sneaky way of me trying (unsuccessfully) to get bug 944325 fixed.
[22:05] <ubot5`> bug 944325 in MAAS "no separation of instance id from node id" [Wishlist,Triaged] https://launchpad.net/bugs/944325
[22:05] <smoser> "maybe if i find a security vulnerability they'll add a feature for me"
[22:23] <blackboxsw> smoser: just responded, on your branch. I know the approach you are taking looks like it makese sense for how cloud-init currently looks like it will work,  but I wonder about a couple of things noted in the comments