=== liqw_ is now known as liqw
=== shardy is now known as shardy_lunch
=== shardy_lunch is now known as shardy
[14:29] <ys__> Hi I am running openstack with cloudinit pre-installed in all images. Just wondering how cloudinit recognize whether a vm is a new vm or it is a old vm if vm is rebooted?
[15:14] <rharper> ys__: hi, cloud-init uses an instance-id, provided by OpenStack's metadata service, this is stored (along wtih other per-instance variables) in /var/lib/cloud/instances/<instance-id>;  if you boot the same image in a new instance (say you captured the disk and booted a  new one) Openstack assigns a new instance-id, and cloud-init knows that it's not run as the new instance id and does it's init as the first time
[15:24] <ys__> rharper: Hi thanks. What I actually wants to do is that, I am running openstack and boot vm from cinder volume. After I terminate the vm, I can keep the root volume and boot a new instance from this volume. However, I don't want to reset the password for that instance since the password has been configured previously. Is there a way I can do this?
[15:28] <rharper> ys__: I'm not sure how we'd do that; but let me look; in general, openstack is going to give you a new instance-id when you boot a new instance
[15:31] <ys__> rharper: Thanks!
[16:01] <blackboxsw> ok couple minutes folks and we'll be starting the cloud-init meeting
[16:02] <blackboxsw> smoser: is out this week so I'll chair. Just digging up the meeting details now
[16:05] <blackboxsw> #startmeeting
[16:06] <blackboxsw> hi all, Ok here we go cloud-init status meeting starts now we'll go through recent changes, inprogress work and office hours for open discussions
[16:06] <blackboxsw> #topic recent changes
[16:07] <blackboxsw> here is a list of content that has landed on tip  over the last 2 weeks
[16:08] <dpb1> o/
[16:08] <blackboxsw> - We no longer run locale-gen if requested system locale is default . saving time on instances which are already configured with the appropriate locale
[16:08] <blackboxsw> - small SRU verifcation of #1690430
[16:08] <blackboxsw> - Removed Yakkery tests due to series end of life
[16:09] <blackboxsw> - cc_ntp timesyncd- support (when missing ntp in environment) Ubuntu Core or any image that doesn't have package manager  bug#1686485
[16:09] <blackboxsw> - Fix /etc/resolve.conf comment added on each reboot (LP: #1701420)
[16:09] <blackboxsw> - Fix integration test building local tree (avoid symlink chasing of sockets)
[16:09] <ubot5> Launchpad bug 1701420 in cloud-init "Created by cloud-init comment being added on every reboot to /etc/resolv.conf" [Low,Confirmed] https://launchpad.net/bugs/1701420
[16:10] <blackboxsw> rharper: dpb1 can you think of anything else I've missed here?
[16:11] <blackboxsw> if not, we can head to InProgress work
[16:12] <dpb1> nothing here
[16:13] <blackboxsw> ok yeah looks like smoser also landed a quick one related to     archlinux: Fix bug with empty dns, do not render 'lo' devices.
[16:13] <blackboxsw> ok next topic
[16:13] <blackboxsw> #meetingtopic Ongoing (In-progress) Work
[16:15] <blackboxsw> As smoser mentioned last meeting we were trying to get AWS IPv6 support in shape to land. 2/3's of that work is up, DataSourceAWSLocal will proform an initial dhcp4 discovery on a nic and query the meta-data source to find any ipv6 support designated for the nics on the instance.
[16:16] <blackboxsw> we will likely land the 3rd branch (querying the ipv6 from ec2 metadata and configuring all nics accordingly).
[16:17] <blackboxsw> this change required the datasource to know about a newer datasource 2016-08-02 I think instead of the current default which is 2009-04-04.
[16:17] <dpb1> newer "metadata format" ?
[16:18] <ajorg> this still relies on DHCPv6 to configure the address, right?
[16:18] <blackboxsw> correct dpb1 , the 2016-08-02 version surfaces ipv6 info we care about consuming.
[16:19] <blackboxsw> ajorg: the branch you looked over does the initial dhcpv4 on the fallback nic(eth0) then hits the 2016-08-02 metadata if it exists.
[16:20] <blackboxsw> ajorg: the followup (work in progress) branch will check ipv6 configuration defined in the metadata and then properly configure all devices which need dhcpv6  because the metadata service itself doesn't quite give us enough information to fully (statically) configure ipv6
[16:20] <ilianaw> is it just missing the gateway?
[16:21] <blackboxsw> ilianaw: I believe it was minimally missing gw. let me see what else (....digging up the ec2 metadata description page)
[16:21] <ajorg> okay, that's correct, afaik. configuring it statically probably isn't the right choice anyway.
[16:23] <blackboxsw> ajorg: ok thx. Yeah so this dhcp4 discovery we do initially is only required because we can't contact the link-local metadata addr 169.254.169.254 with our own statically set link-local 169.254.x.y address. Something in aws only allocated/authorizes a specific source dhcp IP address to allow communication to the metadata service
[16:24] <blackboxsw>   so our discovery lifecycle is now   : cloud-init init-local   - dhcp4 discovery eth0 - > wait for base to metadata 2009-04-04 up -> check for 2016-08-02  -> check ipv6 true/false - > dhcp6 on desired interfaces
[16:24] <rharper> it could be cool to pass in the correct ip address that was assigned via non-network methods (like DMI table entry)
[16:24] <ajorg> right.
[16:25] <blackboxsw> right on DMI/smbios or some systrem env variable. it would save the first couple steps (timewise)
[16:25] <blackboxsw> but that's a cloud platform change and would take some time we assume, so expedience of ipv6 support is currently the above
[16:26] <blackboxsw> so ajorg I noticed comments on my branch. I'll add more docs to it, but that's the general approach for now
[16:27] <blackboxsw> and as mentioned before DigitalOcean datasource I believe also does the init-local timeframe link-local network setup to discover more net config info from their metadata service too. So this approach is probably one we will grow in other datasources in the future
[16:27] <blackboxsw> if there are other question, feel free to fire 'em off.  I'll move to the next topic, but we can still chat about it
[16:28] <blackboxsw> We started another ubuntu SRU getting some bug fixes back into xenail and zesty https://trello.com/c/CAjwe8LX/273-cloud-init-sru-zesty-xenial
[16:28] <blackboxsw> hopefully that trello board we use is public for all folks. If you can't see it holler :)
[16:29] <ajorg> yup, it's visible
[16:29] <blackboxsw> feel free to track what we are doing there. we should have most interesting things updated
[16:29] <blackboxsw> sweet
[16:29] <blackboxsw> we hope to get that SRU in this week. waiting on some checks and balances to queue it up for consumption.
[16:30] <blackboxsw> we also started work on pulling in a cloud-init analyze tool rharper wrote into the cloud-init commandline tool so folks can debug their cloud runs
[16:30] <blackboxsw> Pulling cloud-init analyze scripts into cloud-init devel sub-command
[16:30] <blackboxsw> https://trello.com/c/k5F3KftA/126-cloudinitanalyze-mp-and-land-blackboxsw-smoser
[16:30] <blackboxsw> this'll be really cool for tracking (and blaming) what costs clouds instances the most time during boot/discovery etc.
[16:31] <blackboxsw> so we can improve effeciency across all datasources
[16:32] <rharper> *cough* locale-gen *cough*
[16:32] <blackboxsw> rharper: - We no longer run locale-gen if requested system locale is default . saving time on instances which are already configured with the appropriate locale ?
[16:33] <ajorg> this was discovered using the analyze thing?
[16:33] <blackboxsw> what should we chat about there do you think?
[16:33] <rharper> blackboxsw: right, if the image pregenerates and configures locales, we'll skip unless user sets it (and it differs from the in-image setting)
[16:33] <blackboxsw> rharper: do you recal how much time that costs to rebuild?
[16:34] <rharper> it depends on the system, but it was roughly 1 second or so on a 3 year old laptop (with ssd)
[16:34] <blackboxsw> I thought it was on the order of seconds
[16:34] <blackboxsw> yeah
[16:36] <blackboxsw> so anything we can do to expedite is a big win.   ohh I BTW the Ec2 init-local extra dhcp4 query/response seems to cost around a 10th of a second (so not too bad)
[16:37] <blackboxsw> though it'd be great if we could either use link local, or be told by the environment somehow what the approporiate address is in order to contact the metadata service.
[16:37] <blackboxsw> anyhow. I guess that's it for initial topics we can move to office-hours for anyone that has questions, concerns, requests etc.
[16:38] <blackboxsw> #meetingtopic Cloud-init open office-hours
[16:40] <blackboxsw> regarding these #commands, I filed a request to get a meetingbot to join this channel.  I'll followup on that to see if we can't get something in here to harvest links/notes etc and publish them somewhere for easy reference in the future
[16:41] <blackboxsw> I think this probably wraps up our meeting if there are no pressing needs to discuss.  If anyone has any bugs in their bonnet, please raise them here. We'll be hitting up a small summit in a couple weeks and plan a working session to squash bugs.
[16:42] <blackboxsw> Otherwise thank you all for your eyes/cycles and have a good one
[16:42] <blackboxsw> #endmeeting
[16:42] <ajorg> :)
[16:43] <blackboxsw> thx again ajorg ilianaw
[16:46] <ilianaw> blackboxsw: i do worry about not enabling dhcpv6 on interfaces because at a later point someone can go add an IPv6 address to that interface
[16:47] <ilianaw> i.e. not running dhclient (or equivalent) because at boot there weren't any v6 addrs
[16:48] <rharper> ilianaw: are you thinking of hotplug? or something else?
[16:52] <ilianaw> rharper: a simple `aws ec2 assign-ipv6-addresses` call on an already-plugged-in interface
[16:52] <ilianaw> that didn't have v6 addrs at boot
[16:53] <rharper> ilianaw: does that update the metadata service configuration ?
[16:53] <ilianaw> it does (i think? i hope so)
[16:54] <ilianaw> the only way it informs the instance, as far as i know, is an ICMP6 router advertisement
[16:54] <rharper> ok, I Don't think there are expectations that an OS reconfigure itself at runtime *yet* but we do want to have cloud-init react to metadata changes
[16:54] <blackboxsw> sorry was grabbing coffee
[16:54]  * ilianaw nods
[16:55] <rharper> if cloud-init was watching the metadata service, and someone issues the aws update, that could trigger cloud-init to update network-config (at least rerender it);  possibly trigger a network-update;
[16:55] <rharper> it's not clear if it's desirable (dhcp6 could take over default gateway)
[16:55] <rharper> which could affect applications/routes
[16:55] <rharper> so automatically doing that reconfigure has impacts that need to be weighed
[16:55] <ilianaw> yup
[16:56] <blackboxsw> it'd be interesting if we surfaced a configuration option to allow the instance to reconfigure network on metadata change...
[16:57] <ilianaw> although would detecting a change just be "hit the metadata service every five minutes looking for updates"?
[16:57] <rharper> well, that's certainly under discussion w.r.t cloud-init handling metadata changes
[16:57] <blackboxsw> but this work would probably fall into the camp of some of the hotplug feature discussions you've already started w/ smoser rharper.
[16:57] <rharper> ilianaw: normally we'd do this through something like a long-pool
[16:57] <rharper> poll
[16:58] <rharper> websocket
[16:58] <rharper> so, consumes some resources, but more dynamic (and lower latency) than regular polling intervals (and lower resource cost w.r.t socket setup/teardown)
[17:01] <blackboxsw> it's an interesting proposal, so does anyone have familiarity w/ cfn_hup? I just google hit
[17:01] <blackboxsw> http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/updating.stacks.walkthrough.html
[17:02] <blackboxsw> was just wondering generally in the aws case whether a pub/sub service for watching metadata deltas was "out there" somewhere
[17:02] <ilianaw> i need to test what all cfn-hup listens at
[17:03] <ilianaw> i was pretty sure it was only cloudformation related but i suppose i ought to check
[17:04] <blackboxsw> yeah per the docs it looks fairly specific, but was wondering if the foundation is uses is something generic to all instances.
[17:05] <ilianaw> i would probably not be as hopeful. but i'll definitely check
[17:05] <blackboxsw> .... but seems like the same general concept rharper mentioned. occasional polling loop on data we care about, check for deltas and react
[17:43] <blackboxsw> welcom meetingology :)
[17:43] <blackboxsw> ok ready for next week
[17:44] <blackboxsw> s/week/meeting/
[18:19] <msaikia> Hi, I just submitted a changeset addressing a review comment. But it is failing continuous integration test for tip-pyflakes. It states that subprocess is imported but not used in tools/read-dependencies. That is not a part of my changeset. What should I do?
[18:33] <rharper> msaikia: do you have a link to your changeset ?
[18:36] <msaikia> yes. https://code.launchpad.net/~msaikia/cloud-init/+git/cloud-init/+merge/322991/+index?ss=1
[18:36] <rharper> let me see
[18:46] <rharper> msaikia: it looks like you need to rebase your branch to origin/master ;
[18:46] <msaikia> i actually did that as well. let me try it one more time
[18:47] <rharper> if you ahve, you may need to force push to your launchpad git repo
[18:47] <msaikia> ok..
[18:48] <rharper> what I cloned from your repo reproduces the failure;  so on your local rebased branch, you can run the check directly yourself, tox -e tip-pyflakes
[18:48] <rharper> once that passed locally, you can push to up date your launchpad repo;  then the jenkins ci job should pick up when you push and confirm it's fixed as well
[19:33] <msaikia> rharper: Thanks. its passing locally. so i will force push it now.
[19:33] <rharper> great!
[20:10] <blackboxsw> ajorg, I just ran some additional tests on AWS with the Ec2 init-local with the dhcp initialization to talk to the metadata service. Overall cloud-init reaps a 1 second benefit from setting up in init-local phase instead of the init-network phase, even despite the dhclient setup & teardown.
[20:11] <blackboxsw> ajorg, I added comments to my merge proposal. Even with the dhclient discovery in init-local  and the attempt to load minmal metadata version 2009-04-04 before checking 2016-09-02 , we still are a big win with this approach.
[20:11] <blackboxsw> meh, he's not here at the moment.
[20:12] <blackboxsw> rharper: I'm pretty happy about this metadata approach. Things are happy.  https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/328241 we'll await ajorg's feedback on the 'Needs information' but I think it's a go (so I'll pull together that IPv6 metadata query/setup branch which should be minimal.
[20:13] <blackboxsw> s/things are happy/things appear fast/
[20:13] <rharper> blackboxsw: looking
[20:14] <blackboxsw> msaikia: will put some eyes your branch now too. thanks.
=== meetingology` is now known as meetingology