[15:30] <rharper> Odd_Bloke: thanks for the reminder about the upcoming meeting
[15:30]  * rharper goes and gets the meetingology commands 
[16:11] <robjo> I am going to have to miss the meeting today, sorry
[16:11] <robjo> rharper: I'll catch up to the comments in the next couple of days
[16:12] <rharper> robjo: np
[16:15] <rharper> #startmeeting Cloud-init bi-weekly status
[16:15] <meetingology> Meeting started Mon Jul 22 16:15:03 2019 UTC.  The chair is rharper. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
[16:15] <meetingology> Available commands: action commands idea info link nick
[16:15] <rharper> \o/
[16:15] <rharper> #chair Odd_Bloke
[16:15] <meetingology> Current chairs: Odd_Bloke rharper
[16:15] <rharper> hi folks, welcome to another cloud-init community status meeting. All discussions and interjections welcome.
[16:16] <rharper> cloud-init upstream uses this meeting as a platform for community updates, feature/bug discussions, and an opportunity to get some extra input on current development.
[16:16] <rharper> our format is the following topics: Previous Actions, Recent Changes, In-progress Development, Office Hours
[16:16] <rharper> anyone is welcome to participate, interject, make suggestions or ask questions
[16:16] <rharper> we host the meeting every two weeks at the date and time  indicated in the IRC channel topic  ^
[16:17] <rharper> #topic Previous Actions
[16:17] <Odd_Bloke> o/
[16:17] <rharper> We had a few previous action items to look at
[16:18] <rharper> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381
[16:18] <ubot5> Ubuntu bug 1832381 in cloud-init (Ubuntu) "vm fails to boot due to conflicting network configuration when user switches from netplan to eni" [Undecided,Incomplete]
[16:18] <rharper> AnhVoMSFT was looking to collect logs from this scenario;
[16:19] <Goneri> hey!
[16:19] <rharper> it appears that getting an instance where the MAC address changes is harder so fewer folks trip over this; however, we agreed that cloud-init can track which renderer it used and if it switches it can clean up the config it wrote;
[16:19] <rharper> #action rharper to update https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381 status
[16:19] <meetingology> ACTION: rharper to update https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381 status
[16:19] <ubot5> Ubuntu bug 1832381 in cloud-init (Ubuntu) "vm fails to boot due to conflicting network configuration when user switches from netplan to eni" [Undecided,Incomplete]
[16:20] <rharper> The other action was to review https://code.launchpad.net/~vtqanh/cloud-init/+git/cloud-init/+merge/369785
[16:20] <rharper> This was completed the other week while we worked toward the 19.2 release; that branch is currently work-in-progress, awaiting feedback/changes from submitter
[16:21] <rharper> that's all of the action items from previous meeting
[16:22] <rharper> previous meeting status found here:
[16:22] <rharper> #link http://ubottu.com/meetingology/logs/cloud-init/2019/cloud-init.2019-07-08-16.16.html
[16:23] <rharper> normally at the cloud-init.github.io status page; looks like we didn't push the logs there.
[16:23] <rharper> #action rharper to followup with blackboxsw on pushing status minutes up to cloud-init.github.io page
[16:23] <meetingology> ACTION: rharper to followup with blackboxsw on pushing status minutes up to cloud-init.github.io page
[16:23] <rharper> #topic Recent Changes
[16:25] <rharper> % git log --oneline  --since 2019-07-08
[16:25] <rharper> a02c0c9 (HEAD -> master, origin/master, origin/HEAD) cloud_tests: updates and fixes
[16:25] <rharper> 5498107 Fix bug rendering MTU on bond or vlan when input was netplan.
[16:25] <rharper> b3a87fc net: update net sequence, include wait on netdevs, opensuse netrules path
[16:25] <rharper> 060b1a1 (tag: 19.2, raharper/release/19.2, release/19.2, fix/fs_setup_custom_command_lp1801790) Release 19.2
[16:25] <rharper> 07b1723 net: add rfc3442 (classless static routes) to EphemeralDHCP
[16:25] <rharper> 1404817 templates/ntp.conf.debian.tmpl: fix missing newline for pools
[16:25] <rharper> a785462 Support netplan renderer in Arch Linux
[16:25] <rharper> a066ccd Fix typo in publicly viewable documentation.
[16:26] <rharper> d9769c4 Add a cdrom size checker for OVF ds to ds-identify
[16:26] <rharper> 9c47c68 VMWare: Trigger the post customization script via cc_scripts module.
[16:26] <rharper> a24550a Cloud-init analyze module: Added ability to analyze boot events.
[16:26] <rharper> a6faf3a Update debian eni network configuration location, retain Ubuntu setting
[16:26] <rharper> e5f5421 net: skip bond interfaces in get_interfaces
[16:26] <rharper> 217c893 Fix a couple of issues raised by a coverity scan
[16:26] <rharper> biggest item in there is the 19.2 release
[16:26] <rharper> #link https://discourse.ubuntu.com/t/cloud-init-19-2-release/11873
[16:26] <rharper> a big thank you from the cloud-init team to everyone who helped contribute to the release
[16:27] <Goneri> do you have time for a little BSD update?
[16:27] <rharper> Yes, let's talk about In progress developement
[16:27] <rharper> #topic In Progress Development
[16:27] <rharper> Goneri: go ahead
[16:28] <Goneri> #topic FreeBSD/NetBSD status
[16:28] <Goneri> so there is two active branches, the first one is: https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368507
[16:28] <rharper> #link https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368507
[16:29] <Goneri> it has started with a tiny fix to address a configuration difference with FreeBSD (there is no chpasswd there)
[16:29] <Goneri> and it's now a slightly bigger refactoring now, I believe it clarify the code base and I would like to land it like that.
[16:30] <Goneri> a discussion is ongoing with rharper on the PR
[16:30] <Goneri> second PR is https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641
[16:30] <Goneri> #link https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641
[16:30] <Goneri> this one is much bigger, and I've just addressed the last comment from rharper, I test it often and it works fine for me
[16:31] <Goneri> if you want to give it a try, I pushed some pre-built images here: http://bsd-cloud-image.org/
[16:31] <rharper> nice!
[16:31] <Goneri> I test it with OpenStack and NoCloud, a friend who maintains CBSD also test it on Bhyve (FreeBSD)
[16:32] <Goneri> finally, the last one is https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368508
[16:32] <Goneri> #link https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368508
[16:32] <Goneri> No active merge request yet because it depends on the two actives PR that I just mentioned
[16:32] <Goneri> this patch brings NetBSD support (7 and 8)
[16:33] <Goneri> I would like to work on OpenBSD later, but it's still a low priority
[16:34] <rharper> Goneri: thanks for the update
[16:34] <Goneri> finally, I've a bunch of scripts that I use to build my images
[16:35] <Goneri> it's still rather raw, but I would like to integrate that at some point with your CI
[16:35] <Goneri> that's all
[16:35] <rharper> thanks
[16:36] <rharper> #topic In Progress Development
[16:36] <rharper> #link https://code.launchpad.net/~tribaal/cloud-init/+git/cloud-init/+merge/369516
[16:36] <rharper> Adding a new datasource for Exoscale
[16:36] <rharper> #link https://code.launchpad.net/~xiaofengw/cloud-init/+git/cloud-init/+merge/367889
[16:36] <rharper> vmware user-defined-scripts
[16:37] <rharper> #link https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369783
[16:37] <rharper> Allow datasources to configure the order of network-config sources
[16:37] <rharper> #link https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+ref/feature/stage_threadpool
[16:38] <Odd_Bloke> I'm expecting to have a response to Ryan's review comments on that today, and whatever conclusion we reach shouldn't be too much work to implement.
[16:38] <rharper> definitely
[16:38] <Odd_Bloke> And then I'll have a follow-up to split apart the "cmdline" network data source in to "cmdline" and "initramfs", which are currently conflated.
[16:39] <Odd_Bloke> (Neither of these should cause behavioural changes, they're just setting us up for some data source work down the line.)
[16:40] <rharper> The threadpool branch is a more general approach to handle running modules async from the mainthread;  there were some limitations depending on systemd;  and there is a desire for more than just disk_setup to run async;  this branch I'm working on would allow modules to be tagged async and they run in a separate thread allowing the next module to proceed; we then join at the end of stage to ensure completion of threads;
[16:41] <rharper> any other upstream development I'm missing?
[16:43] <rharper> ok, I think that's it then;
[16:43] <rharper> #topic  Office Hours (next ~30 mins)
[16:43] <rharper> feel free to ask for help, reviews, discussions on any cloud-init items you're looking at.
[16:44] <metsuke> Is Ubuntu the recommended, or most maintained, distribution of cloud-init?
[16:46] <rharper> metsuke: hi; cloud-init in Ubuntu is the most-up-to-date as we're both the upstream maintainers (working for Canonical) and handle getting the latest upstream into Ubuntu images
[16:47] <rharper> metsuke: we also help produce daily rpm builds for RedHat/Centos/Fedora in our copr repo
[16:47] <rharper> #link https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
[16:48] <rharper> #link https://download.opensuse.org/repositories/Cloud:/Tools/
[16:48] <rharper> suse's cloud:Tools keeps a really recent cloud-init as well
[16:48] <metsuke> great, thanks for the info!
[16:48] <rharper> sure
[16:49] <metsuke> I'm looking to distribute standardized VMs to 100+ sites running ESXi so I'm trying to find the best way to do that =)
[16:52] <Odd_Bloke> metsuke: If you have any questions, please don't hesitate to ask in here; we're generally around US working hours, for reference.
[16:53] <rharper> yes, cloud-init can help keep your base image generic, allowing customization to happen at boot time;
[16:53] <metsuke> thanks, I'm trying to do some preliminary investigation so I have an intelligent question to ask =P
[17:15] <rharper> #endmeeting
[17:15] <meetingology> Meeting ended Mon Jul 22 17:15:54 2019 UTC.
[17:15] <meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2019/cloud-init.2019-07-22-16.15.moin.txt
[17:16] <rharper> Thanks everyone !
[17:24] <Odd_Bloke> Yes, thanks folks!
[18:16] <Odd_Bloke> rharper: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369783 <-- I have hopefully responded properly to your comment here, let me know if you have any questions!
[18:29] <rharper> Odd_Bloke: thanks!
[19:09] <AhosmanMSFT> Hi I'm Ahmed from Microsoft. I'd like to get more information on cloud testing with cloud init?
[19:10] <meena> AhosmanMSFT: testing, what, exactly?
[19:11] <AhosmanMSFT> upstream cloud tests
[19:12] <AhosmanMSFT> https://cloudinit.readthedocs.io/en/latest/topics/tests.html
[19:12] <rharper> AhosmanMSFT: hi
[19:13] <rharper> AhosmanMSFT: I think the platforms is the most interesting part
[19:14] <rharper> if you look at tests/cloud_tests/platforms/ec2/platofrm.py ; for Azure, you'd need a platform subclass which talks Azure API to implemeant each of the platform methods
[19:15] <rharper> that's mostly image related and then handling the instance creation/destruction in instance.py for the platform.
[19:17] <AhosmanMSFT> rharper I see, so the base code is there it just needs to be inherited by us
[19:18] <rharper> AhosmanMSFT: yeah; the structure of the tests framework is there, you'd be adding a new platform,
[19:19] <rharper> the basic flow is 1) get an image 2) boot it  3) copy in the cloud-init package to be installed/upgraded  4) install test cloud-init package 5) clean-up  6) shutdown and capture image for boot;
[19:20] <rharper> this is saved as a snapshot image; and then the collect phase boots the snapshot for each test-case we have; and then waits to ssh'in and collect output;
[19:20] <rharper> and the verify stage runs the unittests against the collected data;
[19:30] <AhosmanMSFT> rharper: Thanks, trying this now. Can I contact you for assistance?
[19:33] <Odd_Bloke> AhosmanMSFT: Feel free to ping rharper or myself if you need anything.
[19:33] <AhosmanMSFT> Odd_Bloke: 👍
[20:55] <metsuke> does cloud-init directly compare to hashicorp's Packer for its base functionality?  I'm trying to get a sense of scope of features and use case.
[20:56] <rharper> https://events.linuxfoundation.org/wp-content/uploads/2017/12/cloud-init-The-cross-cloud-Magic-Sauce-Scott-Moser-Chad-Smith-Canonical.pdf
[20:57] <rharper> metsuke: ^^
[20:58] <rharper> metsuke: I would say that packer creates custom _images_ where as cloud-init customizes _instances_ ;  starting with a generic image, at _runtime_ you pass in user-data/configuration and cloud-init helps customize that instances consistently (allowing a common image and config to run the same on many platforms)
[20:59] <metsuke> Interesting. So does cloud-init need access to the platform that the images are being spun up on?
[21:00] <metsuke> or rather, are you executing cloud-init in the same network in which your VM platform resides (including cloud)?
[21:02] <Odd_Bloke> metsuke: cloud-init runs within the instance.
[21:03] <Odd_Bloke> So during boot, it will run, determine which platform it is on, gather configuration from various sources, and then apply that configuration.
[21:04] <metsuke> hm, that's kind of backwards from the other model.  Cool
[21:04] <AhosmanMSFT> For the files __init__.py, image, instance, platform and snapshot.py in cloud_tests. Should I work on these in any specific order, since some of the files depend on others. Also after implementing the files, what would be the next steps?
[21:17] <Odd_Bloke> AhosmanMSFT: So I haven't written a cloud test platform myself, so take this with a grain of salt, but I think you're going to need to work on them all together.  That said, I would do this by working out a cloud_tests command line that works, changing the platform to "azure" and then iterating on the missing pieces reported by the framework.  (I _expect_ that would go in the order platform, image,
[21:17] <Odd_Bloke> snapshot, instance based on reading cloud_tests/collect.py.)
[21:18] <Odd_Bloke> So, for example, `tox -e citest -- tree_run --verbose --os-name bionic --preserve-data --data-dir ~/scratch/citest --platform lxd --test-config tests/cloud_tests/testcases/modules/ntp.yaml` runs the NTP tests in lxd for me.  So I would change "lxd" to "azure" and see what happens, and then iterate.
[21:19] <Odd_Bloke> AhosmanMSFT: Is that a useful answer?  If not, I'll ping the folks with more domain knowledge. :)
[21:22] <AhosmanMSFT> That helps. It would be great to get some more insight from them too.
[21:23] <Odd_Bloke> rharper: powersj: Perhaps you can flesh out my answer to help AhosmanMSFT get started on Azure cloud tests?
[21:51] <rharper> Odd_Bloke: that's exactly where'd I'd start as well.
[23:48] <powersj> AhosmanMSFT, I would start with platform. That is how you setup how we interact with Azure. Any work that is needed before an instance is done goes there
[23:48] <powersj> After that, go after instance. There all the basic operations of start stop and getting the instance log
[23:49] <powersj> then go to snapshot to generate a... snapshot of the instance :)
[23:49] <powersj> and finally image
[23:50]  * powersj disappears for a while