=== r-daneel_ is now known as r-daneel
=== shardy is now known as shardy_mtg
[15:40] <rharper> https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/341662
[15:40] <rharper> smoser: blackboxsw ^
[15:51] <smoser> blackboxsw: cloudinit/config/cc_puppet.py:143
[15:51] <smoser> did you l ook at that more closely ?
[15:51] <smoser>  cloudinit/config/cc_puppet.py:143: [W1505(deprecated-method), handle] Using deprecated method readfp()
[15:51] <smoser> hm..
[15:51] <smoser> need to fix one way or another.
[16:01] <blackboxsw> smoser: nope, the puppet fix is one I had wanted to come back to today/tomorrow
[16:01] <smoser> that was a dude-on-the-internet-broke-our-ci change.
[16:01] <blackboxsw> we need to sort it across releases :/
[16:01] <blackboxsw> I just put a bandaid on it
[16:02] <blackboxsw> ...and it's that time again
[16:02] <blackboxsw>  #startmeeting Cloud-init bi-weekly status meeting
[16:02] <blackboxsw> #startmeeting Cloud-init bi-weekly status meeting
[16:02] <meetingology> Meeting started Mon Mar 19 16:02:30 2018 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
[16:02] <meetingology> Available commands: action commands idea info link nick
[16:03] <blackboxsw> ok, let's kick off this cloud-init bi-weekly meeting. welcome all!
[16:04] <dpb1> o/
[16:04] <blackboxsw> it's been a busy couple weeks for a few of us w/ planning meetings and vacation, but let's see what progress we've made on cloud-init.
[16:05] <blackboxsw> #topic Recent Changes
[16:05] <dpb1> smoser vacation specifically
[16:05] <blackboxsw> hehe. Generally we're tracking high-points of what  lands in our trello board
[16:05] <blackboxsw> #link http://trello.com/b/hFtWKUn3/daily-cloud-init-curtin
[16:06] <blackboxsw> but from changelogs folks have made progress on azure, vmware and FreeBSD deployment targets
[16:06] <blackboxsw>     - netplan: render bridge port-priority values (LP: #1735821)
[16:06] <blackboxsw>     - net: recognize iscsi root cases without ip= on kernel command line.
[16:06] <blackboxsw>       (LP: #1752391)
[16:06] <blackboxsw>     - util: Fix subp regression. Allow specifying subp command as a string.
[16:06] <blackboxsw>        (LP: #1755965)
[16:06] <blackboxsw>     - This commit fixes get_hostname on the AzureDataSource.
[16:06] <blackboxsw>       [Douglas Jordan] (LP: #1754495)
[16:06] <ubot5`> Launchpad bug 1735821 in nplan (Ubuntu Artful) "netplan needs bridge port-priority support" [Medium,Fix committed] https://launchpad.net/bugs/1735821
[16:06] <blackboxsw>     - shellify: raise TypeError on bad input.
[16:06] <blackboxsw>     - FreeBSD: Set hostname to FQDN. [Dominic Schlegel] (LP: #1753499)
[16:06] <blackboxsw>     - Make salt minion module work on FreeBSD.
[16:06] <blackboxsw>       [Dominic Schlegel] (LP: #1721503)
[16:06] <ubot5`> Launchpad bug 1752391 in cloud-init "cloud-init does not recognize initramfs provided network config in all cases" [Medium,Fix committed] https://launchpad.net/bugs/1752391
[16:06] <blackboxsw>     - set_hostname: When present in metadata, set it before network bringup.
[16:06] <blackboxsw>        (LP: #1746455) VMWare
[16:06] <ubot5`> Launchpad bug 1755965 in cloud-init "util.subp regression: no longer accept commands as string" [High,Fix committed] https://launchpad.net/bugs/1755965
[16:06] <blackboxsw>     - cc_snap: Add new module to install and configure snapd and snap
[16:06] <blackboxsw>       packages.
[16:06] <blackboxsw>     - doc: fix all warnings issued by 'tox -e doc'
[16:06] <blackboxsw>     - tests: Make pylint happy and fix python2.6 uses of assertRaisesRegex.
[16:06] <ubot5`> Launchpad bug 1755965 in cloud-init "duplicate for #1754495 util.subp regression: no longer accept commands as string" [High,Fix committed] https://launchpad.net/bugs/1755965
[16:06] <blackboxsw>     - tests: fix run_tree and bddeb
[16:06] <blackboxsw>     - tests: Fix some warnings in tests that popped up with newer python.
[16:06] <ubot5`> Launchpad bug 1753499 in cloud-init "hostname in FreeBSD should prefere FQDN" [Undecided,Fix committed] https://launchpad.net/bugs/1753499
[16:06] <blackboxsw>     - tests: fix flakes warning for unused variable
[16:06] <blackboxsw>     - tests: patch leaked stderr messages from snap unit tests
[16:06] <ubot5`> Launchpad bug 1721503 in cloud-init "salt module not able to be used on FreeBSD" [Medium,Fix committed] https://launchpad.net/bugs/1721503
[16:06] <blackboxsw>     - tests: Centralize and re-use skipTest based on json schema presense.
[16:06] <ubot5`> Launchpad bug 1746455 in cloud-init "cloud-init vSphere cloud provider DHCP unique hostname issue" [High,Fix committed] https://launchpad.net/bugs/1746455
[16:08] <blackboxsw> a big thanks to dojordan (Azure) and Dominic Schlegel (FreeBSD) for patching some gaps in support as cloud-init master progresses
[16:10] <blackboxsw> On the ubuntu side of the house we got tip of tree published into Bionic thusday & friday, we are awaiting cloud-image builds which look like they are stale at 03-15-2018. once those builds are published, all clouds should be getting latest cloud-init on Bionic
[16:11] <blackboxsw> I think that's probably it for 'done' work. We have a few things in flight at the moment
[16:12] <blackboxsw> #topic In-progress Development
[16:13] <blackboxsw> Ubuntu is getting a number of new cloud-config modules:
[16:13] <blackboxsw>  - new cc_snap module (deprecated cc_snappy and cc_snap_config modules) the ability to install and manage snap package
[16:14] <blackboxsw> - new cc_ubuntu_drivers: support to install 3rd party drivers on install
[16:15] <blackboxsw> - new cc_ubuntu_advantage: manage Ubuntu Advantage subscriptions for services such as Extended Security Mainenance (trusty), canonical livepatch and FIPS PPAs
[16:15] <blackboxsw> these should be landing in the week(s) to come
[16:15] <blackboxsw> and cc_snap landed already
[16:17] <blackboxsw> also there are a couple of branches that we are trying to wrap up for first class chrony support (per rharper, inspired by robjo's work)
[16:17] <rharper> blackboxsw: smoser: on the lander emails, the subject could include the git hash (or branch name); it's currently only in the body;
[16:17] <smoser> rharper: i had suggested to blackboxsw that it should acutally change to *not* send a subject. so it threads in your email reader with the other MP mails.
[16:18] <rharper> heh
[16:18] <blackboxsw> maybe we can toggle between the two modulus 2 :)
[16:18] <rharper> sorry, didn't meant to disturb the flow
[16:19] <rharper> continue
[16:19] <blackboxsw> yeah, we've also touched a little bit of our code landing automation this last week. powersj also is working on a git lander plugin that we might be able to use to automate landing of approved branches w/ tox test runs
[16:19] <blackboxsw> anything to free up developer time will give us more time for reviews/code
[16:19] <ajorg> should vendor-specific modules be shipped in a separate package?
[16:20] <smoser> vendor specific modules ?
[16:20] <ajorg> ubuntu_advantage
[16:22] <blackboxsw> good question/point. I hand't thought about that separation as a lot of the modules cloud-init delivers support a subset of distros
[16:23] <blackboxsw> each module has a distro attribute defined as to whether or not it will even run
[16:23] <blackboxsw> so we have spacewalk, zypper_repos etc
[16:24] <smoser> and cloud.cfg is rendered based on knowledge of the distro
[16:24] <smoser> so ubuntu_advantage wont even be in the list of config modules
[16:24] <smoser> having a static config module list is WIN in this case (but pain elsewhere)
[16:25] <smoser> at some point whe may have a more dynamic config module list.
[16:25] <smoser> but anyway... at the moment the only cost to non-ubuntu of that module being shipped is bytes on disk.
[16:26] <blackboxsw> if/when we do define that more dynamic config module list, I'd like us also to look at having configurable/separate plugin directories defined for folks providing vendor-specific content.
[16:26] <ajorg> agree that having a more dynamic config module list is prerequisite to being able to parcel out modules to other packages
[16:26] <blackboxsw> so that we don't expect folks to add plugins directly into /usr/lib/python3/dist-packages/cloudinit/config/ for instance
[16:26] <smoser> yeah. at the point when it is dynamic, the module would still declare its support for a list of distros and would be filtered out.
[16:29]  * ajorg is satisfied
[16:30] <blackboxsw> :).    the only other thing I can think of in progress two more datasources  softlayer cloud support by smoser and hetzner cloud
[16:30] <blackboxsw> so cloud-init is getting it's grubby hands into a couple of more clouds shortly.
[16:31] <blackboxsw> s/it's/its/
[16:31] <blackboxsw> it's nice to see the adoption continue to grow
[16:31] <smoser> looks like someone followed up on hetzner
[16:31] <smoser> so that hopefully is ready to land
[16:31] <blackboxsw> #link https://code.launchpad.net/~lp-markusschade/cloud-init/+git/cloud-init
[16:32] <blackboxsw> rharper: also has a couple of branches to allow cloud-init to work a bit better when rendering netplan configuration
[16:33] <blackboxsw> I *think* that's all for in-progress development at the moment.
[16:33] <smoser> man we need to fix that pylint thing.
[16:33] <smoser> did you mention ?
[16:33] <rharper> blackboxsw: yeah, I just pushed a fix for v1 global dns entries to get rendered under interfaces without any dns configuration
[16:33] <blackboxsw> Anything else that should be noted by anyone?
[16:33] <blackboxsw> #link https://code.launchpad.net/~lp-markusschade/cloud-init/+git/cloud-init/+merge/338439
[16:33] <blackboxsw> oops
[16:34] <smoser> cloudinit/config/cc_puppet.py:143: [W1505(deprecated-method), handle] Using deprecated method readfp()
[16:34] <blackboxsw> #link https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/341662
[16:34] <smoser> that needs fixing. it has come to us due to a new version of some of our tox environemnts.  we do not fully pin the versions , only the top level packages.  Ie, pylint's dependencies changed, but we only pin pylint version.
[16:35] <blackboxsw> yeah how much should we freeze our deps?
[16:35] <blackboxsw> it's kindof annoying to have your branch locally pass ci, and a fresh build of CI deps fail when you try to land
[16:36] <blackboxsw> but I don't really know whether it's worth us 'pinning' everything
[16:36] <smoser> i think pinning everything is generally the best practice for this sort of thing now.
[16:37] <smoser> oh my.
[16:37] <blackboxsw> so if we pin the world, should we also just make it a habit to occasionally tox -e tip-pylint?
[16:37] <smoser> blackboxsw: did you know you accidently fixed that in trunk ?
[16:38] <blackboxsw> smoser: I know I intentionally added a pylint ignore on that to come back and address it today.
[16:38] <ajorg> i tend to believe it's better to stay current and take your punches a few at a time so you don't have a major upset when you have to upgrade.
[16:38] <smoser> oh ko. i see.
[16:39] <smoser> ajorg: well, sor tof. if you have c-i that you want to be green, and consider it bad when it is not, then you dont want dude-on-the-internet to break you
[16:40] <smoser> there is the "good" break, where new upload to pypi identifies some lingering bug
[16:40] <blackboxsw> +1 ajorg, but I'm good (on avoiding a avalanche) if we agree to run tip-pylint target fairy regularly to avoid the landslide
[16:40] <smoser> but also the "bad" break where some upload breaks your c-i for invalid reason.
[16:42] <smoser> one huge advantage to pinning is the ability to re-create things.
[16:42] <blackboxsw> it definitely felt like last week was a lot of c-i breaks  for changes unrelated to the code up for review
[16:42] <smoser> ie, if you were looking to it bisect something...
[16:42] <smoser> git bisect
[16:43] <blackboxsw> I should transition to the office hours topic so we can continue discussion
[16:43] <smoser> you can't really do that if trunk from a point in the past does not pass C-I because an external dependency changed.
[16:43] <smoser> sure we can transition to office  hours.
[16:43] <blackboxsw> #topic Office hours (next ~30 mins)
[16:43] <smoser> but yeah... i want c-i on tip to not just start failing.
[16:44] <stanguturi> @blackboxsw, I have couple of requests. First, https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1724128 discussed this in last meeting as well. Any help is greatly appreciated
[16:44] <ubot5`> Ubuntu bug 1724128 in open-vm-tools (Ubuntu) "Need a Success / Failure notification mechanism when cloud-init finishes." [Undecided,New]
[16:44] <blackboxsw> that's fair. smoser can we maybe add a jenkins job to run tip-pylint then weekly. So, we don't have a huge backlog of lint failures against tip?
[16:45] <smoser> blackboxsw: i'mi fine with that... thats why we added the tip-* targets. so it was easy enough to keep current.
[16:45] <blackboxsw> +1 smoser I'll put a card up for that
[16:46] <smoser> stanguturi: your suggestion there is not a bad idea at all.
[16:46] <blackboxsw> #link https://trello.com/c/vD1em9WP/698-jenkins-job-to-run-tox-tip-pylint-weekly-pin-all-lint-versions-otherwise
[16:46] <smoser> the desire to have cloud-init tell the platform/datasource that it failed or succeeded is valid.
[16:47] <smoser> with MAAS, that his done via reporting
[16:47] <blackboxsw> stanguturi: hiya. I think we talked after that meeting about trying to allow the datasource to subscribe to a callback when cloud-init exists
[16:47] <blackboxsw> stanguturi: hiya. I think we talked after that meeting about trying to allow the datasource to subscribe to a callback when cloud-init exits
[16:47] <smoser> cloud-init reports status and results via a Reporter.
[16:47] <smoser> dojordan (i think) had also put up a request for a reproter module on azure.
[16:47] <smoser> so we *do* kind of have the function you're after  in place.
[16:48] <stanguturi> smoser: ok. Any inputs / examples of using it will be really great.
[16:50] <smoser> stanguturi: well, the reporter interface is pretty simple. you can cloudinit/reporting/handlers.py
[16:51] <smoser> blackboxsw: http://paste.ubuntu.com/p/6KjDX8WHQH/
[16:51] <smoser> did you intend boht of those changes ?
[16:51] <stanguturi> smoser: ok. Then do I need to write a new handler for our DataSource?
[16:52] <smoser> stanguturi: well you write a reporting Handler, ankd then either system confi or optionally datasource config would turn that reporter on.
[16:52] <blackboxsw> wow smoser, no
[16:52] <blackboxsw> wow, ok, I'll put up a branch to fix that
[16:52] <stanguturi> smoser: ok. Will work on that.
[16:52] <smoser> ok.
[16:53] <stanguturi> I have another quick request about https://code.launchpad.net/~sankaraditya/cloud-init/+git/cloud-init/+ref/set_hwclock_module
[16:53] <smoser> stanguturi: do you find this is actually needed ?
[16:53] <smoser> to my knowledge the only time anyone would ever set their hardware clock to something other than UTC would be dual booting with windows.
[16:54] <smoser> which i can't seem to believe is all that a common situation in VMs
[16:54] <stanguturi> smoser: Yeah. We need this for 'VMware guest customization workflow'.
[16:54] <dpb1> smoser: oh man, I hope not
[16:54] <stanguturi> smoser: if you think, this is not worth for the base cloud-init modules, I can modify to do this change in our datasource specific modules.
[16:54] <smoser> stanguturi: does it actually solve a *current* problem for you ?
[16:55] <stanguturi> smoser: For 'VMware managed VMs', customers can specify in the specification file if they want UTC or localtime for the hardware clock.
[16:55] <smoser> or one that originally came in from a decade ago
[16:55] <stanguturi> smoser: Our existing customization (non cloud-init) engine does it. If we want to move to cloud-init, we want to port all our changes from our engine to our datasource in cloud-init.
[16:55] <smoser> hm... so my sugestion is really to stop allowing that :)
[16:56] <stanguturi> smoser: Oh ok. Can you please add a comment to that merge request just for the record.
[16:56] <smoser> i very well could be wrong
[16:56] <smoser> but the only time that i ever had to deal with this was when dual booting
[16:56] <smoser> with windows specifically
[16:57] <stanguturi> smoser: ok.
[16:57] <smoser> am i wrong there ?
[16:57] <smoser> i really *could* be.
[16:58] <stanguturi> smoser: I can discuss this within our team. But to be on par with our existing engine, want to port the changes.
[16:58] <smoser> and even if you get it wrong, generally speaking you havhe some sort of ntpdate or ntp that will fix your system clock anyway.
[16:58] <smoser> stanguturi: yeah. i understand that.
[16:58] <stanguturi> I have another request. For Ubuntu 18.04, we are planning to set 'disable_vmware_customization' flag to False by default in /etc/cloud/cloud.cfg file.
[16:59] <stanguturi> Want to know your opinion, shall we set it in cloud-init installation phase or request Ubuntu maintainers to set it in 18.04
[17:00] <stanguturi> smoser: And when is the cloud-init 18.2 scheduled for release? 3/22?
[17:01] <blackboxsw> probably a good time for us to bring that up
[17:01] <smoser> yeah. whoops.
[17:01] <smoser> :)
[17:01] <smoser> 18.2 is scheduled for 3/22 (thursday)
[17:02] <blackboxsw> We cloud-init 18.2 have it scheduled for an arbitrary 3/22 date, we'd like to slip that out to next week Tuesday 3/27
[17:02] <smoser> but amoung our internall team we decided to push that to 3/27
[17:02] <stanguturi> smoser: ok. Thanks for the update.
[17:02] <blackboxsw> there a a few in flight branches, azure, softlayer etc that we'd like to get in and get tested before 18.2
[17:02] <smoser> we will send an email today or tomrrow witih "pending release" like subject like we've done before.
[17:03] <blackboxsw> dpb1: others any objections to cutting the 18.2 release on Tuesday 3/27?
[17:03] <dpb1> none
[17:04]  * blackboxsw adds the upcoming date to the topic
=== blackboxsw changed the topic of #cloud-init to:   Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 4/2 16:00 UTC | cloud-init 18.1 released (02/22/18) | cloud-init 18.2 (03/27/18)
[17:05] <blackboxsw> ... ok, folks interested in discussing today?
[17:08] <blackboxsw> #link https://cloud-init.github.io
[17:09] <blackboxsw> The above link will have our captured logs for this meeting.
[17:09] <blackboxsw> thanks again for tuning in
[17:09] <blackboxsw> #endmeeting
[17:09] <meetingology> Meeting ended Mon Mar 19 17:09:23 2018 UTC.
[17:09] <meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2018/cloud-init.2018-03-19-16.02.moin.txt
=== dhill__ is now known as amanjo
[19:33] <blackboxsw> smoser: good point on ubuntu-advantage common code between cC_snap and cc_ubuntu_advantage. Shall it live in cloudinit.config.helpers? cloudinit.config.common? or somewhere else?
[19:34] <blackboxsw> or cloudinit.config_util?
[19:35] <smoser> blackboxsw: hm... wonder where to go with it.
[19:35] <blackboxsw> rharper: same question for common functions that are reused only in cc modules, where would you expect them to live? I don't want to bloat cloudinit.util any more that it already is
[19:35] <smoser> not the monolith 'util'
[19:35] <blackboxsw> yeah
[19:36] <blackboxsw> yeah we still need to decouple functionality from util into smaller modules if we can
[19:36] <rharper> for sure
[19:36] <rharper> I wanted to unbundle the network bits out of util
[19:36] <blackboxsw> +1
[19:36] <rharper> so we pull in fewer modules for cloud-init local
[19:37] <rharper> I sort of like cloudinit.config.util
[19:44] <smoser> for the functions you are adding, they're subprocess related.
[19:45] <smoser> it might make sense to have a cloudinit.subprocess
[19:45] <smoser> tjhat had
[19:45] <rharper> yeah
[19:45] <smoser> subp
[19:45] <smoser> runparts
[19:45] <smoser> shellify
[19:50] <blackboxsw> that works for me. so we create a new module cloudinit.subprocess then.
[19:52] <blackboxsw> smoser: rharper will move runparts shellify into subprocess as a separate branch ok?
[19:52] <Odd_Bloke> Bikeshedding, but semi-shadowing a builtin module could be confusing.
[19:53] <Odd_Bloke> (from cloudinit import subprocess -> confusion)
[19:53] <smoser> yeah. name subprocess isnt great.
[19:53] <Odd_Bloke> (Also from .subprocess import blah would work and also be confusing to read.)
[19:55] <blackboxsw> subp_utils || util_subp?
[19:55] <blackboxsw> we have cloudinit.util and cloudinit.temp_utils currently
[19:56] <blackboxsw> and ssh_util
[19:56] <blackboxsw> hrm cs_utils should probably be under cloudinit/source/helpers
[19:58] <rharper> cloudinit.supb would be decent and avoid Odd_Bloke good point
[19:58] <Odd_Bloke> SUP B
[19:58] <blackboxsw> :) all caps on typos works much better
[19:58] <blackboxsw> sin boldly
[19:58] <rharper> *whoosh*
[19:59] <Odd_Bloke> This is a great definition: http://onlineslangdictionary.com/meaning-definition-of/'sup,-b%253f
[19:59] <rharper> maybe cloudinit.whasssuuuuuuuuuuuuuuuup
[19:59] <blackboxsw> heh
[19:59] <Odd_Bloke> cloudinit.sup.b
[19:59] <Odd_Bloke> cloudinit.sup.what.is
[19:59] <rharper> lol
[20:30] <smoser> that reminds me of garrent holmstrom irc/launchpad name
[20:30] <smoser> https://launchpad.net/~gholms
[20:30] <smoser> gee homes
[22:37] <dpb1> smoser: thankfully mutt threads by in-reply-to:, not the subject, but I see what you mean in gmail.