[07:25] <meena> And so it begins again! (I hope)
[09:04] <tribaal> hi all! I would need some pointers on how to get a more recent cloud-init version on CentOS (eg. CentOS 7.6). Is using a COPR repository the exepcted way to get the latest binairies or is there some kind of process to refresh the cloud-init package in CentOS base similar to Ubuntu's SRU?
[09:07] <meena> tribaal: which version are you getting from COPR?
[09:08] <meena> or, "would you be getting" … if you haven't dared to try yet.
[09:19] <tribaal> meena: the idea would be to have a centOS 7.6 with 19.3+
[09:20] <tribaal> but I don't know what the best way to do this would be (we'd probably update our default templates to include the right binaries ourselves if there's no SRU-like process for CentOS)
[09:25] <meena> tribaal: https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/ you'd get 19.4 if you https://docs.pagure.org/copr.copr/how_to_enable_repo.html#how-to-enable-repo `dnf copr enable @cloud-init/el-testing`
[09:26] <tribaal> meena: ack, thanks. That's not really "production" builds though, is it?
[09:28] <meena> tribaal: i don't know (anything about CentOS / Fedora etc… i haven't used them in a long time, and in $$jobs$$ we use RHEL)
[09:30] <tribaal> meena: ack - same here (but we us Ubuntu :) )
[09:30] <tribaal> so our Ubuntu templates have the latest and greatest but we'd need our centOS templates to have the latest/greatest as well
[09:30] <tribaal> and I'm not too sure how/where to look :)
[13:42] <Odd_Bloke> meena: tribaal: o/ Happy new year!
[13:43] <Odd_Bloke> I can't remember exactly the arrangement of our COPR repos, we'll have to wait for Ryan/Chad to help, I'm afraid.
[13:43] <meena> Odd_Bloke: happy monday
[13:54] <meena> Odd_Bloke: i'll need to quiz your brain some more wrt net refactor
[13:54] <meena> but, IIRC, i've written the ideas / questions down. now lemme just find the URL…
[13:55] <meena> 23:47 <meena> updates: https://hackmd.io/3-YBj1t9TAeKhmfLBQUjXQ?view#Revisiting-modules
[13:55] <meena> 27th of December.
[13:56] <Odd_Bloke> I haven't even opened my inbox yet, so I don't know how soon I'll get to it, but it's on my list now! :p
[14:09] <tribaal> Odd_Bloke: heya! happy new year to you too :)
[16:32] <blackboxsw> happy 2020 cloud-initers. Time to dig out from holiday emails
[16:37] <blackboxsw> tribaal: we (upstream cloud-init) don't work with distros directly, other than Ubuntu, to push upstream changes into the distribution. We rely on the distro vendors to vet cloud-init themselves. That said, I believe otubo is the right person to ask about how to coordinate updates into CentOS.   The cloud-init upstream team only provides a our COPR repos as a facility for CentOS users to check if more recent
[16:37] <blackboxsw> upstream versions of cloud-init help resolve bugs seen on the current (older) cloud-init in CentOS stock images.  Also note, upstream only publishes to https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/  when we are undergoing an SRU for ubuntu, so expect that the "el-testing" repo is fairly stable.
[16:38] <blackboxsw> we SRU cloud-init into Ubuntu every month or two.
[16:41] <tribaal> blackboxsw: ack, so I guess building our CentOS templates with a newer binary (for instance built in your COPR repo - or building one ourselves) might be what we end up doing
[16:41] <tribaal> blackboxsw: also: happy new year :)
[16:41] <blackboxsw> tribaal: definitely, for the short term and otubo may have pointers for feature-requesting an update of cloud-init into CentOS. I'm not sure where to start there.
[16:42] <blackboxsw> same to you 🎉
[16:42] <tribaal> otubo: out of curiosity - do you know/can you explain what the process of updating cloud-init in CentOS looks like? is it periodic? is it "whatever was current at the time of the CentOS release"?
[16:42] <blackboxsw> emoj snap for the win
[16:43] <tribaal> I mean updating in the base CentOS
[17:32] <ananke> I'm trying to locate where in the official documentation various config options are documented, specifically for 'users' section, and I must be missing something obvious. where can I find the canonical info on items such as the ones described here? https://www.zetta.io/en/help/articles-tutorials/cloud-init-reference/
[17:35] <smoser> its more than a bit obnoxious that you cannot attach tarballs to pull requests
[17:35] <smoser> https://github.com/canonical/cloud-init/pull/128
[17:36] <rharper> ananke: https://cloudinit.readthedocs.io/en/19.2/topics/modules.html#users-and-groups  and https://cloudinit.readthedocs.io/en/19.2/topics/examples.html#including-users-and-groups
[17:37] <smoser> oh. maybe ignore my noise
[17:37] <ananke> rharper: ahh, thanks! didn't realize the modules section expanded
[17:38] <rharper> ananke: well, the latest version doesn't do that, we have a bug to revert that back to the other releases
[17:38] <ananke> ohh, that would explain
[17:39] <blackboxsw> rharper: yeah I was finding the same.
[17:39] <rharper> https://bugs.launchpad.net/cloud-init/+bug/1852456
[17:39] <blackboxsw> user_groups docs get generated but not included for some reason
[17:39] <blackboxsw> thanks ryh
[17:39] <rharper> low hanging fruit
[17:39] <blackboxsw> yeah
[17:39] <ananke> yep, no wonder. I've been blindly fumbling through the search and menu
[17:40] <rharper> ananke: yeah, sorry for the trouble
[17:40] <rharper> blackboxsw: I don't think we should wait for powersj to sort out a better solution  as mentioned in the bug; we should re-add the Left-hand-side toc  as it was; it's *much* friendlier
[17:41] <powersj> rharper, blackboxsw +1
[17:41] <blackboxsw> smoser, I thought I saw github could allow tar.gz attachments https://github.com/dear-github/dear-github/issues/150#issuecomment-369744370
[17:42] <blackboxsw> testing that theory
[17:44] <blackboxsw> yeah test.tar.gz works, test.tar is rejected due to invalid file type
[17:45] <blackboxsw> and yeah, ridiculous that they'd support tar.gz, but not tar :P0
[17:51] <blackboxsw> Just published to focal [ubuntu/focal-proposed] cloud-init 19.4-16-gf8950d63-0ubuntu1 (Accepted)
[17:51] <blackboxsw> Inbox
[17:54] <blackboxsw> rharper:/Odd_Bloke I think we'll need to fix build-and-push script as we can no longer push branches direct to upstream (as they require review) https://trello.com/c/MdP53rtp/1239-github-upstream-release-script-build-and-push-needs-to-push-up-a-pr-for-review
[17:54] <smoser> blackboxsw: yeah, ignore my noise
[17:55] <blackboxsw> I've put up a PR for the upstream release (which was already dput and accepted by my build-and-push run) https://github.com/canonical/cloud-init/pull/151
[17:55] <blackboxsw> smoser:  no problemo..... but just this time my friend
[18:01] <smoser> i wont mess up again
[18:01] <blackboxsw> heh
[18:25] <Odd_Bloke> blackboxsw: Should we just disable branch protection on those branches?
[18:26] <Odd_Bloke> I'd prefer to have exactly the git commit used for the upload as tip of those branches, and I don't think any of GH's merging strategies would do that.
[18:26] <Odd_Bloke> (GH's rebase never fast-forwards even when it could IIRC.)
[18:36] <blackboxsw> Odd_Bloke: that's a good idea on branch protection disabling for ubuntu/devel at least. We can talk at a planning meeting in 1.5 hrs
[18:55] <Odd_Bloke> blackboxsw: Do we need to wait for a meeting?
[18:55] <Odd_Bloke> rharper: powersj: Thoughts on disabling branch protection on the ubuntu/* branches, so we can push the uploaded git revision directly?
[18:59] <powersj> Odd_Bloke, I'm good with that
[18:59] <powersj> not hearing any objections
[19:03] <blackboxsw> +1 Odd_Bloke
[19:03] <rharper> Odd_Bloke: +1 on allowing direct landing
[19:04] <blackboxsw> and Odd_Bloke earns a card :) https://trello.com/c/MdP53rtp/1239-github-upstream-release-script-build-and-push-needs-to-push-up-a-pr-for-review
[19:05] <blackboxsw> rharper: ananke powersj https://github.com/canonical/cloud-init/pull/153  for review of the doc fix in modules https://github.com/canonical/cloud-init/pull/153
[19:07] <blackboxsw> I added a TOC to the top of modules page. make doc seems to show all modules now, with links
[19:13] <Odd_Bloke> blackboxsw: Done. :)
[19:14] <ananke> thanks!
[19:14] <ananke> and looks like trello is back in working order too
[19:19] <blackboxsw> starting out the year right :). New year's resolutions and all
[19:31] <blackboxsw> Odd_Bloke: thanks, just pushed ubuntu/19.4-16-gf8950d63-0ubuntu1 to origin
[19:32] <blackboxsw> ubuntu/devel should represent the commitish that was uploaded
[19:34] <Odd_Bloke> Nice
[20:40] <ananke> am I just confused, or is the example backwards? 'Update apt database on first boot
[20:41] <ananke> from https://cloudinit.readthedocs.io/en/latest/topics/examples.html shows 'package_update: false
[21:03] <Odd_Bloke> ananke: It's not _wrong_ per se, in the sense that it's just illustrating how you would give this configuration option, but it is _definitely_ confusing, so we should address it.
[21:06] <ananke> Odd_Bloke: it's certainly very confusing, considering the very next example shows an option _adjusted_ to match the description
[21:06] <Odd_Bloke> Yep, agreed.
[21:06] <Odd_Bloke> ananke: Interested in proposing a fix?
[21:07] <ananke> certainly, I'm just not familiar with the procedure. do I have to actually do a PR?
[21:09] <ananke> on a side note, is there a way to use variables in cloud-init configs? not sure if YAML permits that
[21:11] <Odd_Bloke> ananke: https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html#using-instance-data details how to use Jinja for templating, is that what you're looking for?
[21:11] <Odd_Bloke> ananke: You'd need to submit a PR, and sign the CLA (which, unfortunately, involves having a Launchpad account).
[21:12] <ananke> indeed, it's close. I am looking for using it with runcmd, since I have to refer to a version number in multiple commands. I wasn't prepared for the entire config to be be a potential jinja2 template, but I think it will work
[21:13] <ananke> Odd_Bloke: I see. I'd love to, but I'm short on time to complete few other things, and getting those items lined up would put me behind. maybe later this week
[21:15] <ananke> hmm, I don't think the instance data will work though, since all I'm doing is supplying a single cloud init config file
[21:15] <Odd_Bloke> ananke: What do you mean by "config file"?
[21:15] <Odd_Bloke> cloud-init has several things that might be. :p
[21:16] <ananke> Odd_Bloke: good point. it's what I can provide with packer & multipass when an instance is launched
[21:17] <Odd_Bloke> ananke: That's user-data (at least for multipass), so I think you should be good.  But your best bet would be to try and see, if you can. :)
[21:17] <ananke> forgive my ignorance, still trying to wrap my head around how cloud-init works, and how we can leverage it
[21:17] <Odd_Bloke> No forgiveness necessary, we're more than happy to help. :)
[21:20] <ananke> is there an architectural diagram of cloud-init that would show the stages and what's being ingested at each one? or something that would differentiate between user and instance data (and if there are any other ones)
[21:22] <Odd_Bloke> ananke: https://github.com/canonical/cloud-init/pull/154 :)
[21:22] <Odd_Bloke> I'm not aware of such a diagram.
[21:22] <ananke> thanks!
[21:22] <Odd_Bloke> Once again, I agree that it would be very useful! :p
[21:22] <blackboxsw> just descriptive docs that I know of https://cloudinit.readthedocs.io/en/latest/topics/boot.html about what each stage does. differentation of user/meta/instance  data  https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html
[21:23] <ananke> the diagram here is somewhat helpful: http://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instance
[21:25] <blackboxsw> ananke: a brief overview of cloud-init from a presentation we gave in OSSEU https://events19.linuxfoundation.org/wp-content/uploads/2017/12/cloud-init-The-cross-cloud-Magic-Sauce-Scott-Moser-Chad-Smith-Canonical.pdf
[21:27] <ananke> blackboxsw: thank you, this looks great. exactly the high level stuff I need to get me oriented. I've been suffering from 'can't see the forest for the trees'
[21:28] <blackboxsw> ananke: also I forgot it's linked here too with other context https://cloudinit.readthedocs.io/en/latest/topics/faq.html#where-can-i-learn-more
[21:28] <ananke> cheers!
[21:28] <blackboxsw> surely
[22:19] <meena> packer can use cloud-init?
[22:45] <Odd_Bloke> meena: It depends on which backend you're using, I think, but yes.
[23:08] <ananke> meena: yes. you can pass the cloud-init config via its user_data or user_data_file options
[23:10] <ananke> it's certainly not something easy to figure out if you're not familiar with that nomenclature. none of its documentation mentions cloud-init outright
[23:12] <Odd_Bloke> rharper: On the Scaleway PR, is your intent that Louis step back and find a way to log better messages?  (It looks, to me, like a more general statement about what we should do as a project, but might read as a more specific request.)
[23:13] <rharper> Odd_Bloke: the latter; for now;
[23:13] <rharper> I'm I won't -1 the current change to the datasource; but we have other variants, DatasourceSmartOS does a if self._network_config is UNSET: self._network_config = None;
[23:13] <rharper> which might match better in Scaleway since they __init__() to self._network_config = None
[23:14] <rharper> it works out the same, but I was sort of ranting that we've not given datasources common methods for this non-datasource-specific scenario
[23:15] <rharper> I was quite surprised that it was set to UNSET until I walked through the logs and code
[23:15] <rharper> so I kind of want to make it clear in the future via the logs; and potentially have better methods around manipulation/checking of ds.network_config state
[23:39] <Odd_Bloke> Ack
[23:50] <meena> ananke: aaah. i was looking for it under the provisioning methods, and could not find it
[23:54] <ananke> meena: bingo. and it runs before any provisioners kick in
[23:55] <meena> cool cool
[23:57] <ananke> though I wish it had better integration with cloud-init, for example an easy way to tell packer to wait until cloud-init is done