[07:25] And so it begins again! (I hope) [09:04] hi all! I would need some pointers on how to get a more recent cloud-init version on CentOS (eg. CentOS 7.6). Is using a COPR repository the exepcted way to get the latest binairies or is there some kind of process to refresh the cloud-init package in CentOS base similar to Ubuntu's SRU? [09:07] tribaal: which version are you getting from COPR? [09:08] or, "would you be getting" … if you haven't dared to try yet. [09:19] meena: the idea would be to have a centOS 7.6 with 19.3+ [09:20] but I don't know what the best way to do this would be (we'd probably update our default templates to include the right binaries ourselves if there's no SRU-like process for CentOS) [09:25] tribaal: https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/ you'd get 19.4 if you https://docs.pagure.org/copr.copr/how_to_enable_repo.html#how-to-enable-repo `dnf copr enable @cloud-init/el-testing` [09:26] meena: ack, thanks. That's not really "production" builds though, is it? [09:28] tribaal: i don't know (anything about CentOS / Fedora etc… i haven't used them in a long time, and in $$jobs$$ we use RHEL) [09:30] meena: ack - same here (but we us Ubuntu :) ) [09:30] so our Ubuntu templates have the latest and greatest but we'd need our centOS templates to have the latest/greatest as well [09:30] and I'm not too sure how/where to look :) === cpaelzer__ is now known as cpaelzer [13:42] meena: tribaal: o/ Happy new year! [13:43] I can't remember exactly the arrangement of our COPR repos, we'll have to wait for Ryan/Chad to help, I'm afraid. [13:43] Odd_Bloke: happy monday [13:54] Odd_Bloke: i'll need to quiz your brain some more wrt net refactor [13:54] but, IIRC, i've written the ideas / questions down. now lemme just find the URL… [13:55] 23:47 updates: https://hackmd.io/3-YBj1t9TAeKhmfLBQUjXQ?view#Revisiting-modules [13:55] 27th of December. [13:56] I haven't even opened my inbox yet, so I don't know how soon I'll get to it, but it's on my list now! :p [14:09] Odd_Bloke: heya! happy new year to you too :) [16:32] happy 2020 cloud-initers. Time to dig out from holiday emails [16:37] tribaal: we (upstream cloud-init) don't work with distros directly, other than Ubuntu, to push upstream changes into the distribution. We rely on the distro vendors to vet cloud-init themselves. That said, I believe otubo is the right person to ask about how to coordinate updates into CentOS. The cloud-init upstream team only provides a our COPR repos as a facility for CentOS users to check if more recent [16:37] upstream versions of cloud-init help resolve bugs seen on the current (older) cloud-init in CentOS stock images. Also note, upstream only publishes to https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/ when we are undergoing an SRU for ubuntu, so expect that the "el-testing" repo is fairly stable. [16:38] we SRU cloud-init into Ubuntu every month or two. [16:41] blackboxsw: ack, so I guess building our CentOS templates with a newer binary (for instance built in your COPR repo - or building one ourselves) might be what we end up doing [16:41] blackboxsw: also: happy new year :) [16:41] tribaal: definitely, for the short term and otubo may have pointers for feature-requesting an update of cloud-init into CentOS. I'm not sure where to start there. [16:42] same to you 🎉 [16:42] otubo: out of curiosity - do you know/can you explain what the process of updating cloud-init in CentOS looks like? is it periodic? is it "whatever was current at the time of the CentOS release"? [16:42] emoj snap for the win [16:43] I mean updating in the base CentOS [17:32] I'm trying to locate where in the official documentation various config options are documented, specifically for 'users' section, and I must be missing something obvious. where can I find the canonical info on items such as the ones described here? https://www.zetta.io/en/help/articles-tutorials/cloud-init-reference/ [17:35] its more than a bit obnoxious that you cannot attach tarballs to pull requests [17:35] https://github.com/canonical/cloud-init/pull/128 [17:36] ananke: https://cloudinit.readthedocs.io/en/19.2/topics/modules.html#users-and-groups and https://cloudinit.readthedocs.io/en/19.2/topics/examples.html#including-users-and-groups [17:37] oh. maybe ignore my noise [17:37] rharper: ahh, thanks! didn't realize the modules section expanded [17:38] ananke: well, the latest version doesn't do that, we have a bug to revert that back to the other releases [17:38] ohh, that would explain [17:39] rharper: yeah I was finding the same. [17:39] https://bugs.launchpad.net/cloud-init/+bug/1852456 [17:39] Ubuntu bug 1852456 in cloud-init "doc: list of modules is no longer present" [Medium,Triaged] [17:39] user_groups docs get generated but not included for some reason [17:39] thanks ryh [17:39] low hanging fruit [17:39] yeah [17:39] yep, no wonder. I've been blindly fumbling through the search and menu [17:40] ananke: yeah, sorry for the trouble [17:40] blackboxsw: I don't think we should wait for powersj to sort out a better solution as mentioned in the bug; we should re-add the Left-hand-side toc as it was; it's *much* friendlier [17:41] rharper, blackboxsw +1 [17:41] smoser, I thought I saw github could allow tar.gz attachments https://github.com/dear-github/dear-github/issues/150#issuecomment-369744370 [17:42] testing that theory [17:44] yeah test.tar.gz works, test.tar is rejected due to invalid file type [17:45] and yeah, ridiculous that they'd support tar.gz, but not tar :P0 [17:51] Just published to focal [ubuntu/focal-proposed] cloud-init 19.4-16-gf8950d63-0ubuntu1 (Accepted) [17:51] Inbox [17:54] rharper:/Odd_Bloke I think we'll need to fix build-and-push script as we can no longer push branches direct to upstream (as they require review) https://trello.com/c/MdP53rtp/1239-github-upstream-release-script-build-and-push-needs-to-push-up-a-pr-for-review [17:54] blackboxsw: yeah, ignore my noise [17:55] I've put up a PR for the upstream release (which was already dput and accepted by my build-and-push run) https://github.com/canonical/cloud-init/pull/151 [17:55] smoser: no problemo..... but just this time my friend [18:01] i wont mess up again [18:01] heh [18:25] blackboxsw: Should we just disable branch protection on those branches? [18:26] I'd prefer to have exactly the git commit used for the upload as tip of those branches, and I don't think any of GH's merging strategies would do that. [18:26] (GH's rebase never fast-forwards even when it could IIRC.) [18:36] Odd_Bloke: that's a good idea on branch protection disabling for ubuntu/devel at least. We can talk at a planning meeting in 1.5 hrs [18:55] blackboxsw: Do we need to wait for a meeting? [18:55] rharper: powersj: Thoughts on disabling branch protection on the ubuntu/* branches, so we can push the uploaded git revision directly? [18:59] Odd_Bloke, I'm good with that [18:59] not hearing any objections [19:03] +1 Odd_Bloke [19:03] Odd_Bloke: +1 on allowing direct landing [19:04] and Odd_Bloke earns a card :) https://trello.com/c/MdP53rtp/1239-github-upstream-release-script-build-and-push-needs-to-push-up-a-pr-for-review [19:05] rharper: ananke powersj https://github.com/canonical/cloud-init/pull/153 for review of the doc fix in modules https://github.com/canonical/cloud-init/pull/153 [19:07] I added a TOC to the top of modules page. make doc seems to show all modules now, with links [19:13] blackboxsw: Done. :) [19:14] thanks! [19:14] and looks like trello is back in working order too [19:19] starting out the year right :). New year's resolutions and all [19:31] Odd_Bloke: thanks, just pushed ubuntu/19.4-16-gf8950d63-0ubuntu1 to origin [19:32] ubuntu/devel should represent the commitish that was uploaded [19:34] Nice [20:40] am I just confused, or is the example backwards? 'Update apt database on first boot [20:41] from https://cloudinit.readthedocs.io/en/latest/topics/examples.html shows 'package_update: false [21:03] ananke: It's not _wrong_ per se, in the sense that it's just illustrating how you would give this configuration option, but it is _definitely_ confusing, so we should address it. [21:06] Odd_Bloke: it's certainly very confusing, considering the very next example shows an option _adjusted_ to match the description [21:06] Yep, agreed. [21:06] ananke: Interested in proposing a fix? [21:07] certainly, I'm just not familiar with the procedure. do I have to actually do a PR? [21:09] on a side note, is there a way to use variables in cloud-init configs? not sure if YAML permits that [21:11] ananke: https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html#using-instance-data details how to use Jinja for templating, is that what you're looking for? [21:11] ananke: You'd need to submit a PR, and sign the CLA (which, unfortunately, involves having a Launchpad account). [21:12] indeed, it's close. I am looking for using it with runcmd, since I have to refer to a version number in multiple commands. I wasn't prepared for the entire config to be be a potential jinja2 template, but I think it will work [21:13] Odd_Bloke: I see. I'd love to, but I'm short on time to complete few other things, and getting those items lined up would put me behind. maybe later this week [21:15] hmm, I don't think the instance data will work though, since all I'm doing is supplying a single cloud init config file [21:15] ananke: What do you mean by "config file"? [21:15] cloud-init has several things that might be. :p [21:16] Odd_Bloke: good point. it's what I can provide with packer & multipass when an instance is launched [21:17] ananke: That's user-data (at least for multipass), so I think you should be good. But your best bet would be to try and see, if you can. :) [21:17] forgive my ignorance, still trying to wrap my head around how cloud-init works, and how we can leverage it [21:17] No forgiveness necessary, we're more than happy to help. :) [21:20] is there an architectural diagram of cloud-init that would show the stages and what's being ingested at each one? or something that would differentiate between user and instance data (and if there are any other ones) [21:22] ananke: https://github.com/canonical/cloud-init/pull/154 :) [21:22] I'm not aware of such a diagram. [21:22] thanks! [21:22] Once again, I agree that it would be very useful! :p [21:22] just descriptive docs that I know of https://cloudinit.readthedocs.io/en/latest/topics/boot.html about what each stage does. differentation of user/meta/instance data https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html [21:23] the diagram here is somewhat helpful: http://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instance [21:25] ananke: a brief overview of cloud-init from a presentation we gave in OSSEU https://events19.linuxfoundation.org/wp-content/uploads/2017/12/cloud-init-The-cross-cloud-Magic-Sauce-Scott-Moser-Chad-Smith-Canonical.pdf [21:27] blackboxsw: thank you, this looks great. exactly the high level stuff I need to get me oriented. I've been suffering from 'can't see the forest for the trees' [21:28] ananke: also I forgot it's linked here too with other context https://cloudinit.readthedocs.io/en/latest/topics/faq.html#where-can-i-learn-more [21:28] cheers! [21:28] surely [22:19] packer can use cloud-init? [22:45] meena: It depends on which backend you're using, I think, but yes. [23:08] meena: yes. you can pass the cloud-init config via its user_data or user_data_file options [23:10] it's certainly not something easy to figure out if you're not familiar with that nomenclature. none of its documentation mentions cloud-init outright [23:12] rharper: On the Scaleway PR, is your intent that Louis step back and find a way to log better messages? (It looks, to me, like a more general statement about what we should do as a project, but might read as a more specific request.) [23:13] Odd_Bloke: the latter; for now; [23:13] I'm I won't -1 the current change to the datasource; but we have other variants, DatasourceSmartOS does a if self._network_config is UNSET: self._network_config = None; [23:13] which might match better in Scaleway since they __init__() to self._network_config = None [23:14] it works out the same, but I was sort of ranting that we've not given datasources common methods for this non-datasource-specific scenario [23:15] I was quite surprised that it was set to UNSET until I walked through the logs and code [23:15] so I kind of want to make it clear in the future via the logs; and potentially have better methods around manipulation/checking of ds.network_config state [23:39] Ack [23:50] ananke: aaah. i was looking for it under the provisioning methods, and could not find it [23:54] meena: bingo. and it runs before any provisioners kick in [23:55] cool cool [23:57] though I wish it had better integration with cloud-init, for example an easy way to tell packer to wait until cloud-init is done