[00:34] <holmanb> caribou: poking at NetworkManager and systemd-networkd configs they both seem to have support for directly connected gateways (no surprise there, given the netplan config setting). It sounds like both of those renderers and per klnikita's comment also debian's eni renderer should be able to support this - just requires a bit of plumbing (I think)
[11:00] <Dev040> Hello Guys!
[11:00] <Dev040> I'm trying to use write_files.
[11:00] <Dev040> write_files is trying to write into a file which depends on a mount.
[11:00] <Dev040> What is the order of cloudinit? Is write_files executed before mounts?
[11:00] <Dev040> If so, how to handle this problem I have?
[11:04] <aciba> Dev040: Hello
[11:05] <aciba> to see the modules order you can execute `cloud-init query merged_cfg`
[11:06] <aciba> there you can see what modules run in what stage (https://cloudinit.readthedocs.io/en/latest/topics/boot.html)
[11:07] <aciba> `write_files` and mount run tipically in the init stage
[11:07] <aciba> but there is an option to defer `write_files` and execute it in the final stage: https://cloudinit.readthedocs.io/en/latest/topics/modules.html#write-files
[11:09] <aciba> defer: true in the file object under write_files, see the examples in the previous link
[11:09] <aciba> Example5 more concretely
[11:25] <meena> aciba: how do i access a defaultdict? and was this way of building it: https://github.com/canonical/cloud-init/compare/main...igalic:cloud-init:bsd/fix_duplicate_macs even sensible?
[11:25] <meena> I mean, good morning
[11:30] <aciba> meena: morning one can access a defaultdict as a normal dict, but one cannot distinguish between already added key and not added. IN typically convert the defaultdicts to dicts after they are built to be able to do so.
[11:30] <aciba> I think that's sensible :)
[11:41] <meena> aciba: the documentation seems a bit thin: https://docs.python.org/3/library/collections.html#collections.defaultdict how do I convert it?
[11:43] <meena> oh, you just say: dict(your_default_dict)
[11:43] <meena> duh…
[11:44] <meena> that'll instantly fix like 14 failing test cases
[11:44] <aciba> yeap
[11:50] <meena> nope, doesn't, cuz I'm still building lists for this…
[11:55] <aciba> convert to dict while reading and convert to defaultdict while writing ?
[11:57] <aciba> meena: if a lot of conversions are expected, one can modify my_default_dict.default_factory attibute instead of converting to/from dict https://docs.python.org/3/library/collections.html#collections.defaultdict.default_factory
[11:58] <meena> aciba: my plan is to only use it for building, because it seems easier…
[11:58] <aciba> nice
[12:09] <Dev040> aciba write_files with defer worked great!
[12:09] <Dev040> How does write_files work? Does it override existing files or doesnt do anything when the given path already exists?
[12:13] <aciba> Dev040: nice to here that. That's not defined in the docs. By a quick look to the code, it looks like it does override existing files.
[12:13] <aciba> hear *
[12:22] <aciba> PR up clarifying that: https://github.com/canonical/cloud-init/pull/1916
[12:22] -ubottu:#cloud-init- Pull 1916 in canonical/cloud-init "doc: improve cc_write_files doc" [Open]
[12:24] <meena> boohyaaah!
[12:24] <meena> that was … actually pretty easy
[12:24] <meena> https://gist.github.com/igalic/6fbff08788f474c1279de5d1ee209a61
[12:24] <meena> aciba ^ cjp256 ^
[12:25] <meena> I should add some tests for this case…
[12:25] <cjp256> awesome :D
[12:25] <aciba> nice :)
[12:27] <meena> I think my target will be to refactor get_interfaces() and get_interfaces_by_mac()… before the end of the year :P
[12:27] <meena> (*or fiscal year, maybe whenever that is)
[12:28] <meena> so the fact that get_interfaces() is currently broken on BSD for at least Azure compels me to hurry up with this
[12:33] <ahasenack> hi, does "cloud-init status --wait" block forever?
[12:35] <meena> ahasenack: if it does, something must be broken.
[12:35] <ahasenack> I'm considering wrapping a call to it in timeout(1)
[12:35] <ahasenack> (shell)
[12:37] <meena> ahasenack: there has to be a reason why it's blocking forever. What's the logs say?
[12:37] <aciba> ahasenack: it waits until the status is different from { running , not_run }
[12:38] <ahasenack> meena: it's not blocking forever, I'm just guarding this script against that eventuality
[12:38] <ahasenack> so I was looking for docs on it, or maybe its own little --timeout parameter
[12:40] <aciba> the difficulty may be to define a default timeout suitable for every one. but we could probably introduce an optional flag to specify the timeout time as you suggest
[12:41] <ahasenack> blocking forever is usually a nono, that's my thought
[12:41] <ahasenack> this is in a script that runs tests, so specially here I don't want forever blocks
[12:45] <aciba> I agree with you
[12:48] <aciba> I think the original intention of the --wait flag was to let others block until cloud-init finishes. But for a testing context, a supplemental --timeout makes completely sense.
[13:02] <meena> cjp256: https://github.com/canonical/cloud-init/pull/1917
[13:02] -ubottu:#cloud-init- Pull 1917 in canonical/cloud-init "BSD: fix duplicate macs in Ifconfig parser" [Open]
[13:21] <meena> how did i break the readthedocs build?
[13:25] <falcojr> you didn't. It's broken atm
[14:30] <meena> in that case, that's ready to review then, falcojr
[14:31] <meena> uuuh, I can do another assert, actually
[15:52] <blackboxsw> meena: you can git fetch upstream; git rebase -i upstream/main now to fix readthedocs CI. We just merged https://github.com/canonical/cloud-init/pull/1912
[15:52] -ubottu:#cloud-init- Pull 1912 in canonical/cloud-init "read-version: When insufficient tags, use cloudinit.version.get_version" [Merged]
[15:52] <seanbright> hi there. i am using the nocloud datasource and providing a static network configuration via the network-config file. i want to change the IP after first boot, so i've updated the network-config file and rebuilt the ISO and rebooted the VM but it doesn't re-render the netplan config when i do that. is there a way i can force that?
[15:58] <blackboxsw> seanbright: typically datasourcenocloud renders network only PER_BOOT_NEW_INSTANCE. If you are rebuilding the ISO and want to trigger cloud-init to run everything again (including network config). You could change the 'instance-id: value'  in your meta-data file and that will indicate to cloud-init to re-run everything including network config. 
[15:58] <seanbright> blackboxsw: perfect
[15:59] <seanbright> blackboxsw: thank you!
[15:59] <blackboxsw> seanbright: I'm trying to think through whether we can override the default_update_events https://github.com/canonical/cloud-init/blob/main/cloudinit/sources/__init__.py#L224-L228  to include eventtype.BOOT for NoCloud datasource, but I'm not sure it's possible without code changes
[16:00] <blackboxsw> seanbright: be wary the new instance-id: in metadata tells cloud-init to re-run everything including any user-data and SSH host key regeneration on the next boot. so the identity of your system in terms of SSH will change.
[16:00] <meena> blackboxsw: I used the GitHub sync button. but i still wanna add that yet, however, I'm away from the laptop until 18:45
[16:01] <minimal> blackboxsw: to be honest I'd assumed NoCloud network-config would always be a one-shot thing, never thought anyone would want to change/update it
[16:02] <seanbright> blackboxsw: so maybe i am better off just manually updating the network config.
[16:02] <blackboxsw> minimal: yes generally that should be the case. because it's not a cloud :) and shouldn't be changing metadata or config after that first boot. 
[16:03] <blackboxsw> seanbright: if you are creating the iso yourself, it would be best to seed it with all the data you intend for final configuration
[16:03] <minimal> especially in the scenario of using cloud-init for *physical* servers (which I use it for) as there's a security risk if someone sticks in a USB device with cidata and reboots (i.e. powers off/on) a machine to unexpectedly (for the "admin" person) reconfigure it
[16:04] <seanbright> blackboxsw: sure. makes sense. thanks for your help.
[17:45] <holmanb> minimal: curious, which approach do you use to install cloud-init on the servers?
[17:47] <holmanb> one of the default modes from the alpine wiki? from a cloud image directly? something else?
[17:51] <meena> holmanb: minimal is gone
[17:52] <holmanb> ah, thanks
[17:56] <meena> some people aren't on IRC 24/7. weirdos
[17:56] <holmanb> heh
[17:56] <holmanb> my quassel server is always "on", but I'm not always "on"
[17:57] <meena> same with my TheLounge
[17:58] <meena> I used to run Quassel until i decided to consolidate my chat/social stuff into a Web server: https://igalic.co/thoughts/2020-05-30-social-apps-in-firefox.html
[17:58] <holmanb> another vote for TheLounge :) - I keep hearing good things but haven't tried it
[17:58] <holmanb> looks nice
[17:59] <meena> the Quassel Web interface(s?) at the time were pretty horrid
[17:59] <meena> holmanb: he warned: there's currently zero full-time devs working on it
[17:59] <holmanb> I use the mobile/desktop apps, didn't even know a web client existed
[17:59] <meena> s/he/be/
[18:00] <falcojr> meena: looks like we took a similar journey. I used franz and got frustrated with the same issues. I used the lounge until very recently and now have consolidated some of my chats into a matrix server
[18:00] <holmanb> +1 thx, after initial install, which was pretty painless, I've had zero issues (was initially on web clients then matrix for irc)
[18:01] <holmanb> the UI isn't completely seemless on gnome, but it is a qt app iirc so to be expected I guess
[18:01] <holmanb> gnome UI isn't completely seemless on gnome either, so /shrug
[18:57] <blackboxsw> holmanb: or falcojr I minor bump because my previous commit 'fixing read-version' for shallow git clones didn't drop a 2nd call to the git descibe HEAD --long. https://github.com/canonical/cloud-init/pull/1918
[18:57] -ubottu:#cloud-init- Pull 1918 in canonical/cloud-init "tools: read-version drop extra call to git describe --long" [Open]
[18:57] <blackboxsw> this would be needed for any PRs which are failing RTD test runners
[18:58] <blackboxsw> thx. I only recognized because my nocloud PR continued to fail after a git rebase
[19:04] <holmanb> blackboxsw: I think the RTD test runner issue should be resolved now. Are we seeing that still?
[19:05] <blackboxsw> holmanb: the last test run failed on that 2nd line that I should have redacted line 103
[19:05] <blackboxsw> even through the failure happened on shallow clone it still fell through and re-ran the --long cmd
[19:05] <holmanb>  I messaged a short bit ago in a different channel about that, I should have reflected that more broadly
[19:07] <blackboxsw> +1 that said, the changeset we landed before still has an ommission in that it'll still try to run --long twice on a shallow clone
[19:10] <blackboxsw> Strange as the passing RTD CI builds for are still sing --depth 50... not sure why that is getting enough context for tags now. 
[19:10] <blackboxsw> event without the 'fix' in main https://readthedocs.com/projects/canonical-cloud-init/builds/1222922/
[19:13] <holmanb> a couple of CI runs have made it through in the last day or so as well - I don't think this flakyness is new behavior
[19:14] <blackboxsw> I only was able to get the RTD pass on https://github.com/canonical/cloud-init/pull/1879 after I rebased pulling in changes from 1918
[19:14] -ubottu:#cloud-init- Pull 1879 in canonical/cloud-init "nocloud: add support for dmi variable expansion for seedfrom URL" [Open]
[19:15] <blackboxsw> just waiting on the final integration test to complete
[19:15] <holmanb> I still don't have visibility to see the RTD logs, but if anyone wants to check whether we are shallow cloning on new PRs I made a dummy: https://github.com/canonical/cloud-init/pull/1919
[19:15] -ubottu:#cloud-init- Pull 1919 in canonical/cloud-init "ignoreme" [Open]
[19:15] <blackboxsw> @holmanb checking now
[19:15] <blackboxsw> holmanb: woo hoo 
[19:16] <blackboxsw> git fetch origin --force --tags --prune --prune-tags pull/1919/head:external-1919
[19:16] <blackboxsw> no --depth
[19:16] <holmanb> beautiful
[19:16] <blackboxsw> ok that works for our project in general. that said, the read-version fixes are still 'valid' for shallow clone environments
[19:16] <blackboxsw> in that we want to fallback to cloudinit.version if you can't see tags
[19:18] <holmanb> +1, thanks :)
[19:19] <holmanb> is there any reason we shouldn't try to eventually get RTD logs visible for non-members of the org?
[19:19] <holmanb> for the driveby spelling fixers and whatnot
[19:20] <blackboxsw> It'd be nice if non-org members can see our CI failures and debug them themselves. I don't like the fact that our results are hidden behind firewalls 
[19:20] <blackboxsw> I get it for or jenkins jobs as jenkins has a bad surface area for plugins and security breaches
[19:20] <blackboxsw> but anything we can make public we should
[19:21] <holmanb> +1 agreed
[23:51] <minimal> holmanb: Hi. You were looking me?
[23:52] <minimal> ah, you were asking about how I install physical servers using cloud-init
[23:53] <minimal> I create a suitable disk image, using my own script: https://github.com/dermotbradley/create-alpine-disk-image
[23:53] <minimal> for physical machines it has a small cidata partition with the YAML files
[23:54] <minimal> I boot the machines using USB, "dd" the disk image onto the HDD/SDD, edit the YAML files accordingly (e.g. set static IP etc in network-config.yaml) and then reboot the machine
[23:55] <minimal> I've been meaning to set up PXE for the same, rather than using a USB device