[11:06] <toabctl> hey cloud-init team. I want to inform you that the focal & bionic lxd metadata updated the nocloud-net templates. Chad told me that I should inform you when this change lands. See https://bugs.launchpad.net/cloud-images/+bug/1988401 for details.
[11:06] <ubottu> Launchpad bug 1988401 in cloud-images "LXD image metadata: Update nocloud-net templates in compressed LXD image metadata for releases older than 22.04 Jammy" [Medium, Fix Released]
[12:09] <aciba> thanks toabctl. I am seeing some integration tests failing due to the change of behavior (datasource detection) on that change: https://jenkins.canonical.com/server-team/view/cloud-init/job/cloud-init-integration-bionic-lxd_vm/102 and https://app.travis-ci.com/github/canonical/cloud-init/jobs/582765054
[12:12] <toabctl> aciba, https://jenkins.canonical.com/server-team/view/cloud-init/job/cloud-init-integration-bionic-lxd_vm/102/consoleFull uses serial=20220902 . that should be unaffected so this might be a different problem
[12:15] <toabctl> the error from travis is correct. the error from jenkins seems exactly the opposite. the jenkins error should actually be fixed with the change . maybe you did unconditionally backport the datasource changes in the CI? travis shows:
[12:15] <toabctl> if ImageSpecification.from_os_image().release in [
[12:15] <toabctl> 6110                "bionic",
[12:15] <toabctl> 6111                "focal",
[12:15] <toabctl> 6112            ]:
[12:15] <toabctl> 6113                datasource = "DataSourceNoCloud"
[12:15] <toabctl> 6114            else:
[12:15] <toabctl> 6115                datasource = "DataSourceLXD"
[12:21] <aciba> I assumed both were caused by the same. We will have a look, thanks!
[14:10] <SDes91> Hi all, I am making a bootable disk image using docker containers. I have installed `cloud-init` using the APT package manager. I wish to know how can I configure cloud-init so that it picks up the `user-data` file from a specific partition like that in Raspberry Pi 4 (`/boot/firmware`) partition?
[15:10] <meena> SDes91: cloud-init does a lot of probing to figure out what kind of cloud it's on, and based on that, decides where to look for its user-data
[15:17] <meena> SDes91: you can (re-)run that probing (manually) with /usr/local/lib/cloud-init/ds-identify or wherever that is on Linux
[15:20] <SDes91> I am actually planning to use `NoCloud`. So maybe I can symlink the files in the partition to their respective files in the `/var/lib/cloud` directory?
[15:22] <holman> SDes91: not sure exactly what you're doing, but symlinking files shouldn't be necessary for normal cloud-init operation. 
[15:23] <holman> as meena suggested, ds-identify (with a certain env var) should print out NoCloud as an option if your NoCloud partition is detected
[15:26] <holman> something like this: DEBUG_LEVEL=2 DI_LOG=stderr /usr/lib/cloud-init/ds-identify
[15:26] <holman> add --force if you already ran it (otherwise it uses a cached result)
[15:28] <holman> also without the correct partition label and filesystem type, NoCloud will not be detected, so that detail is really important
[15:29] <SDes91> holman in a nutshell, I have a custom ubuntu image for ARM64 which is practically is built from the ground up. This includes cloud-init via APT package. I also have to setup custom boot partition for this image, I am assuming based on documentation for NoCloud if my boot partition is of type VFAT, cloud-init will be able to pick the cloud-config
[15:29] <SDes91> files in them and setup my hardware device. Am I right to understand this?
[15:31] <holman> SDes91: hmm, I've never tried re-using the boot partition for this, I've always had a separate partition, but I can't think of a reason your approach wouldn't work (but make sure you have the label, per that docs: "The filesystem volume label must be cidata or CIDATA."
[15:33] <holman> SDes91: We have a little demo for NoCloud using qemu that might be helpful to poke at if you have questions.
[15:33] <SDes91> I am hoping I can achieve that via `sfdisk` or even with some Hashicorp Packer plugin.
[15:34] <SDes91> *fingers-crossed* seems a bit daunting but hope it works out.
[15:34] <holman> SDes91: if you grep the NoCloud doc page for `echo "instance-id` you'll hit a section that shows you how to set up a local file-backed NoCloud disk for qemu that contains the required files.
[15:34] <holman> Good luck :)
[15:41] <meena> nothing pre-baked: https://learn.hashicorp.com/search?query=cloud-init&product=packer
[15:42] <SDes91> meena I am aware of that hence I had to work through a Repository https://github.com/shantanoo-desai/packer-ubuntu-server-uefi albeit for AMD64 Ubuntu images.
[15:45] <SDes91> Packer in essence creates a server where the cloud-init files will be fetched by the emulated image and then be configured accordingly. Only difference for ARM64 (I am basing this on Raspberry Pi) is that typically you add the user-data file already on the boot partition on the SD Card.
[15:49] <meena> SDes91: well, yes, unless your ARM64 is in a different sort of cloud, like oracle's
[15:51] <SDes91> True. Packer does provide two builders for ARM images, one of them offers partitioning, so maybe I might be able to create `cidata` partition with the files in them and hope that cloud-init will be able to pick the stuff up "out-of-the-box"
[16:57] <blackboxsw> falcojr: holman have you seen the LXD VM recent focal integration test error E       `AssertionError: Unexpected warning count != 0. Found: ["WARNING]: Running ['netplan', 'apply'] resulted in stderr output: \x1b[0;1;31mFailed to connect system bus: No such file or directory\x1b[0m", 'WARNING]: Used fallback datasource']`  I haven't seen it on current `lxc launch ubuntu-daily:focal  --vm --profile pycloudlib-vm-focal-v3`
[16:58] <blackboxsw> yet it seems I can reproduce that error with our integration tests. ` CLOUD_INIT_PLATFORM=lxd_vm CLOUD_INIT_OS_IMAGE=focal tox -e integration-tests tests/integration_tests/modules/test_lxd.py`
[16:59] <blackboxsw> Will dig in now and report my findings if it wasn't already talked through yesterday. I'm thinking this was the "other" integration test failure aciba mentioned trying to peek at
[17:01] <falcojr> yeah, I was wondering if he'd have a status before EOD for him 
[17:02] <falcojr> we have other integration test failures I plan on looking at too for triage 
[18:16] <blackboxsw> falcojr: holman I agree we with cjp256 may need better handling of the `DataSource.update_metadata_if_supported` in the event that the datasource is not longer accessible or cloud-init introduces a regression that results in get_data == False across upgrade path. But, I don't think it ecessarily should be fixed in 22.3.2 as it preexists 22.2 and probably needs a more pragmatic approach to how we "stage" metadata updates 
[18:16] <blackboxsw> before persisting them to the obj.pkl. https://github.com/canonical/cloud-init/pull/1725#issuecomment-1247096562
[18:16] <ubottu> Pull 1725 in canonical/cloud-init "define new attribute for pre-22.3 pickles" [Merged]
[18:17] <blackboxsw>  I don't think it necessarily should be fixed in 22.3.2*.....   cjp256 falcojr holman aciba: Do you think it's reasonable then that we just cherry-pick (instance-id persistence for Azure) b861ea8a5e1fd0eb33096f60f54eeff42d80d3bd and (azure: define new attribute for pre-22.3 pickles) 51351f24d80637b0a58a4c706003e64dd32579b3 to create our 22.3.2 upstream release?
[18:27] <falcojr> sounds reasonable to me
[19:00] <holman> no objections from me
[20:17] <falcojr> blackboxsw or holman: integration tests are blocked until #1732 (or #1730 if my machine is crazy) merges
[20:17] <falcojr> can one of you take a look, but first test we have the right series
[20:18] <holman> will do, looking
[20:18] <falcojr> I looked through the LXD changelog and didn't see anything regarding this change
[20:18] <blackboxsw> I also think per cjp256's comment on Azure raising InvalidMetadataException we actually "want" that type of behavior from the Azure datasource when we get past is_platform_viable checks because no non-azure platforms should get into this part of the code, and we want to raise visible errors when that happens instead of "return False" https://github.com/canonical/cloud-init/pull/1725#issuecomment-1247256748
[20:18] <ubottu> Pull 1725 in canonical/cloud-init "define new attribute for pre-22.3 pickles" [Merged]
[20:21] <blackboxsw> @falcojr when testing 1732 earlier today I hit a prob on focal (ran  fine on Bionic) with`Failed to connect system bus: No such file or directory` ..... I'll check with 1730
[20:27] <falcojr> other way around? I just put up 1732. But it runs fine on Bionic? It ran for me on Focal but failed on Bionic still using the NoCloud datasource
[20:33] <cjp256> blackboxsw: we are on the same page, pulling in those two fixes would be best for now
[20:35] <holman> @falcojr @blackboxsw: confirmed #1732 fails on bionic and not focal locally for me
[20:35] <falcojr> hmmm, so my machine is the weird one
[20:36] <falcojr> and #1730 is fine for both? (other way to verify is launch bionic-daily and check if it detects LXD or NoCloud)
[20:36] <holman> s/1732/1730/
[20:36] <holman> sorry mixed up PR numbers there
[20:36] <holman> @falcojr @blackboxsw: confirmed #1730 fails on bionic and not focal locally for me
[20:37] <holman> I had both open locally and read the wrong one as typing
[20:38] <falcojr> ok, that scares me less then
[20:40] <holman> :D
[20:45] <holman> and 1732 passes on both 
[20:48] <holman> +1 @cjp256, sounds good thanks
[20:49] <falcojr> thanks for the reviews
[21:47] <cjp256> any idea when a new -proposed will hit to test?
[22:01] <falcojr> hopefully tomorrow if all goes well
[22:04] <meena> I could've sworn we had a cloudinit/parsers directory. is that gone? or am i delusional and it never existed?
[22:34] <cjp256> falcojr: thanks!
[23:34] <holman> meena: are you thinking of the one under distros? https://dpaste.org/7Jkcj
=== holman is now known as holmanb
=== holmanb is now known as holman