[19:20] <holmanb> Odd_Bloke/falcojr/minimal/meena: another big reason for a non-bash language is to have a proper yaml parser in ds-identfy
[19:20] <holmanb> we likely have lots of hidden bugs related to this hacky parsing we do
[19:20] <holmanb> I just came across one
[19:21] <falcojr> yes, it would definitely be nice to have real yaml parsing
[19:22] <falcojr> holmanb: are you saying https://stackoverflow.com/a/21189044 isn't good enough ;)
[19:23] <holmanb> heh
[19:23] <holmanb> lets say you want a single cloud.cfg that descibes behavior on multiple clouds - you configure maas and ec2, for example: https://dpaste.org/e9JJh
[19:24] <holmanb> oops, now ds-identify identifies MAAS as your datasource
[19:25] <holmanb> and ds-identify produces a /run/cloud-init/cloud.cfg with the following:
[19:25] <holmanb> datasource_list: [ MAAS, LXD, None ]
[19:25] <holmanb> (tested on lxd obviously)
[19:30] <minimal> silly question, how does one test that ds-identify is actually working? (beyond looking at /run/cloud-init/dsidentify.log)
[19:31] <holmanb> minimal: good question, the answer isn't pretty
[19:31] <holmanb> minimal: tests/unittests/test_ds_identify.py
[19:31] <holmanb> that is our unit test suite
[19:31] <minimal> was playing "spot the difference" when looking at "with ds-identify" and "without ds-identify" cloud-init.log files
[19:32] <holmanb> minimal: the big different is going to be that /run/cloud-init/cloud.cfg will exist, which contains a datasource_list that overrides the one in /etc/cloud/cloud.cfg
[19:34] <holmanb> minimal: ds-identify will usually cut the list in cloud.cfg down to just one datasource (and None)
[19:34] <minimal> ok, but if my datasource_list is "[ NoCloud, None ]" and cloud.cfg also has "[ NoCloud, None ]" then how do I *know*/prove the decision came from ds-identify and not the existing config file? ;-)
[19:35] <minimal> in that scenario boottime is actually slightly slower as ds-identify is being run lol
[19:36] <holmanb> minimal: do you want to know that for testing / validation? or is there some reason at runtime that you would want to know that?
[19:36] <holmanb> minimal: right, ds-identify running is going to add time over it not running (especially on not-systemd)
[19:37] <minimal> I'm testing my proposed changes in a VM and was expecting to see some evidence in the cloud-init.log
[19:37] <minimal> i.e. some logline showing it using /run/cloud-init/cloud.cfg
[19:38] <holmanb> ds-identify also leave behind /run/cloud-init/.ds-identify.result, which is like a "persistent" return code of ds-identify
[19:38] <holmanb> *leaves
[19:40] <minimal> ok, just about to test a "generic" VM, i.e. where the datasource_list has a whole lot of entries compared to the NoCloud-only VM I've been testing so far
[19:40] <holmanb> I don't know whether anything will show up in cloud-init.log
[19:40] <holmanb> maybe something about config merging, if that code logs which files it is merging
[19:40] <minimal> ok, I just assumed something would appear there
[19:43] <holmanb> minimal: yeah I don't think you would see this line in cloud-init.log without ds-identify:
[19:43] <holmanb> Reading from /run/cloud-init/cloud.cfg
[19:44] <holmanb> it isn't obvious 
[19:52] <minimal> side question: should "None" be specified any more in a datasources_list? I got lost with various changes related to this over the past few months
[19:52] <minimal> I mean in any datasources_list that I might put into a config file
[20:16] <minimal> holmanb: ok, tried with "generic" VM and yes cloud-init.log shows "NoCloud" is determined/used which I guess comes from ds-identify
[20:18] <minimal> cloud-init-local however still runs blkid on the cdrom device to check its label, fs type etc even though /run/cloud-init/ds-identify.log contains ISO9660_DEVS=/dev/sr0=cidata
[20:18] <minimal> FS_LABELS=cidata,alpine-root, got_label=cidata etc
[20:19] <minimal> should'nt cloud-init-local use this info provided by ds-identify rather than search for a NoCloud YAML source?
[21:45] <minimal> holmanb: another unrelated question, I'm adding a new shell script to tools/ that is installed to /usr/lib/cloud-init/ during c-i install, however it is not executable. I guess I could do a chmod +x BEFORE I add that new script to the git repo but I'd really expect the c-i install to be able to specifically make it executable. It added the script to setup.py's data_files section next to "tools/ds-identify"
[21:48] <minimal> i.e. I don't expect the PR I'm planning to raise to add this would "flag" the execute permission for the file newly added to git main...