| === benj_0 is now known as benj_ | ||
| dch | I'm writing up some notes on how to *test* cloudinit, specifically on FreeBSD, using jails. | 10:05 |
|---|---|---|
| dch | is there a clever trick (tm) where people can drop their user-data file, and pass this in as a parameter somehow? | 10:05 |
| dch | I have already used `cloud-init schema --annotate -c /var/tmp/userdata` , so something similar | 10:06 |
| dch | I have tried using DataSource NoCloud and None, and then running `cloud-init --debug init --local` per https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html without success | 13:13 |
| dch | aha there is a subtle error in https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html#method-4-local-filesystem a missing `/` makes the path relative, not absolute | 13:40 |
| minimal | isn't that how file references work in general? If they don't begin with "/" then they're relative | 14:07 |
| minimal | dch: it's not clear exactly what you're referring to | 14:09 |
| dch | minimal: now I'm past getting NoCloud to work, my original issue is fixed. | 14:10 |
| dch | IMHO the example in the docs should use an absolute path, not a relative one. | 14:10 |
| dch | I will log a ticket for it | 14:11 |
| minimal | dch: which example? | 14:11 |
| dch | https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html#method-4-local-filesystem | 14:11 |
| dch | `seedfrom: file://10.42.42.42/cloud-init/configs/` should use `file:///...` | 14:11 |
| minimal | which example specifically in that section? | 14:11 |
| dch | that string occurs only once | 14:11 |
| minimal | ok, I wondered if it was the file url was the issue | 14:12 |
| dch | its *super handy* using this for testing, in CI or just locally. | 14:12 |
| minimal | also the "seedfrom" example there is incorrect as the url includes host IP | 14:13 |
| dch | yeah its a bit of a weird copypasta I guess | 14:18 |
| === benj_2 is now known as benj_ | ||
| dch | so everything with NoCloud works as expected on 23.3, but ... 24.1.4 it appears to be broken. | 16:43 |
| dch | I have found a few similar looking issues in github, so will try with git master and hope for the best. | 16:44 |
| minimal | define "broken", also note that 24.2 came out a few weeks ago | 16:44 |
| dch | NoCloud datasource appears not to run | 16:44 |
| dch | and it still tries to fetch the network related ones | 16:45 |
| dch | it's going to take me a while to put up a proper issue for this | 16:45 |
| minimal | check /var/log/cloud-init.log to see what happened? | 16:45 |
| minimal | in there you'll see which DataSources cloud-init considered using and which one it settled on | 16:46 |
| dch | https://gist.githubusercontent.com/dch/a7c0c546fba7f65240fb681c5f1c9f1d for details | 16:51 |
| dch | it just seems to settle on OpenStack with no explanation | 16:51 |
| dch | 2024-07-25 16:48:33,540 - main.py[DEBUG]: No local datasource found | 16:53 |
| minimal | Looking for data source in: ['OpenStack', 'None'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM'] | 16:53 |
| minimal | so it only checks OpenStack and None | 16:53 |
| minimal | that would be baed on any datasources_list definition in /etc/cloud/cloud.cfg or inside /etc/cloud/cloud.cfg.d/ typically | 16:54 |
| dch | there are no datasources_list anywhere | 16:56 |
| dch | is there some code-based default? | 16:56 |
| minimal | yes, lemme find it | 16:56 |
| dch | tyvm | 16:56 |
| dch | datasource_list: [ NoCloud, None ] | 16:56 |
| dch | I should be able to add this to my 00_nocloud.cfg file then.. let's see | 16:57 |
| dch | it still seems to ignore the `network: disabled` line in my 00_nocloud.cfg tho | 16:58 |
| minimal | in /usr/lib/python3.12/site-packages/cloudinit/settings.py (or similar path) | 16:59 |
| minimal | the definition of datasource_list | 16:59 |
| dch | ok its not coming from that, this is the same as the release version | 17:00 |
| minimal | how did you get (or create) the OS you're booting? | 17:01 |
| minimal | ids this a VM using an official freebsd cloud disk image? | 17:01 |
| dch | its a fresh container specifically for troubleshooting ci | 17:01 |
| dch | so yes its 100% "normal" freebsd | 17:01 |
| minimal | container? created how? | 17:02 |
| dch | how well do you know freebsd? if not, just trust me, its vanilla | 17:02 |
| minimal | I'm not even sure if cloud-init on FreeBSD supports containers or not | 17:02 |
| minimal | thge "resident" cloud-init FreeBSD isn't in this channel currently | 17:03 |
| minimal | s/FreeBSD/FreeBSD person/ | 17:03 |
| dch | I know them well, they're taking a well deserved break | 17:03 |
| dch | zfs clone zroot/jails/templates/14.1-RELEASE-amd64-amd64@ready zroot/jails/instances/14.1-RELEASE-amd64-amd64/clowninit | 17:03 |
| minimal | basically I'd expect the default list of DataSources to be used unless it is being overridden inside /etc/cloud/ | 17:04 |
| dch | yes thats what I see here too | 17:04 |
| dch | I don't think the /usr/local/etc/cloud/cloud.cfg.d/ files are being processed properly | 17:05 |
| dch | I can see they're read in the log | 17:05 |
| dch | 2024-07-25 16:58:17,782 - util.py[DEBUG]: Reading from /usr/local/etc/cloud/cloud.cfg.d/00_nocloud.cfg (quiet=False) | 17:05 |
| dch | but somehow that data no longer ends up in the final merged config | 17:06 |
| minimal | oh hang on, does cloud-init on FreeBSD use ds-identify? | 17:08 |
| dch | yes, but we fixed that running in ~ 2023 IIRC | 17:08 |
| minimal | if it does then it's likely an issue in there | 17:08 |
| minimal | cloud-init only decides a list of DSes (whether default or otherwise) if ds-identify doesn't provide it with a list | 17:09 |
| dch | https://www.irccloud.com/pastebin/EznjjOis/ds-identify.log | 17:10 |
| dch | ok I found ds-identify.log now | 17:10 |
| dch | NoCloud is in the DSLIST and in the defaults | 17:10 |
| dch | so does ds_identify not handle files in /usr/local/etc/cloud/cloud.cfg.d/*.cfg perhaps? | 17:11 |
| minimal | look at the last couple of lines | 17:11 |
| dch | another yaml-in-shell-script footgun | 17:11 |
| minimal | AFAIK /etc/cloud/ is only used by cloud-init "proper" | 17:12 |
| minimal | 2nd last line: "1 datasources returned maybe: OpenStack" | 17:12 |
| dch | so, moving the 00_nocloud.cfg contents into cloud.cfg *directly* now works | 17:13 |
| dch | DSLIST=NoCloud None | 17:13 |
| dch | dsidentify should never have been written in shell | 17:14 |
| minimal | from a quick look at ds-identify I see it can detect FreeBSD jails | 17:15 |
| dch | yes cloudinit has nothing to do with jails really | 17:15 |
| minimal | but I don't see any indication of which datasource will be selected when in a jail | 17:15 |
| dch | its simply broken if there is a merged config in cloud.cfg.d/* | 17:15 |
| dch | would that work on linux either? | 17:16 |
| dch | we set PATH_ETC_CLOUD but not PATH_ETC_CI_CFG_D= | 17:17 |
| minimal | would what work? I use multiple files in /etc/cloud/cloud.cfg.d/ on Linux all the time and it works | 17:17 |
| dch | exactly that | 17:17 |
| minimal | e.g. when I create disk images for specfic DSes I have a file in /etc/cloud/cloud.cfg.d/ to override the datasources_list | 17:18 |
| dch | and on FreeBSD this directory should point to /usr/local/etc/cloud.cfg.d/ instead | 17:18 |
| dch | which is why this isn't being picked up | 17:18 |
| minimal | I think the issue is that ds-identify when it encounters a jail doesn't decide what datasource(s) should apply for the jail | 17:19 |
| dch | no it has nothing to do with a jail | 17:19 |
| dch | its simply looking at /etc/cloud/cloud.cfg.d instead of /usr/local/etc/cloud/cloud.cfg.d | 17:19 |
| minimal | ds-identify doesn't look there at all AFAIK | 17:19 |
| minimal | if you look at dscheck_NoCloud in ds-identify it checks DMI serial for "ds=nocloud=" | 17:20 |
| minimal | it also checks for seed dir etc | 17:21 |
| dch | if I move /usr/local/etc/cloud/cloud.cfg.d in /etc/cloud/ this works | 17:21 |
| dch | so the issue is earlier - its not identifying the seed dir correctly because its not reading the right config files | 17:21 |
| minimal | it works because you've enable cloud-init itself to use the override you provide? that's a separate issue from whether ds-identify first doesn't detect NoCloud | 17:22 |
| dch | bbl, gotta feed the fam | 17:22 |
| minimal | there are 2 distinct actions here | 17:22 |
| dch | yes dsidentify should run first. the shell script is looking in the wrong place. if I use the linux dir, this part succeeds. | 17:23 |
| minimal | (1) is ds-identify detecting correctly? (2) is cloud-init reading config files from right place | 17:23 |
| dch | but, then during "normal" cloudinit, this is now broken and doesn't read the seed. | 17:23 |
| minimal | so has the ds-identify script in FreeBSD been modified to use the /usr/local/etc/cloud path? | 17:28 |
| minimal | or variables like PATH_ETC_CLOUD, PATH_ETC_CI_CFG and PATH_ETC_CI_CFG_D set before ds-identify is run to override the defaults to /usr/local/... | 17:30 |
| dch | back | 17:56 |
| dch | more the latter | 17:56 |
| dch | currently just these 3 variations, mina has upstreamed the rest already | 17:57 |
| dch | https://cgit.freebsd.org/ports/tree/net/cloud-init/files | 17:57 |
| dch | these small remaining patches allow people to relocate all of /usr/local to whereever they fancy like /opt/mystuff or whatever | 17:58 |
| dch | so not really relevant | 17:58 |
| minimal | I'm confused as to what the issue is - if FreeBSD is using /usr/local/etc/cloud prefix instead of /etc/cloud then shouldnt' the FreeBSD "package" of cloud-init change all references accordingly? | 18:15 |
| === arif-ali_ is now known as arif-ali | ||
| dch | TLDR yes. but these settings are encoded in cloudinit itself, this is not the problem. | 21:23 |
| dch | after a bit more refining, the issue is that: | 21:23 |
| dch | dsidentify can parse `datasource_list: ['k', v']` but not the same thing when it is spread over more than 1 line, which is both valid yaml, and canonical (ie formatters will print it that way) | 21:24 |
| dch | and thats what is causing /etc/cloud.cfg.d/00_nocloud.cfg to not be read | 21:25 |
| dch | and probably other things too, but this has lost me from 10:00 to 21:30 today with a couple hours off for food/family | 21:26 |
| dch | and similar lost time last year when dsidentify was introduced | 21:26 |
| dch | maybe dsidentify could just borrow python to handle the yaml correctly? shell is not our friend here | 21:27 |
| minimal | dch: oh "that", I thought that was fixed in ds-identify some time ago | 22:02 |
| dch | no, just finished checking in 24.2 and its still problematic | 22:07 |
| minimal | dch: you missing the comment in /etc/cloud/cloud.cfg about using a single line? lol | 22:30 |
| dch | yes I made the mistake of just reading the docs, and not modifying the main file | 22:31 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!