| === cpaelzer_ is now known as cpaelzer | ||
| === janitha75 is now known as janitha7 | ||
| === jrm2 is now known as jrm | ||
| blackboxsw | meena: sorry for the question out of left field. Do you know if kenv kernel env var names are synonymous across freebsd, openbsd, netbsd, dragonflybsd etc? I ask because I'm wondering if this check should actually be is_BSD() https://github.com/canonical/cloud-init/blob/main/cloudinit/dmi.py#L161 | 21:27 |
|---|---|---|
| dch | I'm trying unsuccessfully to get userdata scripts to execute. | 22:36 |
| dch | - `cloud-init query --all | jq .userdata` returns "#cloud-config\nruncmd:\n ...' as expected | 22:37 |
| dch | - passing a shell script also produces the expected result `loud-init query --all | jq .userdata "#!/bin/sh\...` | 22:38 |
| dch | but logs and filesystem don't show these as executing | 22:38 |
| dch | there is no `allow_userdata: false` key, and AFAICT I'm following https://cloudinit.readthedocs.io/en/latest/topics/format.html#user-data-script | 22:45 |
| === SirScott6 is now known as SirScott | ||
| === kenyon_ is now known as kenyon | ||
| falcojr | dch: can you provide a pastebin of /var/log/cloud-init.log and/or your userdata? | 23:11 |
| dch | hmmm `cloud-init status` returns "running" but no processes remain | 23:11 |
| blackboxsw | systemctl list-units --type=service --state=active # probably will also show you someting is hung or incomplete etc. | 23:12 |
| dch | this is FreeBSD 13.1-RELEASE amd64 with py39-cloud-init-22.3.4 btw | 23:13 |
| dch | I don't have a cloudinit.log for some reason, but here is the various userdata etc | 23:14 |
| dch | userdata https://git.sr.ht/~dch/cloud-confused/tree/main/item/var/lib/cloud/instances/5f05e50c-79bc-4d26-841c-4176dd41566d/user-data.txt | 23:14 |
| dch | falcojr: btw this repo is the snapshot of /var/lib/cloud and the config /usr/local/etc/cloud/ off the system post-cloudinit running | 23:15 |
| blackboxsw | hrm, ok BSD. `sudo cloud-init schema --system` should check that user-data provided is right format and header | 23:15 |
| dch | status.json https://git.sr.ht/~dch/cloud-confused/tree/main/item/var/lib/cloud/data/status.json shows no errors AFAICT | 23:15 |
| blackboxsw | ahh sorry cloud-init schema --system is only for #cloud-config specifically :/ | 23:16 |
| blackboxsw | script looks fine to me | 23:16 |
| dch | # cloud-init schema --system Error: Cloud config schema errors: format-l1.c1: File None needs to begin with "#cloud-config" | 23:16 |
| dch | "computer says No" | 23:16 |
| blackboxsw | heh | 23:16 |
| blackboxsw | too true now that I know your user-data isn't #cloud-config different set of tools :/ | 23:17 |
| dch | let me restart one and get the console log, it has all the cloudinit logs that we expect to see | 23:17 |
| blackboxsw | your https://git.sr.ht/~dch/cloud-confused/tree/main/item/var/lib/cloud/data/status.json also shows that none of the other boot stages ran | 23:17 |
| dch | interesting. I'm not yet familiar enough with all these files to know what I should expect | 23:18 |
| dch | I'll stick with a basic shell script atm, in case #cloud-config has bsd-related bugs | 23:19 |
| blackboxsw | I'd expect to see both a start and finished time for modules-config, modules-final and init-local. So those separate rc.d services are not being started in your image for some reason | 23:19 |
| blackboxsw | it looks like the only boot stage that ran was 'init' stage in this image in order to discover the Ec2 datasource. modules-(config|final) is where all the user-data is acted upon | 23:19 |
| blackboxsw | per https://canonical-cloud-init.readthedocs-hosted.com/en/latest/topics/boot.html | 23:20 |
| blackboxsw | typically these stages are skipped in Linux if there is a system ordering dependency cycle that forces cloud-init's separate systemd init scripts from being run in the current boot target. | 23:21 |
| dch | ok so once this next fresh boot finishes, I will re-run cloudinit locally without rebooting, this should be a bit easier to deal with | 23:22 |
| dch | blackboxsw: btw I can't read that rtd-hosted URL, it requires GH auth and then 404 | 23:22 |
| dch | https://cloudinit.readthedocs.io/en/latest/topics/boot.html is presumably close enough? | 23:23 |
| blackboxsw | oops. my bad, we just started setting up a paid offering without RTD adds and not up for public consumption yet. It is exactly the same, sorry about mis-link there | 23:25 |
| blackboxsw | both are a direct mirror of each other. | 23:25 |
| blackboxsw | yes dch, there should be four rc.d scripts to get cloud-init functioning properly (and those init scripts should run the following in order | 23:27 |
| blackboxsw | sudo cloud-init init --local | 23:27 |
| blackboxsw | sudo cloud-init init | 23:27 |
| dch | this is annoying, it was working a few months ago, and our new release broke something... hope to track bug down | 23:27 |
| blackboxsw | sudo cloud-init modules --mode=config | 23:27 |
| blackboxsw | sudo cloud-init modules --mode=final | 23:27 |
| dch | ok, now with logs! https://git.sr.ht/~dch/cloud-confused/tree/main/item/console.log | 23:28 |
| dch | https://git.sr.ht/~dch/cloud-confused/tree/main/item/console.log#L630 | 23:28 |
| dch | L764 we got metadata ok | 23:28 |
| dch | L816 we skip network deliberately | 23:29 |
| blackboxsw | what I see in that boot log is https://git.sr.ht/~dch/cloud-confused/tree/main/item/console.log#L631 only the 'init' stage was run, no 'init-local' 'modules:config' or 'modules:final' stage log header when starting each boot stage | 23:30 |
| blackboxsw | +1 on proper meta-data detected for Ec2 datasource discovery in L 764, yeah just nothing seems to have called `sudo cloud-init modules --mode=config` or --mode-final for the final two boot stages | 23:31 |
| blackboxsw | +1 on proper meta-data detected for Ec2 datasource discovery in L 764, yeah just nothing seems to have called `sudo cloud-init modules --mode=config` or `--mode-final` `for the final two boot stages | 23:32 |
| dch | hmm | 23:33 |
| dch | "normally" on FreeBSD we just use `cloudinit_enable=YES` and that's all that's required to run the full gamut | 23:33 |
| blackboxsw | Here are our systemd services templates for modules:config stage[1] and modules:final (each is separate in Linux) | 23:34 |
| blackboxsw | 1. https://github.com/canonical/cloud-init/blob/main/systemd/cloud-config.service.tmpl | 23:34 |
| blackboxsw | 2. https://github.com/canonical/cloud-init/blob/main/systemd/cloud-final.service.tmpl | 23:34 |
| dch | blackboxsw: thanks for your diag help here, is there a log of a working one you can point me towards? | 23:34 |
| blackboxsw | I'd expect something in rc.d for each of those | 23:34 |
| dch | doesn't have to be FreeBSD, I just need to see what line we're missing | 23:34 |
| blackboxsw | sure. and cloud-init analyze show typically organizing this output based on boot stages. one sec | 23:34 |
| dch | on bsd here I have cloudinit, cloudinitlocal cloudfinal, cloudconfig | 23:34 |
| dch | for example, this config is what we deploy in Oracle Cloud. Let me check if that still works... | 23:35 |
| dch | ok, `cloud-init analyze blame|show|dump` all blow up | 23:39 |
| dch | https://www.irccloud.com/pastebin/Zy4Osq8R/cloudinit_analyze_dump | 23:39 |
| dch | presumably this is because there's no log file to digest | 23:43 |
| blackboxsw | sorry crashed my browser: https://paste.opendev.org/show/bQuuSdQMl12zyPz9DQtT/ | 23:44 |
| blackboxsw | on linux: absence of /var/log/cloud-init.log isn't a trace | 23:46 |
| blackboxsw | root@dd-j:~# cloud-init analyze show | 23:46 |
| blackboxsw | Cannot open file /var/log/cloud-init.log | 23:46 |
| dch | ok! | 23:46 |
| dch | we got it | 23:46 |
| blackboxsw | but that could be just a newer commit | 23:46 |
| dch | you are (all) right | 23:47 |
| dch | cloudfinal needs to be run | 23:47 |
| dch | this also causes `cloudinit status` to be "done" | 23:47 |
| blackboxsw | ok so I'm curious when you find out what's off if you can give feedback on which rc.d script wasn't running or why | 23:47 |
| blackboxsw | just for our context | 23:47 |
| dch | yeah I think we can improve the port by adding a short message "enable these things or you will fail" | 23:48 |
| blackboxsw | yeah good note: also `cloud-init status --format json`should give more information about the reasons we are in waiting state (it'll be in cloud-init version 22.4 ) | 23:49 |
| blackboxsw | doesn't help today, but we could raise a bigger warning message if we are blocked on different stages not yet run | 23:50 |
| blackboxsw | https://paste.opendev.org/show/biPxW0DUiq8xbx6lZkIl/ | 23:50 |
| dch | ok now we have this all figured out, I think there are 3 things: | 23:54 |
| dch | 1. the FreeBSD port should be clear that you need to run all the commands now | 23:54 |
| dch | 2. it shouldn't put stuff in /run/cloud-init/ but use /var/run/cloud-init/ like other BSD stuff | 23:55 |
| dch | 3. it should (by default) produce logs. they are useful. | 23:55 |
| dch | 4. the existence of the 4 boot rc.d scripts should be clarified. | 23:55 |
| dch | I think all of these can be addressed downstream | 23:56 |
| dch | 5. I owe you all some sort of beverage. Perhaps I should come to FOSDEM for the first time. | 23:56 |
| blackboxsw | sounds like a great plan, and anything that makes sense upstream we would like to incorporate | 23:56 |
| blackboxsw | Boom FOSDEM'd be nice that'd make two of us for first timers. | 23:57 |
| blackboxsw | -> have to head out for dinner EndofDay. Take care thx for the help on BSD | 23:58 |
| dch | I checked, I copypasted my cloudinit stuff from the FreeBSD openstack deployment. | 23:58 |
| dch | I guess none of these worked fully. | 23:58 |
| dch | although, reading my OCI patch, I definitely tested this and had it working. | 23:59 |
| dch | https://reviews.freebsd.org/D34746 | 23:59 |
| dch | that was ~ a year ago maybe? | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!