| === EugenMayer5968 is now known as EugenMayer596 | ||
| tiaz | Hi. I'm messing around with #include directives. Right now it's pulling a trivial config that just adds a pair of CAs. However, it appears to be stuck in `running`, even though there are no processes named anything like 'cloud' running anymore. | 20:45 |
|---|---|---|
| tiaz | Finding that `cloud-init status` examines /run/cloud-init/status.json, I checked that out ... the errors arrays are empty, and "v1" and "init-local" both have start and finish times, but modules-{init,config,final} all have neither | 20:46 |
| tiaz | er, sorry, the "init" and "init-local" (rather than v1 and init-local) | 20:46 |
| tiaz | anyway, there's also a stage: null key/value. I don't see anything in the logs indicating where it failed or gave up. Can anyone advise as to where I might dig deeper to find out where this broke? | 20:47 |
| tiaz | I really don't want to restart cloud-init through strace ;) | 20:47 |
| blackboxsw | tiaz: what distribution are you on? That makes me think something in your image has potentially ejected cloud-init.service, cloud-config.service and cloud-final.service from the systemd boot goals due to systemd ordering cycles/incompatibilities with cloud-init's other boot stages. Each bootstage of cloud-init is a separate systemd unit that could be dropped if the ordering of services at initial boot is unable to resolve. | 20:51 |
| tiaz | blackboxsw: I'm running ubuntu:focal through lxc almost exactly like the quickstart/demo recommends | 20:52 |
| tiaz | except instead of #cloud-config/runcmd: it's #include/someurl. logs show it pulled someurl successfully, and it applied the CAs I expect | 20:52 |
| tiaz | cloud-init is active/exited, -config is inactive/dead, -final is inactive/dead | 20:53 |
| falcojr | tiaz: any problems from "systemctl list-units | grep cloud" or "systemctl list-units --failed"? | 20:53 |
| tiaz | there are failed units: e2scrub_reap, systemd-hostnamed, systemd-remount-fs. grep cloud w/o failed shows active/active for cloud-config.target, then as described (init active/exited, init-local active/exited, config,final inactive/dead, none failed) | 20:55 |
| tiaz | -and cloud-init.target inactive/dead | 20:56 |
| tiaz | oh, I have an idea, just a moment | 20:56 |
| tiaz | ah, no joy (thought it might be remove_defaults:true to make it easier to see it'd been applied, but I started it without that and no change) | 20:58 |
| tiaz | https://pastebin.com/T5UawKKa is the cloud-config that's being #include-d (censored, ofc, the certs are valid in the real one). as you can see it's pretty trivial | 21:00 |
| blackboxsw | tiaz also cloud-init analyze show can quickly confirm whether cloud-init boot stages got skipped by showing you what boot stages were actually run on the latest boot | 21:00 |
| tiaz | blackboxsw: https://pastebin.com/ayBGrffe doesn't seem to say anything about skipped, are there other values I should expect in this output? | 21:02 |
| blackboxsw | alrighty, well that confirms what your results.json file states. no modules-config or modules-final stage. in latest boot. What does `systemctl status cloud-config.service` and `systemctl status cloud-final.service` say? also journalctl -b 0 | grep "ordering cycle"` to ensure those systemd units are active/health. | 21:21 |
| tiaz | blackboxsw: both inactive/dead but enabled: https://pastebin.com/tZ05CEwG , no joy on journalctl | 21:58 |
| tiaz | truncated the next shell line by accident. 'ordering cycle' (insensitive) does not occur in journalctl output | 22:00 |
| === sam_ is now known as sam | ||
| === sam is now known as sam_ | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!