=== esv_ is now known as esv
holmanbminimal: sorry, missed your comment before. Adding support in eni seems reasonable, at least without digging into details too much. Are you looking for any input/thoughts in particular?17:10
minimalholmanb: I guess I'm wondering about the impact of this on other renderers and why things like IPv6 Privacy Extensions are not currently catered for in network config v217:36
minimalI guess I'm thinking about IPv6 network configurations that are perhaps not typical in cloud environments but may be in VM or physical host environments17:55
axinohey there, cloud-init is giving me https://pastebin.ubuntu.com/p/mNBzMpkRdP/ on a physical machine that's getting commissioned by MAAS. Apparently it's not breaking anything though.21:09
axinohappy to provide debugging output to help understand this21:09
blackboxswaxino: sorry to have missed this. Does `sudo cloud-init  schema --system` present an error due to invalid YAML being provided as user-data?21:41
axinoblackboxsw: hi o/ looking21:41
axinoblackboxsw: Cloud config schema errors: format-l1.c1: File None needs to begin with "#cloud-config"21:45
axinoblackboxsw: that's all this command gives me21:45
blackboxswaxino: meh, ok user-data provided to this instance isn't #cloud-config specifically (could be b64encoded or scripts or something else ) you can see it with: sudo cloud-init query userdata21:52
blackboxswthat said, the traceback you mentioned is making me think either the network-config.yaml provided to cloud-unit on this instance is bogus YAML or user-data.21:53
axinoblackboxsw: yeah it's a shell script with a payload21:54
blackboxsw+1 ok we should have a bug for cloud-niit query being smarter/more informative about non-#cloud-config user-data.21:54
blackboxswthat said, the mac address provided would likely need to be quoted to avoid YAML parse errors for yaml parse errors I think21:55
axinoblackboxsw: so I can extract this, and it gives me a curtin directory with subdirs21:57
axinoblackboxsw: and the MAC address generating the messages appears only in curtin/configs/config-003.cfg21:58
axinoblackboxsw: which looks like a netplan config with a "network_commands" block appended21:58
axinoblackboxsw: the error appears 3 times, and the MAC address as well, so it looks like a good candidate22:00
blackboxswaxino: is there a /etc/cloud/cloud.cfg.d/50-curtin-networking.cfg  on the system?22:07
blackboxswand is that content shareable(not sensitive)>22:07
blackboxswat least I think that's where curtin via MAAS leaves network config artifacts for cloud-init to apply22:08
blackboxswif network config version provided by MAAS is `version: 2` recent cloud-init should actually pass through that content without trying to load the YAML at all, so I wouldn't have expected that error22:09
axinoblackboxsw: OK I can't answer that now, all the nodes are deployed22:14
axinoblackboxsw: I'll try to circle back22:14
blackboxswaxino: the other error in that log is reminiscent  of the dbus race w/ netplan apply per this bug https://bugs.launchpad.net/cloud-init/+bug/199712422:15
-ubottu:#cloud-init- Launchpad bug 1997124 in cloud-init "Netplan/Systemd/Cloud-init/Dbus Race" [High, In Progress]22:15
blackboxswJames is already toying with systemd dependency changes to avoid the  activators.py[WARNING]: Running ['netplan', 'apply'] resulted in stderr output: Failed to connect system bus: No su22:16
blackboxswch file or directory                                                                                                                       22:16
blackboxswin this PR https://github.com/canonical/cloud-init/pull/193722:16
-ubottu:#cloud-init- Pull 1937 in canonical/cloud-init "Order cloud-init.service after dbus.socket on Ubuntu" [Open]22:16
blackboxswbut I think there is a secondary problem with the YAML processing as you mentioned22:17
waldibah, can't commet. but how does ordering after dbus.socket help`22:18
waldieven if, this should be sockets.target. but dbus.socket does not mean it can run dbus yet22:19
waldiand on debian/ubuntu, there is a cycle between dbus.service and cloud-init if you hold it wrong enough22:21
blackboxswsystemd-networkd talks to dbus I believe and "netplan apply" on ubuntu systems writes out configuration that is ultimately rendered by networkd22:22
waldisystemd-networkd, as all systemd services, can connect to dbus late, so does not depend on dbus22:23
waldias dbus.service depends on sysinit.target, this reorders the whole startup22:25
* blackboxsw starts looking at wonders about cloud-init services only blocking on presence of an active dbus.socket when we happen to need to run `netplan apply` (which happens only for datasources which are only detected after network is already up and have netplan for their network config 'backend').22:29
blackboxswwaldi: we don't want After=dbus.service but were thinking about After=dbus.socket ... trying to see if that dep-chain still pulls us into After=sysinit.target22:30
blackboxswmeh same prob22:33
blackboxswroot@dev-f:~# systemctl show -p Requires,Wants,Before,After dbus.socket22:35
blackboxswRequires=-.mount system.slice sysinit.target22:35
blackboxswBefore=shutdown.target dbus.service systemd-logind.service polkit.service sockets.target accounts-daemon.service22:35
blackboxswAfter=-.mount system.slice sysinit.target22:35
waldiyou have a socket, try to connect to it and systemd schedules a transaction to start the associated service. at this stage it can only kill the connection or wait until the service is up and running. dbus.service depends on sysinit.target22:36
waldithe only thing that fixes all those problems is to move dbus.socket/.service before sysinit.target22:41
waldidebian currently have that problem with firewalld, because it wants to start before network, but after dbus. so sysinit -> dbus -> firewalld -> network -> cloud-init -> sysinit22:42
blackboxswthanks waldi for chiming in. hrm. right, ok it looks like the option of re-ordering dependencies for very early cloud-init boot stages (cloud-init.service or cloud-init-local.sevice) to depend on dbus.socket/service is high risk and impactful to overall general boot times and systemd ordering cycles (I had missed the After-sysinit.target in those cases).  22:49
blackboxswthe alternative we could take in cloud-init for the specific case where the datasource is detected after network is up (in init-network stage via "cloud-init.service") 22:50
blackboxswis to actually just defer the 'netplan apply' call (to apply network configuration) to cloud-config.service22:51
blackboxswwhich is scheduled After=sysinit.target22:52
waldiwhy does netplan apply need dbus?22:52
waldihmpf, even service reload is implemeted via "networkctl reload"22:53
blackboxswI don't think it's netplan per-se, but rather systemd-networkd is complaining about inability to talk to the socket22:54
blackboxswbut I'm digging in both netplan && systemd to double-check that22:55
waldiunlikely. systemd-networkd.service does not depend on dbus.*. this is networkctl22:55
waldianyway, sleep22:56
blackboxswyep https://github.com/canonical/netplan/blob/main/netplan/cli/utils.py#L11622:57
blackboxswthx again. will poke at a more reasonable solution for this.22:58
blackboxswor, simplest interim solution, we could retry netplan apply in cases where dbus.socket isn't up yet and cloud-init hits this specific error condition23:14

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!