[08:48] <acamilleri> Hi, i want to configure a bridge through cloud-init network config on Debian 11, i'm using version 2 of network configuration, in the doc it's said the interface id is opaque when a match operator is used, but in my case, the configuration is used as ENI renderer, so the interface id is used as interface name, i don't understand what cloud-init is
[08:48] <acamilleri> doing, can you help me please ?
[09:17] <meena> acamilleri: can you show us what your config looks like vs what the actual result looks like? 
[09:31] <acamilleri> Yes sure, i'm using this configuration
[09:31] <acamilleri> https://pastebin.com/TCvmrsK7, in /etc/network/interfaces.d/50-cloud-init.cfg file an entry with id0 is created, an entry with vmbr0 is created too
[09:46] <acamilleri> I tested with interfaces: [id0] too
[09:47] <acamilleri> The problem can be reproduce easily with command cloud-init devel net-convert
[09:56] <meena> and what happens in /e/n/i? and what do the logs say?
=== djhankb9 is now known as djhankb
[12:58] <acamilleri> sorry for delay meena ,
[12:58] <acamilleri> you can find config and logs here (i replaced my server): https://pastebin.com/LHEkK5Lr
[13:00] <acamilleri> I add a print to show returned values of iter_interfaces() method from network_state class
[13:00] <acamilleri> {'name': 'eth0', 'type': 'physical', 'mac_address': 'd8:5e:d3:69:d6:90', 'inet': 'inet', 'mode': 'manual', 'mtu': None, 'address': None, 'gateway': None, 'subnets': [{'type': 'dhcp4', 'routes': []}], 'accept-ra': None, 'wakeonlan': None}
[13:00] <acamilleri> {'name': 'id0', 'type': None, 'mac_address': None, 'inet': 'inet', 'mode': 'manual', 'mtu': None, 'address': None, 'gateway': None, 'subnets': [], 'accept-ra': None, 'wakeonlan': None}
[13:00] <acamilleri> {'name': 'vmbr0', 'type': 'bridge', 'mac_address': None, 'inet': 'inet', 'mode': 'manual', 'mtu': None, 'address': None, 'gateway': None, 'subnets': [{'type': 'dhcp4', 'routes': []}], 'accept-ra': None, 'wakeonlan': None, 'bridge_ports': ['id0']}
[13:17] <meena> aciba[m]: https://github.com/canonical/cloud-init/pull/4209
[13:17] -ubottu:#cloud-init- Pull 4209 in canonical/cloud-init "setup: fix generation of init templates" [Open]
[13:46] <minimal> acamilleri: why are you enabling dhcp on BOTH the ethernet interface and the bridge interface on top of it? I'd expect you to only enable dhcp on the bridge interface
[13:48] <aciba[m]> thanks meena for the ping, I will have a look
[13:48] <meena> aciba[m]: pushed one more commit just now
[13:58] <acamilleri> minimal don't know ^^' im trying to remove dhcp on the ethernet interface
[14:05] <minimal> acamilleri: have you tried matching on name ('eth0') rather than mac address to see if that works
[14:32] <acamilleri> minimal no i don't test with name, but with debian it's not eth0, but ensX / enpX, i will testing it
[14:37] <minimal> acamilleri: also does the cloud-init.log have any lines that relating to it trying to match the ethernet interface?
[15:52] <acamilleri> minimal sorry for delay
[15:52] <acamilleri> About testing without dhcp4 on interface : https://pastebin.com/WaE7qmAT
[15:53] <acamilleri> i tried to match with interface name, same :/
[15:59] <minimal> looks like an issue with cloud-init when it internally converts the v2 network format to v2
[15:59] <acamilleri> it seems converting v2 to eni for debian
[15:59] <minimal> 2023-06-28 15:41:01,669 - network_state.py[DEBUG]: v2(ethernets) -> v1(physical):
[15:59] <minimal> {'type': 'physical', 'mac_address': 'd8:5e:d3:69:d6:90', 'name': 'eth0', 'match': {'macaddress': 'd8:5e:d3:69:d6:90'}}
[16:00] <minimal> it converts v2 to v1 BEFORE it selects the "eni" renderer
[16:00] <blackboxsw> falcojr: can you double check my version on https://github.com/canonical/cloud-init/pull/4208 for devel debian/changelog
[16:00] -ubottu:#cloud-init- Pull 4208 in canonical/cloud-init "Ubuntu/devel" [Open]
[16:00] <minimal> but "match" is not defined in the v1 config format
[16:02] <acamilleri> oh yes, i understand
[16:02] <minimal> and then when it converts the v2 bridge portion to v1 it doesn't replace "id0" with "eth0"
[16:03] <minimal> 2023-06-28 15:41:01,672 - network_state.py[DEBUG]: v2(bridge) -> v1(bridge):
[16:03] <minimal> {'type': 'bridge', 'name': 'vmbr0', 'bridge_interfaces': ['id0'], 'params': {}, 'subnets': [{'type': 'dhcp4'}]}
[16:05] <acamilleri> yes, right, maybe i can try to upgrade cloud-init to the latest version and retry, if it's not working it can be a bug ?
[16:05] <minimal> seems like it. You could try replacing all the "id0" references with "eth0" to see if that worksaround the issue
[16:07] <minimal> also the "stp: False" in your v2 config should result in the v1 config that c-i created having "bridge_stp: 'off'" in the "params:" list
[16:08] <minimal> and the "forward-delay: 0" also having "bridge_fd: 0" in the v1 "params:" list
[16:09] <acamilleri> yes, i found the mapping in the code, i will trying to upgrade to the latest version and if not works, i will replace id0 by eth0
[16:09] <acamilleri> maybe i can test a v1 version without match too
[16:09] <acamilleri> but matching can help me, because i have the macaddr instead of the interface name
[16:09] <acamilleri> actually i'm using eth0 because i booted with net.ifnames=0 parameter
[16:10] <acamilleri> but it's need to work without this
[16:11] <minimal> actually, for v2, don't the "stp:" and "forward-delay" options go inside a "parameters:" section?
[16:12] <minimal> so it should be:
[16:12] <minimal>   bridges:
[16:12] <minimal>     vmbr0:
[16:12] <minimal>       interfaces: [id0]
[16:12] <minimal>       dhcp4: true
[16:12] <minimal>       parameters:
[16:12] <minimal>         forward-delay: 0
[16:12] <minimal>         stp: False
[16:16] <minimal> acamilleri: ^^^
[16:16] <acamilleri> oh ok
[16:17] <acamilleri> i don't see the parameters field ^^'
[16:17] <acamilleri> i updated
[16:17] <acamilleri> thanks a lot !
[16:19] <minimal> https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v2.html#bridges
[16:19] <minimal> "parameters:" is mentioned there
[16:20] <minimal> the doc is not clear that the following options should all be inside the parameters: section
[16:21] <acamilleri> yes, not clear, and me not very attentive ^^
[16:30] <minimal> sorinp: going back to your problem, are you expecting that the ethernet interface is already configured in Arch before cloud-init runs? if so, how? do you expect Arch's initramfs to configure it?
[16:57] <minimal> falcojr, holmanb: this line makes no sense if this function is building up phy_cmd to represent a v1 network config equivalent of a v2 config - as v1 does not (from the docs) define "match", "wakeonlan", or "accept-ra" entries
[16:58] <minimal> https://github.com/canonical/cloud-init/blob/main/cloudinit/net/network_state.py#L753
[18:09] <falcojr> minimal: It's not. The network state class uses it's own structure that's supposed to be the intermediary between v1/v2 and the various network renderers
[19:00] <minimal> falcojr: ok, confusing. The log shows acamilleri's v2 ethernet section being allegedly rewritten to v1 physical but this intermediary structure is shown which is a mix of v1/v2
[19:52] <sorinp> minimal: I do not know when cloud-init runs, but I expect that there is a service that runs after the system brings up the network. I saw there are multiple cloud-init services and I hope that if I use network-config=disable, the service that's running early will not error out because it cannot access the datasource.
[19:57] <sorinp> minimal: I'm also preparing a network v2 configuration to bring up the network from the kernel's command line, as a plan b.
[19:59] <blackboxsw> sorinp: minimal we just generated release 23.2.1 with a fix for specifying `ds=nocloud-net;s=http://Some-seed-url` on the kernel cmdline
[20:00] <minimal> sorinp: I was asking that, as you appear to be using PXI/iPXE, whether you are expecting that the network interface setup that the PXE client did would be carried across into the Arch OS "magically"? as you originally had c-i's "network-config" disabled
[20:00] <blackboxsw> I've uploaded it here https://github.com/canonical/cloud-init/releases
[20:00] <blackboxsw> and just finished also publishing that content (plus tip of main) to Ubuntu 23.10 (Mantic) development release as `[ubuntu/mantic-proposed] cloud-init 23.3~1ge5a617fe-0ubuntu1 (Accepted)
[20:00] <blackboxsw> External`
[20:01] <sorinp> blackboxsw: I cannot use it directly, unless I'm building archiso myself and I cannot right now. However, there will be a release on July 1st and they are watching this space.
[20:01] <blackboxsw> caribou: we'll update our downstream SRU for Ubuntu to 23.2.1 and see if we can't get the SRU aging expedited so we can get this out.
[20:02] <blackboxsw> sorinp: +1, just wanted to update on status
[20:02] <sorinp> blackboxsw: Cheers!
[20:13] <sorinp> minimal: I am not sure the net config is carried on magically from the bootloader to the OS. If there is an offline window during the boot process, then the network datasource is not available in that window, so the early cloud-init run will error out. But the late run, if it still runs even if the early one returned an error, than that one should work and configure ssh keys.
[20:15] <sorinp> minimal: I have no prior experience with cloud-init, so I'm learning as I go.
[20:15] <minimal> sorinp: I'm just trying to understand how, and when, you were expecting Arch's networking to be configured, especially as you want to use seed URLs
[20:17] <sorinp> minimal: I am expecting to be configured as it is when cloud-init is not used. If cloud-init does not detect a datasource, it is not used and the system is configuring the network itself.
[20:18] <sorinp> minimal: even if cloud-init is used for some setup, the system is not aware that cloud-init is trying to take control and it configures the network like cloud-init does not exist.
[20:19] <sorinp> minimal: cloud-init is an optional feature of archiso.
[20:37] <sorinp> minimal: The actual problem with cloud-init network is missing mDNS support. I sent a patch to archiso to include mDNS in its network configuration, but if the network is configured by cloud-init, it does not enable mDNS responder. Without cloud-init, archiso is registering as archiso.local on the network, so one does not need to find the IP address on the router/dhcp server.
[20:40] <sorinp> minimal: https://dpaste.org/OAGRN/raw
[20:40] <minimal> sorinp: well if you use Arch without cloud-init then I expect that Arch sets up/uses DHCP during boot to configure network - but with cloud-init in use and trying to use seed URLs with cloud-init then Arch will *not* have set up networking at the point in booting where cloud-init needs network config to retrieve the seed URLs
[20:41] <minimal> why would cloud-init want/need mDNS support?
[20:44] <minimal> if you mean "zero configuration networking" then that is basically what Ephemeral networking is in cloud-init - except it isn't used for the NoCloud-Net stuff, at least not currently
[20:46] <sorinp> minimal: https://dpaste.org/4GJFs/raw This is the configuration when cloud-init configured the network.
[20:47] <sorinp> minimal: I need other machines on the network to be able to find the archiso booted machine by its name.
[20:47] <sorinp> *hostname
[20:47] <sorinp> I will look into Ephemeral network.
[20:50] <sorinp> minimal: Here https://cloudinit.readthedocs.io/en/latest/reference/network-config.html I cannot find the word 'ephemeral'.
[20:51] <sorinp> minimal: I can only find it in relation with filesystems.
[21:01] <minimal> sorinp: it is not a config settings, it is part of the cloud-init code, but as I said it is *not* currently used for nocloud-net
[21:01] <sorinp> minimal: If by "zero configuration networking" you mean apipa, then no. I do not need that.
[21:02] <minimal> cloud-init does not support mDNS
[21:02] <sorinp> minimal: Yes. That is my problem.
[21:03] <sorinp> minimal: That is why I need network-config=disable
[21:04] <sorinp> *network-config=disabled
[21:04] <minimal> that will not solve your problem as with "network-config=disable" then cloud-init in its init-local stage then wouldn't create a ENI file for Arch to then use so that networking is up before cloud-init (in "init" stage) tries to access the seed URLs
[21:06] <sorinp> minimal: I can put a network configuration in the kernel parameters and download the seed and using runcmd do some stuff to enable mDNS.
[21:08] <sorinp> minimal: I don't even think I need to put any netconfig in kernel cmdline, but just take this new version of cloud-init and download the seend, then enable mDNS.
[21:08] <sorinp> *seed
[21:09] <minimal> you will still have a /etc/system/network/10-cloud-init-eth0.network file created I assume
[21:09] <sorinp> minimal: rm -f /etc/systemd/network/10-cloud-init*; systemctl restart systemd-networkd
[21:10] <minimal> also your seed url is using a name rather than an IP address so unless your DHCP server provides DNS settings then that will likely fail host/DNS lookup
[21:12] <sorinp> minimal: I will even go to the length of adding a copy of 10-cloud-init-eth0.network as 0-eth0.network with the MulticastDNS=yes line added and then restart systemd-networkd, if all else more nice actions fail.
[21:12] <minimal> runcmd only runs once-per-instance, not every boot, so on the 2nd boot cloud-init will again set up networking to get to the seed URLs...
[21:13] <sorinp> minimal: every boot is the first boot. archiso is the arch installation medium, it is not the installed system.
[21:15] <minimal> ah ok, you're trying to "misuse" cloud-init as an OS installer (like subquity) rather than as a OS configuration tool for a running OS
[21:15] <sorinp> minimal: all changes cloud-init does are in tmpfs.
[21:16] <sorinp> minimal: It's not only me. It's everyone that is downloading the archiso to use it to install arch.
[21:17] <sorinp> minimal: I'm using cloud-init to configure the installation medium. I will use ansible as the os installer.
[21:17] <minimal> ok, but who is "developing" this particular use of cloud-init by Arch?
[21:17] <sorinp> The archiso developers.
[21:18] <minimal> who are separate from the arch developers? The Arch Wiki page on cloud-init doesn't mention anything about this
[21:18] <sorinp> They included cloud-init as a means to get a ssh key so that the installation env is accessible remotely by ssh.
[21:18] <sorinp> https://wiki.archlinux.org/title/Install_Arch_Linux_via_SSH
[21:19] <sorinp> https://wiki.archlinux.org/title/Install_Arch_Linux_via_SSH#Prepare_cloud-init_configuration_files
[21:21] <minimal> that Wiki page looks fine. It does not mention seed URLs use however
[21:22] <sorinp> minimal: That is my making: I'll be running archiso from a boot server.
[21:22] <minimal> it also mentions that mDNS and DHCPv6 are not enabled - however DHCPv6 could be enabled with a suitable network-config file, they just don't document that
[21:23] <sorinp> https://wiki.archlinux.org/title/Netboot
[21:23] <minimal> sorinp: right, so *you* are the person "developing" the seed usage, not archiso.
[21:24] <minimal> that Netboot page does not mention cloud-init at all
[21:24] <sorinp> I'm developing it and will write an article on their wiki on how to set it up.
[21:25] <minimal> ok, I'd asked you earlier who was developing this and you said the archiso devs
[21:25] <sorinp> https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/330
[21:25] -ubottu:#cloud-init- Merge 330 in archlinux/archiso "Add support for mDNS." [Merged]
[21:25] <sorinp> I understood you asked who is developing the cloud-init support in archiso.
[21:25] <minimal> anyway as I explained yesterday there are some problems with nocloud-net and how it sets up a "temporary" interface in order to retrieve the seed URLs
[21:26] <minimal> as I encountered them some months ago
[21:27] <minimal> so I believe this needs to be resolved for seed-url use to work as expected
[21:29] <sorinp> I think it will work properly in this latest version, but just after the seed files are run, the network can be changed to ad mDNS support.
[21:29] <minimal> nope, that change doesn't stop it rendering OS network config rather than using a Ephemeral/temporary connection
[21:30] <sorinp> Correct.
[21:30] <minimal> which is what is needed for seed support to work correctly
[21:33] <minimal> so someone needs to work on changing nocloud-net to use a ephemeral connection
[23:54] <meena> minimal: how does it do stuff now?
[23:56] <meena> the change shouldn't be too hard, it's literally just a with wrapper, and thanks to the connection check, it shouldn't regress existing setups