=== vrubiolo1 is now known as vrubiolo
[15:07] <Odd_Bloke> split-brain: cloud-init will only configure network interfaces on first boot, might that explain what you're seeing?
[15:07] <Odd_Bloke> (Upgrading cloud-init on an already-booted system will not cause any new configuration to happen.)
[15:07] <split-brain> yes i see
[15:08] <split-brain> will this help to recreate the config?https://github.com/canonical/cloud-init/blob/master/doc/rtd/topics/faq.rst#how-can-i-re-run-datasource-detection-and-cloud-init
[15:09] <Odd_Bloke> split-brain: That will also perform other configuration which you probably don't want to do to a running system.
[15:09] <split-brain> ok
[15:11] <Odd_Bloke> We have network interface hotplug on our roadmap for the 21.04 cycle: that should enable the plugging in of network interfaces after boot, and having them configured as expected.
[15:12] <Odd_Bloke> But that's still a ways away from landing in trunk, nevermind being available in your distro package.
[15:13] <split-brain> yes i understand
[15:14] <split-brain> so what do you suggest to do in order to properly configure the secondary ipv4s? using the official centos 8 ami
[15:29] <Odd_Bloke> split-brain: There isn't a great way for us to do it via cloud-init, unfortunately.  If you run `python3 -c 'from cloudinit.stages import _pkl_load; import yaml; ds = _pkl_load("/var/lib/cloud/instance/obj.pkl"); print(yaml.dump(ds.network_config))'` then you can see cloud-init's internal representation of the network metadata it fetched from the IMDS.  Are your secondary interfaces in there?
[15:32] <split-brain> i'm using just the primary network interface with one secondary private ipv4
[15:32] <split-brain> yes i can see the secondary ip listed there
[15:37] <minimal> Odd_Bloke: not to distract from the current conversation too much, I started looking at AWS's ec2-net-utils this week (which is what Amazon Linux uses for dynamic address handling) and was wondering how to get it to interwork with cloud-init
[15:38] <minimal> clarify: to see how to get it to work on Alpine and Debian with cloud-init
[15:39] <split-brain> yes this package looks promising. there are even ports to distributions other than amazon linux
[15:40] <minimal> split-brain: yes I found a couple of Github repos for these ports
[15:43] <split-brain> minimal i'll try it and see if i'll get any luck with it
[15:45] <minimal> I'm assuming on Amazon Linux they've sorted out the interaction with cloud-init. Have been meaning to download their ISO to have a look to figure it out. I assume they're using cloud-init to handle the primary IP on the 1st interface and ec2ifscan for secondary IPs on the primary interface and for all other interfaces
[15:53] <Odd_Bloke> minimal: Interesting, I hadn't come across that before; cloud-init supports configuring non-primary interfaces already (but it only performs network configuration at first boot).  The hotplug support I mentioned would extend that to support configuring interfaces that appear after boot.
[15:54] <minimal> Odd_Bloke: yeah that's what ec2-net-utils does - but obviously its only intended for AWS
[15:55] <minimal> https://github.com/aws/amazon-ec2-net-utils.git
[15:55] <Odd_Bloke> split-brain: So if you want to see how cloud-init would render that config, `python3 -c 'from cloudinit.stages import _pkl_load; import yaml; ds = _pkl_load("/var/lib/cloud/instance/obj.pkl"); print(yaml.dump(ds.network_config))' > network_config.yaml` and then `cloud-init devel net-convert -p network_config.yaml -k yaml -D centos -O sysconfig -d /tmp/out` will perform a render of the network config into
[15:55] <Odd_Bloke> `/tmp/out/etc/...`.
[15:57] <Odd_Bloke> minimal: Right, interesting; is it legitimately only 5mos old, do you know?  Or is that just when this git history started?
[15:58] <minimal> I believe the package name changed, in Amazon Linux v1 it was naming slightly differently (saw an Issue in their GitHub about someone hitting upgrade problems causing by the renaming)
[15:59] <minimal> so its been around for a lot longer apparently. I only became aware of it a couple of months ago
[16:02] <split-brain> they are officially suggesting using ec2-net-utils for secondary ip configuration https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/MultipleIP.html#StepTwoConfigOS
[16:05] <minimal> yupe, was just about to post a link to that page
[16:08] <split-brain> Odd_Bloke: yes the script yields the basic ip config, which i can set up manually as well. now i'm curious to know, from what version cloud-init is capable to set up the secondary ipv4s on the first instance boot?
=== vrubiolo1 is now known as vrubiolo
[16:14] <minimal> Odd_Bloke: I think the principle Amazon works on is that any IP on an interface or any interface can change (i.e. be added or removed) *except* for the 1st IP on the 1st interface which cannot be changed unless the VM is destroyed. With that in mind I think they're setting that 1st IP on 1st interface during initial boot, using udev hotplug to spot other interfaces and configuring them as they appear, and am not sure yet how they handle
[16:14] <minimal> secondary addrs on primary interface
[17:38] <meena> reminder that this https://github.com/canonical/cloud-init/pull/655#pullrequestreview-527325337 needs a review from someone who actually knows cloud-init
[17:41] <Odd_Bloke> split-brain: It landed in https://github.com/canonical/cloud-init/commit/6600c642af3817fe5e0170cb7b4eeac4be3c60eb#diff-66c4d0c34e626a49273eda62819c3068f5501520b86f28e89e0f998cf4cf2027, so was included in the 20.2 upstream release.  It's possible that the CentOS packaging reverts that behaviour, however: in the Ubuntu packaging, we don't generate secondary NIC config for pre-focal releases by default
[17:41] <Odd_Bloke> (because that would be a change in default behaviour of existing releases).
[17:44] <Odd_Bloke> split-brain: https://cloudinit.readthedocs.io/en/latest/topics/datasources/ec2.html#configuration is the documentation for the configuration flag (`apply_full_imds_network_config`).  Note, however, that changes to that configuration option would need to be present in the image you launch; by the time you can modify that in a launched instance, network configuration has already happened.
[17:53] <meena> Odd_Bloke, except, accidentally, for FreeBSD, where we had dhcp on all interfaces
[18:08] <minimal> Odd_Bloke: so going forward perhaps a realistic way to handle EC2 would be to set apply_full_imds_network_config to False so only the primary IP on primary interfaces gets set up and then implement ec2-net-tools (or equivalent)? An all static initial config doesn't cater for situations like adding extra interfaces after or adding an elastic IP to an interface after
[18:18] <Odd_Bloke> minimal: The plan is that cloud-init will have generic support for hotplugging of network interfaces across clouds, so users don't have to figure out how to do this on each individual cloud that they work on (and also so each individual cloud doesn't have to write tooling which works across the multiple network backends we already support).
[18:19] <Odd_Bloke> (I think whether or not the full config is applied at first boot is orthogonal to later interface configuration?)
[18:20] <minimal> Odd_Bloke: right, makes sense. got a pointer to any work or design done so far?
[18:22] <Odd_Bloke> rharper did the initial work in https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/356152, I'm not 100% sure if that's the latest iteration of it.
[18:22] <split-brain> Odd_Bloke: thank for the info. as i currently don't want to prepare my images by myself, i'll stick with ec2-net-utils. after installing it and reboot, my secondary private ipv4 did show up.
=== ijohnson is now known as ijohnson|lunch
=== ijohnson|lunch is now known as ijohnson
[20:15] <Odd_Bloke> split-brain: Understood, thanks for letting me know.  Hopefully we'll be improving this for folks soon. :)
[20:19] <split-brain> Odd_Bloke: cheers & all the best!