[00:47] <greeb> hi all, having difficulty figuring out the cloud-init ecosystem. As far as I can tell, there's a core set of features that the cloud-init utility itself provides, along with metadata that individual cloud providers expose as a datasource to cloud-init. If I were to just download some random cloud image of a Linux-based distribution, where would i begin to dissect what is supported? For example,
[00:47] <greeb> there is a note here: https://cloudinit.readthedocs.io/en/latest/reference/network-config.html that says PhotonOS disabled fallback networking. Where does Photon define that?
[00:47] <greeb> additionally, to figure out what metadata is provided per-cloud provider, would I just consult their docs (assuming the docs actually document this)
[01:18] <minimal> greeb: let's backtrack - what are you trying to achieve?
[01:19] <minimal> you want to create a VM on a particular Cloud Provider?
[01:25] <greeb> I'm running terra(grunt|form) using the libvirt provider. It works well for quickly spinning up local VMs and I am starting to prefer it to vagrant. there is a templatized network configuration that terragrunt consumes. Here's the pastebin: https://pastebin.com/a5RU21A7 . The cloud I'm using is an official Ubuntu xenial image. My question is why does that network drop-in simply work.
[01:25] <minimal> "drop-in"?
[01:26] <minimal> so the cloud disk image (not "cloud") you're using is Ubuntu
[01:27] <greeb> yes I understand the difference
[01:27] <minimal> so it doesn't sound like you have any problems
[01:27] <greeb> I don't. I'm trying to figure out how it's working exactly.
[01:27] <minimal> its like using NoCloud Ds with libvirt providing a cidata ISO
[01:28] <greeb> what does the D mean in what you just typed
[01:29] <minimal> DS = DataSource
[01:29] <greeb> okay, I thought you mean Ds and in plural D
[01:29] <greeb> meant*
[01:29] <minimal> did you try searching online? one of the 1st links I found explains how libvirt works with cloud-init
[01:32] <minimal> and assuming you're using this, https://registry.terraform.io/providers/DevScrewOps/libvirt/latest/docs/resources/cloudinit with Terraform its description mentions it creates a cidata ISO
[01:35] <minimal> but none of this relates to what you were initially talking about
[01:36] <greeb> I'm using this provider: https://github.com/dmacvicar/terraform-provider-libvirt . Let's go back to the original confusion regarding network configuration. See here: https://github.com/dmacvicar/terraform-provider-libvirt/blob/main/examples/v0.13/ubuntu/network_config.cfg I just copied this on faith and it works. Which is fine, but I want to understand why. Is 'ens3' baked into the image? How
[01:36] <greeb> can I check that myself?
[01:36] <minimal> ens is a network interface, the naming of which depends on the distro you are using
[01:37] <minimal> that's nothing specific to cloud-init
[01:37] <greeb> yes I know
[01:37] <greeb> I think I know where I'm not being clear, sorry.
[01:37] <greeb> one sec
[01:39] <greeb> So let's say I roll my own image. Do I bake-in the network interface and put in documentation "use this interface in your cloud-init configurations" or do I leave it up to cloud-init (the documentation for which suggests some link detection magic) or do I do something else? I'm thinking from kind of a managerial perspective, sorry for being unclear earlier.
[01:40] <minimal> no you don't "bake in" the network interface, when any machine boots it will "see" whatever network interfaces are present, again that's nothing specific to cloud-init
[01:40] <minimal> how such network interfaces are named is down ot the particular distro/OS config you are using
[01:41] <minimal> i.e. old style "eth0, eth0" versus newer style scheme
[01:41] <minimal> s/eth0, eth0/eth0, eth1/
[01:43] <minimal> also are you asked specifically about "local" VM hypervisor environments only, not including Cloud Providers?
[01:44] <greeb> yeah I know there's old style vs predictable naming scheme new style. In the example above, how is it guaranteed that ens3 is always going to be there on that xenial image? 
[01:44] <minimal> a interface isn't "on" a OS image, it is on a physical/virtual machine that happens to be running the OS image
[01:45] <greeb> let me rephrase
[01:45] <minimal> can you answer my last last question please
[01:45] <greeb> I have no questions about cloud providers specifically no.
[01:45] <minimal> ok, as network config doesn't apply there
[01:46] <minimal> so have a look at the docs regarding the network-config file: https://cloudinit.readthedocs.io/en/latest/reference/network-config.html
[01:46] <minimal> specifically the links to the NoCloud v1 and v2 file formats
[01:48] <minimal> with NoCloud I have my OS configured to use old style names so the 1st (perhaps only) network interface is eth0
[01:48] <greeb> I see. Thank you! this helps
[01:49] <greeb> I should've kept reading this page before asking here.
[01:49] <greeb> so there's no match keys in the network interfaces I'm using, which makes it seem like magic. So I'm assuming there is something in the xenial image forcing ens3 onto a deteced network interface.
[01:50] <greeb> s/interfaces/configuration/g
[01:51] <minimal> or perhaps it is using the "fallback" of DHCP on the 1st present interface
[01:51] <greeb> I think that's what's happening because no amount of grep'ing or find'ing is turning up the interface configuration on the VM post-boot.
[01:52] <minimal> eh? cloud-init will be setting the config appropriately
[01:52] <minimal> as it's Ubuntu it should be setting the netplan config file
[01:52] <greeb> disregard, nevermind haha
[01:53] <greeb> anyway, thanks for you time. I should focus on this NoCloud type of config.
[01:53] <greeb> s/you/your lol
[01:53] <minimal> cloud-init "renders" any network to the appropriate form for the distro (/etc/network/interfaces, netplan, sysconfig, etc)
[01:53] <greeb> yeah, I realized the mistake of my last message wrt to that
[01:54] <minimal> well you said you're not using Cloud Providers so yes, only NoCloud, is appropriate
[01:54] <minimal> if you enable debugging for the cloud-init logfile you'd see what exactly it is doing
[01:54] <greeb> thanks, I'll try that out
[10:47] <meena> oops https://github.com/NetworkConfiguration/dhcpcd/issues/195
[10:47] -ubottu:#cloud-init- Issue 195 in NetworkConfiguration/dhcpcd "dhcpcd bypasses firewall" [Open]
[15:42] <holmanb> @eb3095-Vultr: I have a question for you regarding __main__ in DataSourceVultr.py if you get a chance. See -> https://github.com/canonical/cloud-init/pull/2122/files#r1166094268
[15:47] <minimal> holmanb: just starting to look at your dhcp changes
[15:48] <minimal> I guess fixing the dhcp stuff in NoCloud (for seed url access) would come via a different PR
[15:57] <meena> holmanb: any
[15:57] <meena> …
[15:58] <meena> do you want me to do a pr with the FreeBSD changes now, against your pr, or after it lands?
[15:59] <holmanb> meena: after - just wanted your awareness for now
[16:00] <meena> aye
[16:01] <holmanb> minimal: what needs to get fixed in nocloud for seed url?
[16:01] <holmanb> minimal: not sure I follow
[16:03] <minimal> holmanb: I mentioned previously that it is using the "fallback" method of adding an entry in /e/n/i and the relying on OS to bring up interface so that c-i can then fetch URLs to then read network config from meta-data to then rewrite /e/n/i
[16:03] <holmanb> ahhh right that part
[16:03] <minimal> but the 2nd (proper) network config is never activated as the interface is already up
[16:03] <minimal> basically NoCloud in general uses fallback in no CIDATA found, which is fine
[16:04] <minimal> however for seed URL use fallback IMHO should *NOT* be used
[16:04] <minimal> rather the interface should be upped and dhcp client run, and then after URLs fetched it 
[16:04] <minimal> addr should be deleted
[16:05] <holmanb> Yep, I remember now. I'd like that issue to be a separate PR. 
[16:05] <holmanb> It probably deserves a LP bug.
[16:06] <minimal> ok, guess I'll raise an LP bug then, I don't currently understand the relevant parts of the code to write a PR yet
[16:07] <holmanb> appreciated, thanks
[16:07] <minimal> BTW with your current WIP I guess the idea of doing DHCP all in python is no longer being considered
[16:10] <holmanb> it's certainly still possible (easier, even with the refactor), but not the current plan
[16:11] <minimal> ok
[16:14] <holmanb> There was some internal hesitation around that approach with maintaining a custom client for only cloud-init, and dhcpcd is required anyways in images for initramfs, so the dhcpcd approach is seen as lowest risk/difficulty and doesn't require maintaining / adding new dependencies that aren't already needed in base images.
[16:16] <holmanb> Like I said, that could certainly change, and this PR doesn't lock us into that approach.
[16:17] <holmanb> Also I see someone just offered a proposal for udhcpc
[16:47] <eb3095-Vultr> I got that going to look when I get home in a few
[16:53] <minimal> holmanb: udhcpc "suffers" from the same limitation as dhclient, 1 process cannot do both IPv4 and IPv6
[16:55] <minimal> but yes it is good to see udhcpc interest, I'm interested in it as it is the default dhcp client for Alpine
[16:58] <meena> honestly, can't wait for dhcpcd in FreeBSD
[18:56] <holmanb> eb3095-Vultr: much appreciated, thanks
[18:58] <eb3095-Vultr> np!
[22:20] <Lvl4Sword> What do you use to verify your cloud-init isn't typoed?