[14:06] <akutz> Sorry I missed the weekly call. I intended to attend, but my VP and a Sr. Staff Engineer were visiting Austin from PA, and I went to hang out with them for a few hours. I've been remote since I joined VMware, and I still have not met most people in person :)
[14:32] <falcojr> what call are you referring to? We usually do a weekly IRC "office hours" on Tuesday, but I unfortunately forgot to start it yesterday so you didn't miss anything
[14:37] <akutz> The one from the channel topic.
[14:37] <akutz> I did notice the lack of an entry for 8/17 at https://cloud-init.github.io :) 
[18:50] <akutz> I noticed VMware isn't listed in "90_dpkg.cfg" by default. Should it be? Did I not add something in the PR? This is on a Focal system where cloud-init was upgraded after install via PPA, so maybe the file isn't updated?
[19:12] <rharper> the 90_dpkg.cfg is modified on package install, but the list of datasources has to be updated in the packaging of the deb as well in the debian/ directory 
[19:12] <rharper> dpkg-reconfigure cloud-init also runs that code; but when we add a datasource, there are two steps, 1) adding it to cloudinit proper 2) updating ubuntu/devel packaging branch to include the new datasource in the list 
[19:12] <rharper> akutz: ^
[19:14] <akutz> Thanks! Should I open a PR to do update the ubuntu/devel packaging branch to include the new DS in the list? Or is that something the maintainers do? I'm not exactly sure how to add it, but I'm happy to figure it out.
[19:14] <rharper> let me look, there's likely a few commits in that branch that show adding a recent data source
[19:15] <rharper> https://github.com/canonical/cloud-init/commit/868105e9b862e2f3b567fd2805d6ddc7d91328b1 
[19:15] <ubottu> Commit 868105e in canonical/cloud-init "Add VMware to default datasource_list (#967)"
[19:16] <rharper> looks like it' already landed, falcojr handled it, so the question is where did your package get built and was it built against ubuntu/devel ?   you should be able to test out the daily-ppa for  i-series 
[19:22] <rharper> akutz: it's present on impish ubuntu images, 
[19:22] <rharper> # lsb_release -rd
[19:22] <rharper> Description:	Ubuntu Impish Indri (development branch)
[19:22] <rharper> Release:	21.10
[19:22] <rharper> root@i1:/etc/cloud/cloud.cfg.d# grep VMware /etc/cloud/cloud.cfg.d/90_dpkg.cfg 
[19:22] <rharper> datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, VMware, Vultr, None ]
[19:24] <rharper> that's landed just last week, so the next cloud-init SRU will bring that back to previous releases;  in the mean time, if you want to test it out on older releases, you'll need to use cloud-init daily ppa, which builds master for previous releases, https://code.launchpad.net/~cloud-init-dev/+archive/ubuntu/daily 
[19:24] <akutz> I updated a focal system with the new cloud-init via the aforementioned PPA link. I guess that doesn't cause the file in question to be updated without re-running dpkg-reconfigure.
[19:24] <akutz> Yes, I did all of that.
[19:24] <akutz> >  I noticed VMware isn't listed in "90_dpkg.cfg" by default. Should it be? Did I not add something in the PR? This is on a Focal system where cloud-init was upgraded after install via PPA, so maybe the file isn't updated?
[19:25] <akutz> I think we're saying the same thing. I think you missed where I said how I updated an existing system. It seems like that just doesn't cause the file to be updated automatically :)
[19:25] <rharper> I didn't see that, right;  let me check the focal package then
[19:25] <akutz> No worries. So it seems everything is working as expected. Thank you for your help rharper!
[19:26] <falcojr> we updated the datasource list in focal too. akutz: are you saying you see it now in focal? Or it's still not there?
[19:27] <rharper> one needs to reconfigure; I need to re-read the post-inst scripts to recall why that's the case; 
[19:28] <akutz> I installed 20.04.3, apt-get update, apt-get upgrade, install ppa, apt-get update, apt-get upgrade (at this point cloud-init was updated to the latest bits). From there I tried to test things, but that's when I realized the VMware DS wasn't in the 90_dpkg.cfg file. It wasn't until I ran dpkg-reconfigure that it showed up.
[19:29] <falcojr> gotcha, yeah for new launches you'll see it there
[19:30] <rharper> I launched daily focal container, it's not present (as expected I believe) add the ppa, install cloud-init focal from ppa; and the datasource list isn't updated unless you reconfigure;  I *though* it would get updated but I'm foggy on the details;  looking at the scripts now
[19:31] <falcojr> I'm guessing it's because some people will change the datasource lists to only show their datasource, and we don't want to mess with that on upgrade
[19:32] <akutz> You should add a line that says "DO NOT MODIFY -- create 99_ds-list.cfg to override" or something
[19:33] <akutz> Not that it would prevent people from modifying it, but it's not called "90_ds-list.cfg" on purpose I imagine. 
[19:33] <falcojr> I mean...how many people are going to want to change the datasource they're running on?
[19:33] <akutz> Just doc that this is auto-generated and if a customer wants to be explicit, use a higher level file to set an override
[19:34] <akutz> I don't know. I was responding to your comment about upgrades
[19:35] <rharper> heh, we do have: # to update this file, run dpkg-reconfigure cloud-init  
[19:36] <rharper> so yeah, we don't add new datasources on upgrade to the ds_list;  that does mean (on purpose) that we won't ever switch to a new datasource on upgrade;  so, akutz in your case, if you want to test it on an instance, you can upgrade, dpkg-reconfigure cloud-init (select vmware) and then cloud-init clean --logs --reboot;
[19:37] <rharper> that'll run as firstboot but now with VMware enabled 
[19:37] <akutz> That's what I did to test it :)
[19:37] <akutz> I didn't say that I didn't get it working. I was only wondering if it was supposed to be part of that list automatically after a ppa upgrade.
[19:37] <rharper> perfecto 
[19:37] <akutz> Y'all answered my question.
[19:37] <rharper> akutz: thanks for catching me up =P  
[19:39] <akutz> And yes, I know you have that comment in the file, but the point is more that a comment should be there that says "do not modify this file directly if you want to restrict datasources" and instead tell people to create a higher-level file. At least if you're concerned about upgrades. But then again, I guess that is what dpkg-reconfigure does already, and the reason you don't do that on ppa upgrade is because you don't want to overwrite it. 
[19:40] <rharper> yeah
[20:33] <akutz> Thank you again for all your time and help rharper!
[20:34] <rharper> akutz: sure, thanks for the contributions! 
[20:35] <akutz> Should we consider adding a call to the same code that occurs as part of "dpkg-reconfigure cloud-init" to "cloud-init clean"? Since the latter makes cloud-init operate in first-boot mode again, should the clean op also update the DS list?
[20:36] <akutz> Or is that still running afoul of a potential customer customization?
[20:43] <akutz> Actually, it doesn't make sense as part of cloud-init clean. Now, if we introduced "cloud-init clobber" as a meatier version of clean... :) 
[20:44] <rharper> I think it's extremely rare in typical cloud instances to switch datasources ... there is one example in IBMCloud in the tree;  where they've done a multi-stage bootstrap from one datasource to the next ... so if you wanted to switch OVF users to VMWare, then I think we'd want to handle that within OVF datasource, detecting that VMWare datasource is present and sorting out what/if/how to migrate that to a VMWare datasource; 
[20:44] <falcojr> lol...yeah, I think it should be considered a separate operation. Is something about the dpkg-reconfigure process making things difficult for you, or are you just thinking about optimal UI?
[20:46] <rharper> I am wondering the user story for migrating the datasource; though I admin that the VMWare cases tend to involve more pets that get "updated" with new configs over the lifetime of the VM vs. a typical cloud instance which is destroyed and recreated ; cloud instances that stick around don't change their datasource (save the IBM cloud bootstrap exception I mentioned); 
[20:47] <akutz> Yeah, this is likely a non-issue that I only encountered due to the ppa upgrade path
[20:48] <akutz> > I am wondering the user story for migrating the datasource; though I admin that the VMWare cases tend to involve more pets that get "updated" with new configs over the lifetime of the VM
[20:48] <akutz> Ironically I am the solutions architect for the recently released VM Service platform that aims to modernize VMs and their workloads to be closer to cattle than pets :)
[20:48] <rharper> \o/  
=== EvaSDK_ is now known as EvaSDK
[20:50] <akutz> https://blogs.vmware.com/vsphere/2021/04/introducing-the-vsphere-virtual-machine-service.html
[20:51] <akutz> Basically, the cloud/VI admin controls allows hardware configs (VM Class) and a DevOps persona selects a VM Class and VM Image, and they get a VM. Think of it has an OVA where the OVF doesn't matter :)
[22:29] <bahamat> I'm running lxc and passing in a network-config as json. cloud-init-local says it's failing with "Unknown network config version: None", but I've got version 2 specified in the file, and cloud-init devel net-convert is fine with it. 
[22:30] <bahamat> Is there a way I can get more debugging info out of it?
[22:32] <rharper> yes, you want to look at /var/lib/cloud/seed/nocloud-net/network-config ;  this file is typically generated by lxd itself before starting the container, and one needs to set the config up in lxd config of the container .   you can lxc exec $container /bin/bash to get into the container even if networking didn't come up 
[22:33] <bahamat> Yeah, I'm in the container. That file is exactly what I expect it to be.
[22:34] <bahamat> and cloud-init devel net-convert happily and correctly converts it
[22:34] <rharper> it should not have network: { your config } , rather it should just be { your config}    
[22:34] <rharper> that is the top-level network: key is not needed in a network-config file 
[22:34] <bahamat> ok, maybe that's what I've got wrong.
[22:34] <rharper> that'd be my guess w.r.t the error 
[22:35] <bahamat> version 1 seems to be fine with network being at the top though.
[22:36] <rharper> At some point I think we fix up if there's a leading network: at the top , but I don't recall if that's also done for 2;  
[22:37] <bahamat> OK, so in either v1 or v2 it's not supposed to be there.
[22:38] <rharper> its not needed for either, the name of the file itself gives cloud-init context as to what's included;  the top-level network: is used to namespace it when it's within other larger config files 
[22:49] <bahamat> Alright, that 
[22:49] <bahamat> that was what it was
[22:49] <bahamat> Cool, thanks for the pointer.