[10:49] <meena> Good point
[10:50] <meena> i can't tell from the documentation a new network interface is considered
[10:51] <meena> faa: i'd have to look at the code, and chances are, before i'm able to do that, someone more experienced with the project will have woken up
[12:03] <faa> meena: thank, good news :)
[12:04] <meena> my hour of laptop time has been wasted, and now i have to go and wake my daughter from her nap
[12:09] <faa> this is not an urgent problem, but I will be glad to any advice, because as far as I can see, now cloud-init does not have the ability to change the network configuration
[13:11] <cjmm6> Hi, I need help with latest cloud-init version. In machines that not needed or not works cloud-init, I use a systemd service (I cannot use boot parameters), that using a Before=cloud-init-local.service and some conditions, then write the file to disable cloud-init /etc/cloud/cloud-init.disable (or remove this file if need use cloud-init)
[13:11] <cjmm6> This works in previous versions of cloud-init
[13:11] <cjmm6> But in the last version not works because exists one previous task called "cloud-init-generator". Then, so i don't know how to make my systemd unit run before that task
[13:13] <cjmm6> My systemd unit is:
[13:13] <cjmm6> [Unit]
[13:13] <cjmm6> Description=Control cloud-init to enable or disable cloud-init dinamically at boot time
[13:13] <cjmm6> DefaultDependencies=no
[13:13] <cjmm6> After=systemd-remount-fs.service
[13:13] <cjmm6> Before=NetworkManager.service network.service network-pre.target
[13:13] <cjmm6> Before=shutdown.target sysinit.target
[13:13] <cjmm6> Before=cloud-config.service cloud-final.service cloud-init-local.service cloud-init.service cloud-config.target
[13:13] <cjmm6> Conflicts=shutdown.target
[13:13] <cjmm6> [Service]
[13:13] <cjmm6> Type=oneshot
[13:13] <cjmm6> ExecStart=/usr/local/bin/control-cloud-init.sh
[13:13] <cjmm6> RemainAfterExit=yes
[13:13] <cjmm6> TimeoutSec=0
[13:13] <cjmm6> # Output needs to appear in instance console output
[13:50] <smoser> cjmm6: you wont really be able to reliably run before a generator
[13:50] <smoser> at least not with a systemd unit
[13:54] <smoser> you can disable the generator and then you'll get cloud-init behavior like before.
[13:55] <smoser> unfortunatlye "fully disabled" isn't really possible without the generator.  cloud-init will still run through all its stages and use the 'none' datasource.
[14:26] <Odd_Bloke> The generator doesn't do much to the system, either, so it probably isn't the end of the world if it runs and then your custom unit disables all the cloud-init units that it just added to the boot graph.
[14:32] <cjmm6> disable is as follows?
[14:32] <cjmm6> ln -sf /dev/null /etc/systemd/system-generator/cloud-init-generator
[14:32] <cjmm6> because early systemd cannot use systemctl
[14:33] <cjmm6> and how to reenable generator without systemctl?
[14:37] <Odd_Bloke> The generator runs before any units run, so you won't be able to disable the generator using a unit.
[14:38] <Odd_Bloke> And I was going to say that you should be able to disable the units in your own unit, but that may be too late for systemd to pick up the change.
[14:39] <Odd_Bloke> Hmm.
[14:41] <cjmm6> I cannot try to disable all cloud-init units
[14:41] <cjmm6> error!
[14:42] <Odd_Bloke> cjmm6: How do you get the systemd unit onto these systems?
[14:42] <cjmm6> i haven't tried disabling the unit systemd of cloud-init, i will test it if it works
[14:43] <Odd_Bloke> cjmm6: Sorry, I meant how do you get the systemd service which was working with older versions of cloud-init onto the system?
[14:43] <cjmm6> Its because I use a VM image compatible with multiple environment systems.
[14:44] <Odd_Bloke> Right, so the conditions in the unit are used to determine enable/disable based on where you're booting?
[14:44] <cjmm6> Yes! Dinamically
[14:45] <cjmm6> Then, I use one unique VM image for multiple systems
[14:45] <cjmm6> https://github.com/cesarjorgemartinez/automate-virtual-machine-linux-images/tree/master/CentOS7Minimal
[14:46] <cjmm6> Until this challenge (Ubuntu):
[14:46] <cjmm6> https://github.com/cesarjorgemartinez/automate-virtual-machine-linux-images/tree/master/Ubuntu20Minimal
[14:47] <smoser> https://github.com/cesarjorgemartinez/automate-virtual-machine-linux-images/blob/master/Ubuntu20Minimal/files/control-cloud-init.sh
[14:47] <smoser> is that what is controlling cloud-init ?
[14:48] <smoser> it looks to me that you enable cloud-init on kvm and aws. and disable elsewhere.
[14:48] <Odd_Bloke> cjmm6: OK, so one thing you could potentially do is configure cloud-init to only consider the data sources that will present on KVM and EC2.  The generator should then disable cloud-init when it fails to find those datasources.
[14:48] <Odd_Bloke> s/find/detect the applicability of/
[14:48] <smoser> but in reality... cloud-init really should do all the right things.
[14:49] <smoser> i'im surprised if it ever enables itself in a scenario where you think it should not run
[14:49] <smoser> or even disables itself where you think it should
[14:51] <cjmm6> Not ...  cloud-init really should do all the right things... In many systems it is even harmful
[14:51] <smoser> like ?
[14:51] <cjmm6> Then, if you can test, Centos7 and 8 works correctly because not use the last cloud-init
[14:52] <smoser> the intent of the generator is to enable cloud-init when there is a "datasource". and do nothing otherwise.
[14:52] <smoser> i'm sorry, i didn't follow.
[14:53] <cjmm6> Centos 7 => cloud-init-18.5-3.el7.centos.x86_64
[14:53] <smoser> i agree that cloud-init was very annoying in the past.
[14:53] <smoser> without the generator, when it ran and chose the "None" datasource, it would have undesireable side effects.
[14:53] <cjmm6> ubuntu 20 => 20.1
[14:53] <smoser> but the generator was intended to *fix* those.
[14:54] <smoser> can you actually provide examples when it ran and you think it should not have ran ?
[14:54] <cjmm6> ad-hoc environments, or cloud-init not supported
[14:55] <cjmm6> + using compatible VMs between environments, as the purpose of this projet
[14:56] <cjmm6> "boot one image in multiple envs"
[14:56] <smoser> can you provide a 'cloud-init collect-logs' from any of those environments ?
[14:56] <cjmm6> without changes
[14:57] <smoser> ubuntu cloud images are "one image" that boots in multiple envs
[14:57] <smoser> and all "just work" (and if they do not, then feel free to file bugs)
[14:57] <smoser> i'm not trying to be rude.  we want to make this work, but we've intended to solve the problem that you're having with the generator
[14:57] <smoser> so we want to fix the generator if it is not behaving correctly
[14:58] <smoser> rather than make users like yourself have to write scripts to do the right thing.
[15:00] <smoser> cjmm6: ^
[15:02] <smoser> Odd_Bloke's suggesting of configuring the datasource_list should also work.  but the goal you're after of "one image" is already a design goal of the project.
[15:04] <cjmm6> but need one thing to disable, always... Only works in my test, if, when the VM is booted, enter to ssh and create /etc/cloud/cloud-init.disable, and second reboot
[15:06] <smoser> i dont understand.  "but need one thing to disable, always"
[15:07] <smoser> in some scenarios you want to stop cloud-init from running ?
[15:07] <smoser> what are those scenarios?
[15:08] <cjmm6> yes, in adhoc envs, when no need to change nothing. New envs without already cloud-init support, etc.
[15:10] <cjmm6> Situations that, exists a big bug with difficult solution, or the Operating Systems not update the cloud-init for their own reasons...
[15:24] <smoser> so how do you determine that this is such an environment?
[15:27] <smoser> as you seem to know, you can disable cloud-init always by touching /etc/cloud/cloud-init.disabled.
[15:37] <rharper> cjmm6: a couple of comments;  the image build process (https://github.com/cesarjorgemartinez/automate-virtual-machine-linux-images/blob/master/Ubuntu20Minimal/postscripts/post-stage-1.sh)  does a lot of work cleaning up the image due to the image being booted in the first place via packer;  you may want to look at using something like mount-image-callback (in Ubuntu's cloud-image-utils package) to allow you to mount the image without booting so you
[15:37] <rharper> can add your various changes (injecting scripts and programs and updating defaults in the image);    second, for Centos 7/8 cloud-init, RedHat merges features/fixes from master; so while 18.5 is "old" the core capabilities around dynamically enabling/disabling cloud-init (via ds-identify);   third; I suspect that the cloud-init control script is not needed; the cloud-init generator using the ds-identify code (/usr/lib/cloud-init/ds-identify, writes
[15:37] <rharper> output to /run/cloud-init/* ) one can examine that to see why cloud-init gets enabled or not;  If you have a scenario where cloud-init ran when you did not want it to; or the reverse; capturing /run/cloud-init/* contents into a bug will let us help sort out if it's expected behavior or if there is a bug
[16:09] <Odd_Bloke> smoser: Can you still disable cloud-init by touching that file after the generator has run?
[16:14] <smoser> Odd_Bloke: well it wont next time
[16:14] <smoser> but the ship has almost certainly sailed when you touch it otherwise.
[16:21] <meena> anyone here seen / answered faa's problem of adding another network interface?
[16:23] <rharper> meena: faa: by default, adding a nic won't trigger any update to existing network configuration rendered;  so that's expected behavior.    the instance-id is set by the meta-data in the NoCloud datasource;  changing the instance-id will boot like a new instance;
[16:28] <rharper>  If you've added a nic, and you know what an updated network-config (v1) format should look like, you can use 'cloud-init devel net-convert'  to render a specific distro/renderer output; and then copy that to where you need ;  Long term, as we land support in Datasources for refreshing metadata and handling hotplug, https://github.com/canonical/cloud-init/pull/47  ;  I could see NoCloud (configured with an HTTP end-point instead of local files) could
[16:28] <rharper> refresh (fetch metadata) on hotplug of nic.
[16:56] <Odd_Bloke> blackboxsw: rharper: Could one of you look over my proposed changes to https://github.com/canonical/cloud-init/pull/358, please?  They're minor, but as mruffell is out this week, I'd like at least one other person to check them before I land them.
[16:56] <rharper> Odd_Bloke: sure
[16:57] <Odd_Bloke> Thanks!
[17:38] <meena> rharper: you set pr has gone stale…
[17:39] <rharper> yes, it's still valid, just needs a refresh and push
[18:29] <smoser> rcj or maybe Odd_Bloke , or maybe one of the microsoft people here...
[18:30] <smoser> it was my impression in the past that when Ubuntu images were uploaded, that the upload was done with sparseness in-tact
[18:30] <smoser> but then that the copy from region to region did not maintain that sparseness.
[18:31] <smoser> it seems that a feature went into some tools https://github.com/Azure/azure-cli/issues/11509
[18:31] <smoser> late in 2019 that may make my knowledge obsolete.
[18:32] <Odd_Bloke> I believe we addressed that a while ago; I remember pairing with SteveZ (of Azure) to fix it at a F2F a few years back.
[18:32] <smoser> My question then... Do infrastructure (user->user or region->region) copies inside azure now retain sparseness ?
[18:34] <smoser> my current employer is in a similar situation where we publish sparse images, but then use of those images was believed to entail non-sparse copies.
[18:34] <smoser> i'd love to have some link to doc that says otherwise, or link to a nice manual that I could RTFM.
[18:34] <Odd_Bloke> The image copy/replication happens on the backend for Ubuntu image publication (kicked off via private APIs), so I don't know if knowledge from Ubuntu publication will transfer over to a third-party image publication flow.
[18:35] <Odd_Bloke> (We are categorically different from images uploaded by users.)
[18:36] <smoser> hm... yeah.
[18:36] <smoser> so what i'm after is
[18:36] <smoser> a.) we publish an image (we do this i think as well as we can, using sparseness)
[18:36] <smoser> b.) user launches that image or imports it into their account
[18:38] <smoser> i'm not actually sure how they do that, but if its "click click" or otherwise somehow underlying infrastructure, then i'd certainly hope it was sparse.  and if it is *not* click click, then i'd like to at least know how to tell someone that they can copy it the right way.
[18:38] <smoser> thanks Odd_Bloke
[18:39] <smoser> i'm disappointed that cjmm6 disappeared. i was honestly interested in his/her use case.
[18:42] <smoser> well, if someone ms knowledgeable could reach out, i'd love to chat some.
[19:49] <Odd_Bloke> rharper: I've updated https://github.com/canonical/cloud-init/pull/358, could you take another quick look pls?
[19:49] <rharper> Odd_Bloke: yes
[19:49] <Odd_Bloke> Thanks!
[20:13] <rharper> Odd_Bloke: one more comment, I just left for you
[20:14] <blackboxsw> falcojr: lucasmoura if either of you get a chance. I've put up a basic SRU doc  from which we'll work to validate this cloudinit SRU https://github.com/cloud-init/ubuntu-sru/pull/97/files
[20:15] <blackboxsw> it basically creates a bunchof  broken links which we will fill in as we verify each cloud or manual work it
[20:15] <blackboxsw> work *item*
[20:18] <Odd_Bloke> rharper: And replied.
[20:21] <rharper> k
[20:46] <blackboxsw> lucasmoura: falcojr rharper the much anticipated quilt patches for xenial https://github.com/canonical/cloud-init/pull/406
[20:46] <blackboxsw> and procedure
[20:46] <lucasmoura> blackboxsw, ack
[20:46] <blackboxsw> I'll do basically the same for bionic and eoan now.
[20:46] <blackboxsw> without the procedural steps
[20:56] <rharper> ok
[21:01] <blackboxsw> rharper: falcojr lucasmoura https://github.com/canonical/cloud-init/pull/407 bionic
[21:02] <rharper> thx
[21:03] <Odd_Bloke> rharper: Updated, thanks again for the review!
[21:08] <blackboxsw> and eoan https://github.com/canonical/cloud-init/pull/409
[21:56] <Odd_Bloke> blackboxsw: rharper: https://github.com/canonical/cloud-init/pull/358 just landed, so we can SRU away now, I think.
[22:34] <blackboxsw> nice Odd_Bloke I don't mind doing another new-upstream-snapshot round
[22:34] <blackboxsw> might as well