=== esv_ is now known as esv
[18:49] <ananke> I don't suppose anybody recalls how user-data in older cloud-init versions was handled? I'll get to the point: I'm getting odd errors when feeding user-data via packer to a very old AMI (Ubuntu 14.04, running cloud-init 0.7.5). The user-data in question is identical to what we use on newer versions, and so is the packer recipe
[18:50] <ananke> the only difference being the very ancient version of ubuntu and its cloud-init version
[18:51] <ananke> the errors imply that cloud-init is trying to execute the user-data, rather than interpret it as its config. Here's just the first two lines of the cloud-init-output.log: 
[18:51] <ananke> Cloud-init v. 0.7.5 running 'modules:final' at Fri, 11 Nov 2022 17:12:06 +0000. Up 126.92 seconds.
[18:51] <ananke>  /var/lib/cloud/instance/user-data.txt: 9: /var/lib/cloud/instance/user-data.txt: manage_etc_hosts:: not found
[18:53] <ananke> https://dpaste.org/ZY8qS is just the beginning of that log, and the user-data we feed is: https://dpaste.org/NTr0v
[18:55] <ananke> I feel like this may be a trivial issue, and I'm missing something obvious. Unfortunately, I haven't worked with cloud-init back in 0.7.5 version days, so I don't know what might have changed
[19:02] <blackboxsw> ananke: hrm I'mbaffled too on this. the user-data looks valid
[19:02] <blackboxsw> and generally user-data in 0.7.5 was still written to /var/lib/cloud/instances/dev-t/user-data.txt
[19:05] <blackboxsw> I would've expected to see logs earlier about Cloud-init v. 0.7.5 running 'init-local' and 'init' and 'modules:config' etc instead of just 'modules:final'. From your logs, it makes me think something in that AMI in cloud-init's final stage  is trying to `source /var/lib/cloud/instance/user-data.txt`in shell script and falling over
[19:06] <blackboxsw> ananke: are there custom modules in that AMI declared to run during `cloud_final_modules:` in /etc/cloud/cloud.cfg ?
[19:07] <blackboxsw> or maybe custom scripts in /var/lib/cloud/scripts on that AMI that cannot be run and reproduce that same error?
[19:07] <blackboxsw> same set of errors
[19:09] <blackboxsw> note if I "source <your_userdata_file>" on the command line I get those same type of errors  (manage_etc_hosts:: command not found
[19:09] <blackboxsw> : command not found) etc
[19:11] <ananke> I'm going to add some debugging steps with packer to dump the contents of /var/lib/cloud/instance/user-data.txt, perhaps with full logs. This issue happens when I try to build a new image with our pipeline, and the next boot has no user-data, so all I have is leftover logs
[19:49] <ananke> now this is interesting: amazon-ebs.CR_amazon_builder: Nov 11 19:24:49 ip-100-64-21-177 [CLOUDINIT] importer.py[DEBUG]: Failed at attempted import of 'cc_runcmd' due to: No module named cc_runcmd
[19:51] <ananke> this is a fun discovery trip. I'll keep poking at things to see what's going on
[20:27] <ananke> found a few issues, seems the 'apt:' config didn't seem to support proxy settings, there was no cloud-init status, etc. 
[20:54] <cjp256> seeing some cases of "duplicate mac".  The Azure datasource patches the distro list, but unfortunately that gets reset: https://github.com/canonical/cloud-init/blob/main/cloudinit/stages.py#L338   any thoughts on how best to address?  I was thinking if a network refactor were on the table to avoid that assertion, we could strip out the blacklist_drivers handling across net and keep it contained to the Azure datasource
[21:55] <blackboxsw> coincidentally, network refactor is in progress from meena :) given that it doesn't look like we have any call-sites using blacklist_drivers  besides Azure, that may make sense.
[22:04] <cjp256> nice :D