[13:07] <elendrys> Hello
[13:10] <elendrys> I need some help with Openstack's vendor data. The log shows that it successfully reads the vendor_data url but the cache vendordata is empty. The data json is a simple structure like {"env": "test}.
[13:10] <elendrys> Thank you
[13:11] <elendrys> (I missed a quote while writing the example: {"env": "test", "dc": "dc2"})
[21:21] <elendrys> After several tests, i can get something only if I set the vendor data to { "cloud-init": "a string" }, which is quite limitating
[21:24] <blackboxsw> elendrys: sorry for missing this earlier, how are you providing vendordata or vendordata2 to openstack?
[21:25] <elendrys> I use the static json provider with a json file
[21:25] <elendrys> I can get the vendor data when I query the metadata api within the instance
[21:26] <blackboxsw> generally cloud-init should pull down vendor_data.json and vendor_data2.json directly from the OpenStack URLs such as openstack/2016-10-06/vendor_data2.json etc.
[21:26] <blackboxsw> ok good. so how are you trying to reference/use that data in cloud-init?
[21:26] <elendrys> I can also see this in the logs: 2023-04-12 21:17:05,982 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254/openstack/2018-08-27/vendor_data.json' with {'url': 'http://169.254.169.254/openstack/2018-08-27/vendor_data.json', 'allow_redirects': True, 'method': 'GET', 'timeout': 10.0, 'headers': {'User-Agent': 'Cloud-Init/20.4.1'}}
[21:26] <elendrys> configuration
[21:26] <elendrys> 2023-04-12 21:17:06,052 - url_helper.py[DEBUG]: Read from http://169.254.169.254/openstack/2018-08-27/vendor_data.json (200, 36b) after 1 attempts
[21:27] <elendrys> but later in the log stages.py[DEBUG]: no vendordata from datasource
[21:28] <elendrys> I wad trying "cloud-init query -a" or "cloud-init query vendordata"
[21:28] <elendrys> and I also look at /var/lib/cloud/instance/vendor-data.txt.i
[21:29] <elendrys> And it give me something only if I have this single cloud-init key with a sting as a value
[21:30] <elendrys> I didn't trying anything else since this doesn't look like it's working
[21:30] <elendrys> try*
[21:32] <minimal> elendrys: looking at the c-i OpenStack DS docs, the json should be "{"cloud-init": "#cloud-config\n<rest of config>"}
[21:33] <minimal> where "<rest of string>" is an embedded user-data style "document"
[21:35] <elendrys> Yes this is what I read today but how I can test that provided variables in this config is rendered from the command line ?
[21:35] <elendrys> Or do I have to set something like bootcmd to validate the content first ?
[21:36] <blackboxsw> elendrys: I expect in your case, and per minimal's comment the format of the network data in openstack is off and it's being ignored by cloud-init as a result. 
[21:36] <minimal> I'm not sure I understand. Have you tried setting such vendor data and checking the /var/log/cloud-init.log (when it's in debug mode)?
[21:36] <blackboxsw> elendrys: what I'd expect is your logs have an inconspicuous message like the following telling us cloud-init didn't like the vendordata format: "2023-04-12 21:34:58,325 - __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: 'b'..... something' during vendordata processing
[21:37] <blackboxsw> try a `grep WARNING /var/log/cloud-init.log ` and I wonder if we'll see it
[21:38] <elendrys> 2023-04-12 21:17:07,698 - DataSourceOpenStack.py[WARNING]: Invalid content in vendor-data: vendordata['cloud-init'] cannot be dict
[21:38] <elendrys> Right I missed it
[21:38] <blackboxsw> roger... yep, what it's expecting is the actual value is a multi-line string that starts with "#cloud-config"
[21:39] <minimal> https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data
[21:39] <minimal> it is documented here
[21:41] <elendrys> I've already tried this and it indeed give me the content that I configured
[21:41] <elendrys> I expected that I could just put JSON instead
[21:43] <minimal> you have, you put JSON that embedded YAML ;-)
[21:44] <meena> JSON is a subset of YAML… :(
[21:47] <elendrys> I mean pure JSON but if it works like this then I'll stick to this
[21:48] <blackboxsw> :) yep, a bit of a pain for openstack datasource..... we could adapt https://github.com/canonical/cloud-init/blob/main/cloudinit/sources/__init__.py#L1089 to be a bit more flexible on that account if the top-level key is cloud-init in vendordata, then accept a dict
[21:50] <elendrys> as you're here, is there somewhere I can get the deb packages for the latest version ? We use Debian 11 mostly, it is packaged with 20, and maybe 22.2 with backports from 12
[21:51] <blackboxsw> elendrys: not sure if this helps https://launchpad.net/~cloud-init-dev/+archive/ubuntu/proposed we keep up to date when we are releasing something to Ubuntu that has passed internal testing.
[21:52] <blackboxsw> to live on the bleeding edge:  add-apt-repository ppa:cloud-init-dev/daily to see latest and greatest from tip of main
[21:54] <elendrys> We had a junior devops that "worked" on two topics last year, one of them being building custom cloud images
[21:55] <elendrys> I just take this recently after he left, and ... my eyes are bleeding every time I look at the code
[21:55] <blackboxsw> I'm not connecting the dots on whether we have something specific for debian downstreams. generally from cloud-init source you should be able to run ./tool/run-container -p debian/11 to gen your own debs locally using LXC
[21:56] <elendrys> So I'm very new to cloud-init
[21:56] <elendrys> Ok thank you I'll give it a try next week
[21:58] <elendrys> @bla
[21:58] <blackboxsw> *./tools/run-container -p debian/11 to get something that kinda looks like what debian has downstream
[22:00] <elendrys> blackboxsw minimal Thanks I just put an echo in the runcmd section of my vendor data and it works
[22:04] <elendrys> One last thing, we use cloud init support in Proxmox as well
[22:04] <elendrys> It looks like it uses the nocloud datasource
[22:06] <elendrys> https://pve.proxmox.com/wiki/Cloud-Init_Support says that it can pass vendor data, but the doc doesn't say if this source supports vendor data as well
[22:07] <elendrys> Howdy it's late, it is written, my ba
[22:07] <elendrys> bad*
[22:09] <minimal> yeah, it's either a vedor-data file (in a "cidata" ISO/filesystem) or a vendor-data seed URL
[22:14] <elendrys> Last question, how can I pass variable via my vendor data ? The idea is to have a generic config in the image with a script that behaves according to the variables provided by the cloud's vendor data
[22:18] <minimal> vendor-data is basically user-data (with a different name), there's no variables in user-data AFAIK
[22:19] <elendrys> I expected that vendor data could provide details about the cloud hosting the instance
[22:19] <minimal> there is however jinja templating for user-data
[22:20] <minimal> not sure if that works for vendor-data
[22:21] <elendrys> As the script is already in the image, maybe I can just use write_files to store the data on the disk and try to read it from the script
[22:23] <minimal> I've never tried using the jinja templating but it seems for NoCloud that things in the supplied meta-data file could then be used by templated vendor-data / user-data
[22:23] <elendrys> Currently my ex-colleague did something like setting a cron with a single task that launch the script and delete the cron ... I don't know why he didn't use runcmd and cloud-init-per
[22:24] <minimal> https://cloudinit.readthedocs.io/en/latest/explanation/instancedata.html#example-user-data-script-with-instance-data
[22:32] <elendrys> Thank you. Time to go to bed
[22:32] <elendrys> I'll probably be arround later to bother you a bit more
[22:32] <elendrys> Bye
[22:33] <minimal> cya