| elendrys | Hello | 13:07 |
|---|---|---|
| elendrys | I need some help with Openstack's vendor data. The log shows that it successfully reads the vendor_data url but the cache vendordata is empty. The data json is a simple structure like {"env": "test}. | 13:10 |
| elendrys | Thank you | 13:10 |
| elendrys | (I missed a quote while writing the example: {"env": "test", "dc": "dc2"}) | 13:11 |
| elendrys | After several tests, i can get something only if I set the vendor data to { "cloud-init": "a string" }, which is quite limitating | 21:21 |
| blackboxsw | elendrys: sorry for missing this earlier, how are you providing vendordata or vendordata2 to openstack? | 21:24 |
| elendrys | I use the static json provider with a json file | 21:25 |
| elendrys | I can get the vendor data when I query the metadata api within the instance | 21:25 |
| blackboxsw | generally cloud-init should pull down vendor_data.json and vendor_data2.json directly from the OpenStack URLs such as openstack/2016-10-06/vendor_data2.json etc. | 21:26 |
| blackboxsw | ok good. so how are you trying to reference/use that data in cloud-init? | 21:26 |
| elendrys | I can also see this in the logs: 2023-04-12 21:17:05,982 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254/openstack/2018-08-27/vendor_data.json' with {'url': 'http://169.254.169.254/openstack/2018-08-27/vendor_data.json', 'allow_redirects': True, 'method': 'GET', 'timeout': 10.0, 'headers': {'User-Agent': 'Cloud-Init/20.4.1'}} | 21:26 |
| elendrys | configuration | 21:26 |
| elendrys | 2023-04-12 21:17:06,052 - url_helper.py[DEBUG]: Read from http://169.254.169.254/openstack/2018-08-27/vendor_data.json (200, 36b) after 1 attempts | 21:26 |
| elendrys | but later in the log stages.py[DEBUG]: no vendordata from datasource | 21:27 |
| elendrys | I wad trying "cloud-init query -a" or "cloud-init query vendordata" | 21:28 |
| elendrys | and I also look at /var/lib/cloud/instance/vendor-data.txt.i | 21:28 |
| elendrys | And it give me something only if I have this single cloud-init key with a sting as a value | 21:29 |
| elendrys | I didn't trying anything else since this doesn't look like it's working | 21:30 |
| elendrys | try* | 21:30 |
| minimal | elendrys: looking at the c-i OpenStack DS docs, the json should be "{"cloud-init": "#cloud-config\n<rest of config>"} | 21:32 |
| minimal | where "<rest of string>" is an embedded user-data style "document" | 21:33 |
| elendrys | Yes this is what I read today but how I can test that provided variables in this config is rendered from the command line ? | 21:35 |
| elendrys | Or do I have to set something like bootcmd to validate the content first ? | 21:35 |
| blackboxsw | elendrys: I expect in your case, and per minimal's comment the format of the network data in openstack is off and it's being ignored by cloud-init as a result. | 21:36 |
| minimal | I'm not sure I understand. Have you tried setting such vendor data and checking the /var/log/cloud-init.log (when it's in debug mode)? | 21:36 |
| blackboxsw | elendrys: what I'd expect is your logs have an inconspicuous message like the following telling us cloud-init didn't like the vendordata format: "2023-04-12 21:34:58,325 - __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: 'b'..... something' during vendordata processing | 21:36 |
| blackboxsw | try a `grep WARNING /var/log/cloud-init.log ` and I wonder if we'll see it | 21:37 |
| elendrys | 2023-04-12 21:17:07,698 - DataSourceOpenStack.py[WARNING]: Invalid content in vendor-data: vendordata['cloud-init'] cannot be dict | 21:38 |
| elendrys | Right I missed it | 21:38 |
| blackboxsw | roger... yep, what it's expecting is the actual value is a multi-line string that starts with "#cloud-config" | 21:38 |
| minimal | https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data | 21:39 |
| minimal | it is documented here | 21:39 |
| elendrys | I've already tried this and it indeed give me the content that I configured | 21:41 |
| elendrys | I expected that I could just put JSON instead | 21:41 |
| minimal | you have, you put JSON that embedded YAML ;-) | 21:43 |
| meena | JSON is a subset of YAML… :( | 21:44 |
| elendrys | I mean pure JSON but if it works like this then I'll stick to this | 21:47 |
| blackboxsw | :) yep, a bit of a pain for openstack datasource..... we could adapt https://github.com/canonical/cloud-init/blob/main/cloudinit/sources/__init__.py#L1089 to be a bit more flexible on that account if the top-level key is cloud-init in vendordata, then accept a dict | 21:48 |
| elendrys | as you're here, is there somewhere I can get the deb packages for the latest version ? We use Debian 11 mostly, it is packaged with 20, and maybe 22.2 with backports from 12 | 21:50 |
| blackboxsw | elendrys: not sure if this helps https://launchpad.net/~cloud-init-dev/+archive/ubuntu/proposed we keep up to date when we are releasing something to Ubuntu that has passed internal testing. | 21:51 |
| blackboxsw | to live on the bleeding edge: add-apt-repository ppa:cloud-init-dev/daily to see latest and greatest from tip of main | 21:52 |
| elendrys | We had a junior devops that "worked" on two topics last year, one of them being building custom cloud images | 21:54 |
| elendrys | I just take this recently after he left, and ... my eyes are bleeding every time I look at the code | 21:55 |
| blackboxsw | I'm not connecting the dots on whether we have something specific for debian downstreams. generally from cloud-init source you should be able to run ./tool/run-container -p debian/11 to gen your own debs locally using LXC | 21:55 |
| elendrys | So I'm very new to cloud-init | 21:56 |
| elendrys | Ok thank you I'll give it a try next week | 21:56 |
| elendrys | @bla | 21:58 |
| blackboxsw | *./tools/run-container -p debian/11 to get something that kinda looks like what debian has downstream | 21:58 |
| elendrys | blackboxsw minimal Thanks I just put an echo in the runcmd section of my vendor data and it works | 22:00 |
| elendrys | One last thing, we use cloud init support in Proxmox as well | 22:04 |
| elendrys | It looks like it uses the nocloud datasource | 22:04 |
| elendrys | https://pve.proxmox.com/wiki/Cloud-Init_Support says that it can pass vendor data, but the doc doesn't say if this source supports vendor data as well | 22:06 |
| elendrys | Howdy it's late, it is written, my ba | 22:07 |
| elendrys | bad* | 22:07 |
| minimal | yeah, it's either a vedor-data file (in a "cidata" ISO/filesystem) or a vendor-data seed URL | 22:09 |
| elendrys | Last question, how can I pass variable via my vendor data ? The idea is to have a generic config in the image with a script that behaves according to the variables provided by the cloud's vendor data | 22:14 |
| minimal | vendor-data is basically user-data (with a different name), there's no variables in user-data AFAIK | 22:18 |
| elendrys | I expected that vendor data could provide details about the cloud hosting the instance | 22:19 |
| minimal | there is however jinja templating for user-data | 22:19 |
| minimal | not sure if that works for vendor-data | 22:20 |
| elendrys | As the script is already in the image, maybe I can just use write_files to store the data on the disk and try to read it from the script | 22:21 |
| minimal | I've never tried using the jinja templating but it seems for NoCloud that things in the supplied meta-data file could then be used by templated vendor-data / user-data | 22:23 |
| elendrys | Currently my ex-colleague did something like setting a cron with a single task that launch the script and delete the cron ... I don't know why he didn't use runcmd and cloud-init-per | 22:23 |
| minimal | https://cloudinit.readthedocs.io/en/latest/explanation/instancedata.html#example-user-data-script-with-instance-data | 22:24 |
| elendrys | Thank you. Time to go to bed | 22:32 |
| elendrys | I'll probably be arround later to bother you a bit more | 22:32 |
| elendrys | Bye | 22:32 |
| minimal | cya | 22:33 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!