[15:25] Good afternoon. [15:28] I am trying to use cloud-init with a Server 22.04 ISO image to build PCs (so bare metal) with PXE. When I create a user-data file that starts with #cloud-config, I can build successfully. When I try to use the #include format for my user-data file, I get: [15:29] (timestamp) - main.py[INFO]: contents of 'http://myserver/user-data' did not start with b'#cloud-config' [15:30] and cloud-init does not attempt to load the URLs listed in the user-data file. [15:31] https://github.com/canonical/cloud-init suggested coming here for help. [15:39] We are using iPXE. My menu says: [15:40] dhcp [15:40] kernel http://myserver/pathto/vmlinuz [15:40] initrd http://myserver/pathto/initrd [15:40] imgstat [15:40] imgargs vmlinuz initrd=initrd url=http://myserver/pathto/ubuntu2204.iso url=http://myserver/pathto/user-data autoinstall ds=nocloud-net\;s=http://myserver/pathto ip=dhcp net.ifnames=0 [15:40] boot [15:41] Is what I'm trying to do even supported using bare metal and nocloud-net? [15:43] I mean, is #include and a list of URLs supported with nocloud-net? [16:36] hi threepistons [16:37] what's " the #include format "? [16:38] Example at https://github.com/canonical/cloud-init/blob/main/doc/examples/include.txt [16:39] Described in #4 of https://help.ubuntu.com/community/CloudInit [16:42] "This content is a "include" file. The file contains a list of urls, one per line. Each of the URLs will be read, and their content will be passed through this same set of rules. Ie, the content read from the URL can be gzipped, mime-multi-part, or plain text." [16:43] I've never seen those before [16:44] (last edited 2021-07-25 16:34:51 by danielbowers) [16:44] that's from this century [16:47] the cloud-init on the PC that won't build is version 22.1-14-g2e17a0d6-0ubuntu1~22.04.5, it came with the ISO. [17:00] threepistons: nocloud-net loads meta-data, user-data, and vendor-data from the base url [17:00] I don't think you can make it load any other urls [17:01] I'm looking at https://github.com/canonical/cloud-init/blob/2134b0487fa4d453d75ca83bd0d3cf008541ebcb/cloudinit/cmd/main.py#L201 It looks like by design #include will not work. That's unfortunate because I have many different user-data files and they have a lot of data in common. [17:02] > many different user-data files [17:02] SATA vs NVMe, LUKS vs plain, GUI vs headless... [17:02] combinatorial explosion [17:03] threepistons: the function that line is in refers to MAAS DataSource, not NoCloud [17:05] also notice line 222 indicates "url" is deprecated [17:05] I searched the repo for the error message I got and that was what I found. [17:06] https://github.com/canonical/cloud-init/search?q=%22did+not+start+with%22&type=code only one result. [17:07] from my recollections of trying using NoCloud-Net there are 2 stages, first the URLs (meta-data, user-data, vendor-data) and downloaded, then 2nd stage they are parsed - if they try to include from another url I don't think there's code to then download from more urls (I could be wrong) [17:08] actually, looking at your cmdline options I'm confused - you have BOTH url= AND s= [17:10] Yeah, it seemed to need both to work when I first set up the iPXE menus for 18.04, and "if it ain't broke don't fix it" after that. [17:12] I haven't yet tried nocloud-net with PXE [17:12] Now that I want to refactor my list of staff SSH keys into *one* file instead of 36, maybe time to review that set of arguments. [17:12] or dynamically create the user-data contents when it is requested? [17:15] CGIing the user-data is my plan B. [17:17] I don't manage the OS of the server that the iPXE service and user-data files sit on, so negotiating the installation of a CGI engine is something I'd rather avoid. [17:18] actually you don't even necessarily need dynamic/CGI creation, you could prepare the content in advance (i.e. use jinja or similar templates to create files) whenever "provisioning" is done for a new machine [17:19] I was thinking of using something like make, never heard of jinja. [17:19] its what Python code (including cloud-init) tends to use [17:21] e.g. cloud-init has template files in /etc/cloud/templates used to create chrony.conf and /etc/hosts [17:23] Removing the s= and replacing the url= with cloud-config-url= leaves me with the same "did not start with b'#cloud-config'" message. But the PC does load the user-data file, so I can at least remove the deprecated arguments from my iPXE menus. [17:24] all I know is that nocloud-net docs indicate using "s=" to specify the base url for it to retrieve meta-data, user-data, and vendor-data [17:32] we don't use meta-data and vendor-data. I've removed the s= and replacing the url= with cloud-config-url= with a known good user-data file and now have a desktop happily building with a laptop's full disk encryption config. [17:35] That helps me if I decide to go down the CGI route because I don't have to try to set parameters twice in the arguments that iPXE sends to the kernel. [17:35] And I don't have to try to set parameters in the path section of a URL. [17:35] threepiston: I mean that whatever url you specify for "s=" is a BASE URL - nocloud appends "/meta-data", "/user-data", and "/vendor-data" to it and tried to fetch all 3 [17:37] Yeah. If I was trying to use CGI generated files, I'd have to somehow put the CGI arguments into the base URL because nocloud-net does that appending thing. [17:38] did you read this? https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html#file-formats [17:38] I'm building now with nocloud-net and no s=baseurl because I've used cloud-config-url instead and the PC accepted that. [17:38] "Bost files must be present for it to be considered a valid seed ISO." [17:38] s/Bost/Both/ [17:39] the use of the word "ISO" is misleading, it should say something like "location" [17:40] Yes, the *empty* meta-data file is there next to the user-data file. I just didn't put the s=baseurl in. [17:40] if you have "ds=nocloud s=http://1.2.3.4/config" and http://1.2.3.4/config/user-data exists but http://1.2.3.4/config/meta-data does not exist then NoCloud won't work [17:41] meta-data is there, but I take your point that it needs to stay there. [17:41] I'm not sure what happens with an *empty* meta-data file, whether that is valid or not [17:41] as usually meta-data provides at least an instance-id [17:41] It's worked for me for a timeframe measured in years. [17:42] this is nocloud-net. [17:42] indeed that same part of the doc says "meta-data is a YAML-formatted file representing what you'd find in the EC2 metadata service" which implies a blank file is not valid [17:43] yes there were changes made to git recently to change nocloud and nocloud-net handling so that "ds=nocloud" covers both with the "s=" value distinguishing between them (file: vs http/https) [17:44] you didn't indicate which cloud-init version you're using [17:44] 17:47:20 the cloud-init on the PC that won't build is version 22.1-14-g2e17a0d6-0ubuntu1~22.04.5, it came with the ISO. [17:44] I appreciate that you might have missed that post [17:46] the docs for 22.1 still say the same things about meta-data being YAML formatted file etc [17:46] have you debugging enabled so that /var/log/cloud-init.log gives more info about what is happening? [17:47] And yet I'm looking at a built PC that I've just fed a LUKS passphrase to... [17:48] there may be a difference between what is documented/intended to work and what does work [17:49] in general just because something works doesn't mean it is intended to work and therefore is likely to keep on working in future versions ;-) [17:49] The undocumented feature... [17:49] or unnoticed bug lol [17:50] I mean, the issue for me is that nocloud doesn't seem to support #include at the start of a user-data file. [17:50] probably because, as I mentioned previously, that by the time it looks at the user-data file it has already fetched any urls it is going to fetch... [17:51] I know that you said that https://github.com/canonical/cloud-init/blob/2134b0487fa4d453d75ca83bd0d3cf008541ebcb/cloudinit/cmd/main.py#L201 applies to MAAS but there is no other instance of that error message in that repo. [17:51] Well, not error, but that WARN/INFO text. [17:52] also I don't see "#include" in the 22.1 docs, I think it was only added a couple of releases ago [17:53] ah, found it, my mistake "include file" [17:53] https://cloudinit.readthedocs.io/en/22.1_a/topics/format.html#include-file [17:54] yes I found it, the search didn't pick it up [17:54] so omitting s= might be an undocumented feature but #include on nocloud is vapourware. [17:55] I've never used include files, only cloud-config files [17:58] I don't think nocloud-net is used by many people, I've already found some issues with it myself [18:01] Yeah, I think the tool is called "cloud-init" because it's meant for cloud platforms, I think using it for PCs is a bit of an edge case. [18:02] well I do use it myself for physical machines, though in my case I've already prepared the disk image for things like LVM, LUKS, etc, and I'm (currently) using a very small partition for the YAML config files [18:02] plus OVH are using it for some of their bare-metal provisioning [18:03] I'm using to replace preseed. [18:04] also that's what Ubuntu's curtin/subiquity installer based is AFAIK [18:06] When Ubuntu stopped letting me inject preseed into Ubuntu Desktop installations, that was for me a disaster because suddenly this fire-and-forget, can get anyone with the BIOS password to do it task became manual and error-prone and we had laptops going out without full disk encryption, root passwords miskeyed, all sorts of problems. [18:07] haven't used preseed for some time as I'm mainly Alpine these days [18:07] I'm at a university, we have a research and teaching need for Ubuntu on the desk, but also we have to comply with GDPR. [18:08] well pretty much every org (in UK/EU) "has" to comply with GDPR but many/most doesn't know or don't care :-( [18:09] Interestingly, if I omit cloud-config-url and just use ds=nocloud\;s=myurl, cloud-init will not even download the user-data file. [18:09] if you're in the UK don't worry the ICO is pretty much unlikely to do anything no matter how badly you breach UK GDPR :-( [18:10] Our chief information security officer takes it quite seriously. [18:10] "ds=nocloud\;s=myurl" isn't correct for 22.1, it would be "ds=nocloud-net;s=url" (yeah I noticed issues with ";" on cmdline in the past)) [18:11] wow, must be one of only a handful of people in UK who care (speaking from experience unfortunately) [18:12] on 22.1 "ds=nocloud" is for file-based seeds, "ds=nocloud-net" is for network-based seeds [18:12] he's fighting an uphill battle with the academics, who really don't care. [18:14] I spent 2+ years highlighting large scale breach of special category personal data by 2 local government bodies, ICO didn't care, investigated little of the matter and despite even then finding the orgs had breached GDPR and previous law they decided to take no actions :-( [18:14] That's universal though, academics dislike anything that interferes with their research. [18:16] Can data subjects sue? [18:16] I'm about to spend a large amount of personal money taking the 2 orgs to court myself [18:17] but think I can only take action about their handling of *my* personal data and not the same (ongoing) mishandling of almost 2 million other peoples' data [18:17] Class action? [18:18] anyway talking about GDPR lack of compliance is depressing :-( [18:18] Problem is, not enough data subjects care. [18:19] that's exactly the issue - class action would require finding how many other people? part of ICO's failure to act on my complaints was "well no one else has complained about this" [18:19] If you have a crowdjustice fundraiser for your case, DM me the link. [18:19] Let's instead talk about a willful misinterpretation if GDPR to mean: We can't share any data, ever. (Germany) [18:20] XD [18:21] threepistons: DM? [18:21] meena: including "we can't share any data about how we can't share any data"? ;-) [18:22] The ${thing} are set at the top of the menu file. `imgargs ${mykernel} initrd=${myinitrd} url=${isourl} autoinstall ds=nocloud-net\;seedfrom=http://${nfs-host}/preseed/ubuntu/22.04/laptop-sda ip=dhcp net.ifnames=0` does not work. [18:22] minimal: i lied. to share data in Germany, all you need is: 📠 [18:22] meena: what's that emoji? [18:23] a fax [18:23] pocket calculator? [18:23] Ah. [18:23] I've not seen one of those in about ten years. [18:23] meena: its been a few years since I left Germany, have they still not phased out faxes? lol [18:23] nope [18:24] They are so secure, not. They spew out the pages with no assurance as to who will see them. [18:25] despite the fact that some people have caught on to the fact that these days faxes are sent via email, you still can't just use email, but you can still use a fax [18:25] have you checked "dmesg" do see that the cmdline values passed from kernel on to initramfs etc is correct, I found in past the "ds=" value was truncated at semicolon unless I quoted the whole "ds=" section [18:31] trying quotes now. [18:31] DM = direct message [18:32] yes [18:32] yeah, was trying to direct message you [18:33] I just spotted that. [18:33] Putting quotes on ds= did not help. [18:34] did "dmesg" show the cmdline looking correct? [18:34] when you say check dmesg, you mean "go behind the Help of the locale menu and launch a terminal"? [18:37] it has the ; there and the full argument. I replaced s with seedfrom for readabiity, will change that back. [18:38] on any machine that has booted if I run "dmesg" then the output includes the likes of: "Command line: BOOT_IMAGE=......." [18:38] which shows the cmdline the kernel got from the bootloader [18:39] when using nocloud-net in the past I has the problem that the cmdline passed to kernel (and therefore to initramfs and full OS) was being truncated that the semicolon in the "ds=nocloud-net;s=..." section [18:39] yes, it's more that I was checking that "from the locale menu in the middle of the by-hand installer wizard that the installer dropped into" was where you meant me to run dmesg from [18:44] it dropped into the wizard again and says the full ds=nocloud;s=http:... in quotes [18:44] hmm. [18:45] I put s= and url= for a reason, probably that the one alone was not working. [18:46] it's been about 2 months since I last looked at/tried to debug nocloud-net stuff [18:48] git log says I had issues and added url= to get around s= not working. [18:49] never tried that, but also not tried PXE either to-date [19:08] It wouldn't be the first time that I've had a globally-unique use case. [19:08] How about Puppet-managing 312 Raspberry Pis? [19:09] I use cloud-init for RPIs initial bootstrapping (with prepared SDcard OS image) and then Ansible for any further config [19:10] Not so unique after all then. [19:11] having finished the full arch - which is where cloud-init's phone-home support is used to sign to central provisioning server [19:11] s/having/haven't/ [19:11] threepistons: BTW when is a Joint Data Controller not a Joint Data Controller? [19:12] When the JDC has a really good lawyer who can shift the blame onto the other DC? [19:15] when the alleged Joint Data Controllers (the vast majority of them) have never agreed to or signed the Data Sharing Agreement that defines the Joint DC relationship between the 600+ orgs for the sharing yet they've been sharing personal allegedly in the JDC relationship with those other orgs for 10 years... [19:15] The fun part will be updating the OS on them this summer. The Pis are fixed to the desks and getting them off to replace the SD cards requires dismantling the desks. Or I can write a new image over the network, but I get one chance to get that right. These are 3Bs, they don't support PXE without some weirdness. [19:16] So how did they acquire the data without someone breaking the law? [19:16] yeah, I remember the fun of network booting RPIs (I was part of the original beta trial way back when) [19:17] Guessing you're also in academia. [19:46] s= doesn't work ,cloud-config-data= does work. I suspect that the docs are wrong.