| Rusty_Almighty | So, to define the question, how do I launch the installer from IPXE with an answer file that was generated and being served from a web address? | 05:26 |
|---|---|---|
| Rusty_Almighty | Example: set params root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://${MYIP}/ubuntu-20.04-live-server-amd64.iso autoinstall ds=nocloud-net;s=http://${MYIP}/ubuntu-yaml.php | 05:26 |
| Rusty_Almighty | the nocloud-net data source is expecting a folder and very specific file names to be in their respective file locations. Is there a way to provide the meta-data and the user-data files as a single .yaml file provided by a single web page? | 05:28 |
| meena | Rusty_Almighty: 🤷🏻♀️ | 07:30 |
| meena | that sounds like a NoCloud specific question, so maybe go looking there for answers | 07:31 |
| meena | it does seem like NoCloud is one of the simpler datasourcres | 07:31 |
| Rusty_Almighty | Is there a specific channel for only the nocloud data source? | 07:36 |
| Rusty_Almighty | Also, is there another source that will allow me to doe what I am talking about? | 07:37 |
| meena | https://pypi.org/project/cloud-init-seed/ this converts an existing image to one where the NoCloud seed is already mounted | 08:24 |
| meena | https://github.com/rmb938/vmw-cloudinit-metadata/ this could be what you're looking for, Rusty_Almighty | 08:28 |
| meena | Although it claims that it's VMware specific | 08:29 |
| meena | you might be able to extract some essence | 08:29 |
| meena | https://gist.github.com/smoser/635897f845f7cb56c0a7ac3018a4f476 this example by our own smoser mounts the Filesystem | 08:33 |
| meena | aaaaand it also says that NoCloud's "upstream" is *cloud-init* | 08:33 |
| meena | now that's pretty embarrassing | 08:33 |
| meena | we really should have a simple Metadaten | 08:34 |
| meena | *metadata Server | 08:34 |
| Rusty_Almighty | I would agree. Looking through the source code there doesn't seem to be any way to define just a single answer file in any of the source providers. | 08:47 |
| Rusty_Almighty | https://github.com/canonical/cloud-init/blob/ubuntu/daily/focal/cloudinit/sources/DataSourceNoCloud.py | 08:47 |
| otubo | langjv, interesting. I'll make sure we document this vmware_extra_config='{"smbios.assetTag":"OpenTelekomCloud"}' on our knowledge base. Or, if it doesn't interfere on other clouds we could push it upstream as well. What do you think? | 12:00 |
| adgud | Hey, how do I regenerate network config (for netplan)? Let's say I changed IP address in cloud-init source file and I would like this change to be applied. | 12:51 |
| toolz | I guess netplan apply ? | 12:53 |
| adgud | But the netplan config file is not updated yet - I should mention that I'm using Proxmox and I changed cloud-init IP address in GUI | 12:54 |
| paride | hey rharper, on https://github.com/canonical/cloud-init/pull/482: I also noticed that sometimes images are exported as a single .tar.gz file. I think it depends in how the images was imported in the first place, and that doesn't seem to happen with imaged imported using the ubuntu: or images: remotes | 13:10 |
| paride | the cloud_tests were limited on squashfs + tar.xz exports even before | 13:10 |
| paride | do we need to support "combined" .tar.gz exports in cloud_tests? | 13:11 |
| paride | rharper, in any case not knowing what is going to be exported and guess is not nice | 13:25 |
| paride | I'm asking the LXD people | 13:26 |
| Odd_Bloke | meena: cloud-init doesn't, but Ubuntu ships all sorts of simple metadata servers: apache2 and nginx are the two most commonly used. ;) | 13:40 |
| Odd_Bloke | Rusty_Almighty: This isn't exactly what you were asking for, but if you used something like `ds=nocloud-net;s=http://${MYIP}/ubuntu-yaml.php?document=` then you should get multiple requests to /ubuntu-yaml.php with different query data each time. | 13:43 |
| Odd_Bloke | (I'm looking to see if we already have a bug for all-in-one.) | 13:43 |
| Odd_Bloke | Rusty_Almighty: And I can't find such a bug, so please feel free to use the link in the /topic to file such a feature request. :) | 13:50 |
| Odd_Bloke | otubo: We shouldn't be using `OpenTelekomCloud` anywhere other than https://open-telekom-cloud.com/en; can we identify why that string works and check if there's an alternative that isn't tied to a specific vendor's platform? | 13:55 |
| Odd_Bloke | (Specifically, we could potentially land changes to implement behaviour specific to OTC, which might then start being applied to people's VMWare VMs; it's unlikely that this is their desired behaviour. :) | 13:56 |
| rharper | paride: thanks; yeah; there's a real lack of formatting via the cli; | 14:03 |
| smoser | Rusty_Almighty: well, the "seed" was essentially intended to be "a single answer file" per source | 14:05 |
| smoser | oh. i see the question Rusty_Almighty was asking. there is no way to provide both user-data and meta-data as a single file. i can see that might be useful, but its not really a big limitation. files/urls are "free". 2 is not really any harder to serve than 1. | 14:07 |
| Rusty_Almighty | But I would at least need to be able to control the name for each file as a PHP with separate variables so that the file can be generated. | 14:09 |
| smoser | https://github.com/rmb938/vmw-cloudinit-metadata/ is interesting. | 14:09 |
| Rusty_Almighty | I would also need to make changes to my web server to forward certain url's to php files which breaks the paradigm a little bit. | 14:11 |
| smoser | Rusty_Almighty: it isn't that hard. | 14:12 |
| smoser | cloud-init 'seed' of some url. and then cloud-init just appends 'user-data' and 'meta-data' to that url | 14:12 |
| smoser | php or any other web server can look at the url and use it. | 14:12 |
| smoser | seed=http://my.address/instance/i-<your-id-here/ | 14:13 |
| smoser | i dont know in php how to have one php file take all requests "under" a certain path. | 14:14 |
| smoser | but certainly you could do: | 14:14 |
| smoser | seed=http://my.address/instance/server.php?id=<your-id-here>&name= | 14:14 |
| smoser | and cloud-init would just append 'user-data' and 'meta-data' | 14:14 |
| smoser | then just look at the 'name' variable in php | 14:15 |
| Rusty_Almighty | I'm not saying it couldn't. I'm just saying that the 53 other automatic installing operating systems that my ipxe server installs don't . However, I'm liking this idea where we hack around the issue by providing a blank "document=" at the end of the get statement. | 14:16 |
| Rusty_Almighty | 23* | 14:16 |
| === vrubiolo1 is now known as vrubiolo | ||
| smoser | fwiw, its not a hack. | 14:19 |
| smoser | cloud-init is doing string concatenation . urls are just strings. | 14:19 |
| smoser | urls no longer have to map 1:1 to a file in a unix filesystem. thats a good thing. | 14:21 |
| Odd_Bloke | smoser: Having support for a single file would simplify things IMO; if people are just writing files to disk then then gain isn't massive, but if people are using some sort of framework (or PHP file) then _requiring_ two GETs makes things more complex than less. | 14:24 |
| Odd_Bloke | Because you can't just implement a single endpoint which returns YAML, you have to either implement separate endpoints or implement logic based on query strings. | 14:26 |
| Rusty_Almighty | Apologies. pidgin on Ubuntu keeps crashing. | 14:26 |
| meena | Odd_Bloke: did you know at i'm an apache httpd committer? | 14:27 |
| Rusty_Almighty | What was that last comment about making a single request? | 14:27 |
| Odd_Bloke | meena: I did not! | 14:27 |
| meena | (i haven't contributed in ages, because the ASF is quite exhausting, but, yeah) | 14:27 |
| Odd_Bloke | Rusty_Almighty: "Because you can't just implement a single endpoint which returns YAML, you have to either implement separate endpoints or implement logic based on query strings." | 14:28 |
| Rusty_Almighty | Some one was saying that it would be easier to make a single web request. | 14:33 |
| Odd_Bloke | Oh right, "Having support for a single file would simplify things IMO; if people are just writing files to disk then then gain isn't massive, but if people are using some sort of framework (or PHP file) then _requiring_ two GETs makes things more complex than less." | 14:33 |
| Rusty_Almighty | Odd_Bloke: Yes. that would simplify things greatly in my specific case. | 14:35 |
| smoser | Odd_Bloke: "single endpoint" don't know what you mean. | 14:36 |
| smoser | this isn't 1999. | 14:36 |
| smoser | s3 is a "single endpoint" | 14:37 |
| Odd_Bloke | OK, well, I'm clearly not using it in that sense? | 14:37 |
| Odd_Bloke | s/endpoint/route/ if that helps. | 14:37 |
| smoser | baked into your thoughts is the idea that '/' means something. | 14:38 |
| smoser | its jsut a character in a string. | 14:38 |
| Odd_Bloke | What? | 14:38 |
| Odd_Bloke | Does the current implementation require 2 GETs or can it only make 1 GET? | 14:38 |
| smoser | you're right about 2 GETs. | 14:39 |
| smoser | and that *is* simpler. and then also "atomic" | 14:39 |
| smoser | so i'm not opposed to it. | 14:40 |
| smoser | but, fwiw, no cloud that I know has chosen to implement things that way. | 14:40 |
| smoser | ie, ec2 metadata, openstack metadata, hetzner ... they're all multiple GETs | 14:41 |
| Odd_Bloke | Oracle has a separate GET for network metadata, but you can get metadata/user-data all as part of one GET from their new APIs. | 14:42 |
| Odd_Bloke | But I agree that it isn't a general pattern. | 14:42 |
| smoser | you'd have to send back in the resonpse some indication to cloud-init that this is a "nocloud combined" response. | 14:43 |
| Odd_Bloke | But if I had to speculate, I'd say that isn't because that's what's best for the guest, it's because organisationally exposing all of the data from multiple different systems run by different orgs via a single GET is Complicated (TM). | 14:43 |
| smoser | well.. i dont know. | 14:43 |
| Odd_Bloke | Alternatively we could introduce a new `seedcombinedfrom=` or whatever, so you explicitly opt into the combined behaviour. | 14:43 |
| smoser | ultimately its a "single endpoint" | 14:44 |
| smoser | and you can either combine all the things when the client requests them individually or all at once. | 14:44 |
| smoser | value in separate is less to deliver now. | 14:45 |
| smoser | anyway | 14:45 |
| smoser | so i'm not opposed to single | 14:45 |
| smoser | i'd like for it to work with directory also. such that you could /var/lib/cloud/seed/nocloud/<single-file> | 14:46 |
| smoser | your suggestion for combined 'seedfrom' means that we don't have to "go fishing" | 14:47 |
| smoser | and see if there is a /<single-file> and then <user-data>/<meta-data> | 14:47 |
| smoser | antoher option would be to justs have cloud-init check 'meta-data' for a given format. | 14:48 |
| smoser | if it starts with '#nocloud-v2' or if yaml/json has 'type: nocloud' 'version: 2' ot something | 14:49 |
| Rusty_Almighty | 2 requests are not impossible, maybe if we just had a way to provide what the file names were by allowing more keys to be passed to the data source. eg. s=http://my-ip/some/folder/;m=meta-data;u=user-data | 15:17 |
| meena | how did we go from providing a NoCloud dataserver that transforms one YAML file (per server) into the correct NoCloud format…to this? | 15:18 |
| Rusty_Almighty | You would think that something like this would be required for localization. I'm pretty sure that some of the poor Russians out there have no idea what meta-data means. | 15:18 |
| meena | or did i misunderstand what Rusty_Almighty was talking about to begin with? | 15:18 |
| Rusty_Almighty | Or maybe I'm just stating to speak gibberish because I have only slept 4 hours for the past 48. | 15:19 |
| Rusty_Almighty | meena: Where is the best place to file a bug for the cloud-init project | 15:20 |
| Rusty_Almighty | ? | 15:20 |
| meena | Rusty_Almighty: in the /topic | 15:21 |
| meena | pull-requests https://git.io/JeVed | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting July 14 16:15 UTC | 20.1 (Feb 18) | 20.2 (Apr 28) | https://bugs.launchpad.net/cloud-init/+filebug | 15:32 |
| Rusty_Almighty | Thanks. I got it from the topic. Filing now. | 15:34 |
| Rusty_Almighty | The way I see this is that this is 2 distinct problems. So maybe I should file 2 bugs. | 15:38 |
| Rusty_Almighty | Problem 1. the file names "user-data" and "meta-data" are hard coded and cannot be changed. | 15:51 |
| Rusty_Almighty | Problem 2. the installer is forced to make 2 seperate web requests which is not always very simple. | 15:52 |
| smoser | "installer" is odd. | 15:53 |
| smoser | that is confusing | 15:53 |
| Rusty_Almighty | smoser: what would you call it? | 15:53 |
| Odd_Bloke | Rusty_Almighty: cloud-init operates in already-installed generic images, to specialise them for this particular boot; something else has to do the "install" (e.g. subiquity, debootstrap, the Ubuntu cloud image build process) and include cloud-init before it can run. | 16:03 |
| Rusty_Almighty | I filed it as a single bug, but would some one do me the pleasure of reading through my bug to make sure that we've documented everything we just discussed since I'm operating at compromised mental capacity at the moment? | 16:19 |
| Rusty_Almighty | https://bugs.launchpad.net/cloud-init/+bug/1886846 | 16:19 |
| ubot5 | Ubuntu bug 1886846 in cloud-init "The nocloud-net data source requires a folder forcing 2 seperate web requests to hard coded files" [Undecided,New] | 16:19 |
| smoser | Rusty_Almighty: it doesn't "require a folder" | 16:23 |
| smoser | it does require 2 GETs | 16:23 |
| Rusty_Almighty | Well, it does require an open ended url which is folder like in nature. | 16:24 |
| Rusty_Almighty | smoser: how would you word it? | 16:24 |
| Rusty_Almighty | Also, if the data source is just plain old nocloud, then the URL is a folder. | 16:25 |
| Odd_Bloke | Rusty_Almighty: It doesn't require a folder because the paths are determined by appending a string, not by path/URL "join"ing; so if the path you specify is /my/metadata- then cloud-init will read /my/metadata-user-data and /my/metadata-meta-data; you aren't forced to use a directory/folder (except insofar as most filesystems require files to be in directories ;). | 16:34 |
| Odd_Bloke | Similarly, if you specified `http://example.com/metadata?type=` then your two GETs would be to `http://example.com/metadata?type=user-data` and `http://example.com/metadata?type=meta-data`. | 16:35 |
| Odd_Bloke | (I may be misremembering and getting the hyphenation of user-data and meta-data here completely wrong; apologies if so!) | 16:36 |
| Rusty_Almighty | Renaming folder to "folder like string" then? | 16:38 |
| Rusty_Almighty | "The nocloud-net data source requires a folder like string forcing 2 seperate requests to hard coded locations" | 16:40 |
| smoser | just drop "folder". requires 2 separate requests. | 17:06 |
| smoser | http has no explicit relation to folders. | 17:07 |
| smoser | this is surprising to me | 17:54 |
| smoser | anyone else? | 17:54 |
| smoser | $ python3 -c 'import io; buf = io.StringIO("he | 17:54 |
| smoser | llo "); buf.write("world"); print(buf.getvalue())' | 17:54 |
| smoser | bah. paste fail. | 17:55 |
| smoser | $ python3 -c 'import io; buf = io.StringIO("hello "); buf.write("world"); print(buf.getvalue())'world | 17:55 |
| smoser | again. | 17:55 |
| smoser | https://paste.ubuntu.com/p/Sz6t7hNsHk/ | 17:57 |
| smoser | there. that seems more sane. | 17:57 |
| smoser | sane paste. i still think that is very odd behavior. | 17:57 |
| Odd_Bloke | The docs do say "The stream is positioned at the start of the buffer." | 17:57 |
| Odd_Bloke | So I think you'd need to do `buf.seek(0, whence=os.SEEK_END)`? | 17:59 |
| Odd_Bloke | (I agree it's unintuitive!) | 17:59 |
| Rusty_Almighty | I believe the seek has to happen before you write. | 18:07 |
| Rusty_Almighty | IE. you sert the buffer to hello and even though you did, the carrot is at 0 still. So when you write world, it overwrites hello. | 18:08 |
| Rusty_Almighty | https://paste.ubuntu.com/p/MVD8VsvyD2/ | 18:10 |
| Rusty_Almighty | Odd_Bloke is correct. "buf.seek(0, os.SEEK_END)" before writing will get you what you want. | 18:16 |
| meena | smoser: what happens when you flush? (at the end? at the beginning?) | 18:44 |
| Rusty_Almighty | meena: flushing at the end or at the beginning doesn't seem to affect the carrot. | 18:51 |
| Rusty_Almighty | Carrot stays the same. | 18:52 |
| smoser | it makes sense if you think of io.StringIO("foo") as open(file, "rw") where 'file' already had "foo" inside it. | 18:55 |
| Rusty_Almighty | sigh.. more problems... | 19:04 |
| Rusty_Almighty | https://paste.ubuntu.com/p/NM8z5KXjVW/ | 19:04 |
| Rusty_Almighty | There is also no logs from my web server that the URL was accessed. | 19:06 |
| === jdoss_ is now known as jdoss | ||
| Odd_Bloke | Yeah, that message means that it didn't identify it as a valid URI to even attempt. | 20:26 |
| Odd_Bloke | I'm not sure why. | 20:26 |
| Odd_Bloke | Rusty_Almighty: What was your kernel cmdline? | 20:28 |
| Odd_Bloke | Rusty_Almighty: I'm leaving for today (and off tomorrow), so I'm afraid this will be my last contribution: my only guess is that you've specified nocloud not nocloud-net, which changes the prefixes that it would consider valid. | 20:32 |
| Odd_Bloke | (I think dsmode=net is a red herring there, but maybe it isn't and my guess is wrong! :) | 20:32 |
| Rusty_Almighty | Sorry. Filing another bug with Debian. FIX ALL THE THINGS. | 20:48 |
| Rusty_Almighty | set theKernelParams root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=${base}/ubuntu-20.04-live-server-amd64.iso autoinstall ds=nocloud-net;s=http://ipxe.localdomain/ubuntuks.php?ip=${ip}&version=${UBUNTUVERSION}&doctype= | 20:48 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!