| === chris14_ is now known as chris14 | ||
| === chris14_ is now known as chris14 | ||
| === justache is now known as justache_Test | ||
| === justache_Test is now known as justache | ||
| === JanC is now known as Guest8325 | ||
| === JanC_ is now known as JanC | ||
| === kostkon_ is now known as kostkon | ||
| === esv_ is now known as esv | ||
| === JanC is now known as Guest1241 | ||
| === xispita_ is now known as xispita | ||
| === vince_ is now known as vince | ||
| MTecknology | dbungert: I heard a rumor that maybe you know some things about the autoinstall stuff? I still can't figure out how to make the installer grab an autoinstall file. | 16:24 |
|---|---|---|
| dbungert | MTecknology: I do. Thanks for sharing your autoinstall in progress. | 16:25 |
| dbungert | I'm in a meeting for the moment so my responses might be slow | 16:25 |
| dbungert | the two general plans are to build the autoinstall into the ISO or to serve it over cloud-init, which one sounds better for your case? | 16:25 |
| MTecknology | I've been at this for a couple weeks, so slow is still faster than me. | 16:26 |
| p3lim | MTecknology: I use kernel args with nocloud, e.g: `linux /casper/vmlinuz autoinstall ds=nocloud\;s=http://myhost/autoinstall.yaml ---` | 16:27 |
| p3lim | if you've packaged the autoinstall file with the ISO or have a separate disk you can get it from that instead, e.g. `s=/cdrom/mypath/autoinstall.yaml` | 16:28 |
| MTecknology | My original goal was to bake it into the iso, but I also need to support remote re-installation so I'm trying to use those same "nocloud" arguments. I thought that the thing needed to end in a / so that it could find the user-data and meta-data files and then read the autoinstall data from the cloud-init config file | 16:29 |
| p3lim | docs: https://cloudinit.readthedocs.io/en/17.2/topics/datasources/nocloud.html | 16:29 |
| p3lim | I don't use separate user-/meta-data that way so can't answer that for you | 16:29 |
| MTecknology | That's what's shown on https://ubuntu.com/server/docs/install/autoinstall and https://canonical-subiquity.readthedocs-hosted.com/en/latest/howto/autoinstall-quickstart.html#write-your-autoinstall-configuration | 16:31 |
| p3lim | question: how can I validate the autoinstall/curtin match targets for storage? trying to match by 'model', which should be ID_VENDOR but it just fails to match one | 16:31 |
| * MTecknology goes off and tries with a full path to that specific file. | 16:32 | |
| p3lim | debug output doesn't really help me here, so just wondering if there are any tools to validate/match that | 16:32 |
| MTecknology | The attempt I just tried -> kvm -no-reboot -m 2048 -drive file=image.img,format=raw,cache=none,if=virtio -kernel linux -initrd initrd.gz -append 'autoinstall ds=nocloud-net\;s=https://deb.domain.tld/ubuntu2004/autoinstall.yaml' | 16:34 |
| p3lim | MTecknology: don't do it on initrd, do it on linux | 16:35 |
| MTecknology | "on initrd"? | 16:36 |
| p3lim | (not too familiar with raw kvm parameters) | 16:36 |
| p3lim | I assume `-append` appends it to `-initrd` here? | 16:36 |
| MTecknology | nope, it's the appended arguments in grub | 16:37 |
| MTecknology | well, not "in grub", but what's handed to the kernel | 16:37 |
| MTecknology | I can see the arguments listed in the kernel command line, but they don't appear to have any impact on the installer. | 16:38 |
| p3lim | e.g. you see them in /proc/cmdline? | 16:38 |
| MTecknology | I also can't find any logs to suggest any web activity was attempted. | 16:38 |
| MTecknology | yep | 16:38 |
| p3lim | show us exactly what /proc/cmdline contains please | 16:38 |
| p3lim | mask your s= if you want :P | 16:40 |
| MTecknology | heh ... I don't know how to take a screenshot from this. It's "autoinstall ds=nocloud-net;s=https://deb.corp.tld/ubuntu2004/user-data" where "corp.tld" is a typed replacement. Everything else is char4char. | 16:41 |
| p3lim | I made a mistake in my paths above, I had some part of the ISO builder that did something, I'll explain it here: | 16:42 |
| p3lim | my /proc/cmdline contains "BOOT_IMAGE=/casper/vmlinuz autoinstall ds=nocloud;s=/cdrom/nocloud/foo ---" | 16:43 |
| p3lim | within /cdrom/nocloud/foo directory I have two files; meta-data and user-data | 16:43 |
| p3lim | user-data is the autoinstall.yaml file | 16:43 |
| p3lim | in your case you should point to https://deb.domain.tld/ubuntu/2004 which contains those two files | 16:45 |
| dbungert | ^ | 16:45 |
| dbungert | was just about to say the same, thanks p3lim | 16:45 |
| MTecknology | That's what I was trying first, before you mentioned autoinstall.yaml ;) | 16:45 |
| p3lim | also keep in mind that https _might_ be problematic depending on how you've signed the certificate on that domain | 16:45 |
| MTecknology | > I thought that the thing needed to end in a / so that it could find the user-data and meta-data files and then read the autoinstall data from the cloud-init config file | 16:46 |
| dbungert | https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html#nocloud - the examples are showing a trailing slash, I don't know if it matters | 16:47 |
| p3lim | you might need to end your url/path with "/" indeed, I know I've done that before | 16:47 |
| MTecknology | There's a spot in documentation somewhere that claims the trailing slash must be there | 16:47 |
| p3lim | use it to be safe :) | 16:47 |
| MTecknology | I have been | 16:48 |
| p3lim | dbungert: do you know how I can verify/test the "match" keypair from curtin without running the entire autoinstall process? can't seem to get a match against "model" (which is supposed to be ID_VENDOR in udev according to the docs) | 16:48 |
| MTecknology | I guess ... I'm sharing the arguments that are absolutely definitely being passed to the installer, so the real question is why the installer seems to do nothing with that. | 16:49 |
| MTecknology | I'm not sure if it's trying and not being logged, but netstat suggests it's not being tried at all | 16:49 |
| MTecknology | tcpdump* | 16:49 |
| p3lim | are you actually using this for 20.04? | 16:50 |
| MTecknology | yep ... I get to target 18.04, 20.04, 22.04, and there's soon to be a fourth in that list. | 16:50 |
| dbungert | p3lim: I'm not sure we have a great way to verify match directives, that sounds like a nice feature improvement. I'm jotting that down as a feature request. | 16:51 |
| p3lim | dbungert: you're on the curtin team? | 16:51 |
| dbungert | I own the Subiquity stack, so all of Subiquity/Curtin/Probert, but I'm not the sole decision maker | 16:52 |
| p3lim | ah cool | 16:52 |
| p3lim | please jot down a feature request for configuring btrfs subvolumes too then :D | 16:52 |
| p3lim | anyways, I cannot for the life of me tell what curtin is actually trying to match against, because it just fails whenever I supply the `match.model: foo` field | 16:53 |
| dbungert | MTecknology: if you're testing newer ISOs, there will be a log line in /var/log/installer/subiquity-server-debug.log load_autoinstall_config | 16:53 |
| dbungert | if subiquity sees the autoinstall, that will have useful information, or maybe just a 'None' if it didn't make it that far | 16:54 |
| p3lim | to be specific, I'm using Dell EMC hardware and ID_VENDOR is "DELL " and I've set `match.model: DELL*` (tried without the glob too) and it doesn't work | 16:54 |
| MTecknology | there's no /var/log/installer on this one | 16:54 |
| dbungert | p3lim: I actually have a branch started but it's not a committed feature at this point | 16:54 |
| p3lim | cool :) | 16:54 |
| dbungert | p3lim: (with respect to btrfs and subvolumes) | 16:55 |
| p3lim | if there are steps I can take to test the btrfs branch let me know, would gladly help once it's kinda-ready for that | 16:55 |
| * MTecknology screams ... this is a horrible mess that is driving me absolutely bonkers. How the frick is it so dang difficult just to feed a configuration file to an installer?!! | 16:55 | |
| p3lim | MTecknology: try using http instead of https | 16:56 |
| p3lim | certificates can pose a problem here | 16:56 |
| p3lim | and ensure it can actually resolve the address (or just use IP) | 16:56 |
| MTecknology | It's not making a web request ... cart in front of the horse | 16:56 |
| p3lim | can you wget/curl the file from the installation image? | 16:56 |
| MTecknology | yep, I *just* tried that, actually. | 16:57 |
| p3lim | 🤷 | 16:57 |
| dbungert | MTecknology: ok, so do you have /var/log/cloud-init.log ? That should have information on the attempt to process the nocloud-net datasource | 16:57 |
| MTecknology | heh.. there's also no /var/log/cloud-init.log. In fact, the only thing at all in /var/log is syslog | 16:59 |
| dbungert | that one is beyond me, something very wrong is going on in the ephemeral install environment | 17:02 |
| dbungert | MTecknology: what install ISO are you looking at? | 17:02 |
| MTecknology | I'm using the ubuntu-server-20.04 iso to try to install a desktop of the same version | 17:03 |
| dbungert | MTecknology: which one? 20.04.6? | 17:03 |
| MTecknology | yep, ubuntu-20.04.6-live-server-amd64.iso / sho256: b8f31413336b9393ad5d8ef0282717b2ab19f007df2e9ed5196c13d8f9153c8b | 17:04 |
| MTecknology | sha* | 17:04 |
| p3lim | super weird | 17:06 |
| p3lim | I just tested that exact same image in libvirt just now, although using localpath, worked just fine | 17:07 |
| p3lim | dbungert: has there been any thought of simplifying the build tooling for ISOs? I currently have 300+ lines mess of a bash script around xorriso, and I feel like it could break at any second :( | 17:09 |
| p3lim | feel free to finish helping MT before answering that | 17:09 |
| MTecknology | lol ... I think canonical is trying to abstract the abstraction as much as abstractly possible | 17:09 |
| MTecknology | I get the impression any attempts to help me have been exhausted without having control of my keyboard. I'm back to the mercy of documentation ... and soon to be comments. | 17:12 |
| p3lim | I still think it's your cmdline being incorrect :/ | 17:13 |
| p3lim | considering the info provided | 17:13 |
| p3lim | have you tried booting the ISO and manually typing in the cmdline in interactive grub? | 17:14 |
| MTecknology | I assumed the backslash was specifically for grub escaping, so by the time the system is booting, the "\;" will be escaped to a ; | 17:14 |
| MTecknology | If it is truly supposed to remain a \; into cmdline, then I have only accomplished this once. | 17:14 |
| p3lim | it is, see the above message from me where I dumped my /proc/cmdline | 17:14 |
| p3lim | it's supposed to be just ";" between ds=nocloud and s= | 17:14 |
| dbungert | MTecknology: where did the kernel and initrd come from? It's best to match them to the one from the iso, and can be found at casper/{vmlinuz,initrd} | 17:15 |
| MTecknology | dbungert: I grabbed them from the iso, per some other instructions that I linked to but can't find again | 17:15 |
| dbungert | ok. I think at this point you need to understand why the install environment seems to not be coming up normally. journalctl may have errors. | 17:16 |
| dbungert | p3lim: ubuntu-image has that as a goal, but it's not supporting ISO creation yet | 17:19 |
| p3lim | so the official images is also built using a messy bash script? :D | 17:20 |
| dbungert | you may or may not be interested in https://pad.lv/c/livecd-rootfs, p3lim | 17:27 |
| dbungert | but I think you already know the answer | 17:28 |
| * MTecknology is curious what's behind that link | 17:28 | |
| p3lim | I just repackage the ISO with the nocloud files | 17:30 |
| p3lim | can't host anything in our prov environment | 17:30 |
| dbungert | p3lim: don't worry about it if you have a working solution, I'm just pointing to some of the build software in question | 17:32 |
| p3lim | it's working now, might not work tomorrow :D | 17:32 |
| p3lim | that's why I'm asking :) | 17:32 |
| p3lim | once it breaks I'll just see if either ubuntu-image has been improved, or look at the actual tooling ya'll use | 17:33 |
| MTecknology | p3lim: If you can tolerate static images, live-build is pretty nifty | 17:34 |
| dbungert | MTecknology: I need to be afk, you're welcome to ping or leave comments here and I'll check later | 17:35 |
| MTecknology | dbungert: Thanks; It occurred to me that I'm probably trying to rely too heavily on inaccurate and missing documentation, and it appears to not exist because it's always changing | 17:40 |
| p3lim | no wonder I can't get anything to work | 17:41 |
| p3lim | the docs are incorrect | 17:42 |
| p3lim | in the docs, disk selection extensions says model=ID_VENDOR, but it's actually ID_MODEL | 17:43 |
| p3lim | https://github.com/canonical/subiquity/blob/server/jammy/subiquity/models/filesystem.py#L999-L1001 | 17:43 |
| p3lim | vendor=ID_VENDOR isn't even documented | 17:44 |
| MTecknology | Did I just ... are rotational and ssd the only options? what about tape? :P | 17:46 |
| MTecknology | s/Did I just .../the answer was no and I was going to delete it | 17:46 |
| MTecknology | ya know ... I /do/ have ipxe readily available. I was using this earlier and it provided absolute proof of network connectivity, since that's where the two payloads came from. It just means I also have to go back inside because I don't want to make a network cable long enough for me to play with that outisde. :/ | 17:50 |
| MTecknology | #ITproblems :P | 17:51 |
| MTecknology | does ubuntu not have build-essential? | 17:54 |
| MTecknology | oooooh, lol ... there's no sources.list present. That must be one of the final setup things offloaded to cloud-init on the first boot. | 17:56 |
| MTecknology | dbungert: Is there any chance you can make this part easy for me and tell me what things cloud-init is actually responsible for on the first startup? So far it seems to be sources.list, one of the lists of package installs, and the first user | 17:57 |
| dbungert | p3lim: new subiquity stable snaps are "backported" to newer ISO builds, so that jammy branch is relevant for 22.04.0 ISOs but https://github.com/canonical/subiquity/blob/ubuntu/mantic/subiquity/models/filesystem.py#L1533 would be relevant instead for 22.04.4 | 18:09 |
| p3lim | good to know, but docs are incorrect regardless | 18:12 |
| p3lim | I've reported the docs issue on launchpad | 18:12 |
| dbungert | MTecknology: user creation does happen at first boot of the target system. the `packages` directive of autoinstall is the same. /etc/apt/sources.list really should be present. It might be interesting to add `while [ ! -f /run/finish ] ; do sleep 1 ; done` to autoinstall.late-commands, which will halt the install just before reboot to the target system and let you poke around to see before | 18:20 |
| dbungert | first-boot stuff runs. | 18:20 |
| MTecknology | Can I provide that (autoinstall.late-commands) as a boot argument? | 18:22 |
| dbungert | no, unfortunately | 18:23 |
| MTecknology | oh ... this isn't doing an automatic reboot. I haven't figured out the autoinstall, but I also need a mental break from it so I'm moving on to doing a manual install that does the same steps. | 18:24 |
| MTecknology | so "late-commands" is just what I type after it says updates are done, and early-commands is before giving the installer any input. | 18:24 |
| dbungert | right, late-commands are at install time but before first boot, one of the last steps. early-commands are before anything has really started. | 18:26 |
| MTecknology | Oh! Is there any way to get input from the user? I'd like to prompt for a password and dump that to a text file for another late-command to pick up ... or is it possible to let the script in the late-command directly request the password? (the command invoked is gpg - https://github.com/MTecknology/teckhost/blob/master/bootstrap#L225-L226) | 18:30 |
| MTecknology | for debian, I use https://github.com/MTecknology/teckhost/blob/master/iso/debconf_early | 18:31 |
| dbungert | MTecknology: for prompting for supported things like the first user password or the LUKS passphrase, that can be done, but for arbitrary inputs like your gpg case I have no answer. | 18:42 |
| MTecknology | oh ... | 18:43 |
| MTecknology | I guess that's something that would have to be offloaded to firstboot and written to interact with the boot process, similar to cryptsetup | 18:44 |
| MTecknology | I don't actually have any "firstboot" in my personal deployment. As soon as the installer shuts down at the end, it's completely configured and ready to ship (usually to family). I can even walk family members through doing a remote installation with any hardware they pick up at some cheap box store | 18:45 |
| MTecknology | dbungert: Oh .. how can I use "curtin" from another tty? I assumed that it would behave similar to in-target, but that doesn't seem to be the case. | 18:56 |
| MTecknology | I guess that's confusing since there's "curtin in-target" vs. "in-target" .. I was using the latter | 18:57 |
| dbungert | on recent ISOs you can get to curtin with `subiquity.curtin`, so for example you can do `subiquity.curtin in-target -i -t /target -- /bin/bash` to get a live shell chrooted in, in exactly the same mount setup | 18:59 |
| MTecknology | I saw lots of warnings against using different releases for different installs. Does that mean I'm stuck without for this? | 19:06 |
| MTecknology | It's mostly the removal of (I think) sg3-utils that causes a problem when run inside of chroot at the end of the install, because it needs /dev to be remounted. | 19:08 |
| MTecknology | lol ... apparently "subiquity-shell" is not what I thought it was | 19:13 |
| MTecknology | that seems to be the user interface bit? | 19:13 |
| dbungert | MTecknology: you can refresh the snap to the latest version if you want newer features. The subiquity snap does not refresh itself by default, or you can say `refresh-installer: { update: true }` if you do want that. | 19:24 |
| dbungert | and yes subiquity-shell is internals stuff, I doubt it's very helpful when directly invoked | 19:25 |
| MTecknology | I had the impression that updating the snap was only an option for newer releases | 19:25 |
| MTecknology | like, update the snap, and get updated ubuntu | 19:25 |
| dbungert | they are separate items - if you use a 22.04.x iso you are getting Ubuntu 22.04. Refreshing the snap helps if you want to get newer features or bugfixes but it doesn't change the OS version being installed. | 19:35 |
| MTecknology | gotcha, thanks for clarifying | 19:36 |
| MTecknology | "Also, please note that software in universe WILL NOT receive any review or updates from the Ubuntu security team." ... but ... isn't that enabled by default? | 19:48 |
| MTecknology | dbungert: Which is actually canonical; security.ubuntu.com or archive.ubuntu.com foo-security? | 19:56 |
| dbungert | archive.ubuntu.com can be a different mirror, security updates hit security.ubuntu.com first. This has more details: https://wiki.ubuntu.com/SecurityTeam/FAQ#What_repositories_and_pockets_should_I_use_to_make_sure_my_systems_are_up_to_date.3F | 19:58 |
| MTecknology | thanks | 19:58 |
| MTecknology | dbungert: oh ... I'm finally getting some logs. Now it seems to be failing because vendor-data is missing. Is that a required-empty file? | 21:26 |
| MTecknology | https://dpaste.com/F7KUQ44MZ | 21:28 |
| dbungert | AIUI vendor-data is optional, I see that also in successful cases. | 21:29 |
| MTecknology | That's the only thing standing out to me right now. It seems like cloud-init is finally trying to grab the "cloud-config" files, but it still seems like there's no impact to the installation | 21:32 |
| MTecknology | oooh ... 2024-03-11 21:21:32,837 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 33 column 49: "mapping values are not allowed here | 21:33 |
| MTecknology | - printf "network:\n version: 2\n renderer: NetworkManager" >/target/etc/netplan/nm.yaml | 21:34 |
| MTecknology | that should be wrapped entirely in quotes | 21:34 |
| MTecknology | oooh next fun challenge | 21:38 |
| MTecknology | I guess "install to largest disk" is the best strategy here | 21:40 |
| minimal | note the WARNING about "netplan apply" failing, that doesn't look good | 23:40 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!