[00:36] <MTecknology> Is https://ubuntu.com/server/docs/install/autoinstall actually maintained? It seems inaccurate and I'm not able to reproduce what they claim should work.
[00:37] <oerheks> yes
[00:37] <oerheks> see bottom 'This page was last modified 22 minutes ago'
[00:38] <oerheks> so, you might have used the old one
[00:38] <MTecknology> It /always/ says it was modified X minutes ago, but nothing else on the page changes
[00:55] <MTecknology> The kvm command they provide doesn't even try to read a data source. I had to use tcpdump to be absolutely sure it wasn't just from unclear documentation.
[00:58] <MTecknology> I guess the "broken" whitespace hops off to a different site with the exact same theme - https://canonical-subiquity.readthedocs-hosted.com/en/latest/howto/autoinstall-quickstart.html
[01:05] <MTecknology> ... it sure seems like I'm trying to follow instructions that never actually worked. I'm not believing that this is maintained.
[01:16] <oerheks> This page was last modified 29 minutes ago.
[01:16] <oerheks> so, it is working
[01:16] <oerheks> not a realtime clock, but not static
[01:17] <oerheks> care to share what is not working?
[01:18] <MTecknology> I can't replicate what they claim should be working. I set it up the same way, but no kvm command I try appears to even try to pull a cloud-init config.
[01:19] <MTecknology> I can use curl to see the python web server returns the correct file, but tcpdump suggests no request is made.
[01:21] <MTecknology> cloud-init documentation suggests it should be ds=nocloud instead of ds=nocloud-net, but the apparent result was the same.
[01:25] <oerheks> i' d like to ask a server genius to join us, but none are available at saturday night :-(
[01:26] <oerheks> i read about single quotes issue https://askubuntu.com/questions/1487703/try-to-setup-autoinstall-in-ubuntu-22-04-lts-booting-through-iso-and-use-https
[01:26] <oerheks>  'ds=nocloud;s=https://x.x.x.x/ubuntu/' 
[01:31] <MTecknology> ah, interesting. I tried the reverse (double inside single) ->  -append 'autoinstall "ds=nocloud;s=http://_gateway:3003/"'  (also with just a backlash and with both quotes and backslash, and all with nocloud-net and nocloud.
[01:31] <oerheks> jups
[01:31] <oerheks> nocloud-net
[01:32] <oerheks> sorry that i have less experience with autoinstall ..
[01:32] <MTecknology> I've been trying with both, since cloud-init documentation shows without the -net
[01:32] <oerheks> really nobody is available right now.
[01:32] <MTecknology> Is there an equivalent to BOOT_DEBUG?
[01:35] <MTecknology> The great thing about IRC is that nerds tend to always be connected. If anyone with any clue comes along in a few days, they'll be able to help. I very much get the impression almost nobody is actually using this mess and just resorts to static image deployment
[01:35] <oerheks> is it possible to hang in here, maybe tomorrow ..
[01:37] <MTecknology> Yeah, I'll just leave it connected. I don't expect anything, but I'm still passively hopeful that maybe someone around here has actually used this thing.
[01:38] <oerheks> i try tomorrow again
[01:38] <oerheks> interesting though..
[01:39] <MTecknology> Anyone that might have a clue is /probably/ going to be connected 24/7, so it's just a matter of them reading my follow-up to your questions and seeing if they have anything else they can contribute.
[01:40] <MTecknology> oerheks: Quick question- How do /you/ handle automatic deployment/provisioning of servers and/or endpoints?
[01:43] <oerheks> i did the old way, preseed files, deploying dedicated servers.
[01:43] <oerheks> not working with canonical though..
[01:47] <MTecknology> Yeah ... that's my personal method. I have a "remastered" debian ISO, with a embedded preseed and customized grub menu. The preseed is pretty basic except for the disk layout and the execution of a "bootstrap" script, which is where my actual device provisioning comes from.
[01:47] <MTecknology> (https://github.com/MTecknology/teckhost)
[01:48] <MTecknology> Converting this to ubuntu is proving to be insanely difficult.
[01:50] <MTecknology> oh .. I think the quoting thing is specifically for grub, which isn't being used for this kvm command. The kvm command basically skips that step.
[01:55] <oerheks> i have break up, see you tomorrow, maybe?
[01:55] <oerheks> this channel is logged and read by many..
[01:55] <MTecknology> I'll be here
[01:55] <oerheks> meanwhile, relax and have fun!
[01:55] <MTecknology> I don't get to relax until this is done.
[01:56] <oerheks> i know, but you need some rest some time
[01:58] <MTecknology> yeah .. I guess I could return to an imaged install and pick this up from a bootstrap.
[01:59] <MTecknology> I was hoping to feel like I made /some/ progress today, but I'm exactly where I started.
[02:04] <minimal> cloud-init docs are for cloud-init, ubuntu server's autoinstall (subiquity) whilst based on cloud has its own YAML config that then "wraps" cloud-init config
[02:06] <MTecknology> It seems like there's some back-and-forth, with cloud-init being how autoinstall is provided, and then autoinstall writing things for a later cloud-init execution
[02:07] <MTecknology> I found some whiteboard sketches documenting how subiquity, cloud-init, autoinstall, and d-i interact with each other, but I can't find that anymore.
[02:07] <minimal> "cloud-init being how autoinstall is provided" - cloud-init knows nothing about autoinstall, that's a subquity thing
[02:07] <minimal> also subquity changes cloud-init's behaviour which the cloud-init docs won't reflect
[02:12] <minimal> I could help with cloud-init but not with subquity/autoinstall as I basically know nothing about it
[02:13] <MTecknology> I don't know which piece is even broken ...
[02:16] <MTecknology> The only thing I know for certain is that I can't get ubuntu to appear to try to load any "data source"
[02:16] <minimal> the bit about nocloud/nocloud-net depends on the cloud-init version - recent cloud-init versions just use "nocloud" (tho I assume "nocloud-net" is still supported for now) whereas older releases needed "nocloud-net" to be specified for a http/https url
[02:17] <minimal> and yes the quoting around the ds= entry is required for some (all?) bootloaders
[02:19] <MTecknology> The last thing I tried, using the contents extracted from the 20.04-server iso ::  kvm -no-reboot -m 2048 -drive file=image.img,format=raw,cache=none,if=virtio -kernel linux -initrd initrd.gz -append "autoinstall 'ds=nocloud-net;s=http://10.41.3.84/ubuntu2004/'"
[02:19] <MTecknology> (happy to push that off to people.d.o if it'd be helpful)
[02:20] <minimal> I can only comment on how cloud-init might behave, I don't know if subquity/autoinstall might try and use that URL
[02:21] <MTecknology> Is there any obvious issue if it were given to cloud-init?
[02:21] <minimal> I'm assuming you're wanting to use that url's contents to drive the autoinstall
[02:21] <MTecknology> yep
[02:21] <MTecknology> There's a user-data and meta-data file in that directory
[02:22] <minimal> so that's not cloud-init then, autoinstall is not a cloud-init feature, that subquity
[02:22] <MTecknology> this?  https://canonical-subiquity.readthedocs-hosted.com/en/latest/howto/autoinstall-quickstart.html
[02:23] <oerheks> 0oh, my bad
[02:23] <MTecknology> hm?
[02:24] <oerheks> subquity is a cloud-init feature
[02:24] <minimal> oerheks: no it is not
[02:24] <minimal> it is a Ubuntu server feature
[02:25] <MTecknology> I'm assuming that everything I think I currently understand is probably wrong, so I don't know what I don't know ... I'm worse than a blank slate.
[02:29] <minimal> oerheks: to be more precise, cloud-init has some limited ubuntu autoinstall support - specifically as cloud-init validations YAML docs against schemas then cloud-init had to add the "autoinstall" bit to the cloud-config schemas as otherwise ubuntu autoinstall YAML docs would fail validation
[02:30] <oerheks> oke. i have limit knowledge of the new subquity installer.
[02:31] <oerheks> thanks for stepping in this late
[02:32] <minimal> cloud-init is designed to run on a disk image (i.e. re-prepared OS install) such as ubuntu (and other OS/distros) cloud images
[02:33] <minimal> Ubuntu then decides to use cloud-init for installing Ubuntu by using it via Subiquity
[02:33] <MTecknology> ooh, neat! It looks like the only modification required for my own (debian) installer to support kvm is adding "elif [ -e /dev/vda ] ..." to if [ -e /dev/nvme0n1 ]; then disk=/dev/nvme0n1; else disk=/dev/sda; fi;
[02:34] <minimal>  /dev/vda is a virtio-blk device
[02:34] <minimal> whereas /dev/sda can be a virtio-scsi, or sata, scsi device
[02:34] <MTecknology> I have almost no experience with kvm outside of things like proxmox, so it seemed like a helpful way to validate the commands should actually work the way I expect them to.
[02:35] <minimal> when you run QEMU on cli you can specify the type of storage device you want
[02:35] <minimal> so I have a wrapper script that lets me specify virtio-blk, virtio-scsi, nvme, etc
[02:36] <MTecknology> That was the only change required and it got through with my bootstrap, which handles installation and execution of "salt-call state.highstate"
=== chris14_ is now known as chris14
[02:37] <MTecknology> At least I know it generally works correctly. It makes me trust at least that much of the documentation.
[02:37] <minimal> this defines the autoinstall YAML format: https://canonical-subiquity.readthedocs-hosted.com/en/latest/reference/autoinstall-reference.html
[02:38] <MTecknology> Yeah, I actually had that open in a tab, but I can't use that because I can't get the installer to try to load anything in the first place.
[02:38] <minimal> and inside that the "user-data" defines what will be passed on to cloud-init
[02:38] <MTecknology> that documentation is a future challenge
[02:39] <minimal> cloud-init's NoCloud expects to be provided with user-data
[02:39] <MTecknology> most documentation just uses a single "autoinstall:" key with everything else nested one level
[02:39] <MTecknology> I have a few versions that I tried, but I stopped playing with that when I realized that tcpdump shows no activity
[02:40] <minimal> I don't know if/how subiquity handles NoCloud-Net
[02:40] <MTecknology> I also tried passing -cdrom and added it manually, but that didn't help.
[02:41] <minimal> i've used NoCloud-Net with cloud-init in the past to fetch meta-data and user-data
[02:41] <MTecknology> This is one of the whitespace issues -> https://canonical-subiquity.readthedocs-hosted.com/en/latest/howto/autoinstall-quickstart.html#write-your-autoinstall-configuration
[02:41] <MTecknology> How did you kick that off?
[02:41] <minimal> I'm not using subiquity
[02:42] <MTecknology> Do you do installations to unknown hardware?
[02:42] <minimal> anything I tell you is how "pure" cloud-init works
[02:42] <minimal> I use disk images that contain cloud-init to then run upon boot
[02:43] <MTecknology> ah, bummer ... that's what I need to avoid and what it seems like everything is now
[02:43] <MTecknology> great for a vps or container, but not great for metal
[02:43] <minimal> as I already said, cloud-init does not do installations
[02:43] <MTecknology> I have zero objections to throwing this giant clusterfuck, but I don't think I actually /can/
[02:43] <minimal> Ubuntu came up with a way (Subiquity) to use/abuse it for installs
[02:44] <minimal> for bare-metal I boot a simple stripped down OS that "dd"s a disk image onto the storage device and then when the machine reboots it runs cloud-init
[02:45] <minimal> I use cloud-init that way on baremetal PCs and Raspberry Pis
[02:46] <MTecknology> That seems to be how live-build works -> static image with a little bit of extra bootstrapping, which /could/ be cloud-init.
[02:47] <MTecknology> it's great if you know the target hardware, but I have to support bring-your-own
[02:47] <minimal> so example I have a disk image for BIOS boot and another for UEFI boot and the appropriate one is dd'ed onto the disk..........on 1st boot then cloud-init grows the size of the root partition to fill the disk, sets keyboard/locate, sets up networking, etc
[02:48] <minimal> well my "simple OS for install" just needs to analyse the hardware (is it BIOS or UEFI? is it SATA or NVME? etc) and then select the appropriate disk image to dd
[02:51] <minimal> I can create "generic" images that have all the relevant drivers for booting (i.e both SATA and NVME in initramfs, both Intel and AMD microcode present) and if I use a hybrid GPT layout then it can be either BIOS or UEFI booted
[02:52] <minimal> but normally I prefer to create/use a more specific image for size and other reasons
[02:53] <MTecknology> My own version of cloud-init is basically the "bootstrap" script that gets salt installed/configured and then runs "highstate," which results in a 100%-completely configured device. I use the *exact* same bootstrap for desktops, servers, and embedded devices. Debian makes portability as simple as "auto=true url=https://raw.githubusercontent.com/MTecknology/teckhost/master/iso/debian12/preseed.cfg"
[02:54] <minimal> well in cloud-init's user-data you can specify an ansible playbook to be run so I use that
[02:55] <minimal> cloud-init has salt support though I've never used it/looked at it
[02:56] <MTecknology> So, my original question is obviously going to get lost to backlog. Can you help me out with a rewritten version of the question that includes your understanding so that I can ask a simple/coherent question next time?
[02:56] <minimal> https://cloudinit.readthedocs.io/en/latest/reference/modules.html#salt-minion
[02:57] <minimal> well I guess you want to know if subiquity supports fetching its config from a https url
[02:59] <MTecknology> Isn't that what it's showing over here?  https://canonical-subiquity.readthedocs-hosted.com/en/latest/howto/autoinstall-quickstart.html
[03:01] <minimal> perhaps, I don't know, as I said before I know noting about subiquity
[03:01] <minimal> I know how cloud-init works by itself
[03:01] <MTecknology> -append "autoinstall 'ds=nocloud-net;s=http://10.41.3.84/ubuntu2004/'"  that should be picked up by ubiquity and handed to cloud-init, which then reads the autoinstall: key?
[03:01] <minimal> perhaps, I don't know, as I said before I know noting about subiquity
[03:02] <minimal> nothing
[03:03] <minimal> it you booted a disk image which had cloud-init enabled to run at boot and you passed "'ds=nocloud-net;s=http://10.41.3.84/ubuntu2004/'" then I'd expect cloud-init to try and fetch the http://10.41.3.84/ubuntu2004/meta-data and ttp://10.41.3.84/ubuntu2004/user-data urls and use them to configure the OS
[03:05] <minimal> but when you're booting the ubuntu server ISO you're not booting cloud-init, you're booting some Subiquity/cloud-init hybrid
[03:07] <MTecknology> It uses cloud-init early in the boot process. There's actually a spot in the installer specifically for manual generation of those files on the ubuntu-server live image; it actually bootstraps itself using cloud-init.
[03:08] <MTecknology> FWIW, I have a few deadlines that absolutely have to be met, so I'm about to start dumping all my time into alternatives. If there's any chance that you'd be willing to take your current knowledge and help me figure out how to get this blasted thing to work correctly, I'd be delighted to pay for that effort.
[03:08] <minimal> we seem to be going around in circles, all I can say is that my understanding is that subquity changes cloud-init's behaviour and therefore, as someone who does not use subiquity, I cannot answer any questions involving subiquity
[03:11] <minimal> there's been at least a couple of Issues recently raised with cloud-init that didn't make sense until people realised that subquity was involved and so was changing how things behaved and those people had to be pointed to go raise Subiquity/Ubuntu issues instead
[03:13] <MTecknology> ah ... maybe I should take a break and start making sure those two words are in all of my searches. I'm sure all the documentation is going to look different after what you taught me. :)
=== iliv_ is now known as iliv
=== apxseemax1 is now known as apxseemax
=== JanC is now known as Guest6359
[17:34] <MTecknology> lol ... someone recommended leaving cloud-init around for the initial startup, but it looks like that conflicts with other tips for removing netplan from the equation on first boot ... I'm not really sure what netplan is, it seems like a cloud-init plugin.
[17:35] <minimal> no, netplan is how Ubuntu configures networking
[17:36] <MTecknology> it looks like a wrapper for configuring other things ...
[17:36] <minimal> its an alternative to /etc/network/interfaces (previously used by Debian), network-manager, sysconfig etc
[17:37] <MTecknology> ah ... yeah, the tip is for s/netplan/network-manager/. It seems like that breaks cloud-init.
[17:37] <minimal> during boot cloud-init will create distro-specific network configuration (so in the case of Ubuntu that's netplan) based on any network config provided to cloud-init
[17:39] <minimal> https://cloudinit.readthedocs.io/en/latest/reference/network-config.html#netplan
[17:40] <minimal> "Introduced in Ubuntu 16.10 (Yakkety Yak), Netplan has been the default network configuration tool in Ubuntu since 17.10 (Artful Aardvark). Netplan consumes Networking config Version 2 input and renders network configuration for supported backends such as systemd-networkd and NetworkManager."
[17:40] <MTecknology> I read through that and it's part of what reinforced my notion that it's a cloud-init plugin (configuration wrapper)
[17:41] <minimal> netplan is NOT a cloud-init plugin, it is Ubuntu functionality
[17:41] <minimal> cloud-init USES Ubuntu netplan functionality if required
[17:41] <MTecknology> I understand that now that you say it; the documentation does not make it clear at all
[17:41] <minimal> which documentation? the netplan documentation?
[17:41] <trippeh> a configuration wrapper (cloud-init) for a configuration wrapper (netplan) :)
[17:42] <minimal> "Netplan has been the default network configuration tool in Ubuntu since 17.10"
[17:42] <MTecknology> k ... gotcha, thanks
[17:42] <minimal> that makes it quite clear that netplan is a Ubuntu thing
[17:42] <MTecknology> yep, it does
[17:42] <minimal> so I'm confused how the documentation was unclear
[17:43] <MTecknology> it doesn't matter anymore ...
[17:43] <MTecknology> thanks for explaining it
[17:44] <minimal> and on that page it mentions how cloud-init can render network config to ENI, sysconfig, netplan, networkd, etc depending on the OS/distro
[17:45] <MTecknology> trippeh: That's pretty much what this is ... cloud-init wrapping autoinstall wrapping cloud-init
[17:46] <MTecknology> it's wrappers all around! I feel like I should be forgiven for not realizing one of these things is not just another wrapper
[17:50] <MTecknology> oh yeah ... at it's all kicked off by subiquity, which appears to be kicked off by debian-installer
[17:55] <trippeh> have we reached peak level of abstractions yet? :)
[17:57] <MTecknology> Wait! There's an xkcd for this ...
[17:57] <trippeh> I mostly just install using a shell script + debootstrap these days. feels simpler somehow.
[17:57] <MTecknology> https://xkcd.com/1406/
[17:58] <MTecknology> It's a million times simpler, but I don't think that works terribly well if you don't know the target hardware.
[17:59] <MTecknology> This is the first xkcd that came to mind, but the other was better. https://xkcd.com/927/
[18:23] <MTecknology> trippeh: For fun, this is my personal setup - https://github.com/MTecknology/teckhost   You can grab the iso and check out my personal desktop experience.
[19:35] <MTecknology> there's also this, which makes netplan seem like a wrapper ->  printf "network:\n  version: 2\n  renderer: NetworkManager" >/target/etc/netplan/plan.yaml
[19:37] <patdk-lap> if your using networkmanager yes
[19:37] <patdk-lap> I have never used netplan that way
[19:38] <patdk-lap> have only used networkd
[19:39] <MTecknology> Isn't networkd the systemd abomination?
[19:39] <MTecknology> part of*
[19:39] <patdk-lap> likely, but I have never liked networkmanager and have never used it on any of my systems
[19:41] <MTecknology> familiarity is worth a lot
[19:59] <minimal> its no more a wrapper than "older" network configuration methods were in other distros, e.g. sysconfig files used by Fedora, /etc/network/interfaces used by Debian
[20:00] <minimal> the netplan file is a *configuration* file used by the appropriate distro tools
[20:32] <MTecknology> Is there a grub command that says "retry a normal boot from this point"?
[20:32] <MTecknology> s/retry/contiue/
[20:34] <MTecknology> cryptsetup was part of initramfs, but it didn't provide a prompt to automatically decrypt the disk, but "cryptsetup open /dev/sda3 sda3_crypt was able to open the disk and then vgchange -a y read the volumes without problem.
[20:48] <JanC> did you try to exit the shell?
[20:53] <minimal> what are you expecting Grub to "continue"? Grub loads kernel + initramfs and passed control over, after than it's nothing more to do with Grub
[20:55] <MTecknology> JanC: I did not! That seems remarkably obvious now.
[20:55] <MTecknology> Here we go, the issue was that I managed to remove initramfs-tools.
[20:56] <JanC> so no new initramfs was built?
[20:56] <JanC> or what?
[20:58] <MTecknology> The missing link was /usr/share/initramfs-tools/scripts/*/cryptroot
[20:59] <MTecknology> (and probably other scripts, but that's what I noticed that lead me to make sure it was installed.
[20:59] <MTecknology> )
[21:01] <JanC> that's part of cryptsetup-initramfs
[21:02] <tomreyn> /usr/share/initramfs-tools/scripts/*/cryptroot are part of package cryptsetup-initramfs. package cryptsetup-initramfs is recommended by package cryptsetup.
[21:02] <tomreyn> maybe you've cusotomized your system so that it does not install recommended packages, which can easily create such issues.
[21:02] <MTecknology> JanC: so, yeah ... it was the lack of rebuilding. The scripts were there to be added, but they weren't.
[21:07] <JanC> are you sure you had that package installed?  I'm pretty sure you can't (easily) remove initramfs-tools, and that should automatically rebuild initramfs if necessary, but not installing or removing cryptsetup-initramfs is more likely...?
[21:08] <MTecknology> tomreyn: "maybe I've customized your system" is an understatement! :D   I'm working on an automated desktop installation (onto "unknown" hardware).  I'm pretty much giving up on an actual automated installation (ds=nocloud-net;s=https://.../ubuntu2024/) and focusing on manually completing the same thing.  (giving up -> getting the installer to download it)
[21:09] <MTecknology> JanC: sec ...
[21:15] <MTecknology> JanC: *sigh* ... Thanks for verifying that I lied to you.  https://0x0.st/HhQu.png   I guess it was the --reinstall that triggered a rebuild and got the scripts added.
=== JanC_ is now known as JanC
[21:23] <MTecknology> JanC: minimal: This is obviously incomplete and not "functional," but this is what I have at the moment. https://dpaste.com/49PP4FXMB