[08:34] <eoli3n_> Hi
[08:35] <eoli3n_> is there any plan to let desktop image use autoinstall ?
[08:45] <eoli3n_> as you removed kickstart support since 20.04, what is the recommended way to automate desktops desployments ?
[08:46] <eoli3n_> deployments*
[08:57] <eoli3n_> what is the nickname of Ryan Harper here please ?
[12:15] <xnox> eoli3n_:  i'm not sure what you mean by kickstart support being removed.
[12:15] <xnox> eoli3n_:  desktop installation automation has not been changed for years, and can be done in 20.04 the same way it has always been possible to do.
[12:15] <eoli3n_> actually i was wrong, just reworking on this. Problem is not that kickstart is removed, problem is that i can't deploy in EFI mode with kickstart
[12:16] <xnox> eoli3n_: what's not working for you?
[12:16] <xnox> i deploy ubuntu desktop with EFI always.
[12:16] <eoli3n_> with kickstart ?
[12:16] <xnox> yes
[12:16] <eoli3n_> hm
[12:16] <xnox> eoli3n_:  your partitioning does have ESP right?
[12:16] <xnox> one kind of needs that =)
[12:17] <eoli3n_> how do you trigger EFI deploy mode with kickstart ?
[12:17] <xnox> by booting the machine in EFI mode
[12:17] <xnox> with either EFI iPXE or EFI Http boot
[12:17] <eoli3n_> yes, but when i tested, my booted EFI host deploy legacy system, but maybe i did forget to adapt my partition table
[12:24] <eoli3n_> xnox what iso do you use ?
[12:24] <eoli3n_> the main desktop one ?
[12:25] <eoli3n_> i used the netboot one, which can't deploy in efi mode
[12:25] <xnox> eoli3n_:  yes i always use the desktop one.
[12:26] <xnox> and i network boot initrd & kernel from it, using shim + network build of grub grubnetx64.efi. And pass url=http://path/to/matching-desktop.iso
[12:26] <xnox> eoli3n_: i only always deploy desktop using the desktop.iso with preseeding.
[12:27] <xnox> that's the only way to deploy Ubuntu Desktop the right way.
[12:27] <eoli3n_> any link which describe the process ?
[12:29] <eoli3n_> xnox is there any chance to adapt it to nfsroot the iso ?
[12:30] <eoli3n_> i manage 800 nodes, deployed by group of 40
[12:30] <eoli3n_> i can't let each node get a full 2G iso
[12:31] <xnox> eoli3n_:  "i can't let each node get a full 2G iso" => but they will need to get 2G of data one way or another to complete desktop install.
[12:31] <xnox> so not sure it matters _when_ they do that =) from initrd on boot, or elsehow.
[12:31] <xnox> But there is nfs root mount option too, if you can export nfs mount.
[12:31] <xnox> https://discourse.ubuntu.com/t/netbooting-the-live-server-installer/14510 => is how to setup pxeboot, it does work with UEFI too if your UEFI supports pxe (see like the first comment). Instead of the live-server.iso you just use the desktop one.
[12:34] <eoli3n_> xnox nfsroot is much more efficient
[12:34] <eoli3n_> because of the server itself
[12:35] <eoli3n_> 40x dl of a big file is worse than letting clients get a lots of little files
[12:35] <xnox> boot=casper netboot=nfs nfsroot=192.168.122.73:/srv/iso-mount/ => make sure that you export the full mounted desktop iso; and that initrd&vmlinuz you boot are from it.
[12:35] <xnox> if you can't mount the iso make sure to copy _all_ of it, including hidden files.
[12:36] <xnox> there are some details on automating the preseed of Ubiquity https://wiki.ubuntu.com/UbiquityAutomation
[12:36] <xnox> i.e. booting with automatic-ubiquity and supplying the preseed.
[12:37] <xnox> i.e. cp -r /mnt/. /srv/nfs/somewhere => note the '.' in /mnt/. to ensure all the hidden files are copied and exported.
[12:37] <eoli3n_> i already does this
[12:38] <eoli3n_> do*
[12:41] <xnox> eoli3n_:  cool. so yeah, my preference is to boot in uefi, either download/mount iso from the initrd (url=http://...) or do nfsroot mount, and automate that. There are no reasons not to boot in EFI mode.
[12:42] <xnox> eoli3n_: if you don't want to use PXE / tftp, you can point your servers to perfrom UEFI HTTP boot => point them at shim + grubnetx64.efi
[12:42] <eoli3n_> but you use kickstart or ubiquity only  ?
[12:42] <eoli3n_> i use ipxe and tftp
[12:42] <xnox> eoli3n_:  there is no native support for kickstart, and never was. it always translated kickstart into d-i/ubiquity compatible preseed.
[12:43] <xnox> eoli3n_:  thus i prefer to use native formats, thus yes, i simply write/serve/boot with preseed for ubiquity.
[12:43] <eoli3n_> i know, but i already have a kickstart script for legacy mode
[12:43] <xnox> or autoinstall.yml for the subiquity live-server iso.
[12:43] <eoli3n_> xnox oula oula oula oula
[12:43] <eoli3n_> lets restart fresh, to let you understand from where i come, to where i want to go, and where i'm now
[12:43] <xnox> eoli3n_: kickstart should work too. I can't remember if we build preseed-kickstart into the ubiquity images.
[12:44] <xnox> let me check that.
[12:44] <xnox> eoli3n_:  you also did say taht your kickstart doesn't work, rigth? =) in EFI... hence if it doesn't work, is it really there?
[12:45] <eoli3n_> i used ubuntu ""kickstart"" to deploy legacy hosts with pxelinux/tftp. Now i need to deploy in EFI mode so i worked a lot on server iso to use autoinstall, i'm really near to get it work
[12:45] <eoli3n_> but i wonder if its the right way
[12:46] <eoli3n_> now i don't have much time so i need to choose a quick and proper way
[12:46] <eoli3n_> there is no good documentation about this
[12:46] <eoli3n_> that's sad
[12:47] <eoli3n_> a kind of guide
[12:47] <xnox> eoli3n_:  when you say you used ubuntu ""kickstart"" can you show me what you mean?
[12:47] <eoli3n_> yes, just a sec
[12:47] <xnox> just a few snippets / quotes of it.
[12:48] <xnox> eoli3n_:  your request is not unique. I am pondering to run a public demo netboot/install server that "works" and then let people copy it.
[12:48] <eoli3n_> xnox https://x0.at/UWZ.txt
[12:48] <eoli3n_> i need to deal with dualboot
[12:49] <xnox> eoli3n_:  the only bad thing about your preseed is the part of "generate partitioning script"
[12:49] <eoli3n_> i can't do without it
[12:49] <eoli3n_> because of dualbooting
[12:49] <xnox> eoli3n_:  if you change the sfdisk <<EOSF bits to use label gpt, add EFI partition, then add disk_partone => it will start to work in EFI mode too.
[12:50] <eoli3n_> ahhh
[12:50] <eoli3n_> ok
[12:50] <xnox> eoli3n_:  you need to adjust that one, to create ESP
[12:50] <eoli3n_> that was my question
[12:50] <xnox> eoli3n_:  that's what i did tell you at the very start of our conversation =)
[12:50] <xnox> "xnox>	eoli3n_:  your partitioning does have ESP right?"
[12:50] <eoli3n_> yep yep
[12:51] <xnox> if you have no ESP in your partitioning, when installer gets to "now i install shim+grub into ESP, it goes oh oh"
[12:51] <xnox> eoli3n_:  and it's best to always create ESP and always use GPT => it works with legacy mode, efi mode, dual boot, with any windows&linux combo, etc.
[12:51] <eoli3n_> ok
[12:52] <xnox> even if the deploy is done in bios or efi mode, it will installs and boot in either.
[12:52] <xnox> our ubuntu desktop is versatile like that, and allows to boot itself irrespective of how one installed it.
[12:52] <eoli3n_> so lets try to adapt this instead of working with autoinstall
[12:53] <eoli3n_> xnox http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/current/images/netboot/netboot.tar.gz
[12:53] <eoli3n_> is this able to deploy in EFI  mode ?
[12:54] <xnox> eoli3n_:  it's incomplete. and does not support secureboot.
[12:54] <eoli3n_> i don't use secureboot
[12:54] <eoli3n_> what do you mean by "incomplete" ?
[12:55] <xnox> eoli3n_:  you'd need to add shimx64.efi / uefi-ipxe / grubnetx64.efi to get secureboot (shim), uefi netboot capability (ipxe or httpboot grubnetx64.efi from focal+)
[12:55] <xnox> eoli3n_: one needs more than just that, to boot in efi mode is what i mean.
[12:56] <eoli3n_> if i extract kernel+initrd and boot it with ipxe, no problem then, right ?
[12:56] <xnox> yeah, that works.
[12:56] <xnox> some expect the shim/signed uefi ipxe/grub/configs to be in that tarball too.
[12:56] <eoli3n_> so in that case, what is the benefit to prefer the main desktop.iso ?
[12:57] <xnox> eoli3n_:  that thing, will never deploy the same desktop as the desktop.iso will.
[12:57] <eoli3n_> lets precise something
[12:57] <eoli3n_> i just need the installer to deploy the minimal system
[12:57] <xnox> eoli3n_:  no option to install minimum ubuntu desktop; no option to install correct matching nvidia drivers; no way to configure and install OEM kernel for the certified hardware
[12:57] <eoli3n_> then i use ansible to deploy everything else
[12:57] <eoli3n_> ansible does all those things for me
[12:57] <xnox> no option to have the correct defautl snaps installed; following the correct tracks
[12:58] <xnox> no oem config...
[12:58] <eoli3n_> hm
[12:58] <xnox> eoli3n_:  some/none of the above maybe of interest to you
[12:58] <xnox> eoli3n_:  and/or you may already be fixing those things separately.
[12:58] <eoli3n_> but then will not be any 22.04 netboot so
[12:58] <eoli3n_> lets switch to the main desktop.iso
[12:58] <xnox> eoli3n_:  i also don't see ubiquity supporting kickstarts. and there is new installer in progress which will have autoinstall.yml support just like server iso
[12:58] <eoli3n_> thanks for your help xnox, i was a bit lost
[12:59] <eoli3n_> "there is new installer in progress which will have autoinstall.yml"
[12:59] <eoli3n_> i need this !
[12:59] <xnox> eoli3n_:  swithcing to the main desktop.iso / and or testing that, may help to futureproof your deployements.
[12:59] <eoli3n_> ok
[12:59] <xnox> eoli3n_:  see the New Installer articles / github repo where it is being developed. At the moment early design stages.
[12:59] <eoli3n_> so two moves : move to desktop.iso, and translate the kickstart part to ubiquity script
[13:00] <eoli3n_> any link about this ?
[13:00] <xnox> eoli3n_:  yes, if you boot / deploy your kickstart, there should be somewhere in /var/log/installer the translation. Cause d-i does translates it into preseed. It should be a good enough starting point.
[13:01] <xnox> eoli3n_:  plus you don't actually do anything special much.
[13:01] <xnox> you pick the right disk; and configure ansible
[13:02] <xnox> https://code.launchpad.net/ubuntu-test-cases => we have many preseeds automation for various things
[13:02] <eoli3n_> does ubiquity automation only, support %post chrooted script and %pre ?
[13:02] <xnox> eoli3n_:  https://bazaar.launchpad.net/~ubuntu-test-case-dev/ubuntu-test-cases/desktop/view/head:/preseeds/default.cfg is more or less all you need.
[13:02] <xnox> but you do need preseed/late_command to do more things kind of like what your %post does to do ansible
[13:03] <xnox> and i think you will want to do a hook before partma to figure out the right drive to autopartition with a shell script
[13:03] <xnox> cause i think you are trying to iterate things and find the one you like.
[13:03] <xnox> for that you can use partman/early_command
[13:04] <xnox> eoli3n_:  or check what your kickstart got translated into preseed terms.
[13:04] <xnox> i am sorry, i have to go, but i will come back to read backscroll here.
[13:06] <eoli3n_> thanks a lot for your help
[13:07] <rbasak> cjwatson: any further thoughts on my git-ubuntu rich history thread please? I think, with your feedback in mind, it still makes sense to implement my proposed spec (but with a singular Vcs-Git-Ref), with a view to switching to "push" at some far point in the future. Do you concur?
[13:10] <cjwatson> rbasak: Since you're treating the rich history as best-effort, I think I concur for now, yes.
[13:11] <rbasak> OK, thanks! And for your detailed feedback.
[13:11] <cjwatson> rbasak: I do think you should add at least some kind of fallback-and-retry for temporary failures even in the short term - they won't be *that* uncommon
[13:12] <cjwatson> rbasak: Even if it's just the kind of fairly basic thing that lazr.restfulclient does (I mean in terms of overall algorithm, rather than specifics)
[13:13] <rbasak> Understood.
[13:14] <rbasak> I can add some short-lived retries that will hold up the importer worker (and therefore other imports) for an hour or two.
[13:15] <rbasak> Leaving the import to move on to others and then come back to it is harder, since I don't currently have a soft-fail mechanism in the queue architecture.
[13:15] <rbasak> That's something I should probably add, but isn't there yet.
[13:33] <cjwatson> rbasak: Seems reasonable.  Most such temporary failures are likely to be service-wide anyway
[14:26] <eoli3n_> xnox any chance to get my ks file to work as is ?
[14:26] <eoli3n_> i found the translation in /var/lib/preseed/log, but it doesnt contains everything
[14:40] <eoli3n_> "xnox: eoli3n_:  i also don't see ubiquity supporting kickstarts."
[14:41] <eoli3n_> hm.. ok
[14:55] <eoli3n_> xnox some questions :
[14:56] <eoli3n_> - how to generate my partman-pre script
[14:56] <eoli3n_> - how to run multiple command in chroot after install
[14:56] <eoli3n_> https://help.ubuntu.com/lts/installation-guide/armhf/apbs05.html#preseed-hooks
[14:56] <eoli3n_> this is really unclear
[15:02] <eoli3n_> found this about multiple command in chroot : https://github.com/dsgnr/Ubuntu-16.04-Unattended-Install/blob/master/preseed.cfg#L195
[15:12] <xnox> eoli3n_:  yeah ';' works for multiple commands. not nice, but it's what it is.
[15:12] <xnox> "how to generate my partman-pre script" => not sure what you mean.
[15:13] <eoli3n_> https://x0.at/UWZ.txt
[15:13] <eoli3n_> see the # PRE # part
[15:13] <xnox> eoli3n_:  if you want to control everything by hand, something like this works => https://secopsmonkey.com/custom-partioning-using-preseed.html
[15:13] <xnox> which disables all the automatic partitioning, and does it by hand.
[15:13] <eoli3n_> i do want automatic partitionning
[15:14] <eoli3n_> i just want to force to restore part1 which is windows
[15:14] <eoli3n_> my pre part is to not overwrite windows if present
[15:14] <eoli3n_> if not present, it prepare the place
[15:14] <eoli3n_> that's why i dump diskid
[15:14] <eoli3n_> and regenerate as is
[15:15] <eoli3n_> it actually clear everything except windows part
[15:15] <eoli3n_> clears
[15:15] <eoli3n_> can i wget that script ?
[15:17] <eoli3n_> d-i partman/early_command string wget -O /tmp/pre.sh http://prodpeda-cesium.infra.domain.fr/preseed/fds/pre.sh; /bin/bash /tmp/pre.sh
[15:17] <xnox> eoli3n_:  so you either have empty disk => wipe everything; or you have windows on part1.
[15:17] <eoli3n_> lets try this
[15:17] <eoli3n_> xnox i don't get your question
[15:17] <eoli3n_> i do have both
[15:17] <xnox> eoli3n_:  when you have part1 => do you have enough disk space after part1? or do you need to resize windows too?
[15:17] <eoli3n_> if this is a new host, first use, then prepare part1, else if windows present, keep it functionnal
[15:18] <eoli3n_> i have enough space
[15:18] <xnox> eoli3n_:  cause i wonder if you can just select use largest free, and it will always do the right thing.
[15:18] <eoli3n_> that's what i use
[15:19] <eoli3n_> preseed partman-auto/init_automatically_partition select biggest_free
[15:19] <xnox> hm
[15:19] <xnox> oh, so you create empty part1, such that later you can have windows on it?
[15:19] <xnox> interesting.
[15:19] <eoli3n_> better than that
[15:19] <xnox> or if it is there already, don't touch it?
[15:19] <eoli3n_> if windows is present, is "recreate" over it to keep it intact
[15:19] <eoli3n_> yes
[15:19] <xnox> interesting.
[15:20] <xnox> do we currently break windows? =/
[15:20] <eoli3n_> that was the only way i found
[15:20] <xnox> cause i would have hoped that biggest_free doesn't break/touch windows that is there anyway.
[15:20] <eoli3n_> i had some issues yes
[15:20] <xnox> ah, netboot.tar breaks things yes.
[15:20] <eoli3n_> windows needs a repair if disk_id changed
[15:20] <xnox> ubiquity doesn't => it has more logic.
[15:20] <eoli3n_> oh !
[15:21] <eoli3n_> lets try without then
[15:21] <xnox> eoli3n_:  ah, interesting. maybe we still break disk_id, but i thought we didn't.
[15:21] <eoli3n_> but with GPT partition table, maybe its different
[15:21] <xnox> eoli3n_:  it would be interesting to try without anything. Imho, it is a bug if it breaks windows and doesn't just use biggest_free.
[15:22] <eoli3n_> problem is that i fix two cases
[15:22] <xnox> true....
[15:22] <eoli3n_> first is windows is present : so without anything it "would" work, as you say
[15:22] <eoli3n_> but if nothing is already present, the place will not be prepared if i drop my script
[15:22] <xnox> hopefully doing "d-i partman/early_command string wget -O /tmp/pre.sh http://prodpeda-cesium.infra.domain.fr/preseed/fds/pre.sh; /bin/bash /tmp/pre.sh" will just work as you mentioned earlier.
[15:22] <eoli3n_> ok
[15:23] <eoli3n_> maybe i can use sfdisk in another way
[15:23] <eoli3n_> to just create the part without restauring the full partition table
[15:23] <eoli3n_> in 2 steps
[15:23] <eoli3n_> restoring
[15:30] <eoli3n_> running
[15:30] <eoli3n_> just a thing : how to disable the filesystem check at startup ?
[15:32] <eoli3n_> that one : https://x0.at/xsf.png
[15:34] <xnox> eoli3n_:  it's turned off by default in hirsute; i think you can boot with fsck.mode=skip and it will not run.
[15:34] <eoli3n_> what is hirsute ?
[15:34] <eoli3n_> lets try "fsck.mode=skip"
[15:34] <xnox> eoli3n_:  Ubuntu 21.04 => development version of the next Ubuntu release.
[15:35] <eoli3n_> oh ok
[15:38] <eoli3n_> fsck.mode=skip doesnt disable the filesystem check
[15:38] <eoli3n_> https://x0.at/zL_.txt
[15:45] <eoli3n_> if you have more exemples of seed files, i take
[15:51] <eoli3n_> (filesystem check seems to still print but does not run (stuck at 0% then disapear))
[15:52] <oSoMoN> xnox, what's the state of https://bileto.ubuntu.com/#/ticket/4491, are you still working on it?
[16:09] <xnox> oSoMoN:  that approach failed, but yes still working on it.
[16:09] <oSoMoN> ack, thanks
[16:10] <xnox> eoli3n_:  stuck at 0% then disappear => is not running =) the visual UX improved later.