[08:56]  * apw is doing a d-i server install and hitting 'Installation step failed' from 'Select and install software' ... where might i look for a log with what actually failed ?
[09:01] <infinity> apw: syslog
[09:03] <cjwatson> and nano is the best viewer available, so 'nano -v /var/log/syslog'
[09:04] <apw> infinity, ta ... 'pkgsel' failed with error code 1
[09:04] <apw> oh joy
[09:04] <cjwatson> no that's a consequential error, go up
[09:04] <cjwatson> it's no doubt some package or other failing to install
[09:04] <apw> looks like we also have a find before that, 'find: unrecongnised: -printf
[09:04] <cjwatson> can you copy out the full log for me?
[09:04] <cjwatson> 'anna-install openssh-client-udeb' gets you scp
[09:05] <apw> cjwatson, nice ... had to do that a few 1000 times i assume :)
[09:06] <infinity> I feel like I might be somewhat spoiled when I was about to complain that I was "only" getting 5MiB/s from github.
[09:06]  * apw slaps infinity ... somewhat
[09:06] <cjwatson> apw: and implemented support for it ;-)
[09:07] <apw> cjwatson, for full disclosure i am tracking a bug here in the kernel, which i believe to be fixed which was blowing up the install before here.  but it is possible i didn't do it right
[09:07] <apw> cjwatson, and this boot is of a
[09:07] <apw> cjwatson, and this boot is of an ISO that i have replaced the kernel in
[09:07] <infinity> apw: S'ok, all is right in the world, it scaled up to 9MiB/s now.
[09:07] <infinity> apw: You can stop feeling sorry for me.
[09:07] <apw> cjwatson, http://paste.ubuntu.com/5861037/
[09:07] <apw> infinity, ok :)
[09:09] <apw> cjwatson, and in that log i would have hit 'return' to make it try again as well
[09:09] <apw> not deliberatly, but you know how focus grabby virtmanager is
[09:11] <cjwatson> what an interesting error
[09:12] <apw> cjwatson, i don't like the sounds of that ... :)
[09:12] <infinity> Since when does busybox provide dpkg?
[09:12] <cjwatson> we don't build that module
[09:13] <infinity> It thinks we do...
[09:13] <apw> Jul 10 08:52:49 in-target: dpkg: invalid option -- 'a'
[09:13] <apw> Jul 10 08:52:49 in-target: BusyBox v1.20.2 (Ubuntu 1:1.20.0-8ubuntu1)
[09:14] <apw> infinity, i assume that is what you are referring to
[09:14] <infinity> Indeed.
[09:14] <cjwatson> oh, only in static
[09:14] <cjwatson> wtf
[09:14] <apw> cjwatson, as this is a VM and the issue reproducible i can redo and get a cleaner log if that is any help
[09:15] <cjwatson> apw: I'm deeply confused, I think I need to see this in action - syncing an image for myself
[09:15] <cjwatson> busybox-static has been in installs for ages imo, dunno why this only just started breaking
[09:15] <cjwatson> busybox hasn't been changed for months
[09:16] <apw> cjwatson, ok now i am unsure if you can reproduce this or not at the moment as i have ended up here trying to find an issue which only hits with multi-lvm.preseed from the QA tests, and only then with 512M VMs
[09:16] <apw> cjwatson, and i have only seen this failure mode with the 'fixed' kernel as well, as without it oops in ext4
[09:16] <apw> cjwatson, so i am somewhat worried i have made this happen and i am wasting your time
[09:17]  * apw fires up another VM with the original image
[09:18] <cjwatson> so, busybox-static doesn't actually install the dpkg or find or whatever links
[09:18] <cjwatson> apw: do you still have the original vm running?
[09:18] <apw> cjwatson, sure do
[09:18] <cjwatson> the paths that might cause this kind of thing are in busybox's slightly tortuous "do I launch an applet or an external program" code
[09:19] <cjwatson> this is chrooted so that *should* be moot
[09:19] <cjwatson> 'chroot /target find --help'
[09:19] <cjwatson> does that give you the GNU findutils help, or something from busybox?
[09:19] <apw> busybox
[09:20] <apw> and /target is mounted off of --vg-root ...
[09:24] <cjwatson> apw: could you install strace in the installer environment ('udpkg -i /cdrom/pool/main/s/strace/strace_*.deb') and try stracing that?
[09:24] <cjwatson> and then copy out the strace for me
[09:25] <apw> stracing the chroot ?
[09:25] <cjwatson> Yeah
[09:25] <cjwatson> It's almost as if PREFER_APPLETS has got turned on
[09:25] <cjwatson> But I don't understand how that could have happened without changing busybox :)
[09:26] <cjwatson> Is this a self-built d-i initrd?
[09:26] <apw> cjwatson, it is hacked indeed, i have in theory only replaced the kernel and modules in it
[09:26] <apw> cjwatson, but i have opened and closed it, so i could easily be to blame
[09:26] <cjwatson> So I suspect you might have accidentally hacked a bit more than that
[09:26] <cjwatson> Let's see
[09:26] <cjwatson> What's the size of /bin/sh?
[09:26] <cjwatson> in the installer env
[09:27] <apw> 389928
[09:28] <apw> http://paste.ubuntu.com/5861092/
[09:28] <apw> cjwatson, ^^
[09:32] <cjwatson> Looking
[09:33] <cjwatson> Is /bin/sh a symlink to /bin/busybox/
[09:33] <cjwatson> ?
[09:33] <cjwatson> Actually maybe you could just put the initrd somewhere for me rather than me playing twenty-questions :)
[09:34] <cjwatson> busybox is following the ENABLE_FEATURE_PREFER_APPLETS path here, but I can't see why, since that's meant to be compiled out
[09:35] <apw> cjwatson, uploading now
[09:36] <apw> it is a big big, for various foolish error reasons on my behalf :)
[09:36] <apw> though for cirtain i am not replacing busybox or anything, deliberatly anyhow
[09:37] <apw> any my proceedure for replacement is cpio not using any tools
[09:38] <apw> cjwatson, http://people.canonical.com/~apw/misc/initrd.gz
[09:41] <cjwatson> cpio with what arguments?
[09:42] <apw> cpio --quiet --dereference -o -H newc
[09:42] <apw> stolen from mkinitramfs-tools sometime back
[09:42] <cjwatson> kill --dereference
[09:42] <cjwatson> that's almost certainly your problem
[09:42] <cjwatson> d-i does not build with that - it doesn't use initramfs-tools
[09:42] <apw> ahh ... great.  i'll retest with that and let you kn
[09:42] <apw> know
[09:43] <apw> i knew it had to be me
[09:43] <cjwatson> not totally obvious why that broke it but it's the sort of thing that messes with busybox's delicate head :)
[09:44] <apw> heh ... that i can believe, thanks for all your help, infinity informs me i am doing this the hard way and is going to educate me
[10:03] <apw> cjwatson, hmmm if i did it right, that didn't help
[10:04] <cjwatson> OK, stick the new initrd up for me and I'll grab it
[10:04] <apw> i wonder if i extracted it in a way which already lost the links ...
[10:05] <cjwatson> That's possible.  How did you extract it?
[10:05] <cjwatson> You might be better off building it using the proper d-i build system, assuming you already have the udebs.
[10:05] <cjwatson> I never unpack/rebuild d-i initrds by hand.
[10:06] <apw> zcat "$iso/install/initrd.gz" | ( cd initrd && sudo cpio -i )
[10:06] <apw> cjwatson, i got into this because the isos i was originally changing were special ones used in QA and normal ones weren't triggering the error so i needed to debug that actual iso
[10:07] <apw> (this time round it is just how i am used to doing it now)
[10:08] <cjwatson> Might need --link, not sure.  Like I say I don't normally do it that way
[10:09] <apw> cjwatson, yeah, i'll play some, now i know its likely my fault and compare the two
[10:09] <cjwatson> Normally I take debian-installer, make sure its kernel versions in build/config/ are right, put the modified udebs I'm using in build/localudebs/, and run 'fakeroot make -C rebuild_cdrom_isolinux'
[10:09] <cjwatson> er
[10:09] <cjwatson> 'fakeroot make -C build rebuild_cdrom_isolinux'
[10:09] <cjwatson> Other build targets are available - 'make -C build' on its own lists them
[10:10] <apw> yeah ... i'll also have that a spin
[10:10] <apw> it is about time i became more ohfay (sp?) with that
[10:10] <infinity> That would be the easier method I was suggesting. :)
[10:10] <infinity> Or, at least, more foolproof.
[10:11] <infinity> apw: And I have no idea what an ohfay is...
[10:15] <cjwatson> au fait
[10:16] <infinity> Oh.  It's so cute when the English try to speak French.
[10:16] <infinity> Or, at least, spell it.
[10:20] <apw> i am renouned for my grasp of both spelling and foreign languages
[12:10] <apw> cjwatson, i am struggling to maek this di build (via make -C build) to use the localudebs, is there some magic i should be aware of
[12:11] <cjwatson> it should do it automatically
[12:11] <cjwatson> at least in cases where the same package name would have been used
[12:12] <cjwatson> is it a case of using the same package name but from the archive instead, or a case of trying to add an extra package?
[12:13] <apw> i am replacing all of the kernel udebs, it is building a Packages file
[12:13] <apw> which seems to list 'em just fine
[12:14] <apw> Package: kernel-image-3.10.0-2-generic-di
[12:14] <apw> Version: 3.10.0-2.10lp1195710v201307101138
[12:14] <apw> and at versions above the archive
[12:18] <cjwatson> and does build/config/amd64.cfg have the right version?
[12:18] <cjwatson> (have to go out now)
[12:19] <apw> cjwatson, i'll check and hastle infinity :)
[12:36] <Riddell> cjwatson: could you verify this fix is done right? http://bazaar.launchpad.net/~ubuntu-core-dev/grub-installer/ubuntu/revision/1235
[13:10] <apw> cjwatson, looking at the build logs it seems to be installing mine _and_ the ones from the archive, the latter last
[13:56] <cjwatson> Riddell: Er, functions and variables are in different namespaces and don't need to avoid sharing names
[13:56] <cjwatson> AFAIK
[13:57] <cjwatson> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_05  "The implementation shall maintain separate name spaces for functions and variables"
[13:57] <cjwatson> Riddell: So nack without a better explanation of what's going on ...