[03:28] <CIA-7> debian-installer: adconrad * r1715 ubuntu/ (build/config/armel/omap4.cfg debian/changelog): Move omap4 kernels to 3.4.0-203.
[08:06] <jibel> xnox, cjwatson if there is no fix for bug 1020574, is there a possibility to revert apt and python-apt (if the problem comes from here) to make the images working again ?
[08:06] <ubot2> Launchpad bug 1020574 in ubiquity "SystemError: Broken pipe while installing language packs" [Critical,Confirmed] https://launchpad.net/bugs/1020574
[08:25] <cjwatson> haha, good luck with that
[08:25] <cjwatson> I don't think that's terribly plausible TBH
[08:27] <xnox> cjwatson: well i *think* i can comment the offending line out of apt (it's just a status update message) to unbreak ubiquity.... but it's still weird why that pipe is closed, it is used to communicate progress back to ubiquity
[08:27] <xnox> 17339 15:57:48.385417 write(79, "pmstatus:dpkg-exec:0:Running dpk"..., 34 <unfinished ...>
[08:27] <xnox> 20429 15:57:48.385458 <... read resumed> "pmstatus:dpkg-exec:0:Running dpk"..., 4096) = 34 <0.143676>
[08:27] <xnox> 17339 15:57:53.068803 write(79, "pmstatus:firefox-locale-en:2:Pre"..., 57) = -1 EPIPE (Broken pipe) <0.000025>
[08:28] <cjwatson> is there a useful subset of the strace you can post somewhere?
[08:28] <cjwatson> I would be happy to help analyse it
[08:30] <cjwatson> 900M is a bit much, but maybe something that starts from the common ancestor of pids 17339 and 20429
[08:31] <xnox> it's only 60MB when xz compressed ;-)
[08:31] <xnox> http://paste.ubuntu.com/1077753/
[08:31] <xnox> i should have split the strace per pid/fork
[08:32] <xnox> ok, let me try to grep that subset.
[08:33] <brendand> cjwatson, any luck reproducing bug 1021293?
[08:33] <ubot2> Launchpad bug 1021293 in checkbox-satellite "Ubuntu 12.04 install stalls when doing apt-get upgrade" [Undecided,New] https://launchpad.net/bugs/1021293
[08:33] <cjwatson> brendand: not as yet sorry
[08:34] <cjwatson> xnox: oh, 60M is practical for me to download if it's practical for you to upload
[08:34] <cjwatson> better to have the whole thing if possible
[08:34] <xnox> cjwatson: link in the bug 1020574
[08:34] <ubot2> Launchpad bug 1020574 in ubiquity "SystemError: Broken pipe while installing language packs" [Critical,Confirmed] https://launchpad.net/bugs/1020574
[08:34] <xnox> cjwatson: you can even ssh into the box....
[08:35] <xnox> http://people.canonical.com/~xnox/ubiquity.trace.xz
[08:35] <cjwatson> ETA 4min
[08:36] <xnox> cjwatson: of the fix uploaded? WOW you are quick ;-)
[08:36]  * xnox kidding
[08:37] <cjwatson> hah, no
[08:52] <cjwatson> well, OK, so the process reading from that pipe has exited
[08:53] <cjwatson> it never does very much; I wonder what it is
[08:54] <xnox> -        write(OutStatusFd, status.str().c_str(), status.str().size());
[08:54] <xnox> +        FileFd::Write(OutStatusFd, status.str().c_str(), status.str().size());
[08:54] <xnox> this is the code in apt, which prints the status line back.
[08:54] <xnox> ... it was changed....
[08:55] <cjwatson> Yeah, but my suspicion currently rests on the other end of the pipe
[08:55] <xnox> ok.
[08:55] <cjwatson> This sort of thing can arise from a missing close-on-exec flag
[08:55] <cjwatson> But just a guess so far
[08:56]  * cjwatson has sudden nostalgia for his first real job interview, which involved on-the-fly strace analysis
[08:56] <cjwatson> 17339 is plugininstall.py
[08:57] <cjwatson> string lengths in this strace are unfortunately short, but I guess that was for practicality?
[08:57] <cjwatson> makes it hard to find some things
[09:01] <xnox> well, it took ~2-3h to get the stack trace. cause ubiquity was slow under it. I can run it again if you know options which will reduce the cruft and increase additional info.
[09:01] <xnox> sorry, if I missed stuff....
[09:02] <cjwatson> I usually bump -s to something bigger, maybe 512 for this; but don't worry for now
[09:03] <cjwatson> So, we're in the call to cache.install_archives in commit_with_verify, clearly
[09:03] <cjwatson> The PROGRESS START message is therefore from DebconfInstallProgress.startUpdate
[09:04] <cjwatson> So the clone of 20429 is from something called immediately after the startUpdate hook
[09:04] <cjwatson> Ah, which is .run of course
[09:08] <cjwatson> Hm, it strikes me as a little odd that we're using "while self.update_interface():" but update_interface does not generally seem to return a vaue
[09:08] <cjwatson> *value
[09:08]  * cjwatson looks at older python-apt
[09:13] <cjwatson> So that might well be the real source of the problem; but how are you supposed to tell from update_interface when it's done?
[09:13] <xnox>    def update_interface(self):
[09:13] <xnox>         apt.progress.base.InstallProgress.update_interface(self)
[09:13] <xnox>         self.frontend.refresh()
[09:14] <xnox> wrapped in ubiquity...
[09:15] <cjwatson> Also wouldn't hurt to update the API
[09:15] <cjwatson> xnox: Not relevantly, that's in auto_update.py
[09:16] <xnox> cjwatson: yeah...
[09:16] <cjwatson> And even then, I'm having trouble seeing a way to detect EOF without copying all of update_interface, not just wrapping it
[09:17] <CIA-7> ubiquity: cjwatson * r5534 trunk/ (debian/changelog ubiquity/install_misc.py): Update DebconfInstallProgress method names for the python-apt 0.8 API.
[09:18] <cjwatson> Ah - python-apt itself uses a select loop
[09:21] <cjwatson> Not sure why it bothers with a timeout
[09:27] <cjwatson> Maybe the simplest answer is (a) use a select loop rather than spinning (b) don't worry about a sensible exit condition and instead just kill the status-to-debconf subprocess when pm.do_install completes
[09:28] <cjwatson> xnox: You're correct, I think, that the apt change you pointed to above exposed this problem, but it was never correct in ubiquity AFAICS
[09:28] <cjwatson> And this probably resulted in horrendous inefficiency and/or wildly incorrect progress reporting
[09:29] <cjwatson> So reverting apt is the wrong answer; we should instead fix the bug it exposed
[09:30] <cjwatson> I'm thinking http://paste.ubuntu.com/1077814/
[09:32] <xnox> hmm... do you want me to test it?
[09:33] <cjwatson> If you can, though I'm trying too
[09:33] <cjwatson> Also a sanity check :-)
[09:33] <cjwatson> (I need to confirm that I can reproduce the bug first)
[09:34] <xnox> =0)))
[10:13] <cjwatson> Hmm, that patch gets me to the slideshow with no progress messages
[10:14] <xnox> cjwatson: you did add missing imports right?! =)
[10:14] <cjwatson> plugininstall isn't even running ...
[10:14] <xnox> import select; import signal...
[10:14] <cjwatson> xnox: err, *cough*, I was misled by pyflakes.vim not telling me about them :)
[10:14] <cjwatson> thanks
[10:16] <xnox> cjwatson: i'm off to the office. we'll be back online in ~40minutes
[10:17] <cjwatson> right.  I have to wait for dpkg to be fixed anyway before we can build anything
[10:19] <cjwatson> bah, syntaxerror too
[10:47] <cjwatson> better now
[10:48] <CIA-7> ubiquity: cjwatson * r5535 trunk/ (debian/changelog ubiquity/install_misc.py):
[10:48] <CIA-7> ubiquity: Rearrange DebconfInstallProgress to not assume that update_interface
[10:48] <CIA-7> ubiquity: returns a value (LP: #1020574).
[10:51] <CIA-7> ubiquity: cjwatson * r5536 trunk/ (d-i/manifest debian/changelog):
[10:51] <CIA-7> ubiquity: Automatic update of included source packages: flash-kernel
[10:51] <CIA-7> ubiquity: 3.0~rc.4ubuntu5, partman-base 158ubuntu1, partman-target 78ubuntu1,
[10:51] <CIA-7> ubiquity: preseed 1.53ubuntu1.
[10:58] <CIA-7> ubiquity: cjwatson * r5537 trunk/debian/changelog: releasing version 2.11.9
[12:26] <xnox> cjwatson: ubiquity became far more chatty now. "removing this" "configuring that"
[12:27] <cjwatson> yeah, it would do
[12:27] <xnox> previously it seemed to me to stick at "installing..." during the langpack installation step =)
[12:27] <xnox> i like it, previously I had to expand the thinggy to see what is going on =)
[12:28] <xnox> cjwatson: is the twitter feed not running intentionally? Can we have it on hashtags #quantal and #12.10 ?
[12:28] <xnox> too user friendly, geeks need scrolling text ;-)
[12:29] <cjwatson> no intention on my part
[12:29] <cjwatson> feel free to make sensible adjustments :)
[12:31] <xnox> cjwatson: well LVM auto-install now succeeds. Upon reboot it drops into initrd shell =) I bet the lvm2 modules were not added... and initramfs was not regenerated....
[12:36] <cjwatson> IIRC d-i does that in base-installer, so you'd need to duplicate that
[12:37] <cjwatson> maybe via register-module?  I don't quite remember
[12:38] <cjwatson> hmm.  looking now I can't find where it's done.
[12:39] <cjwatson> the lvm2 initramfs hook, I guess.  so that's odd because I thought we unconditionally regenerated the initramfs.
[12:39] <cjwatson> maybe an ordering thing?
[12:40] <cjwatson> is the lvm2 package installed in the target?
[12:41] <cjwatson> ah - perhaps you need to copy partman-lvm's post-base-installer hook somewhere useful for ubiquity, or duplicate it
[12:41] <cjwatson> this is a bit tedious, there's piles of crap in debian/rules that deals with similar kinds of stuff, or sometimes it's duplicated in components or scripts
[13:07] <xnox> cjwatson: right. I think it's because lvm2 was not in the squashfs and instead it's just "installed" in the live session.
[13:07] <xnox> because the update-initramfs is done at the end of the install.
[13:08] <xnox> and lvm binaries did not appear in the list of "things not copied into the target"
[13:08] <xnox> and it was not installed on the target
[13:09] <xnox> cjwatson: shall I remaster the ISO or should I do a merge proposal with my work?
[13:09] <xnox> cjwatson: after chrooting into mounted target /boot and / and doing # apt-get install lvm2 & doing a reboot it works like a charm =)
[13:10] <xnox> cjwatson: cause I don't see anything special done, for e.g. btrfs-tools
[13:11]  * xnox maybe I should remaster an ISO for training purposes.
[13:16] <cjwatson> xnox: I'd like to see an MP
[13:16] <xnox> cjwatson: ok.
[13:16] <cjwatson> is apt-install called for lvm2?
[13:16] <cjwatson> you can tell by looking in /var/lib/ubiquity/apt-installed
[13:26] <xnox> cjwatson: in the target or the live cd during installation?
[13:26] <xnox> (i finished the install and have /var/log/installer/*)
[13:29] <xnox> nothing in after logs =(
[13:29] <xnox> only /var/lib/ubiquity/os-prober-cache left.
[13:32] <cjwatson> in the live cd during installation
[13:32] <cjwatson> well, at the end, before rebooting
[13:34] <xnox> ok. let me double check.
[13:35] <xnox> cjwatson: mpt has finished: manual & automatic crypt; manual lvm.
[13:36] <xnox> so that's what I am aiming for the feature freeze.
[13:36] <xnox> cjwatson: will that be enough to drop alternative? (no raid...)
[13:40] <cjwatson> hmm, not sure
[13:40] <cjwatson> maybe, since desktop RAID (not DM-RAID) isn't a big thing really
[13:40] <brendand> cjwatson - we have lab presence again. is there any extra info you'd like regarding the upgrade bug?
[13:41] <cjwatson> might be worth looking around to see if people commonly report using the alternate CD in order to use RAID
[13:41] <cjwatson> brendand: not right now, thanks.  sorry, my day has been (a) critical ubiquity bug in quantal (b) server CD -> squashfs work requested increasingly urgently by Daviey (c) working on a regression in recent Launchpad changes (d) this upgrade bug
[13:42] <brendand> cjwatson, i understand
[13:45] <gema> smoke testing bug, easy to fix I hope, bug 1021718
[13:45] <ppisati> hi guys
[13:45] <ubot2> Launchpad bug 1021718 in debian-installer "Ubuntu Precise ISO test failed in Jenkins due to debian installer failed to get debconf answer 'base-installer/kernel/linux/initrd'." [Undecided,Confirmed] https://launchpad.net/bugs/1021718
[13:45] <ppisati> http://people.canonical.com/~ppisati/Jul062012_omap3_preinstall_brk/
[13:45] <ppisati> while trying preinstall server omap3
[13:45] <ppisati> ubiquity.install_misc.InstallStepError: Plugin console_setup failed with code 1
[13:45] <ppisati> i've both syslog and oem-config.log
[13:46] <ppisati> the installer kept restarting
[13:46] <mpt> cjwatson, next week I'll polish off automatic crypt design and then go on to RAID design.
[13:46] <ppisati> until it gave up and landed me in a # console
[13:46] <xnox> ogra_: i think you might want to see ppisati question ^^^^
[13:46] <ogra_> xnox, i sent him here :)
[13:47] <cjwatson> gema: stuck on a soyuz bug / new kernel upload
[13:47] <cjwatson> not an installer bug
[13:47] <gema> cjwatson: who should I notify?
[13:47] <cjwatson> the kernel team already know about it
[13:47] <gema> cjwatson: ack
[13:47] <cjwatson> I'll nag them
[13:47] <gema> thanks
[13:48] <xnox> ogra_: ping pong eh? =)
[13:48] <xnox> ppisati: can you pastebin the logs?
[13:48] <ogra_> xnox, well, we were in #ubuntu-arm but i think its not arch specific at all
[13:48] <ppisati> xnox: available here - http://people.canonical.com/~ppisati/Jul062012_omap3_preinstall_brk/
[13:48] <cjwatson> how's it not arch-specific?
[13:48] <cjwatson> Jun 27 02:39:55 localhost ubiquity: Unsupported platform.
[13:48] <cjwatson> Jun 27 02:39:55 localhost ubiquity: run-parts: /etc/initramfs/post-update.d//flash-kernel exited with return code 1
[13:49] <xnox> ppisati: ideally run ubuntu-bug ubiquity. or paste. Oh ok =)
[13:49] <ogra_> console_setup exploding is arch specific ?
[13:49] <ogra_> (unsupported arch is fine, just fixed that (pending upload))
[13:49] <cjwatson> console-setup calls update-initramfs
[13:49] <cjwatson> update-initramfs explodes because the flash-kernel hook explodes
[13:49] <ogra_> oh, sigh ... yeah
[13:49] <ogra_> sorry
[13:49] <cjwatson> the top-level error message is just a bit misleading, that's all
[13:50] <ogra_> yep, should have thought about the initramfs bits
[13:50] <ppisati> cjwatson: there some more errors in oem-config
[13:50] <ppisati> sh: 1: killall.sh: not found
[13:50] <ppisati> sh: 1: rdisc6: not found
[13:50] <ppisati> etcetc
[13:50] <ppisati> is it ok?
[13:50] <cjwatson> ppisati: don't care
[13:51] <cjwatson> gema: actually, from https://launchpad.net/~canonical-kernel-team/+archive/ppa/+packages I see that it's already building, so no need/point nagging the kernel tam
[13:51] <ppisati> ack
[13:51] <cjwatson> *team
[13:51] <gema> cjwatson: awesome, thanks
[13:51] <cjwatson> ppisati: that's just bits of netcfg that (a) don't work outside d-i but (b) don't really matter here either
[13:52] <ppisati> ok
[13:54] <cjwatson> gema: https://irclogs.canonical.com/2012/07/06/%23launchpad-ops.html#t07:21 has the gory details if you care :)
[13:54] <cjwatson> (hideous LP internals)
[13:55] <gema> cjwatson: thanks!
[13:58]  * cjwatson grumbles and queue-jumps those PPA builds
[14:07] <xnox> cjwatson: so lvm2 is not in /var/lib/ubiquity/* but in the apt-removed there is cryptsetup.
[14:09]  * xnox mouse is stuck =(
[14:12] <cjwatson> ok, that sounds like you need to hunt down why apt-install wasn't called then; I thought partman-lvm or some such did that
[14:13] <xnox> ok.
[14:13]  * ogra_ sends one of his cats to xnox (that will teach the mouse !)
[14:14] <xnox> cjwatson: well apt-install was called, with a few packages, but not lvm2. but yeah, checking now.
[15:20] <xnox> I am not going to ask why livecd has /lib/systemd/
[15:20]  * xnox *sigh*
[15:21] <cjwatson> bunch of packages shipping systemd service files as well
[15:28] <xnox> cjwatson: so, partman-btrfs does apt-install in the partman/finish.d script
[15:28] <xnox> but crypto,lvm,raid do it in the scripts shipped under /usr/lib/post-base-installer.d/
[15:28] <xnox> cjwatson: should I add a plugin to iterate across scripts in the /usr/lib/post-base-installer.d?
[15:29] <xnox> or should I shove those scripts into partman/finish.d for ubiquity to pick up?
[15:29] <xnox> or is btrfs actually apt-installed from partman/finish.d in ubiquity?
[15:29]  * xnox looks what you said above
[15:30] <cjwatson> standard (perhaps a bit shonky) practice is to do something like invoking them from an InstallPlugin (or whatever it's called)
[15:30] <xnox> let me look.
[15:31] <cjwatson> I was a bit reluctant to iterate across everything since (a) that path is nasty for ubiquity and ought to be under /usr/lib/ubiquity/ or something (b) I wasn't sure all the p-b-i scripts were sane (c) you'd still have to have debian/rules or similar code to arrange for them to be installed
[15:31] <cjwatson> partman/finish.d should be run automatically
[15:32] <cjwatson> there is an argument that at least the lvm one belongs in finish.d
[15:32] <cjwatson> I haven't looked at the others
[15:32] <cjwatson> certainly moving to finish.d would be least code
[15:34] <xnox> what is finish-install.d vs finish.d vs post-base-installer.d ? where can I check when what is called?
[15:44] <cjwatson> finish.d is called from partman itself (well, partman-commit in ubiquity)
[15:45] <cjwatson> post-base-installer.d and finish-install.d are both d-i-specific; they're called, respectively, at the end of base-installer and at the very end of installation
[15:45] <xnox> partman-crypto uses all three: finish.d post-base-installer.d and finish-install.d
[15:46] <xnox> and e.g. d-i-utils has post-base-installer.d to add/blacklist modprobe modules
[15:51] <cjwatson> there's a d-i internals document linked from Installer/Development that IIRC goes through all this
[15:54] <xnox> thanks, let me find it.
[15:54] <xnox> cjwatson: the p-b-i does ugly crap, like raising pointless debconf questions and confirmation dialogs
[20:53] <Sargun_Screen> How would one update a package on the netinstall (PXE) - specifically if they were running it as a netboot
[20:54] <Sargun_Screen> The mdadm package causes my kernel to fail
[21:07] <cm-t> omg! I just successfully installed an array list of package (vlc, vrms) with ubiquity, I was so making something far while the code was allready there !
[21:07] <cm-t> (hi)
[21:07] <stgraber> apt-install vlc vrms
[21:07] <stgraber> :)
[21:07] <cm-t> i mean
[21:07] <cm-t>  getting the good /target
[21:08] <cm-t> it was just  self.do_install(packageArray)
[21:08] <cm-t> >.<
[21:09] <cm-t> was trying so many thing, and did not even scroll at the bottom of the script where this wonderfull method was
[21:11] <cm-t> just need to work this week-end with my group mates to get this with oneconf (they made I think, just some sso problems) and we might consider having something running (not talking about a clean code for the while ^^)
[21:12] <cm-t> I will update blueprint this week-end cjwatson, mpt
[21:23] <infinity> cjwatson: Say, opinion.  Since we fixed #1010708 in -proposed (and soon, updates), and since those files/directories are managed in a dubious and poorly-tracked manner anyway, do you have any objections to just "fixing" the bug in the release pocket by moving the two uI* files where they belong, so there isn't inconsistency between release and updates (and later releases)?
[21:24] <infinity> cjwatson: Oh, I guess that would break the manifest too.  Hrm.
[21:25] <infinity> cjwatson: Oh, wait.  But the *SUMS could just be fixed and re-signed.  That doesn't seem like an awful solution for a port in flux.  Worth a ponder.
[22:02] <cjwatson> I'm not at all wild about changing the release pocket of a stable release, although I guess the number of people who would notice or care would be small
[22:02] <cjwatson> the *SUMS files would be automatically re-signed if they change
[22:24] <infinity> Yeah, I figured that.
[22:24] <infinity> And I'm not a big fan of changing release pockets either, but this isn't a Packages file or anything, and it seems like minimal impact to avoid confusion.
[22:24] <infinity> I think I'll JFDI and ask for forgiveness. :P
[22:31] <cjwatson> Yeah.