[00:04] <ev> Riddell: what partitioning method did you choose in bug 726581?
[00:04] <ubot4> Launchpad bug 726581 in ubiquity (Ubuntu) "install stops half way through (affects: 3) (dups: 1) (heat: 3510)" [Undecided,New] https://launchpad.net/bugs/726581
[00:10] <jdstrand> skaet_: re bug #727478. it is actually worse than I initially thought
[00:10] <ubot4> Launchpad bug 727478 in mysql-5.1 (Ubuntu Natty) (and 3 other projects) "mysql upgrade hang at 'installing new version of config file /etc/init/mysql.conf' during upgrade from 10.10 to 11.04 (affects: 1) (heat: 6)" [High,Invalid] https://launchpad.net/bugs/727478
[00:11] <jdstrand> skaet_: it is an apparmor userspace built on natty's kernel on a maverick kernel issue
[00:11] <jdstrand> skaet_: it shouldn't be horribly hard to fix-- jjohansen is looking at a fix this evening
[00:12] <jdstrand> skaet_: mysql users upgrading will get bit by the bug and the upgrade will interminably hang due to how the upstart job is written
[00:12] <skaet_> jstrand:  thanks,  not quite sure I'm parsing "on natty's kernel on a maverick kernel" - can you expand a bit.
[00:12] <jdstrand> skaet_: other apparmor consumers will be affected as well, but the severit is unknown atm
[00:12] <jdstrand> skaet_: after reboot all is fine
[00:12] <cjwatson> (apparmor userspace built on natty's kernel) on a maverick kernel
[00:13] <cjwatson> bracket it thus and it makes more sense
[00:13] <jdstrand> yes
[00:13] <skaet_> yup that helps.
[00:14] <jdstrand> skaet_: now, this only affects upgrades, not installs
[00:14] <skaet_> jdstrand, so there is a workaround to do a reboot?  does that work for mysql users or just the other cases.
[00:14] <jdstrand> skaet_: but an upload of apparmor means it is on all the CDs
[00:14] <jdstrand> skaet_: well, rebooting isn't a workaround, it is just things are correct then
[00:15] <jdstrand> skaet_: mysql users have to stop the mysql upgrade in some manner and revocer from a partial do-release-upgrade
[00:15] <jdstrand> skaet_: it is not pretty
[00:15] <skaet_> jdstrand: agreed.
[00:16] <jdstrand> skaet_: so the question is timing. people are working on a fix, and we should hopefully have something tonight
[00:16] <slangasek> jdstrand: it's generally ok to have newer packages in the archive than are on CDs at the time of alpha release; I don't think that should block you from fixing a critical upgrade bug
[00:16] <slangasek> (assuming skaet_ agrees)
[00:16] <jdstrand> skaet_: the question is do you want it on the CDs, if so when is the deadline
[00:17] <jdstrand> skaet_: or what slangasek said, which is we upload before alpha3 is announced so upgraders can get it
[00:17] <jdstrand> I'm partial to the second option
[00:17] <skaet_> jdstrand, slangasek,  sorry for the slowness of response,  thinking
[00:19] <skaet_> jdstrand, slangasek,  yeah,  lets keep the image building and have this uploaded to the archive, and document it in the release notes that its a known issue.  Not good, but I think we're running out of options.
[00:20] <jdstrand> skaet_: what time do you plan to announce alpha3?
[00:21] <jdstrand> skaet_: I can just make sure that it is in there before then, so any inspired 10.10 users will get it
[00:21] <skaet_> jdstrand,  alpha3 will be announced on Thursday afternoon, evening, after we know we have reasonable coverage from the QA team.
[00:21] <jdstrand> skaet_: cool, thanks. I'll pass that along and keep you updated
[00:21] <skaet_> thanks jdstrand.
[00:23] <slangasek> waiting for this ubiquity build really makes me want qemu armel builders
[00:23] <skaet_> slangasek,  have the rebuilds triggered yet with that last upload?
[00:23] <skaet_> heh
[00:23] <slangasek> nope, still waiting :)
[00:24] <skaet_> ok,  I'm going to break for dinner,  will be back in an hour or so.
[00:25]  * skaet_ crossing fingers ready we have a good set when jibel wakes up. 
[00:25] <cjwatson> hmm, I should have started building CDs a while back, per pitti's comment
[00:25] <cjwatson> oh, but slangasek has them queued
[00:25] <cjwatson> you know, I think I need coffee
[00:25] <cjwatson> bug 723482 is kicking my ass
[00:25] <ubot4> Launchpad bug 723482 in mountall (Ubuntu) "system hangs on boot after updates from 2011-02-22 (affects: 14) (heat: 78)" [High,Confirmed] https://launchpad.net/bugs/723482
[00:26] <slangasek> :/
[00:28]  * slangasek taps his fingers on the desk and stares intently at the I/O bound armel builder
[00:29] <slangasek> I could put that .deb together faster by hand with a magnet and a pottery wheel
[00:29] <cjwatson> oh
[00:29] <cjwatson> of course
[00:30] <cjwatson> mounted MOUNTPOINT=/usr and mounted MOUNTPOINT=/var => the first mounted event waits forever for alsa-restore to come back => deadlock
[00:30]  * cjwatson stupid
[00:30] <slangasek> doh
[00:31] <slangasek> is that in a package?
[00:31] <slangasek> 'cause neither of those are supposed to appear in shipping upstart jobs, you don't get any mounted events at all for paths that aren't mount points
[00:32] <cjwatson> alsa-utils
[00:32] <slangasek> sigh
[00:32] <cjwatson> it should just be 'filesystem'
[00:32] <slangasek> yep
[00:32] <cjwatson> I'll fix it up in a bit
[00:36] <cjwatson> slangasek: in fact, the whole thing is redundant since it's also 'start on runlevel [2345]'
[00:36] <slangasek> cute
[00:36] <cjwatson> and runlevel [2345] is always after filesystem ...
[00:43] <slangasek> there we go, ubiquity/armel built, publisher going
[00:44] <GrueMaster> slangasek: is that a fix for bug 727468?
[00:44] <ubot4> Launchpad bug 727468 in ubiquity (Ubuntu Natty) (and 1 other project) "oem-config crashed during install on armel (affects: 1) (heat: 8)" [Critical,New] https://launchpad.net/bugs/727468
[00:44] <slangasek> no
[00:44] <GrueMaster> so, what's the point of respinning?
[00:45] <slangasek> I'm respinning desktop cds
[00:45] <slangasek> oh, there aren't actually any arm desktop cds building these days, are there
[00:45] <slangasek> so I guess I didn't need to wait
[00:45] <GrueMaster> Oh.  But I thought you were waiting for ubiquity to publish.
[00:45] <cjwatson> other architectures
[00:46] <GrueMaster> Well, we will need to push another update and respin anyways if we can't get this fixed.
[00:47] <slangasek> I would have thought you need a respin if you /do/ get it fixed?
[00:47]  * cjwatson doesn't have the faintest idea what's going on from the debug log in 727468, I'm afraid
[00:48] <cjwatson> I expect that somebody with some degree of installer experience needs to debug it locally
[00:49] <GrueMaster> I'll light a fire under ncommander and get him on it.
[00:50] <cjwatson> thanks
[01:46] <skaet_> slangasek, will you be publishing the images this evening as they emerge from the builder or do you need my help?
[01:47] <slangasek> sure, I'm around and can publish them to the tracker
[01:55] <skaet_> slangasek,  thanks!   will focus on the Tech Overview, and then get some sleep
[01:55]  * skaet_ expects its going to be a late couple of next nights.
[01:55] <slangasek> skaet_: ok :)
[03:02] <slangasek> ubuntu, kubuntu desktop CDs re-posted
[03:29]  * highvoltage wonders if some of those errors in LP: #727468 are related to bug 723357
[03:29] <ubot4> Launchpad bug 723357 in casper (Ubuntu) "ISO filesystem not mounted properly on Live USB disks (affects: 1) (heat: 503)" [Undecided,New] https://launchpad.net/bugs/723357
[06:02] <slangasek> cjwatson: ah, I see you already knew about bug #727603 still being an issue... ran across it this evening because it was still causing trouble for smoser's UEC builds, and I'm wondering if it could be related to the oem-config troubles
[06:02] <ubot4> Launchpad bug 727603 in dpkg (Ubuntu Natty) (and 1 other project) "/var/lib/dpkg/info/$arch still a directory on new installs (affects: 1) (heat: 6)" [High,New] https://launchpad.net/bugs/727603
[06:06] <slangasek> don't see anything in oem-config that would account for it
[06:26] <slangasek> GrueMaster, ogra: ^^ a test for this would be to boot a preinstalled image, pull up a root terminal, and run 'mv /var/lib/dpkg/info/armel/* /var/lib/dpkg/info/ && rmdir /var/lib/dpkg/info/armel && ln -sf . /var/lib/dpkg/info/armel'; if that solves bug #727468, we probably want another dpkg upload to fix this
[06:26] <ubot4> Launchpad bug 727468 in ubiquity (Ubuntu Natty) (and 1 other project) "oem-config crashed during install on armel (affects: 1) (heat: 8)" [Critical,New] https://launchpad.net/bugs/727468
[06:28] <GrueMaster> slangasek: Thanks, I'll try that shortly.  Currently looking at other possible angles.
[06:45] <GrueMaster> slangasek: btw:  I tried reproducing this on x86 in a vm.  Unfortunately after doing an oem install and reboot, oem-config is gone and there is nothing to indicate that the user needs to configure the system.
[07:04] <pitti> Good morning
[07:08] <pitti> ah, so the new dpkg required rebuilding all the desktops?
[07:08] <pitti> but not the alternates?
[07:23] <GrueMaster> slangasek: Your proposed solution is already there. ls -l /var/lib/dpkg/info/armel
[07:23] <GrueMaster> lrwxrwxrwx 1 root root 1 2011-03-01 11:26 armel -> .
[07:23] <GrueMaster> So that can't be it.
[07:23] <slangasek> pitti: yes, the alternates don't call dpkg --root
[07:23] <slangasek> GrueMaster: ok, thanks for checking
[07:24] <slangasek> I think that means I'm off the hook as far as having broken the preinstalled images is concerned
[07:24] <slangasek> don't see how it can be multiarch's doing
[07:26] <GrueMaster> Awww.  We'll still think of you.  :P
[07:26] <pitti> slangasek: do you think bug 727603 affects the desktops/alternates/DVDs as well? (they mostly seem to work, but there might be suble breakage)
[07:26] <ubot4> Launchpad bug 727603 in dpkg (Ubuntu Natty) (and 1 other project) "/var/lib/dpkg/info/$arch still a directory on new installs (affects: 1) (heat: 6)" [High,New] https://launchpad.net/bugs/727603
[07:27] <slangasek> pitti: it definitely /affects/ some set of these, but the last image it was known to /break/, the uec image, is now sorted
[07:28] <slangasek> I think this is just something to clean up post alpha
[07:29] <slangasek> unless someone shows some other way that it's breaking us, I'd rather we take our time now and fix it the right way once the dpkg upstream changes are finalized, since it seems the subdirectory is going away as quickly as it came
[07:29] <pitti> slangasek: i. e. replace it with a symlink for the 'native' arch, and leave the other $arch directories alone?
[07:29] <slangasek> if there are any other $arch directories
[07:30] <slangasek> nobody should be doing that yet, so yeah, they get to keep both pieces on upgrade
[07:30] <pitti> .. which will only be the case if people actually play with multiarch
[07:30] <pitti> i. e. flashplugin-installer and friends still use the old method with ia32-libs, so it shouldn't hit a lot of people
[07:30] <slangasek> it shouldn't hit anyone who doesn't have my phone number :)
[07:30] <pitti> slangasek: can it happen that dpkg actually puts any files into $arch/?
[07:30] <pitti> lol
[07:30] <slangasek> only for $arch == $native
[07:31] <pitti> I just wonder if we can end up in a situation where both info/ and info/$arch/ have files for the same package
[07:31] <slangasek> right now it puts *all* files there, and $arch/ is just conveniently a symlink in the common case
[07:31] <slangasek> dpkg is writing everything to the info/$arch/ path currently
[07:31] <slangasek> so, not too painful to unwind
[07:32]  * pitti checks his desktop install from yesterday
[07:32] <pitti> right, info/ just has dpkg.list and arch/
[07:33] <pitti> I take it dpkg.list is a leftover from debootstrap
[07:33] <slangasek> yep
[07:33] <slangasek> anyway, sleep for me
[07:33] <pitti> slangasek: thanks for the heads-up, sleep well!
[07:34] <slangasek> here's hoping for smooth testing
[07:34] <slangasek> g'night :)
[07:34] <pitti> slangasek: the live system seems alright, though
[07:35] <pitti> GrueMaster: do you know what's happening with the "rebuilding" arm images? what are these waiting for?
[07:36] <GrueMaster> A fix.
[07:36] <GrueMaster> Currently we die in oem-config, making the images absolutely useless.
[07:36] <pitti> GrueMaster: #727603 ?
[07:36] <GrueMaster> NCommander and I are currently debugging, but we wil be hading to bed soon (midnight here).
[07:37] <GrueMaster> Bug #727468
[07:37] <ubot4> Launchpad bug 727468 in ubiquity (Ubuntu Natty) (and 1 other project) "oem-config crashed during install on armel (affects: 1) (heat: 8)" [Critical,New] https://launchpad.net/bugs/727468
[07:37] <pitti> GrueMaster: that exception rings a bell
[07:38] <GrueMaster> 727603 doesn't appear to affect preinstalled.
[07:38] <pitti> the exception in that log is bug 711225
[07:38] <ubot4> Launchpad bug 711225 in python2.7 (Ubuntu Natty) (and 1 other project) "subprocess.Popen() crashed with TypeError in _cleanup(): an integer is required (affects: 3) (heat: 28)" [Medium,Confirmed] https://launchpad.net/bugs/711225
[07:38] <GrueMaster> I checked and /var/lib/dpkg/info/armel is a symlink to .
[07:38] <pitti> whoops, sorry, no; ignore me
[07:39] <GrueMaster> ignored.  :P
[09:06] <Riddell> ev: for bug 726581 I did use whole disk
[09:06] <ubot4> Launchpad bug 726581 in ubiquity (Ubuntu) "install stops half way through (affects: 3) (dups: 1) (heat: 20)" [Undecided,Incomplete] https://launchpad.net/bugs/726581
[09:20] <ev> Riddell: hm, there goes that theory.  Okay, full debug logs would be greatly appreciated then.
[09:23] <Riddell> ev: running ubiquity with --debug?
[09:26] <ev> correct
[09:28] <ogra> hmm,  doesnt look like GrueMaster and NCommander made any progress at all on bug 727468
[09:28] <ubot4> Launchpad bug 727468 in ubiquity (Ubuntu Natty) (and 1 other project) "oem-config crashed during install on armel (affects: 1) (heat: 8)" [Critical,New] https://launchpad.net/bugs/727468
[09:28] <ogra> not even a syslog for the --debug run
[09:28] <ogra> ev, any idea about that ?
[09:33] <ev> looking
[09:44] <cjwatson> GrueMaster: what version of debootstrap are you using to build your preinstalled images?
[09:49] <pitti> cjwatson: good morning
[09:49] <pitti> cjwatson: I figure we'll need to backport the $arch symlink in debootstrap to lucid?
[09:50] <pitti> I heard from smoser that he builds the ec2 images on lucid
[09:52] <ev> ogra: very strange. I can't see any reason it would be failing in that way, short of something else grabbing hold of debconf.  Any chance I could see a ps auxf from such an oem-config install attempt?
[09:52] <cjwatson> so I can trivially (and with minimum bureaucracy) shove it into {lucid,maverick}-backports
[09:53] <ev> when it fails, that is
[09:53] <mvo> ev: the new upgade-install feature is really excellent, I'm currently using it for a test-upgrade
[09:53] <cjwatson> perhaps that's the easiest approach
[09:53] <ev> mvo: I hope you caught the backups warning ;)
[09:53] <mvo> ev: backups, *pff*
[09:53]  * mvo pats bzr push
[09:54] <ev> mvo: you don't know where I live, right?
[09:54] <ev> oh
[09:54] <cjwatson> one thing is that the new debootstrap breaks old base-installer
[09:54] <ev> :)
[09:54] <cjwatson> but that's probably OK because we don't build installer images from -backports
[09:54] <mvo> ev: no, I don't - but I know where you work ;)
[09:54] <cjwatson> pitti: I dunno, would you prefer that, or a selective backport to -proposed?
[09:54] <ogra> cjwatson, whatever is installed on the livefs builders
[09:54] <ev> heh
[09:55] <cjwatson> ogra: the actual livefs is built inside a whatever-suite-you're-building chroot
[09:55] <ogra> i would expect that to be recent natty
[09:55] <cjwatson> so hopefully that's current, I forget exactly how often it's upgraded though
[09:55] <pitti> cjwatson: I'm fine with an SRU; however, I think we need to have a separate natty script then, instead of changing it all the way back into gutsy
[09:55] <cjwatson> pitti: why?  it's a harmless change for older releases
[09:56] <cjwatson> I'm trying to avoid creating new scripts
[09:57] <pitti> I'm just concerned that the same programs which started failing with the new symlink might also trip over the extra "subdir" (link)
[09:57] <pitti> stuff that iterates over all files might find an unexpected symlink, or might find files twice, etc.
[09:57] <cjwatson> hm, not that I've seen, but OK - in that case, I'd rather go with -backports for now
[09:57] <cjwatson> rather than more divergent code changes
[09:57] <pitti> ack
[10:00] <cjwatson> flushing
[10:14] <Riddell> ev: bug 727690
[10:14] <ubot4> Launchpad bug 727690 in ubiquity (Ubuntu) "install requests change of CD (dup-of: 726581)" [Undecided,New] https://launchpad.net/bugs/727690
[10:14] <ubot4> Launchpad bug 726581 in ubiquity (Ubuntu) "install stops half way through (affects: 3) (dups: 2) (heat: 26)" [Undecided,Incomplete] https://launchpad.net/bugs/726581
[10:15] <ev> thanks a bunch
[10:42] <ogra> ev, all i can see is a denconf-communicate -fnoninteractive ubiquity in ps
[10:43] <ogra> nothing special beyond that
[10:43] <ev> ogra: could you please attach the full listing to the bug?
[10:43] <ogra> from the ps ?
[10:43] <ogra> sure
[10:44] <ev> thanks
[10:44] <ogra> the prob is that it doesnt hang actually it dies, i'm not sure i'm fast enough to capture the exact moment
[10:48] <ogra> attached
[10:49] <ogra> ev, anything special you see there ?
[10:51] <ev> indeed not
[10:52]  * ogra is running out of ideas how to debug that
[10:57]  * ogra will tyr ps in a loop
[11:02] <davmor2> cjwatson, pitti:  is the top bar on install meant to be so small?  it looks to be about 4 px
[11:03] <pitti> davmor2: the panel you mean? no, should be some 30 pixel, the usual panel size
[11:04] <davmor2> pitti: is it a known bug?  and yes panel was the word I couldn't remember
[11:04] <pitti> not to me, anyway; I think you should report it
[11:04] <davmor2> pitti: will do
[11:06] <cjwatson> davmor2: ev deals with that stuff nowadays
[11:06] <pitti> thanks
[11:07] <davmor2> cjwatson: cool thanks
[11:08] <davmor2> ev: it's all your fault cjwatson said so ;)
[11:08] <ev> bug please
[11:08] <davmor2> ev: doing it now I have photos
[11:25] <davmor2> ev: https://bugs.launchpad.net/ubuntu/+source/gnome-panel/+bug/727726
[11:25] <ubot4> Launchpad bug 727726 in gnome-panel (Ubuntu) "gnome panel is about 4px instead of 30 on install (affects: 1) (heat: 6)" [Undecided,New]
[11:25] <davmor2> only happens durring install panel is right size post install
[11:27] <seb128> it's an ubiquity bug
[11:27] <seb128> the "install" mode bar is not gnome-panel
[11:28] <seb128> it's a custom ubiquity thing
[11:30] <davmor2> seb128: My bag
[11:31] <ogra> ev, attached three new files to the bug, i see a defunct debconf process while it dies, but i dont expect that to be much more informative
[11:35]  * ogra wonders if anyone else succeded doing an oem install on non armel
[11:49] <ogra> davmor2, i see the small panel too on armel oem installations
[11:50] <ogra> i think GrueMaster filed a bug for it
[11:50] <davmor2> ev: Yay I'm not the only one either is appears ^
[11:51] <ogra> davmor2, is that oem-config or plain ubiquity
[11:51] <davmor2> plain
[11:51] <ogra> k
[12:03]  * ogra is desparate
[13:11] <ogra> cjwatson, so how does some basic installer knowledge help me, debconf dies silently underneath the installer, setting DEBCONF_DEBUG=developer doesnt reveal any additional info beyond using --debug for oem-config
[13:11] <ogra> ps only shows a defunct debconf process when dieing
[13:12] <ogra> i'm really out of ideas how to debug that and would be happy for any kind of suggestion
[13:12] <ogra> (i even upgraded to the most recent ubiquity and dpkg versions in my preinstalled image without any results)
[13:16] <cjwatson> strace?
[13:16] <ogra> of what ? the debconf-communicate subprocess call ?
[13:19] <ogra> what i see is that debconf-communicate survives but the backend is gone
[13:20] <cjwatson> of everything
[13:20] <cjwatson> strace -f -o /root/oem-config.trace -s 1024 oem-config, or similar
[13:20] <cjwatson> modify whatever starts it
[13:20] <ogra> ah, from the upstart job  then
[13:20]  * ogra will try that, thanks !
[13:20] <cjwatson> I think I'd recommend a bit lower than that, otherwise you'll end up stracing X
[13:21] <ogra> oh, indeed
[13:21] <cjwatson> oem-config-wrapper should do
[13:22] <ogra> yep, just inspecting
[13:23] <ogra> the fun is that all the other debconf communication just works, its so weird to have it die later then
[13:26] <jibel> I have a different issue with oem on Ubuntu desktop - bug 727783
[13:26] <ubot4> Launchpad bug 727783 in ubiquity (Ubuntu) "oem-config is not installed after initial system installation, and the user can't proceed with the step 'prepare for shipping' (affects: 1) (heat: 6)" [Undecided,New] https://launchpad.net/bugs/727783
[13:54] <jibel> 727783 is confirmed by pedro on i386 as well
[13:54] <jibel> ev, ^
[14:09] <ogra> ev, cjwatson, strace info attached to the bug, i see a sigpipe even before the traceback kicks in there .... during some access to /etc/apt/apt.conf.d/ files
[14:10] <ogra> 2274  unlink("/etc/apt/apt.conf.d/00AllowUnauthenticated") = -1 ENOENT (No such file or directory)
[14:10] <ogra> 2274  write(1, "PROGRESS STOP\n", 14)   = -1 EPIPE (Broken pipe)
[14:10] <ogra> 2274  --- SIGPIPE (Broken pipe) @ 0 (0) ---
[14:11] <ogra> but the output doesnt seem to be completely in order, its very hard to read
[14:12] <jibel> skaet_, OEM install is broken on desktop images
[14:13] <ogra> and on preinstalled images
[14:13] <ogra> :(
[14:14] <cjwatson> ogra: we need to find out what happened to the other end of that pipe
[14:14] <ogra> cjwatson, yeah, thats what i'm trying since yesterady :)
[14:14] <mterry> Heyo!  Are images for A3 done?
[14:14]  * cjwatson looks at the trace
[14:14]  * ogra knows what he is looking for, just not how to get it 
[14:15] <pitti> mterry: we might rebuild the armel ones (if we get a fix in time), but so far the others look okay
[14:15] <mterry> I have a patch for a bad crasher in unity if you have a11y turned on, but not sure whether it would make it or should make it
[14:15] <ogra> i made an excerpt, the fully syslog is 65M
[14:15] <ogra> *full
[14:15] <mterry> pitti, ok, so no reason to respin just for this probs
[14:15] <pitti> mterry: upload away, but I don't think we'll respin; it's not an install failure, people can upgrade afterwards
[14:15]  * cjwatson is not interested in strace excerpts
[14:16] <ogra> k
[14:16] <mterry> pitti, agreed
[14:16]  * pitti needs to disappear for supermarket, bbl
[14:16] <ogra> i thought i should filter by PID
[14:16] <cjwatson> that's counterproductive - the thing we need to find out is what's happening to processes *other* than plugininstall.py
[14:16] <ogra> oh, k
[14:20] <cjwatson> the other end of the pipe is pid 728
[14:20] <cjwatson> which was killed by SIGSEGV a bit further up
[14:21] <cjwatson> after a huge pile of cacheflush calls
[14:25] <cjwatson> it is not clear to me exactly what it's doing, apart from talking to X
[14:26] <cjwatson> FWIW that's the top-level oem-config process
[14:26] <cjwatson> it might be useful to attach gdb to the top oem-config process from a separate terminal before it segfaults, and hopefully get a stack trace
[14:26] <cjwatson> failing that, 'ulimit -c unlimited' before starting it so that you get a core dump
[14:29] <ogra> oh my ...
[14:56] <ogra> oh, i just noticed i have a crash report from ubiquity-dm in /var/crash
[14:56] <ogra> (unlikely thats helpful, but i'll attach it to the bug anyway)
[15:54] <ogra> hmpf, trying to produce a core file is pretty unsuccessfull here
[15:55] <ogra> it dies but moves on removing ubiquity and doesnt produce a core at all
[15:58]  * ogra has to redo the image now ubiquity is gone ... sigh
[16:09] <GrueMaster> Interesting.  I am able to get oem-config running on x86 after some image manipulation, but I am not able to reproduce the crash seen on armel.
[16:21] <doko> pitti: is there a reason removing the milestone for 705689?
[16:22] <pitti> doko: see https://bugs.launchpad.net/ubuntu/+source/qt4-x11/+bug/705689/comments/32
[16:22] <ubot4> Launchpad bug 705689 in qt4-x11 (Ubuntu Natty) (and 6 other projects) "Qt applications crash with segfault error on armel when Qt is built with gcc 4.5 on natty (affects: 2) (heat: 30)" [High,Invalid]
[16:22] <doko> pitti: ScottK seems to disagree
[16:22] <cjwatson> pitti: it seems like it should be milestoned somewhere though
[16:23] <cjwatson> beta-1?
[16:23] <cjwatson> if it's blocking KDE using gcc-4.5 ...
[16:24] <ScottK> pitti: I agree it's not an Alpha 3 blocker, but I don't think we want to release Natty with Qt/KDE built on GCC 4.4, so it should be milestoned somewhere.
[16:24] <ScottK> Although lack of it now means we'll have to build with 4.4 when we upload KDE 4.6.1.
[16:25] <ScottK> doko: When in March is it due out?
[16:25] <doko> linaro releases are always the 2nd Tue of the month
[16:26] <doko> ScottK: is there a reason to use 4.4 on all archs?
[16:26] <ScottK> doko: No.  Just on armel.
[16:28] <pitti> well, *shrug*, feel free to add beta-1, if you prefer
[16:28] <pitti> but it'd be fine to fix after beta-1 as well from my POV
[16:28] <pitti> it's still a release critical bug for natty, after all
[16:29] <ScottK> pitti: The problem is we'll have to rebuild a large number of packages on arm after it's fixed to get them built with gcc4.5.  That's not something we want to do at the last moment.
[16:29] <pitti> ScottK: so, beta-1 then?
[16:29] <ScottK> I'll mark it.  Hopefully it's resolved next week anyway.
[16:30] <pitti> nice; my primary objective was to clean up the alpha-3 blocker list; this one isn't one
[16:30] <ScottK> Agreed.  It just would have made the next week a lot easier if we'd gotten it.
[16:50] <Daviey> smoser Is having a hard time at the moment testing potential cloud image candidates for A3, as some of the regional AWS hosted mirrors seem broken.  Canonical IS who maintains the mirrors has an RT ticket open.
[16:51] <smoser> well, that will cause some tests to fail. we still dont have all images published yet. and IS is working on it, the cause has been elusive before.
[16:52] <smoser> i expect that ~ 3:30 US/Eastern I will have all images published.  I've one so far and it seems fine.
[16:56] <cjwatson> bug 727783 is another "can I just shoot myself now" bug
[16:56] <ubot4> Launchpad bug 727783 in ubiquity (Ubuntu Natty) (and 1 other project) "oem-config is not installed after initial system installation, and the user can't proceed with the step 'prepare for shipping' (affects: 2) (heat: 12)" [High,Confirmed] https://launchpad.net/bugs/727783
[16:56] <cjwatson> the interaction with apt-cdrom has got confused *again*
[16:56] <cjwatson> I'll see what I can do but can't make any promises about timescales, since I have to go out shortly
[16:58] <skaet_> cjwatson,  understood.   Anyone else able to look into it?
[17:01] <cjwatson> ev if he has time but I don't know if he does, we're on the same timezone
[17:01] <cjwatson> I'm attempting to strace it now
[17:15] <cjwatson> hmm, this 1GB strace file may not be uploadable to a bug ...
[17:38] <cjwatson> might have a fix, testing now
[17:48]  * skaet_ keeping fingers crossed.
[17:48]  * ogra gave up on crossing fingers ... makes typing so hard 
[17:49] <skaet_> lol
[17:49] <cjwatson> I think bug 723357 has a similar cause, but would rather try to limit the amount of stuff I upload for a3
[17:49] <ubot4> Launchpad bug 723357 in casper (Ubuntu) "ISO filesystem not mounted properly on Live USB disks (affects: 1) (heat: 6)" [Undecided,New] https://launchpad.net/bugs/723357
[17:58] <cjwatson> grr.
[18:07] <skaet_> grr ??  hmm, should I be worried?
[18:07] <cjwatson> first approach didn't work
[18:11] <charlie-tca> skaet_: I would like to release note the bug 71234
[18:11] <ubot4> Launchpad bug 71234 in eog (Ubuntu) (and 1 other project) "Eye of GNOME (eog) grabs keyboard when in full-screen. (heat: 2)" [Low,Invalid] https://launchpad.net/bugs/71234
[18:11] <charlie-tca> valid workaround is to do a full shutdown after first login.
[18:11] <charlie-tca> Then run update-manager
[18:13] <skaet_> charlie-tca,  thanks for flagging.   ok will add now.
[18:13] <charlie-tca> Thank you
[18:38] <skaet_> charlie-tca is bug 71234 the right bug number??    looks pretty stale/invalid.
[18:38] <ubot4> Launchpad bug 71234 in eog (Ubuntu) (and 1 other project) "Eye of GNOME (eog) grabs keyboard when in full-screen. (heat: 2)" [Low,Invalid] https://launchpad.net/bugs/71234
[18:44] <charlie-tca> nope
[18:44] <charlie-tca> I seem to have missed that when I copied it
[18:44] <charlie-tca> the right one is bug 712346
[18:44] <ubot4> Launchpad bug 712346 in update-manager (Ubuntu) "update-manager crashed with TypeError in _get_last_apt_get_update_text(): unsupported operand type(s) for /: 'NoneType' and 'int' (affects: 16) (dups: 3) (heat: 202)" [Medium,Triaged] https://launchpad.net/bugs/712346
[18:44] <charlie-tca> I guess the 6 on the end matters
[18:47] <skaet_> heh,  thanks charlie-tca that makes more sense now ;)
[18:48] <charlie-tca> thanks for catching that mistake
[19:06] <doko> ogra: is openjdk on any arm image?
[19:06] <ogra> doko, not that i can think of, no
[19:07] <ogra> iirc it went away with OO.o
[19:07]  * ogra checks manifest
[19:09] <ogra> doko, confirmed, not on the images
[19:13] <GrueMaster> Just tested http://cdimage.ubuntu.com/kubuntu/daily-preinstalled/20110301/natty-preinstalled-desktop-armel+omap4.img.gz and oem-config doesn't even start.  Boots straight into kdm.  Looking over boot logs now, but may be a race condition with unity.
[19:13] <GrueMaster> I was trying it as a debug measure for our netbook images.
[19:22] <mvo> thanks charlie-tca I look into the u-m crash tomorrow
[19:23] <ogra> cjwatson, ha !
[19:24] <ogra> cjwatson, looks like uninstalling the slideshow fixes the armel bug (seems to be webkit related)
[19:25] <charlie-tca> No problem, mvo
[19:26] <ogra> skaet_, ^^^
[19:27] <skaet_> :)
[19:27] <ogra> skaet_, NCommander is preparing a hack in livecd-rootfs that removes the slideshow from arm builds
[19:27] <ogra> we will need to get that into the archive and need a respin then
[19:27] <skaet_> ogra, heh, that was going to be my next question... are we getting updates for new images.
[19:28] <rsalveti> skaet_: ogra: yup, was able to install it successfully after removing this package
[19:28] <rsalveti> seems the bug is inside webkit
[19:28] <ogra> oh my
[19:28] <NCommander> ogra: skaet_ someone will have to smack the livecd builders to pull from the archive
[19:28] <ogra> our error reporting in ubiquity needs to become better
[19:29] <ogra> it should have told us "hey webkit is broken" in the first place :P
[19:29] <ogra> NCommander, they do that automatically since maverick
[19:29] <ogra> only BuildLiveCD isnt autosynced afaik
[19:29] <NCommander> oh, aweseome
[19:30] <rsalveti> ogra: hard to say that when python is crashing with sigsegv
[19:30] <ogra> heh, indeed
[19:30] <ogra> i wasnt serious
[19:30] <skaet_> rsalveti,  thanks.  :)
[19:30] <ogra> i'm just so happy after two days of debugging
[19:30] <rsalveti> :D
[19:31] <ogra> that was really painful
[19:31] <rsalveti> for sure it was, but cool, at least I learned more about oem-config :-)
[19:32] <ogra> :)
[19:33] <NCommander> ogra: http://paste.ubuntu.com/574629/ - how's that for sanity?
[19:33] <pitti> ogra: I recently fixed webkit on amd64, it tried to allocate 1 GB of RAM; but on arm it should only try 16 MB, which doesn't sound too much
[19:34] <pitti> ogra: I'm curious, how is the slideshow involved in arm preinstalled builds at all
[19:34] <NCommander> pitti: used as part of oem-config
[19:34] <pitti> ah, of course
[19:35] <rsalveti> pitti: once python tries to use the webkit bindings, it crashes with sigsegv
[19:35] <pitti> it would be interesting if /usr/lib/webkitgtk-1.0-0/libexec/GtkLauncher crashes by itself on an arm system
[19:35] <rsalveti> and then oem-config is gone
[19:35] <pitti> if so, that'd be a lot easier to debug
[19:36] <rsalveti> pitti: nops, working fine
[19:36] <rsalveti> could be related with the python bindings
[19:36] <pitti> *nod*
[19:36] <rsalveti> ooops, too soon to tell, got a sigsegv
[19:37] <pitti> rsalveti: with a particular web page? or just at startup?
[19:37] <rsalveti> pitti: it loads google, and then after some seconds it crashes
[19:37] <pitti> rsalveti: do you get an apport .crash?
[19:37] <NCommander> rsalveti: is it the python bindings or webkit in general?
[19:37] <rsalveti> webkit in general
[19:37] <pitti> /usr/lib/webkitgtk-1.0-0/libexec/GtkLauncher is webkit itself
[19:38] <NCommander> ah
[19:38] <NCommander> right :-)
[19:38] <rsalveti> pitti: will check
[19:38] <pitti> rsalveti: thanks; perhaps it can be replicated in xvfb on a porter box or so
[19:38] <rsalveti> argh, gdb kills the board when running with webkit
[19:38] <rsalveti> webkit is huge
[19:39] <rsalveti> pitti: yup, got the crash file
[19:39] <ogra> pitti, bug 727468
[19:39] <ubot4> Launchpad bug 727468 in ubiquity (Ubuntu Natty) (and 3 other projects) "oem-config crashed during install on armel (affects: 1) (heat: 8)" [Critical,Confirmed] https://launchpad.net/bugs/727468
[19:41] <NCommander> ogra: test spin going with hack to remove slideshow
[19:41] <ogra> NCommander, packagename is wrong
[19:41] <rsalveti> ogra: should I open a new bug for this webkit issue? or just link the oem-config with webkit?
[19:42] <rsalveti> argh, corrupted stack
[19:42] <rsalveti> need to install webkit dbg symbols, but first need space for it :-)
[19:42] <ogra> NCommander, and you should actually put your hack *after* the code that installs LIVELIST ;)
[19:43] <ogra> helps a lot to remove somethig *after* it was installed ;)
[19:43] <rsalveti> hehe :-)
[19:43] <ogra> rsalveti, i just changed the title of the bug ... we need other tasks and close the ubiquity one
[19:43] <ogra> i think it helps that we have the debug data on it already
[19:44] <ogra> even though if it might only show fallout
[19:44] <pitti> rsalveti: right, we don't have retracers for armel ATM :/
[19:44] <ogra> pitti, on NCommander's todo, but he was busy fixing mono
[19:44] <pitti> rsalveti: but if you can salvage the .crash file anyway, we might be able to fake it on the porter boxes
[19:44] <ogra> we might have them back right on release day ;)
[19:45] <rsalveti> pitti: np, can install the dbg symbols and try to get a proper trace
[19:45] <pitti> we can do that the plain old way with the core dump and installing -dbg in the porter box dchroots
[19:45] <pitti> rsalveti: even better :)
[19:45] <pitti> ogra: just to triple-check is removing ubiquity-slideshow ok? on amd64 our workaround was to remove ubiquity-slideshow-ubuntu
[19:46] <ogra> pitti, thats what i meant above
[19:46] <pitti> ubiquity-slideshow is purely virtual usually
 NCommander, packagename is wrong
[19:46] <pitti> ah, ok
[19:46] <pitti> sorry
[19:46] <ogra> well, michael didnt get that either i think
[19:46] <ogra> NCommander, ^^^^
[19:46] <ogra> NCommander, ubiquity-slideshow-ubuntu is the package
[19:47] <ogra> NCommander, and move it down a bit ... right above "# remove our diversions" i'd say
[19:57] <NCommander> ogra_: yeah, saw that mistake myself, replacedit with ubiquity-slideshow-*, although I'm happy with its position as its just below where we do the last package manipulation on the image
[19:58] <smoser> skaet_, http://uec-images.ubuntu.com/server/natty/20110302.2/published-ec2-daily.txt has the ec2 images data to populate tracker
[19:58] <ogra_> NCommander, not really
[19:58] <smoser> i know that cjwatson and slangasek have done that before, not sure if you have.
[19:58] <ogra_> NCommander, chroot $ROOT apt-get -y --purge install $LIVELIST </dev/null
[19:58] <ogra_> NCommander, that line actually installs the slideshow
[19:59] <ogra_> NCommander, your hack needs to be below it
[19:59] <skaet_> smoser,  ok,  will go in and populate it.
[19:59] <ogra_> NCommander, thats why the checkpoint says "Installing live packages" ;)
[20:00] <pitti> skaet_: need anythign from me tonight still?
[20:00] <pitti> ogra_: or you? (I think you can retrigger the armels yourself, right?)
[20:00] <ogra_> i can do it myself, yeah
[20:00] <ogra_> but would like to finish my day too at some point
[20:00]  * pitti hugs ogra_, great to see this working at last
[20:00] <ogra_> but NCommander can do that too
[20:01] <slangasek> skaet_: you have the post-amis-to-iso-tracker script for this, right?
[20:01] <skaet_> pitti, can you take a pass through the TechOverview?
[20:01] <pitti> ogra_: I can also set up a trigger to start building them once a fixed livefs-build package is published for arm
[20:01] <skaet_> slangasek, its in the iso build loads, but if that doesn't work, I'll let you know.
[20:01] <ogra_> pitti, nah, NCommander and i can take care
[20:01] <pitti> ok
[20:02] <slangasek> skaet_: hmm, iso build loads?
[20:02] <skaet_> iso tracker,  sorry.
[20:02] <slangasek> skaet_: doing this by hand is *verry* tedious and error-prone if you want to be able to accurately track the individual AMIs on the iso tracker, so I hope you're using the script :)
[20:03] <skaet_> slangasek,  nope what I was trying didn't work.  so I'm missing something.
[20:03] <NCommander> ogra_: gah, hoops
[20:03] <skaet_> slangasek, wasn't using the script, so time for a bit of training for me I guess. ;)
[20:04] <slangasek> skaet_: lp:~ubuntu-archive/ubuntu-archive-tools/trunk/; script called 'post-amis-to-iso-tracker.py'; takes smoser's linked file as input
[20:04] <slangasek> though fwiw I've always snarfed that file via ssh instead of http, since the http connection provides no assurance of data integrity... I'd hate to accidentally post a link to an OpenSuSE AMI to the tracker
[20:04] <slangasek> :)
[20:04] <skaet_> heh.
[20:05] <skaet_> slangasek,  ok, trying.
[20:07] <pitti> skaet_: TechOverview> for content? or also for language already?
[20:08] <skaet_> pitti,  both please ;)
[20:08] <pitti> skaet_: (still missing quite a lot, so I propose to leave the language fine-tuning for tomorrow)
[20:08] <skaet_> sure,  just get me some contents, and I'll do more scrubs on it later.
[20:24] <pitti> skaet_: not too much new contents, but some clarifications and language fixes: https://wiki.ubuntu.com/NattyNarwhal/TechnicalOverview?action=diff&rev2=102&rev1=101
[20:25] <skaet_> thanks pitti.    Glad its had your eyes over it.    When you get in tomorrow,  take a look through the bugs too.  They'll be being added today.
[20:25] <skaet_> have a nice evening.
[20:25] <pitti> skaet_: yup
[20:25] <pitti> skaet_: almost -- just saw the new load of kernels coming in..
[21:19] <pitti> good night
[21:21]  * slangasek waves. 'night, pitti!
[21:22] <skaet_> good night pitti
[21:23] <highvoltage> night pitti
[21:23] <skaet_> smoser, can you confirm the images are available now?
[21:23] <smoser> yeah, tests are being run.
[21:23] <smoser> did someone tell you they weren't available ?
[21:24] <smoser> because for a small point, they were not. but they're back.
[21:26] <skaet_> just confirming that the script actually did what it should have.  :)
[22:14] <smoser> skaet_, i have the tets running now, i'll fill in the results sometime tonight.  they're looking fine.
[22:14] <smoser> and i've got the "pre-publish" for alpha3 running on those images. so we'll be all set tomorrow.
[22:16] <skaet_> smoser,  good to know. :)
[22:17] <skaet_> thanks for the update.
[23:01] <jibel> skaet_, bug 728088 iSCSI test in server amd64
[23:01] <ubot4> Launchpad bug 728088 in debian-installer (Ubuntu Natty) (and 1 other project) "iscsi root (amd64) with or without auth fails to boot (affects: 1) (heat: 6)" [High,New] https://launchpad.net/bugs/728088
[23:01] <skaet_> thanks jibel,  adding it to the hot list
[23:03] <skaet_> smoser, is anyone around from the server team who can look at this new one?  ^  or do we need to wait for UK to come online?   At this stage, we'll probably release with it, and document.
[23:05] <charlie-tca> skaet_: only change I see for release notes is to add gmusicbrowser replaces exaile in xubuntu
[23:30] <GrueMaster> skaet_: Ok, we have an update to livecd-rootfs to remove ubiquity-slideshow-ubuntu from the armel images.  This should at least get us a working image for A3.
[23:30] <GrueMaster> I have been testing with this workaroung and haven't seen any other gotchas.
[23:33] <cjwatson> iscsi> for the record I have a feeling that we need to update open-iscsi to a new upstream version to match kernelspace.  probably not something that can be done in a day.
[23:53] <skaet_> cjwatson,  ack.   we'll go with what we have at this point.
[23:55] <skaet_> GrueMaster,  sounds good.   What is the upload status?
[23:56] <skaet_> charlie-tca, please go ahead and add/make changes for Xubuntu overview (and any other signifcant bugs you want to highlight)
[23:56] <GrueMaster> Uploaded livecd-rootfs, waiting to be published.  Once it gets published, someone can kick image rebuild.
[23:57] <GrueMaster> Once the images are there, I can test them and get the data in to the tracker.
[23:57] <charlie-tca> okeydokey
[23:58] <skaet_> Gruemaster,  sounds good.
[23:59] <NCommander> skaet_: saw that GrueMaster updated you. Publisher should start running in 5 minutes. once its done, i'll kick the image build and publish a new armel+omap4 image