[05:17] Good morning [07:21] good morning [07:21] hey ara, good morning [07:28] pitti, when do we expect to have the first images in the tracker? [07:29] ara: in about 15 minutes, I think [07:29] I'm currently building all alternates/servers [07:29] pitti, nice :-) [07:29] and I prepared the tracker for maverick a1 [07:29] pitti, yes, I saw that ;-) [07:29] we won't have desktops due to bug 587888 [07:29] Launchpad bug 587888 in linux (Ubuntu Maverick) (and 1 other project) "aufs oops in au_do_open() on maverick live system boot (affects: 1) (heat: 6)" [Critical,Triaged] https://launchpad.net/bugs/587888 [07:30] pitti, that's the one we hit yesterday, isn't it? [07:30] right [07:30] "It's not the end of the world if we release alpha-1 without desktop images" :D [07:34] ScottK, Riddell: I suppose http://people.canonical.com/~ubuntu-archive/livefs-build-logs/maverick/kubuntu/20100601/livecd-20100601-i386.out will hit alternate installs as well? [07:36] seems this needs a kdeplasma-addons rebuild against libqalculate5 [07:42] ok, so kdeplasma-addons failed on libmarble-dev, which needs kdeedu to build, which again needs glew in main [07:42] I re-promoted glew, it was in main until jaunty [07:43] but it also needs avogadro in main [07:55] filed as bug 588150 [07:55] Launchpad bug 588150 in kdeedu (Ubuntu Maverick) (and 1 other project) "build-depends on avogadro, which is in universe (affects: 1)" [High,New] https://launchpad.net/bugs/588150 [08:07] ara: ok, added some images to the tracker now [08:07] pitti, OK, thanks, I'll sync now [08:07] kubuntu alternate still building, but it'll be uninstallable anyway [08:07] pitti, :) [08:08] ah, it just finished building, too [08:20] I fixed the xubuntu oversizedness in the seeds, next build shold be okay [08:23] Riddell, ScottK ^ same for kubuntu, I dropped pt [08:37] I reported two RC bugs for the Kubuntu issues now [08:45] pitti, any reason why "rescue mode" now is called "alternative desktop environments" in the menu? [08:45] pitti, or is it a bug? [08:45] ara: that doesn't sound related at all [08:46] pitti, ok, I'll ask ev and file a bug [08:46] thanks [09:44] my installation of ubuntu alternate i386 in my mini9 is taking ages... [09:44] on ext4? soudns like the dpkg performance regression? [09:45] pitti, yes, ext4 (full disk, no manual partitioning) [09:45] pitti, is there a bug number for that? [09:45] ara: bug 570805 [09:45] Launchpad bug 570805 in dpkg (Ubuntu Lucid) (and 3 other projects) "[regression] dpkg fsync cause massive regression in Ubuntu Server and Alternate installation times (affects: 11) (dups: 1) (heat: 86)" [High,Triaged] https://launchpad.net/bugs/570805 [10:03] pitti, it has to be, because it is taking very very long [10:04] seb128, Riddell: I prepared https://wiki.ubuntu.com/MaverickMeerkat/TechnicalOverview for alpha-1; if you have something to mention for GNOME/KDE, please do [10:04] I'll add the "known bugs" based on the iso tracker feedback [10:43] * Riddell removes kdebase-plasma from kubuntu seed [10:45] looks like kdeplasma-addons needs a recompile for that qalculate issue [10:45] hey Riddell, good morning [10:46] Riddell: kdeplasma-addons is FTBFS on too old libmarble-dev (kdeedu), which is dep-wait on avogadro -> bug 588150 [10:46] Launchpad bug 588150 in kdeedu (Ubuntu Maverick) (and 1 other project) "build-depends on avogadro, which is in universe (affects: 1) (heat: 6)" [High,New] https://launchpad.net/bugs/588150 [10:47] Riddell: but seems the libqualculate4 rdepends is plasma-widgets-addons [10:47] (for unseeding?) [10:47] unseeding would be the quick workaround [10:48] if it's working without, sure [10:48] doing [10:48] Riddell: so this would also work around bug 588163 at the same time, as it seems? [10:48] Launchpad bug 588163 in kdebase (Ubuntu Maverick) (and 1 other project) "kdebase-plasma and plasma-widget-folderview depend and conflict on each other (affects: 1) (heat: 6)" [High,New] https://launchpad.net/bugs/588163 [10:49] kdebase-plasma is obsolete, I removed it already from the seed [10:50] ah, good, so 588163 can be closed with a kubuntu-meta rebuild [10:53] kubuntu-meta 1.176 uploaded [10:58] Riddell: thanks; buildds are clogged, so won't make this publisher run, I'm afraid; but we aren't under high pressure yet [10:58] pitti: cdimage carries over images from the previous build, but it doesn't know to get rid of the images from the previous release cycle when we s/lucid/maverick/g - I'm going through and cleaning up the stale lucid-* images now [10:59] cjwatson: is that a configuration change, or just rm in www.full? [10:59] I'm going through with rm [10:59] ah, good to know for next time, tanks [10:59] "thanks" [11:06] done [11:28] pitti: aufs: tempting to reintroduce unionfs-fuse temporarily ... [11:33] cjwatson: how much effort would that be? [11:35] some [11:36] hopefully: re-promote unionfs-fuse temporarily, and re-seed it [11:36] of course we don't know for sure that it still works [11:36] but it hasn't changed since karmic [11:36] it can hardly get any worse [11:37] but don't we also need to change casper for it? [11:37] i think it has the code still [11:57] yes, I left it there as a fallback [11:58] precisely in anticipation of this kind of problem :-) [12:01] cjwatson: ah, so if it's there, it uses and prefers it? [12:07] oh, actually, only if aufs *isn't* there [12:07] so we'd have to add union=unionfs-fuse as a kernel argument too [12:56] ok, kubuntu-meta is published, I build a new kubuntu alternate to verify that it's installable now [12:56] I'll rebuild the rest (xubuntu etc.) once new d-i lands; new xubuntu will fix the oversizedness [12:57] cjwatson: ok, want me to give this unionfs a try, or are you on it already? that'd go into platform/live-common? [12:57] oh, and needs a cdimage change for the kernel argument [13:05] http://cdimage.ubuntu.com/kubuntu/daily/20100601.2/report.html \o/ -> empty [13:05] ara, Riddell: kubuntu alternates posted to tracker for smoketesting [13:06] pitti, ok, thanks! [13:07] pitti: I'm not on it just now [13:07] live-common seems plausible [13:07] cjwatson: does that also need a publisher run? (I guess not, doesn't sound like a task) [13:08] it will do, it's incorporated into the live task [13:08] well, all the live tasks [13:08] so it'll need two publisher runs [13:08] ah, ok [13:10] committed, I'll verify the Tasks: headers in two hours then and attempt a build [13:12] cjwatson: does that go to KERNEL_PARAMS in CONF.sh or debian/CONF.sh? [13:16] ah, not debian/, that's old [13:17] no no not KERNEL_PARAMS [13:17] let me do it :) it goes in the tools/boot stuff [13:18] ok, thank you [13:57] cjwatson, pitti, is there an aufs issue? [13:57] apw: yes, see bug 587888 [13:57] Launchpad bug 587888 in linux (Ubuntu Maverick) (and 1 other project) "aufs oops in au_do_open() on maverick live system boot (affects: 2) (heat: 12)" [Critical,Triaged] https://launchpad.net/bugs/587888 === barry` is now known as barry_ [13:59] pitti, oh was the first CD yesterday then === barry_ is now known as barry [14:00] pitti, do you have a recipe for what casper does when it mounts ? [14:02] * pitti RTFS [14:05] mount -t aufs -o noatime,dirs=/cow=rw:/rofs:rr aufs "$rootmnt" [14:05] apw: ^ something liek that [14:14] for the record, rebuilding xubuntu to fix oversizedness [14:15] Thanks [14:15] charlie-tca: I unseeded some langpacks earlier on, but wanted to wait for the new d-i [14:16] ara: did you already invest a lot in ubuntu alternates? [14:16] no problem [14:16] ara: your "rescue mode" bug got fixed, but would need a respin [14:16] not that bugs of this magnitude would matter much right now [14:30] charlie-tca: http://cdimage.ubuntu.com/xubuntu/daily/20100601.1/ [14:30] added to tracker now [14:30] thank you. downloading [14:32] well, rsyncing, anyway [14:36] pitti, so, are we respining? [14:36] ara: did anything else turn up in testing so far which we need to fix? [14:36] pitti, not really, only minor issues, update-notifier shows the systray icon, i.e. [14:36] ara: I think we'll respin in any case; if we don't have other bugs (ugh, how unusal), then "because we can" :) [14:37] ara: nice [14:37] ara: we'll still try to build desktops, but that'll still take ~ 3 hours [14:37] pitti, OK [14:40] for the record, update-notifier with fix is uploaded [14:45] Pitti, is there an easy way to make a squashfs filesystem [14:46] apw: certainly, but why not just take the one from the CD? [14:46] pitti, DOH [14:46] pitti, cause i am terminally stupid [14:46] apw: the magic runes are in the livecd-rootfs package, FYI [14:46] in case you want to play with a very small one [14:46] apw: does it work with two normal directories? [14:47] apw: i. e. does it only crash on aufs-on-squashfs, not on aufs-on-dirs? [14:47] pitti, well i just tested with two local directories and its ok there [14:47] (or aufs-on-loop or whatnot) [14:47] hense my question :) [14:47] ah, ok [14:47] apw: perhaps the ubuntu squashfs has exactly the wrong combination of bits then? :-/ [14:47] aufs 75210728 3580784 67809464 6% /root/ROOT [14:47] so now i want to swap in the right sorts of FS and see if that changes things [14:48] i have an update for aufs which i want to test, but i need to reproduce first [14:49] apw: mksquashfs /tmp/mychroot/ /tmp/img.squashfs -sort filelist [14:49] apw: so, doesn't seem to be rocket science either, in case you need it [14:49] sounds good [14:53] pitti, ok so i can mount a squashfs foo under an ext4 filesystem ok [14:53] whats the top layer [14:53] tmpfs ? [14:53] I suppose [14:53] apw: do you get the crash as well if you boot that ubuntu CD? [14:54] hrm ... [14:55] pitti, you arn't the only person who has reported the disk as bad [14:55] and an aufs2 crash into the bargain [14:55] apw: happened to ara, too [14:55] so i am sure its real, just not managing to get it to pop yet [14:55] /dev/loop0 128 128 0 100% /root/RO [14:55] tmpfs 1021280 20 1021260 1% /root/RW [14:55] aufs 1021280 20 1021260 1% /root/ROOT [14:59] * pitti -> TB meeting [15:30] xubuntu appears to have three panel applets flashing in the middle of the desktop now. jockey, gnome-volume-control, and network-manager. They are icon sized windows that won't quit flashing over and over [15:31] well Kubuntu installs but then boot doesn't get past plymouth (in virtualbox) [15:41] cjwatson, Riddell: I need to leave in ~ 10 minutes (sorry, working on early hours this week, will be back to normal next week) [15:41] cjwatson: unionfs-fuse seems to have Task: ubuntu-live, edubuntu-dvd-live now, but not yet Kubuntu [15:42] can someone please trigger desktop builds later on, and check if they now work with unionfs-fuse? [15:43] that's odd, they should be simultaneous [15:43] hm, kubuntu live doesn't have the right Task-Seeds. Let me check the effects of fixing that [15:46] fixed in kubuntu, but I'll check the other derivatives too [15:50] pitti, so, the respin is happening tomorrow, I guess [15:52] we can do Ubuntu today - I'll take care of that [15:52] just waiting for a baseline image to finish downloading here === ogra_ is now known as ogra [17:11] pitti, is there a trivial way to inject a new kernel into an iso image? [17:12] pitti, this is on a writable media obviously [17:14] is it ABI-compatible with the one you're replacing? [17:15] if so, overwrite /casper/vmlinuz [17:32] cjwatson, it is abi numbered the same, though its the modules i am moding [17:32] am trying a kernel-replacement now [17:33] but remaking the initrd is prooving painfully slow [17:36] apw, use a chroot (with casper installed in it) [17:36] for mount testing that should suffice [17:38] ogra, well i need it to do the testing as it works perfectly in any testing i do [17:39] right, create a chroot, install casper inside and there install your test .deb and run update-initramfs ... then replace the initramfs and vmlinuz in the image [17:41] ogra, so where will it make the casperiszed initramfs's ? [17:41] in 7boot inside the chroot [17:41] */boot [17:42] making the normal ones unbootable (normally) or making additional ones [17:42] *inside* the chroot [17:42] it doesnt write to your /boot if you are chrooted [17:43] ogra, yeah, but if i installed it outside a chroot, it would eat my real initramfs ? [17:43] it doesnt do any harm [17:43] just working out how it works, in my head [17:43] boot=casper is needed to actually tell init to use casper [17:43] else you just bloat the initrd [17:43] with the casper scripts [17:44] ah [17:44] then i can just install casper on this test box then [17:44] as i can live with a bit of bloat on there [17:44] indeed [17:44] more than i can be bothered to make a new initrd [17:44] chroot even [17:52] cjwatson, pitti, ok it looks like an aufs update could well sort out the CD booting ... did you switch over to fuse already? [17:52] yes (pending rebuilds), but it's easy to revert that [17:52] I'd rather stick with aufs if it's easy [17:54] cjwatson, i am currently booted off of of my kernel with the aufs update, boy was it a bugger to get in there, but it seems to be booted ok on an amd64, which was showing the panics before [17:54] cjwatson, the problem is a kernel respin takes about 14 hours these days [17:55] it's not a major deal if a1 is a bit late [17:55] cjwatson, ok i'll get the patches out to our list and let ogasawara know etc etc [17:55] re [17:55] I think I'd rather have slightly late with aufs than on time with unionfs-fuse [17:55] ok [17:55] apw: you can also break in casper's initramfs and insmod it manually; I think I did something similar in the past in a similar case [17:56] we can leave unionfs-fuse in there for the time being as insurance in case it breaks, and remove that after a1 [17:56] apw: 14 hours on i386/amd64 as well? we don't care about armel for a1 yet [17:56] pitti, no they are more like 5 [17:56] cjwatson: right, now we can just toggle in cdimage, no seed/task changes necessary, right? [17:56] cjwatson: so we could even switch (for testing) on one and the same iso [17:57] if you can do that, then i think its very easy for me to offer up these patches [17:57] as if they don't work we're no worse off and still can say 'use fuse dude' and it'll work ... if i am understanding you [17:58] yes, in fact er cough I think I forgot to make the cdimage change before respinning [17:58] too many distractions [17:58] I'll not bother now then :-) [17:58] apw, you shuld know that nobody uploaded any omap kernel yet :) so armel is out of the question [17:58] apw: *nod* [17:59] ogra, you sure? i thought that omap3 was standard in our main kernel now, and indeed why it takes so long to build [17:59] apw, oh, i wasnt aware [17:59] * ogra checks binaries [18:00] i was looking for omap3 uploads [18:00] apw, heh, indeed ! [18:00] there should be an omap flavour [18:00] yeah [18:00] i wonder if it works on the XM and zoom2 [18:01] anyway, dinner time [18:11] apw: so if that kernel won't change anything else, it wouldn't need an ABI bump, right? otherwise we'll have to change a ton of other stuf [18:11] f [18:12] pitti, indeed, i built it here without a bump and noone would use the interfaces it might change anyhow, they are aufs interfaces so a forced not-bump would likely be appropriate anyhow [18:17] cjwatson, apw: so, sounds great to me; I'll get up early tomorrow again (0430 UTC) and can start wrestling and testing early [18:18] pitti, that sounds awful [18:19] apw: if you can tell me the version of the package upload, I can set up a trigger on cdimage to build live images once that kernel hits [18:21] pitti, ok i expect it would be the just an increment of the upload number [18:21] will need to confirm with ogasawara [18:21] apw: yah, lets mumble after our irc meeting and get everything coordinated [18:21] ogasawara, ack [18:22] apw: 2.6.32-22.34 ? [18:23] pitti, this would be a 2.6.34 kernel [18:23] erk, of course [18:23] there's an error inserting ramzswap that shows up briefly while booting the live CD, too [18:23] is that known broken [18:23] ? [18:24] trying to boot with union=unionfs-fuse at the moment to confirm the fallback option [18:24] apw: 2.6.34-5.13 I mean [18:24] cjwatson, i wonder if thats cause we have both now in .34 [18:24] pitti, that looks right to me [18:26] hmm, "unknown parameter `disksize_kb'" [18:26] maybe the interface changed [18:26] in which case it's probably appropriate to lob in a casper fix [18:26] or initramfs-tools or wherever it is that lives [18:27] cjwatson, i think there was an interface change coming, i think we knew about it ... i seem to remember us updating it during lucid and you spotted the difference and we backed it out for lucid [18:27] I don't remember that but brain like a thing with holes in [18:27] ScottK: I'm curious now... have we actually hit any cases of builds taking too long where it _didn't_ come down to "excessively abusive use of lzma where it should not be" ? [18:28] lamont: I don't know of any that weren't lzma builds. [18:28] cjwatson, apw: ok, I set a trigger for 2.6.34-5.13 which then builds all live CDs; if something bad happens, just kill the wait-for-package process on antimony [18:28] since I need to leave now [18:28] ScottK: I find I'm starting to lean towards "fix this when it's actually an issue" then. or should some of the failing lzma builds not fail and still be lzma? [18:29] lamont: Can you do an ia64 test build for me if I give you a package? [18:29] ScottK: sure [18:29] OK. I'll continue fiddling. [18:29] The only place I've seen the timeouts is on armel and there space tends not to be an issue. [18:30] So in theory lzma should work there, but in practice it's just some added complexity in the packages to not do lzma on armel. [18:30] * pitti waves goodbye now; please don't hesitate to call my mobile on any urgencies [18:50] ramzswap change is non-trivial, requires pulling in a package with rzscontrol; I won't do that for a1 [18:52] cjwatson: Please let me know when you have a few minutes free. I still have my question for 'the release manager' from last Friday. [18:53] ScottK: oh, sorry, I just reached the end of my day [18:53] cjwatson: No problem. It's still not a rush. [19:11] marjo, slangasek i would like data from http://uec-images.ubuntu.com/maverick/20100601/ uploaded to tracker [19:12] smoser: ack [19:14] smoser: hggdh will work with you on this request [19:43] smoser, marjo: I'll take care of it; quicker for me to do it with the script [19:43] slangasek: ok, thx [19:47] smoser: is ap-southeast-1 meant to be posted to the tracker at this stage? [19:47] slangasek, yes. [19:47] sorry for failure to raise that. [19:48] we will need those additional 4 entries (2 arch * 2 roots) [19:48] ok, let me get those in the db then :) [19:57] stgraber, marjo: has anything changed regarding the db setup of iso.qa that I'm not aware of? My changes to the qatracker_testcase table on quandong aren't being reflected on the website [19:58] and indeed, Maverick Alpha 1 isn't in this db [19:59] iso.qa seems to point to cranberry, which I don't have access to, so I can't add the new Asia-Pacific EC2 products smoser needs [20:00] slangasek: limequat [20:01] it was the release day madness, not yet reversed [20:02] elmo: ah, thanks - can I assume that if and when it's reversed, you guys'll take care of the db migration? [20:02] elmo: may i leave this issue in your capable hands? [20:03] no, please don't [20:03] I have no idea what to do with the DB [20:03] but if slangasek had access to the DB for, he still does [20:03] on limequat [20:04] elmo: ack [20:04] s/for/before/ [20:05] (if he doesn't/didn't, I can puppet for someone and/or give them access) [20:06] smoser, marjo, elmo: all done, Asia-Pacific is posted to the tracker [20:08] thank you [20:51] cjwatson, pitti, apw: just fyi, the 2.6.34-5.13 kernel with the aufs update is uploaded. looks like the i386 build has already started and amd64 has an ETA of about and hour to begin building. [21:06] thanks - I'm going to watch a film and I'll give things a poke later on [21:25] cjwatson, excellent thanks [21:25] ogasawara, fingers crossed :) [21:38] slangasek: oh btw, I broke your armel/live machine until londontime tomorrow. (needs a good powerstabbing) [21:39] well, the /I/ in that is supposition on my part, but I fear I'm right. unless you were hammering on the poor thing a little while ago [22:03] lamont: wasn't me [22:05] well, that just makes it near certain it was me then [22:06] :(