[00:03] could someone approve that please? ^^^ :-) [00:18] chrisccoulson: accepted [00:18] slangasek, thanks [00:18] ScottK: did you need someone else to wave the kde no-change builds through? [00:39] Can I get my wine1.3 upload approved? It removes a binary package and I figure the sooner we get that out of the archive the less problems we get. [01:04] YokoZar: this looks like a new upstream release that's subject to feature freeze; has this been ok'ed already by ubuntu-release? [01:19] * NCommander debugs antimon [01:19] y [01:20] why are we building an iso for mx5? [01:20] oneiric-preinstalled-desktop-armel+mx5.iso 14-Sep-2011 04:02 1.5G Preinstalled desktop CD for Freescale i.MX5x computers (standard download) [01:20] thats wrong in like 5 different ways [01:42] slangasek: Err, no, but I can vouch for it being better since I've been testing them all cycle [01:42] I'll mark the bug [01:47] fwiw I've always felt that Wine point releases generally fall under the "only bugfixes" general exception [01:55] oops, seems like that update-grub call in friendly-recovery's postinst is failing quite miserably in the livefs chroots... [01:59] apparently memtest86 installs fine when checking for /boot/grub/grub.cfg too, guess I'll do that too then [02:02] uploaded [02:02] skaet: don't we get another week of being unfrozen after beta 2 before final freeze? [02:03] micahg, yes [02:03] * skaet wonders if one of her recent emails was unclear... sigh. [02:03] just checked, same wording was in the beta 1 freeze notice... [02:04] This line prompted the question: From now on, all uploads to the archive and any user user interface [02:04] changes will have to be approved manually by the release team. [02:04] good point, was only until beta 2 comes out. [02:05] when beta 2 is announced, please remind me to send something out to set the context for the last few weeks. probably needed. [02:07] skaet: I will try to remember :) [02:07] micahg, me too. But I suspect I'll need to be reminded. :) [02:09] would be nice to have that friendly-recovery change reviewed as soon as LP generated the diff. It's currently making livefs builds fail :( [02:14] stgraber, done [02:20] skaet: thanks [02:20] skaet: I'm guessing pretty much all the livefs failed to build? [02:20] stgraber, thanks for the fix. :) [02:20] skaet: I'm only getting the e-mails for Edubuntu [02:22] stgraber: ?? [02:23] do we have problem with one of the mail lists? [02:24] no, I was just wondering if you indeed got a lot of livefs build failures in your inbox. I'm only marked as a contact for Edubuntu dvd so I never get the others. [02:24] I've been working through the launchpad bugs all day, and yeah there are lots of failures to worry about. [02:25] http://reports.qa.ubuntu.com/reports/kernel-bugs/reports/rls-mgr-o-tracking-bugs.html [02:27] ouch, that's a pretty long list of foundations... I guess I'll be doing some more ubiquity bug fixing then. [02:27] stgraber, yes please. [02:27] :) [02:56] slangasek: Please feel free to wave them through, but generally I can get it, I was just away at a social engagement tonight. [03:03] skaet: Now I'm confused. IIRC, last cycle we left the queue frozen between beta 2 and final freeze. Are you saying we're going to do it differently this time? [03:05] ScottK, there's one more week right now between beta 2 and final freeze, that wasn't there in Natty [03:07] Did I miss where this was discussed? [03:08] If the queue is opened up, then I think that reduces the value of the Beta 2 testing since anything can get in between the two freezes. [03:10] ScottK, lets talk about it tomorrow in the meeting then. If the other members of the release team agree, I'm fine with leaving it frozen, balancing work on approving each fix, against getting as many fixes in as possible. Bug list is rather intimidating right now. [03:14] I tend not to make the meetings (clashes with another meeting I have), but for the record, I'm in favour of just keeping us frozen until release at this point. It's more work for -release and -archive, but I agree with the idea of keeping control on what gets in from here on in. [03:15] (Unless people reject my uploads, then I think this whole idea is madness) [03:15] :P [03:15] skaet / ScottK : --^ [03:15] infinity, noted. :) [03:56] ScottK: sorry, my memory must be faulty then, I was just trying to be consistent across releases [04:40] No problem. We'll get it sorted tomorrow. [05:31] Agenda bugs sorted for the meeting tomorrow. [05:31] https://wiki.ubuntu.com/ReleaseTeam/Meeting/2011-09-16 [05:31] for those who want an early look at it. [05:31] * skaet --> zzz === chrisccoulson_ is now known as chrisccoulson [08:51] ^ fixes an oversight I made [09:23] NCommander, if you would read your backlogs regulary you would know why there is an iso for mx5 ... it was largely discussed in #ubuntu-arm when it showed up :) [09:24] (and yes, its a mistake) [09:27] Laney: accepted, though I'm slightly confused why that showed up as an upload rather than a sync [09:27] because I did syncpackage [09:27] --no-lp [09:27] I sometimes do that when uploading to Debian and Ubuntu simultaneously otherwise I risk forgetting [09:27] ah [09:28] I make notes :) [09:28] yeah [09:28] the home → work context switch can enbadden my mind though [09:28] anyway, thanks [10:28] cjwatson: hey, can you accept the oneconf upload please? I start to get quite some duplicate because of a crash on unavailable apps.ubuntu.com [10:39] didrocks: done [10:40] cjwatson: thanks a lot :) [10:59] would be good if someone could review/allow the apt upload, I'm afraid without it translationed package descriptions are broken === Ursinha-afk is now known as Ursinha === doko_ is now known as doko [12:35] mvo_: accepted, thanks - sorry for the delay [12:35] no worries [12:35] and thanks! [12:38] cjwatson, please have a look at gcc-4.6 [12:43] mkay [12:44] you know one of these days queue diff generation will be responsive enough to be useful [12:45] doko: pleasantly readable, thanks. accepted [12:47] skaet: hey-- I may be a few minutes late to the release meeting [12:48] skaet: I think I'll be on time, but may not [14:16] jdstrand, ok, will move you to the end of the round table then, no worries. [14:31] fwiw, pandafarm coming offline to add two more lp-buildds to it [14:43] cjwatson, please have a look at gcc-4.4. -9 is more or less -8ubuntu2. once that's approved, I plan to upload {gcj,gnat,gdc}-4.4 over the weekend [14:47] I'd appreciate it if someone would accept kubuntu-meta. Kubuntu dvd's won't build without it. [14:51] skaet: I've twisted Laney's arm to represent MOTU tonight [14:52] please put me ~first though, because I have an important date with a pint of beer [14:52] thanks for the head's up tumbleweed - will put Laney on the agenda then. [14:52] Laney: :) [14:52] Laney, ok, you can swap with jdstrand, he's going to be late. :) [14:52] Laney: tsk [14:52] groovy [14:52] too less :P [14:53] doko: accepted, thanks [14:53] nigelb: hey, the pub has a new brewery on. it's out of my hands really [14:53] Laney: :) [14:54] and pandafarm back online, I'll be adding the other two lp-buildds $LATER (need to finish the installs on them) [15:01] skaet: I'm actually here [15:01] skaet: I did say I *might* be late ;) [15:01] skaet: but I don't mind swapping [15:01] :) [15:21] cjwatson: to further add misery, Xubuntu alternate images install today [15:22] charlie-tca: that adds perplexity, but not misery [15:23] and the errors appear to have gone away from the log, too [15:23] Oh, well, okay [15:23] Ah, well, tomorrow is another day ;) [15:23] This is the first day this week they worked [15:56] slangasek: We're going to have to take kamoso off the dvd due to digikam as well (thus the pending kubuntu-meta upload). I'd appreciate it if you would accept kubuntu-meta and demote kamoso as we've got an upload of it to do and need it out of main first so it doesn't get stripped. [15:56] ack, poking [15:58] Thanks. [16:02] ScottK: all done [16:02] slangasek: Thanks. [16:03] * Laney found out today that libproxy is a bit broken and outdated [16:03] do people think it's too late now to get the new upstream in? there's a minor api break and new soname that i'd need to check out [16:03] hi skaet, there was just a new major chromium release with security fixes, it's the default browser now for mythbuntu, lubuntu, would this be ok to get uploaded before Monday? [16:04] Laney: New API *and* new SOVER? What could possibly go wrong? [16:04] :-) [16:04] skaet, could I get the linux-backports-modules-3.0.0-11.3 package approved in the queue. It's only an ABI bump to match the current kernel in the archive. [16:04] luckily it has very few revdeps [16:04] basically it currently will crash your app if you are using a PAC ile [16:04] heh, my interpretation of the facts is slightly different [16:05] "OMG it has revdeps in main" [16:05] :P [16:05] Laney: Is the PAC fix backportable? [16:05] I have no idea [16:05] Laney: no chance of cherry-picking the crasher fix? (What's a PAC file?) [16:05] but upstream sound a bit stroppy that we are still on the 0.3.x series [16:05] (https://bugs.launchpad.net/ubuntu/+source/libproxy/+bug/547106) [16:05] Launchpad bug 547106 in libproxy (Debian) (and 1 other project) "Please upgrade libproxy to 0.4.x (affects: 6) (heat: 38)" [Unknown,New] [16:05] slangasek: PAC files are how many/most large corporate networks configure proxy support remotely. [16:05] slangasek: So, y'know, no big deal. [16:05] it's moved on somewhat, so I don't know how easy it would be to tease out [16:06] heh [16:07] it does seem a rather chunky update though [16:08] ogasawara: we should definitely have the new lbm for archive consistency [16:08] http://paste.debian.net/130422/ [16:08] Laney: Ugh. This discussion goes all the way back to 2010? [16:09] one of Those, yes [16:09] infinity: large corporate networks and weirdos like me [16:09] cjwatson: Seriously? [16:09] yeah, it means I can configure my browser to auto-detect and I don't have to change proxy by hand when going between home and elsewhere [16:10] Laney: I think there's no question we should have upgraded to 0.4.x long ago. I think it's unfortunate that we're discussing it this close to release. :P [16:10] Laney: But if you're willing to hand-massage the revdeps and take ownership of the fallout...? [16:10] I only found out today when a semi-angry user came onto IRC [16:10] though admittedly I sometimes turn off auto-detect on untrusted wifi anyway since I'm not convinced I want my browser executing javascript handed to me by an untrusted network [16:10] but ANYWAY [16:11] cjwatson: But a good idea, in theory, right? ;) [16:11] right [16:11] * Laney would support more apps using libproxy in the future though [16:11] consistent proy support, yay [16:11] working x keys too [16:11] infinity: I'll look at it over the weekend [16:11] if the new libproxy can fix my keyboard, I'm all for it :P [16:12] I'm kinda hoping the new libproxy can make me soup. [16:12] no, that's libsoup. HTH [16:12] now I have to sample the delights of http://www.maypolebrewery.co.uk/ [16:12] I hear they now depend on each other. [16:12] good day. [16:12] Laney: you know there's no more xulrunner, right? [16:12] wait, what? no more xulrunner? [16:13] hyperair: we axed it about a month ago, not supportable under rapid release [16:13] ah right. i knew that. [16:13] LALALA CAN'T HEAR YOU [16:15] argh scons [16:16] why do you have to have pointless abstraction around everything [16:16] I don't much care why scons does any of the things it does, but I do wonder why people use it. [16:17] you used to maintain PHP, I wouldn't think you'd need to ask ;P [16:17] I've reformed. [16:17] TO use ruby instead? ;) [16:17] infinity: I shouldn't have to say this, but it wasn't my idea [16:17] cjwatson: That seemed like a safe assumption to make. [16:20] I think people use it because they want to be unable to cross-compile their software without taking excessive quantities of prescription medication [16:20] but maybe that's just me [16:21] ogasawara, approved. [16:23] micahg, ok by me if the mythbuntu and lubuntu leads are comfortable picking it up. [16:24] gilir, ^ [16:25] cjwatson: That's just unfair to the thousands of hard-working non-prescription drug vendors around the world who feed the rest of the Free Software world. [16:25] skaet: just asked superm1 in #mythbuntu-dev as well [16:35] skaet, micahg ok for me [16:35] gilir: thanks [17:06] skaet: Do you know if cdimage has been fixed to not purge candidate ISO's? [17:07] (aka, delete-released-images fun) [17:07] no, cdimage doesn't know what the candidates are [17:07] Daviey, no [17:08] we'll just have to be more alert and temporarily bump purge-days where necessary [17:08] Daviey, now that you're on the release team, feel free to help us figure out how to get this part a little more robust. ;) [17:09] It's currently 3 days? [17:09] http://paste.ubuntu.com/690961/ [17:09] I think the test is > rather than >= [17:10] we can't increase it generally right nw because we're getting uncomfortably close to disk limits again [17:10] *now [17:15] cjwatson: So, one i have a candiate - would it be bad to just chmod? [17:15] candidate* [17:16] I would really prefer you didn't. [17:16] We've had images go awol twice now, at the hour of release. [17:16] We should have a way to nominate a candidate from a shell on antimony, which could then maintain a local file listing the candidates and purge-old-images could respect that. [17:17] (One being for B1, the other for Maverick final release cloud images, for the same reason) [17:17] I have no idea whether just chmodding would (a) work or (b) cause other problems, and I don't particularly want to find out. [17:18] Well yeah, that was why i was asking :) [17:18] I don't mind committing to working on that issue btw. [17:18] What I'd need is a script that posts a candidate to iso.qa, given some appropriate authentication. I can deal with integrating it into cdimage (although not for B2). [17:19] Having a way to mark and save candidates would be nice. On the other hand, if you think that a new build will break your image, your candidate's a bit useless, IMO. [17:19] (In the sense that "we'll just ship something old because the archive is now broken" is a pretty bad place to be in) [17:19] Yep, agreed. [17:19] It's an emergency fallback. [17:19] Yeahp. [17:20] And also just to save on testing if the image actually doesn't change. Which is valuable. [17:20] But I suspect things might change between now and, say, Tuesday. ;) [17:21] infinity: Well, i see your point.. but we have to have a snapshot that can be QA'd. [17:22] Daviey: Yes, I realise that. See above. [17:23] Daviey: Anyhow. Some way to mark them would be shiny, I agree. Bonus points if it can export a candidate manifest (maybe a dotfile) that we can pull to auto-populate the ISO tracker. [17:23] (Which, I guess, is the opposite direction from how cjwatson was going...) [17:23] And his probably makes more sense, so non-cdimage people can mark cadidates for their products. [17:24] cjwatson: So, cdimage queries iso tracker to see if it shouldn't delete? [17:24] no, it should have a local copy [17:24] I don't want it reliant on network when building images [17:24] well, not that network [17:24] You can, as of now, 'export CDIMAGE_NOPURGE=1' to prevent a manual image build from purging old images [17:24] That should suffice for the time being [17:25] A candidate manifest would do if we can pull it reliably. I'd like to have a way to push from cdimage too, but maybe that isn't reuired [17:25] *required [17:25] We can then pull it from cron or something [17:26] And just keep the last version if iso.qa is inaccessible [17:26] archive-team.internal seems like a good place to pull from iso.qa, and then publish to cdimage? [17:26] If you like [17:26] Hmm.. iso tracker doesn't host the iso.. so what purposes does posting it there have? [17:26] If we don't want cdimage to be reliant on yet another sketchy remote host. [17:26] I have no idea what archive-team.internal is [17:26] Daviey: it lets people register tests [17:27] cjwatson: p.c.c/~u-a :) [17:27] oh right [17:27] sure [17:27] cjwatson: Ah, so when we enter freeze - all builds appear on the iso tracker automagically? [17:27] Daviey: no, they're posted manually right now [17:27] Being able to auto-populating the ISO tracker and saving a copy that's on the ISO tracker off somewhere has a certain appeal, and it unifies with the manifest and test results that way. [17:27] though it would help if there were a way for them to be posted automatically; it's tedious manual work right now [17:27] Daviey: Product owners should post their candidates to the ISO tracker (as they do now), but the difference would be that cdimage would now refuse to purge those. [17:28] Daviey: You fail to nominate, we might purge, not our problem? ;) [17:28] cjwatson: Yes, i understood it to be manual - but you'd like it to do it for all, when we ramp up to release ? [17:28] infinity: product owners don't typically post right now [17:28] cjwatson: I guess a way to do one batch post at the beginning of a freeze/release cycle would be cool. [17:28] it's normally the person building the images [17:28] Daviey: in general yes [17:28] cjwatson: I know, I was living in a fantasy world while verbally speccing that. Product owners should be the ones nominating candidates... [17:29] (In theory) [17:29] infinity: I think i'm missing how them being on the iso tracker would stop it purging, if it doesn't query the remote resource? [17:29] It seems two different issues? [17:29] Daviey: we should pull a manifest of candidates every so often and check that [17:29] rather than having to block while actually building the new image [17:29] cjwatson: ahhh [17:29] Daviey: It wouldn't query during purge, obviously we'd consolidate the info and keep a local cache. [17:30] does the iso tracker currently expose that data? [17:30] no idea [17:30] I mean it's on the front page and could be HTML-scraped [17:30] whether there's a more sensible mechanism, I don't know [17:30] iso tracker has a parsable export [17:30] The 'API' (i hate calling it that), has worked pretty well for the coud images, http://cloud-images.ubuntu.com/query/ [17:30] and of course all the names the ISO tracker has for things are gratuitously different from cdimage's [17:30] (not documented though :)) [17:31] stgraber: Have a URL handy? [17:31] Daviey: yep, hang on a sec [17:31] so there would be some ridiculously tedious thing to map them into the right form, much as there is in publish-image-set [17:31] Daviey: http://iso.qa.ubuntu.com/qatracker/dllist [17:31] Daviey: a bit empty at the moment [17:32] I'd have to see what it looked like with real content ... [17:33] if you don't mind me adding the current daily of Edubuntu (low number of subscribers), you'll see the real content for that image [17:34] * cjwatson -> dinner [17:34] oh, actually I can just change it in the DB for a few minutes, that way no e-mail get sent at all [17:36] Daviey: look now [17:36] stgraber: ah neat! [17:37] Oh, that's quite usable, I suspect. [17:37] stgraber: Can the date stamp be added as a field? [17:37] oh, yeah, that's not bad [17:37] yeah, adding stuff in there should be pretty easy, getting the changes deployed may take a while though [17:37] Daviey: cdimage wouldn't need that [17:38] cjwatson: just query the hash? [17:38] it can trivially construct the URL and just compare when deciding whether to purge an image [17:38] wouldn't even need that [17:38] I suppose the hash is an option too [17:38] good thinking that man. [17:39] stgraber: cool, thanks, that should actually be doable [17:39] anyway, really dinner [17:39] I can maybe have a go at that next week in my CFT [17:40] Dandy. [17:40] I am concerned how many beer tokens i now owe cjwatson . [17:42] Grr, my co-lo machine's network is being DoSed... [17:43] stgraber: Would it break anything to add a magic EOF string to that list? [17:43] stgraber: So we could determine if an empty document was actually an empty list, rather than a broken webserver returning an empty document? [17:45] stgraber: That would be about all that would be required to make it a robust enough thing for us to use it to do filesystem matches and purge with impunity. [17:46] adconrad: I don't think anyone is actually using that page at the moment, so adding an EOF string should be fine as long as it doesn't respect the format of the other lines (so it'd likely get skipped by existing parsers, if there's any) [17:46] Yeah, something simple like "=== end of list ===" or whatever. Just something we can match on to see if we actually got a list, or something broken. [17:47] or maybe have it return something when the list is actually empty? I think it'd actually be easier to implement (looking at the (ugly) code) [17:47] so we know that if it's empty and doesn't contain the magic string, it's because something failed [17:47] or are you worried of getting a partial list too? [17:48] Partials are a possible issue that this would solve too, yeah. [17:48] Mostly, this is just more discoverable, though. [17:48] Only returning a magic string on an empty list requires an API doc. Returning it at the end of the list unconditionally is discoverably simple to write against. [17:55] updated the DB to fix the existing netboot images in dllist [17:56] Why no hashes for the ARM netboots? [17:57] Daviey: I'm not :-) [17:57] infinity: doesn't much matter since cdimage isn't going to be purging those anyway [17:57] cjwatson: No, was just a curiosity. :) [17:58] well, I'm rather wondering where the hashes for the non-arm ones come from ;) [17:58] Hah. [17:58] should be up a couple of levels [17:58] e.g. http://archive.ubuntu.com/ubuntu/dists/oneiric/main/installer-i386/current/images/MD5SUMS [17:59] basically go up until you hit images/ [18:00] Looks like that md5 is for netboot/gtk/netboot.tar.gz ... No sure how useful that is, but whatever. [18:00] it lists all of them [18:00] ok, and it's using the one for "./netboot/gtk/netboot.tar.gz" instead of "netboot/mini.iso" or "netboot/netboot.tar.gz" [18:00] oh right, you mean iso.qa is using the wrong one [18:00] cjwatson: No, I mean on iso... Yeah. [18:01] has been way too long since I wrote that code... ;) [18:02] stgraber: Has it fallen into "so old, it embarrasses you" or "so old, you don't understand your own black magic", or a bit of both? [18:03] well, the current ISO tracker was a PHP project for one of my programming classes 4 years ago ;) [18:03] so I guess, the answer would be, both ;) [18:11] maybe I'll see if I can spend a bit of time of the 12.04 cycle improving the ISO tracker a bit. Upgrading it to a supported version of Drupal and getting the LP integration finally working would be a good start ;) [18:11] then maybe get a basic API would be nice (did plenty of that with weblive, so easy enough to re-use for the iso tracker) [18:12] that shouldn't take much more than 1 or 2 weeks of work and I guess would be useful for quite a few [18:12] stgraber, very useful indeed. yes please. [18:13] I'll make sure we have a blueprint for UDS. I know we discuss replacing it entirely at every UDS for the past 3 years but that still didn't happen, so maybe fixing it is a better and cheaper option ;) [18:14] :) sounds good. [18:16] evolution not revolution [18:18] ^ [18:18] We reinvent enough wheels around here. :P [18:18] stgraber: Or rewrite in Django :-) [18:18] * nigelb runs fast. [18:20] doing a little debugging on arm, manualizing for buildd selection with anger === chuck_ is now known as zul [18:23] nigelb: was that an offer to spearhead the new django version??? stgraber looks like your off the hook dude ;) [18:24] davmor2: Technically, that would be ISD's job...... [18:25] I won't be at UDS to get to that hook anyway ;-) [18:27] remote participation lets you have action items, too [18:29] heh, I think I took enough action items last time to lsat a yar. [18:29] *year [18:42] well, we've been talking about a rewrite in python for the last 3 years and it's still no there. So for now I'll stick with making the php one a bit more usable ;) [18:43] if ISD has time to rewrite something that covers all the current usecase, I have no problem with that ;) [18:45] I love how davmor2 went totally silent there :P [18:46] nigelb: Not an ISD project at all it's a QA tool and they'll like pass it back to stgraber :P [18:46] davmor2: good thing I don't work for QA then ;) [18:46] heh [18:47] Seriously though, If I had more time I'd love to help. [18:47] stgraber: I didn't say you did, I said they'd pass it back yo you ;) [18:47] But alas there's only 24 hours in a day. [18:47] nigelb: no there aren't there are 36 how long do you sleep man ;) [18:48] anyway, I don't think 12.04 is really the time where we'd like to switch to a completely different tool, so I'll have a look at how much changes I can make by alpha-1 and hopefully then just do a few bugfixes after that. [18:48] that's the tricky thing with the ISO tracker, it needs to work starting with alpha-1, not feature freeze ;) [18:48] stgraber: We could spec it and target it for 12.10. [18:49] * nigelb ponders [18:49] nigelb: I don't think it needs speccing, we already have at least 2 complete blueprints on it ;) it needs implementation [18:50] stgraber: Oh. lol. Okay. [18:50] I agree with you that we probably don't want to swtich before 12.04. [18:52] nigelb: https://wiki.ubuntu.com/QATeam/Specs/TestTracker [18:54] back on auto [19:09] * skaet --> appt, back on later tonight. [20:26] cjwatson: Is there a reason you haven't shoved that curl through binary NEW for the xmlrpc-c/udeb business, or were you just hoping for a less biased set of eyes? [21:25] infinity: I don't know if cjwatson saw it, i uploaded it. [21:52] skaet: when do we plan to disable the crontabs? I think we're back up to full armel image building capabilities after a test last night and lamont fixed the root cause this morning [21:53] I guess we at least need to let everything build tonight as all the livefs failed yesterday [21:56] might be easier to just do that in manual mode TBH [22:01] skaet had planned to just let the dailies keep going all weekend. [22:02] Given that we know we're getting an influx of new GNOME suff on Monday anyway. :/ [22:05] oh lovely [23:19] infinity, NCommander - lamont has a fix for https://bugs.launchpad.net/launchpad/+bug/851934, we're trying to figure out least impact for deploying it. [23:19] Launchpad bug 851934 in launchpad "umount-chroot fails to umount the chroot (affects: 1) (heat: 6)" [Undecided,New] [23:19] infinity: NCommander so... the fun bug where things seem to take a minute to die... [23:19] I just proposed a merge (lp:~lamont/launchpad/bug-851934) that at least keeps it from killing arm boards [23:20] the question of the day is, do we want to roll that lp-buildd to the buildds on monday (US), or do we want to wait until after beta2 [23:20] skaet: ugh, what a headache :-/ [23:20] with the attendant challenge that llvm may knock over the pandafarm [23:23] lamont: Assuming that loop doesn't have any syntax errors (have you at least locally tested to make sure it DTRT?), I can't see how rolling it out would do any harm. [23:23] lamont: Still, fixing this in the process killing script sounds saner, if we can figure out WTF is going wrong. [23:23] infinity: it's live on ain and shedir, and llvm built on ain [23:24] in terms of my work load, I'm going to let someone else take the "figure out what llvm is doing to sbuild-package" task [23:24] lamont: Ahh, kay. Your merge proposal said it was untested. ;) [23:24] yeah - I need to update the bug [23:25] and actually, I pushed the branch, but I haven't actually done the merge req yet.. :* [23:25] lamont: I'm all for rolling it out all over then, if it's working. But your call really. You get to fix the world if if breaks because the patch isn't in place. ;) [23:25] true dat. [23:26] Still grumpy that this is happening at all, mind you. I assume the bug appears on all arches, the others are just fast enough to not get hit by the race here, whatever that race may be. [23:26] but it's "roll out everywhere" because someone will doubtlessly go get the world current wrt package installs, just like we do all the time [23:26] we have only seen the bug on PANDA [23:26] oh, that could be [23:26] but haven't seen it on any other arm boards than panda [23:26] Yeah, doesn't mean much that it's only happened on Panda. [23:27] SMP+slow would be sufficient to explain that [23:27] Could be an SMP-specific race dealing with processes falling out of proc before closing fds, or who knows what. Either way, it's worth someone looking into. [23:28] Probably worth keeping the bug open even after your patch lands, with a big "this is a workaround, not a fix" comment. [23:28] https://code.launchpad.net/~lamont/launchpad/bug-851934/+merge/75831 <-- comments on the merge, if you would? [23:29] "only a workaround" comment added to the bug [23:31] The kill script is pretty anal about not exiting until everything's (supposedly) dead, according to proc, so this is still disconcerting. [23:33] Given the reproducibility, though, this might be debuggable locally. Maybe. [23:33] Happy birthday to me? [23:36] lamont: What prompted you to test with llvm-defaults and llvm-py? Neither has been uploaded or built in ages... [23:36] those were the packages that doko's rebuild of oneiric/arm did that knocked over the buildds [23:36] Ahh. [23:37] at least I'm guessing that's where they came from [23:37] Probably. [23:37] That would make some sense. [23:37] but the packages were identified based on the logfiles of dead buildds [23:37] Well, defaults is a quick build. And I'm assuming it's just the act of installing build-deps that breaks it, since the build itself doesn't seem to do much. [23:37] So, this might not be TOO painful to track. [23:38] *shrug* [23:38] * infinity fiddles. [23:38] https://launchpad.net/~lamont/+archive/non-virt/+build/2789423 [23:38] for your package copying fun [23:39] It's days like today that I remember why I have a local mirror... [23:40] ^^ that friendly-recovery isn't critical for beta, but it's required for friendly-recovery to be usable on cryptsetup-using systems [23:41] slangasek: We haven't stopped the daily cronjobs yet, if it makes a shiny new feature actually testable/usable on certain systems, I'm all for it. :P [23:45]