[00:23]  * wgrant fires off a full test suite run in an oneiric container.
[00:25] <wgrant> Hmm.
[00:26] <wgrant> mailman does not like the distutils installation :(
[00:26] <wgrant> Ahh.
[00:26]  * G looks to see who the OCR is today
[00:29] <wgrant> G: I'm not, but what needs reviewing?
[00:29] <G> oh, not online yet
[00:29] <G> wgrant: I've got a branch for Bug #697157
[00:29] <_mup_> Bug #697157: Streamline "You have assigned this bug to yourself" message  <bugs> <trivial> <ui> <Launchpad itself:In Progress by dev-nigelj> < https://launchpad.net/bugs/697157 >
[00:30] <G> existing message for all cases except "You have assigned this bug to yourself" if you assigned it to yourself
[00:30] <G> I'm just going to propose it
[00:44] <G> wgrant: https://code.launchpad.net/~dev-nigelj/launchpad/bug-697157/+merge/74035
[00:47] <wgrant> G: That seems like it will miss the case where it's assigned then unassigned quickly, once we move to queuing these (as we should). But you probably don't have enough information to detect that, so I guess it's reasonable.
[00:48] <G> wgrant: hmmm I hadn't thought of that case
[00:49] <G> wgrant: that said, wouldn't the method that e-mail is generated with change slightly if/when that happens
[00:50] <G> I didn't poke too much into how the scripted bugmail is generated, so I'd have to take another look at that but just thinking along that line
[01:20] <StevenK>  /wrists
[01:21] <wgrant> And since we probably won't deploy for 48 hours or so, we should break the graph shortly :)
[01:47] <G> ouch
[01:49] <StevenK> wgrant: Declaring destroy-openidrp ready for a review. I will poke stub when he surfaces.
[01:52] <wgrant> StevenK: But it doesn't have DB changes, does it?
[01:54] <StevenK> wgrant: It does not, but stub is OCR. :_P
[01:55] <G> I'm heading out, can someone ask stub to have a look at my MR?
[01:57] <StevenK> wgrant: Do you think it's safe or unsafe to destroy the bazaar-experts team?
[01:58] <wgrant> StevenK: Safe. We've deployed everything except librarian1 since then, AFAIK.
[01:58] <wgrant> And if librarian1 needs bazaar-experts, then it should blow up, because something is seriously screwed.
[03:01] <wgrant> wallyworld_: Have you tried to run tests on oneiric yet? rabbitfixture doesn't support rabbitmq-server 2.5.0.
[03:01] <wgrant> I think I have a fix.
[03:01] <wgrant> Need to test it on lucid too, though.
[03:02] <wallyworld_> wgrant: i've only run some yui tests and they went ok
[03:02] <wallyworld_> but nothong with rabbit
[03:02] <wallyworld_> wgrant: i'm pissed off ay qas. it keeps timing out for a simple operation :-(
[03:03] <wgrant> :(
[03:04] <wallyworld_> hard to know whether to qa-ok it. the sql is fine
[03:09] <wgrant> wallyworld_: Is this the bug privacy thing?
[03:09] <wgrant> That's not a simple operation :)
[03:09] <wgrant> Aha, got a rabbitfixture regex that works on both lucid and oneiric.
[03:09] <wgrant> But is a bit evil.
[03:11] <wgrant> Test attempt #2
[03:16] <wallyworld_> wgrant: yes, the bug privacy thing. the sql before and after the change is similar, except the fix has a couple of extra selects to update the subscription portal.
[03:17] <wallyworld_> wgrant: it works fine if i edit a bug that's on a smaller project than lp
[03:17] <wallyworld_> so i'm marking it as ok
[03:17] <wallyworld_> but, in testing, i found a bug that i confirmed is also on lp.net
[03:18] <wallyworld_> but it's not a result of any disclosure work so i'm not sure if we'll end up fixing it
[03:36] <wgrant> wallyworld_: What's the bug?
[03:36] <wallyworld_> wgrant: bug 841511
[03:36] <_mup_> Bug #841511: javascript error when changing a bug's security status <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/841511 >
[03:37] <wallyworld_> wgrant: it's in the same area of code as i was qa'ing so i had to look at diffs and also try it out on lp.net and found that it's an unrelated problem
[03:38] <wallyworld_> i should just assign myself to the bug and fix it
[04:06] <StevenK> Someone promote a high to critical ...
[04:07] <wgrant> I'm just looking for one.
[04:07] <nigelb> Morning
[04:07] <nigelb> 255? :(
[04:11] <nigelb> BAH!
[04:12]  * ajmitch thought that number was meant to be going down
[04:12] <nigelb> Actually, I'm not sure if 256 was right.
[04:12] <nigelb> Because there were bugs fixed and newer bugs added.
[04:13] <wgrant> http://webnumbr.com/launchpad-critical-bugs is what that number goes by. I just added two because I bumped two OOPSes that had been forgotten to critical.
[04:13] <wgrant> The real number is significantly higher, as several are private.
[04:14] <nigelb> No, I meant the 256 that acted as the target from which you startred
[04:15] <wgrant> Ah/
[04:16] <nigelb> Now, it seems as though there was no progress.
[04:16] <nigelb> That's bad.
[04:16] <wgrant> There was no progress; that is bad :)
[04:17] <nigelb> Huh?
[04:17] <nigelb> But Francis had this graph which said there were bugs closed as well as new ones opened.
[04:17] <wgrant> Well, yes, lots were closed. But lots more were found, and not all of them were new.
[04:17] <nigelb> I refuse to believe there was no progress. You guys aren't /that/ bad ;)
[04:26] <poolie> there has been progress, things have improved, but they haven't progressed in the intended direction of reducing the number of criticals
[04:30] <nigelb> Ah :(
[04:34] <G> so what, focus urgently on Critical bugs eh?
[04:37] <nigelb> I spent two hours on this bug yesterday, 66345
[04:37] <nigelb> eventually I realized its probably a bug in that library.
[04:37] <wgrant> Bug #66345
[04:37] <_mup_> Bug #66345: alt="" in dependency charts makes them poorly accessible <easy> <lp-blueprints> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/66345 >
[04:38] <wgrant> Hmm.
[04:38]  * StevenK looks for a critical, since the DSP vocab work is still blocked, and his two other branches are waiting for others
[04:39]  * G spent ages trying to understand the API to fix a Medium bug, must try again now
[04:41] <StevenK> Isn't bug 669296 mostly fixed?
[04:41] <_mup_> Bug #669296: App servers die and hang <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/669296 >
[04:41] <wgrant> StevenK: three died over the weekend.
[04:41] <wgrant> But it is much better now that logrotation is fixed.
[04:41] <wgrant> First time it's happened in weeks, I think.
[04:42] <wgrant> Rarrrgh buildbot is broken again.
[04:42] <wgrant> Tempting to roll back the three bzr revisions from the last couple of weeks.
[04:42] <wgrant> Hmm. Except that this looks like a dependency issue.
[04:43] <nigelb> Is this okay to fix? bug 99593
[04:43] <_mup_> Bug #99593: Package names in search results should be linked <bug-columns> <easy> <lp-bugs> <search> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/99593 >
[04:43] <nigelb> I'm not sure if I want it fixed :D
[04:43] <wgrant> nigelb: The next scheduled feature involves reworking bug listings.
[04:44] <wgrant> Ah, and it's even tagged (bug-columns).
[04:44] <wgrant> So I would avoid it.
[04:44] <nigelb> okay :)
[04:44]  * nigelb is /very/glad he asked
[04:46] <nigelb> I can't reproduce bug 750384 in my setup.
[04:46] <_mup_> Bug #750384: After muting with Bug:+mute, you get redirected to +subscribe <story-better-bug-notification> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/750384 >
[04:46] <nigelb> (devel)
[04:46] <nigelb> Could someone else confirm so I can close it.
[04:46] <G> yeah I looked at that one
[04:46] <G> can't reproduce either
[04:46] <nigelb> Excellent.
[04:47] <G> I meant to note it, but couldn't help but think I was doing something majorly wrong :)
[04:47] <StevenK> poolie: O hai -- you reviewed the Twisted patch for bug 703807, should we cherry-pick it, or wait for a upstream release?
[04:47] <_mup_> Bug #703807: launchpad sometimes serves download files as content-type text/html <regression> <Launchpad itself:Triaged> <pyOpenSSL:Invalid> <Twisted:Fix Released> <twisted-web (Ubuntu):Confirmed> < https://launchpad.net/bugs/703807 >
[04:47] <poolie> hi stevenk
[04:47] <StevenK> I have this feeling that there are multiple bugs about that issue
[04:47] <poolie> good question
[04:48] <poolie> well, it has certainly been duped several times
[04:48] <poolie> and perhaps caused some confusion or questions short of filing a bug
[04:48] <poolie> so python at the moment comes from an egg?
[04:50] <poolie> stevenk i guess the questions are
[04:50] <poolie> - how soon will we get an upstream release
[04:50] <StevenK> bug 174224 looks like a dupe
[04:50] <_mup_> Bug #174224: launchpadlibrarian sends wrong content-type header <librarian> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/174224 >
[04:50] <poolie> - how much do we care about getting it done soon vs later
[04:50] <poolie> - how much more work is it to cherry-pick?
[04:50] <StevenK> possibly bug 645924
[04:50] <_mup_> Bug #645924: .tar.xz files are served with text/plain content-type by launchpadlibrarian <content-type> <launchpadlibrarian> <lp-foundations> <mime-type> <xz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/645924 >
[04:51] <wgrant> StevenK: None of those are dupes.
[04:52] <poolie> in the thing i helpde fix in twisted, it would specifically give you text/html i think
[04:52] <wgrant> Right.
[04:53] <poolie> i suspect the first of them is just obsolete
[04:54] <StevenK> poolie: So, I don't think we can answer how soon we get an upstream release; I'm not sure about making a cherry-pick release of Twisted ...
[04:54] <StevenK> It is easier to just wait
[04:55] <nigelb> Can someone help me understand bug 294050?
[04:55] <_mup_> Bug #294050: Please add links for entries in the revision feed for a person <feeds> <lp-code> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/294050 >
[04:55] <nigelb> I think it means the feeds in branches.
[04:55] <nigelb> But I already see links to loggerhead
[04:55] <poolie> whoa, feeds
[04:56] <nigelb> heh, that's not improving my sinking feeling :)
[04:56] <poolie> so, i can help you understand it
[04:57] <nigelb> Excellent.
[04:57] <StevenK> OH! I know what I can remove
[04:57] <poolie> which is that basically it's about inserting a hyperlink, i guess pointing to loggerhead, into the revision feed data
[04:57] <StevenK> DELAYED COPIES
[04:57] <poolie> however, i don't know if revision feeds are a very useful feature
[04:57] <poolie> i never look at them any more
[04:57] <nigelb> poolie: I already see it everywhere I look.
[04:57] <poolie> oh, in the feed?
[04:57] <nigelb> yeah
[04:58] <poolie> then maybe you should just close this
[04:58] <nigelb> so, is it something wrong in how I perceive the bug.
[04:59] <nigelb> For example, in here I can see links http://feeds.launchpad.net/~launchpad-pqm/launchpad/devel/branch.atom
[04:59] <nigelb> Aha, let me close the bug then :)
[04:59] <poolie> go you
[04:59] <poolie> StevenK, i can still reproduce bug 645924
[04:59] <_mup_> Bug #645924: .tar.xz files are served with text/plain content-type by launchpadlibrarian <content-type> <launchpadlibrarian> <lp-foundations> <mime-type> <xz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/645924 >
[05:00] <poolie> i suspect it actually does have c-t text/plain in the librarian
[05:00] <StevenK> poolie: What's the LFA ID?
[05:00] <poolie> i guess it's 56261240 from the url?
[05:01] <StevenK> SELECT mimetype from libraryfilealias where id = 56261240; => text/plain
[05:01] <poolie> yep
[05:01] <wgrant> So the sniffer is (or was) bad.
[05:01] <poolie> so
[05:01] <poolie> if you add a new upload what does it get?
[05:01] <wgrant> I don't recall if we trust the browser or not.
[05:03]  * poolie looks at other .tar.xz files
[05:03] <StevenK> Hold on, I can do it faster
[05:03] <poolie> i'm looking through the staging db
[05:04] <poolie> so, only if you can type faster
[05:04] <poolie> or know the schema better
[05:04] <poolie> either is possible
[05:04] <StevenK> I have a query running, so let's see :-)
[05:05] <StevenK> SELECT distinct(mimetype) from libraryfilealias where filename like '%.tar.xz';
[05:05] <poolie> almost all are application/octet-stream, it seems.
[05:06] <StevenK> application/octet-stream and text/plain
[05:06] <G> I've been looking at https://bugs.launchpad.net/launchpad/+bug/674854 can't help notice that https://api.launchpad.dev/devel/ubuntu/hoary/architectures is a bit funny... total_size: 3, entries: []
[05:06] <_mup_> Bug #674854: [API] Add a way to get the list of "supported" architectures for each release <api> <lp-soyuz> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/674854 >
[05:06] <StevenK> xz doesn't exist in /etc/mime.types
[05:06] <StevenK> application/octet-stream is debatable, but loads better than text/plain
[05:06] <wgrant> G: DistroArchSeries doesn't have a launchpad.View permission defined.
[05:07] <G> wgrant: ahhh thanks
[05:07] <wgrant> G: Since they're all public, they need an adapter class deriving from AnonymousAuthorization (in canonical.launchpad.security)
[05:07] <wgrant> G: There's possibly already a bug for this.
[05:07] <wgrant> G: (they'll already appear if you're logged in, just not to anonymous users)
[05:08] <poolie> anyhow, it's not a dupe
[05:09] <poolie> if launchpad is sniffing the type, it seems to be mostly getting it right now
[05:09] <G> wgrant: apparently not
[05:10] <StevenK> poolie: So, we can probably correct that mimetype and close the bug, then
[05:11] <G> wgrant: http://pastebin.ubuntu.com/682351/ - but I will look at existing DistroARchSeries bugs
[05:11] <poolie> i guess
[05:11] <poolie> just fix it in the db?
[05:12] <poolie> ironically that really may interact with the caches
[05:12] <poolie> because this is exactly the unusual cache of the ct changing without the content changing
[05:12] <poolie> so it will probably stay cached for a while
[05:12] <poolie> arguably there's a bug here if lp trusts the client's ct even when it shouldn't
[05:13] <poolie> StevenK: so how would we handle a cherrypick to something that currently comes from an egg?
[05:14] <G> wgrant: oh I get you now
[05:15] <G> wgrant: thanks for the pointer :)
[05:15] <StevenK> poolie: We make a new egg and update the version in versions.cfg
[05:15] <poolie> called 0.12.0+0launchpad1 or something?
[05:15] <poolie> ok
[05:15] <wgrant> StevenK: Aha, I know how to fix the Jenkins failure, and I'm just going to redisable the failing buildbot test (it was only reenabled a weekish ago)
[05:16] <wgrant> G: Great. If you run into any issues, ask :)
[05:17] <G> wgrant: yeah, although I do like to have a big play w/ stuff before asking, because if it is a simple thing, I learn other stuffs/tidbits about the code by stuffing up :)
[05:17] <StevenK> poolie: I hope not. :-)
[05:18] <StevenK> poolie: Since the version of Twisted we have currently is 11.0.0 :-)
[05:18] <poolie> you know what i mean
[05:18] <StevenK> We have done cherry-picks of Twisted before, I just can't remember their format
[05:22] <G> wgrant: so really, http://pastebin.ubuntu.com/682356/ is all that is needed if I understand it correctly
[05:23] <wgrant> jelmer: Ah, worked out that bzrdir import error
[05:23] <wgrant> jelmer: The daemon was old, from a previous test run.
[05:23] <wgrant> jelmer: So its tree had been deleted.
[05:23] <wgrant> G: You're missing a newline, but yup.
[05:24] <G> missing, don't you mean included by accident?
[05:24] <poolie> StevenK: so can i leave that with you?
[05:24] <poolie> i'm going to try to fix bug 643223 this afternoon
[05:24] <_mup_> Bug #643223: should accept dkim based on from address and signing address belonging to the same person <dkim> <lp-foundations> <mail> <Launchpad itself:Triaged by mbp> < https://launchpad.net/bugs/643223 >
[05:24] <wgrant> G: Two empty lines between classes.
[05:25] <G> wgrant: oh good point, thanks :)
[05:25] <G> I am also missing something else...
[05:25] <G> a test!
[05:25] <wgrant> Mmm.
[05:38] <wgrant> Hah. Old bug for test_import_bzrsvn's disablement was 541526. New one is 841556. A bit too similar...
[05:39] <poolie> 4 out of 14 sampledata people still work at canonical
[05:39] <poolie> not counting foo bar
[05:40] <nigelb> is that good or bad? :)
[05:40] <poolie> i don't know
[05:40] <poolie> it's pretty old
[05:40] <poolie> not too bad
[05:40] <poolie> i know all but 2 of them
[05:41] <nigelb> sampledata = stuff that rf installs onto my machine right?
[05:41] <poolie> yes
[05:41] <poolie> database/sampledata
[05:41] <nigelb> I only saw mark as a familiar name
[05:41] <StevenK> No, the sampledata is stuff that make schema stuff into the development database
[05:41] <StevenK> rf installs the developer dependencies
[05:42] <nigelb> Ah.
[05:42] <poolie> well, strictly, make schema not rf, yes
[05:42] <nigelb> But poolie got what I meant :)
[05:49] <nigelb> Hrm, today's Google Doodle is pretty awesome.
[05:49] <G> I don't know, I haven't been impressed by Google's Doodle's for a while
[05:55] <poolie> i had no idea he was Parsi
[05:56] <nigelb> Neither did I :)
[06:17] <G> hmmm think I may be able to put another bug in the 'fixed' basket
[06:23] <poolie> stevenk, wgrant, in a test, i need a user account with multiple email addresses in different domains
[06:23] <poolie> what's the tasteful way to get this?
[06:23] <wgrant> makePerson()
[06:23] <wgrant> makeEmailAddress()
[06:23] <wgrant> makeEmailAddress()
[06:23] <wgrant> makeEmailAddress()
[06:23] <G> one at canonical.com one at ubuntu.com in makePerson?
[06:23] <poolie> on TestCaseWithFactory?
[06:23] <G> poolie: yeah, there is another test like that, that I was reading
[06:24] <StevenK> poolie: self.factory.makePerson() ...
[06:24] <poolie> thanks
[06:24] <poolie> am i right that in general you prefer tests to always make the person (or whatever) rather than relying on the sample data?
[06:26] <wgrant> poolie: Yes.
[06:26] <wgrant> poolie: It is somewhat slow, but means we're not relying on fragile, rotten sampledata.
[06:26] <wgrant> Which is largely invalid in great parts.
[06:30] <StevenK> I'd like to remove most sampledatsa
[06:30] <StevenK> s/sa/a/
[06:31] <wgrant> 258 :(
[06:32] <StevenK> What is the most recent?
[06:33] <wgrant> Bug #841556
[06:33] <_mup_> Bug #841556: test_import_cvs, test_import_bzrsvn and test_import_subversion disabled <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/841556 >
[06:36] <G> wgrant: I got it! http://bazaar.launchpad.net/~dev-nigelj/launchpad/bug-674854/revision/13863 allows me to do: http://pastebin.ubuntu.com/682386/
[06:36] <G> wgrant: thanks massively for your help
[06:40] <StevenK> poolie: This cherrypick is not so easy, since the diff from trac won't apply at all to 11.0.0
[06:46] <poolie> my patch for 721166 passed an ec2 test
[06:46] <poolie> bug 721166
[06:46] <_mup_> Bug #721166: Tests sometimes fail on EC2 due to _LockWarner garbage <build-infrastructure> <spurious-test-failure> <Launchpad itself:In Progress by mbp> < https://launchpad.net/bugs/721166 >
[06:46] <poolie> should i land it?
[06:47] <poolie> i think the tests can now be reenabled since lp has bzr 2.4
[06:47] <StevenK> Didn't wgrant just roll that back?
[06:47] <poolie> rolled back bzr?
[06:48] <wgrant> I didn't roll back bzr.
[06:48] <wgrant> I redisabled the undisabled tests.
[06:48] <poolie> the ones that gary changed?
[06:48] <poolie> they might have been different
[06:50] <wgrant> Nah, different ones.
[06:50] <wgrant> Did he actually end up disabling any?
[06:50] <wgrant> I think the bzrdir import issues were due to the three I just disabled.
[06:50] <wgrant> Leaving ancient processes around.
[06:51] <poolie> so my question is, really
[06:51] <poolie> if i land this, is there an unreasonable chance that either bzr will be downgraded or spurious failures will disrupt things?
[06:51] <wgrant> 15 minutes ago the chance of a spurious failure disruption was probably around 0.95.
[06:51] <wgrant> Now it's hopefully lower.
[06:52] <wgrant> But please avoid any even slightly disruptive changes until late in the week.
[06:52] <wgrant> We've just had about 20 failures.
[06:52] <poolie> ok, there's no rush
[06:52] <poolie> it's only a tidying-up fix
[06:52] <wgrant> What with the three invasive bzr infrastructure changes lately, it's a bit of a mess.
[06:53] <poolie> oh, what were the three?
[06:53] <wgrant> There was the 2.4 upgrade, the puller SafeBranchOpener unification, and some other codeimport/puller change, the details of which I forget.
[06:54] <poolie> did they break a lot of stuff?
[06:54] <wgrant> It's not clear.
[06:54] <wgrant> Because also around that time three tests were undisabled.
[06:54] <wgrant> They spuriously failed in spectacular ways occasionally.
[06:54] <wgrant> But then lots of other tests blew up in even more spectacular ways.
[06:55] <wgrant> but I speculate that that is because of debris left by the original three broken tests.
[06:55] <wgrant> So I've disabled them, will get buildbot cleaned up, and then hope that stuff stops failing dammit.
[06:55] <wgrant> Otherwise I'll just revert all the bzr changes in the last two weeks and work out wth is going on.
[07:00] <poolie> StevenK: you could just take my patch from http://twistedmatrix.com/trac/ticket/4156#comment:7
[07:10] <wgrant> StevenK: How do I restart a Jenkins build? Just kill the executor and force?
[07:10]  * wgrant tries.
[07:11] <LPCIBot> Project devel build #1,031: ABORTED in 21 min: https://lpci.wedontsleep.org/job/devel/1031/
[07:11] <wgrant> We may have success in 4 hours.
[07:17] <LPCIBot> Project devel build #1,032: STILL FAILING in 6 min 34 sec: https://lpci.wedontsleep.org/job/devel/1032/
[07:19] <wgrant> Lies.
[07:19] <wgrant> urllib2.URLError: <urlopen error [Errno 2] No such file or directory: '/mnt/test/launchpad/workspace/devel/download-cache/dist/setuptools-0.6c11-py2.6.egg'>
[07:19] <wgrant> How unpleasant.
[07:20] <wgrant> StevenK: You broke everything.
[07:20] <wgrant> Seems I can't destroy the executor.
[07:20] <wgrant> :(
[07:21] <LPCIBot> Project devel build #1,033: STILL FAILING in 39 sec: https://lpci.wedontsleep.org/job/devel/1033/
[07:22] <wgrant> :(
[07:25] <G> ouch, that doesn't sound good
[07:26] <wgrant> Well, the test suite has sped up by 360x. That can only be a good thing, right? :)
[07:26] <rvba> hehe
[07:27] <G> eek GPG tests just OOM'd in my VM
[07:27] <wgrant> How much RAM is assigned?
[07:27] <G> and that is 2GB RAM
[07:27] <wgrant> amd64?
[07:28] <wgrant> There may be left over librarians and stuff eating a couple of hundred megabytes each.
[07:28] <wgrant> amd64 with 2GiB is doable, but pushing it.
[07:29] <G> yeah, amd64
[07:29] <G> I might download lucid-server i386 and put on that VM tbh
[07:30] <G> laptop is Core2Duo w/ 4GB
[07:33] <wgrant> My laptop with the same specs runs it OK outside a VM, but if I want to do it in a VM i386 is better.
[07:36] <nigelb> Is bigjools going to be around today?
[07:36] <nigelb> There's a bunch of bugs that I want to talk to him first before touching :)
[07:38] <adeuring> good morning
[07:38] <wgrant> nigelb: I think he should be.
[07:38] <wgrant> let me check.
[07:39] <G> wgrant: yeah, don't blame you for using i386
[07:41] <wgrant> nigelb: Looks like he should be here today.
[07:42] <rvba> nigelb: Yes, he will be here today.
[07:42] <nigelb> wgrant / rvba: Thanks!
[07:46] <LPCIBot> Project devel build #1,034: STILL FAILING in 44 sec: https://lpci.wedontsleep.org/job/devel/1034/
[07:48] <G> wgrant: fast but still failing?
[07:50] <wgrant> Yeah, the slave is broken.
[07:50] <wgrant> Need StevenK to fix.
[07:55] <StevenK> You do?
[07:57] <wgrant> StevenK: I can't work out how to kill the executor.
[07:57] <wgrant> Only the build.
[07:58] <poolie> would it be tasteful to add a script to lp that just processes one mail message from stdin?
[07:58] <poolie> at least for the sake of testing
[07:59] <StevenK>  Oh. Delete the slave.
[07:59] <poolie> it seems a lot easier than configuring a mailbox
[07:59] <wgrant> poolie: It is distasteful to not have one.
[07:59] <wgrant> StevenK: warthogs doesn't seem to have permission to do that.
[07:59] <poolie> or, to add a parameter to process-mail to do this (though that seems a bit hard)
[07:59] <wgrant> Or at least I can't find a relevant button.
[07:59] <poolie> :)
[08:01] <StevenK>  Navigate to the slave, hit the button on the left.
[08:01] <wgrant> https://lpci.wedontsleep.org/computer/i-225ca042/ has only four links on the left.
[08:01] <wgrant> Back to List
[08:01] <wgrant> Status
[08:01] <wgrant> Build History
[08:01] <wgrant> Load Statistics
[08:05] <poolie> wgrant, ok, will do
[08:05] <poolie> we could potentially even get the mta/mda to feed messages directly in to this
[08:05] <poolie> though startup time may make that suck
[08:06] <wgrant> "may"
[08:06] <poolie> well
[08:06] <poolie> will it be worse than the current latency from running from cron?
[08:06] <poolie> and, will the wasted cpu matter?
[08:07] <poolie> i don't know how busy the machine that currently does this is, and how much mail it gets
[08:07] <wgrant> It's about 10s of CPU time per invocation.
[08:07] <wgrant> That's not insignificant.
[08:07] <poolie> true
[08:07] <poolie> especially if we get mail more often than every 10s during peak time
[08:07] <wgrant> Yes.
[08:08] <poolie> yes we do?
[08:08] <wgrant> And if someone sends an email to a few bugs, the world ends.
[08:08] <poolie> hmm
[08:08] <poolie> if the mail is exploded apart before it gets to us
[08:08] <wgrant> It used to be frequent to email hundreds of bugs at once, but that is probably reduced by the API.
[08:08] <poolie> which it probably wouldn't be
[08:08] <nigelb> Is it known that loggerhead fails on first try to see launchpad code?
[08:08] <poolie> but could be
[08:08] <wgrant> Wouldn't it be?
[08:08] <wgrant> nigelb: Yes.
[08:08] <poolie> at any rate it's not optimized
[08:08] <poolie> i mean, it's not worth optimizing
[08:09] <wgrant> poolie: Unless it's treating process-mail as a relay, it should be exploded.
[08:09] <nigelb> wgrant: Loggerhead suckage?
[08:09] <poolie> yes, just that
[08:09] <poolie> it needs to warm the cache
[08:09] <wgrant> nigelb: Yeah, plus LP is insanely huge.
[08:09] <G> bigjools: hey, is https://code.launchpad.net/~dev-nigelj/launchpad/bug-674854 anything like what you were thinking of to fix bug 674854?
[08:09] <_mup_> Bug #674854: [API] Add a way to get the list of "supported" architectures for each release <api> <lp-soyuz> <trivial> <Launchpad itself:In Progress by dev-nigelj> < https://launchpad.net/bugs/674854 >
[08:09] <nigelb> And, looking the launchpad devel branch also timesout on first try.
[08:10] <nigelb> Probably related I guess.
[08:17] <bigjools> G: why did you need the security adapter change?
[08:17] <bigjools> did anon access not work?
[08:18] <G> bigjools: correct
[08:18] <G> it would reply w/ count: 3, entries: []
[08:18] <wgrant> bigjools: Remember that lazr.restful filters items out of collections unless launchpad.View is held.
[08:18] <wgrant> bigjools: And DAS has no launchpad.View.
[08:19] <bigjools> G: ah bugger.  Well the bug is essentially already fixed, you could note that and/or file a new one.
[08:19] <wgrant> So it reverts to the default one, which is checkAuthenticated -> True, checkUnauthenticated -> False
[08:19] <lifeless> wow, 2 non-staff contributors. Excellent :)
[08:19] <G> wgrant: it didn't even work when authenticated
[08:19] <bigjools> lazr.restful is teh awsum
[08:20] <nigelb> lifeless: both of us even have the same first name ;)
[08:20] <wgrant> Evening lifeless.
[08:20] <lifeless> 'lo y'all :)
[08:20] <nigelb> (For added confusion)
[08:20]  * bigjools disappears for calls and mountains of email
[08:20] <G> bigjools: http://pastebin.ubuntu.com/682351/
[08:20] <G> lifeless: howdy
[08:20] <lifeless> I saw a mountain of cron spam this morning
[08:20] <wgrant> lifeless: Chameleon has that effect on people :(
[08:20] <nigelb> lifeless: Glad to hear that everything is now fine :)
[08:20] <wgrant> The broken eggs need fixing :(
[08:21] <lifeless> wgrant: these are setuptools non-dpkg eggs again ? -how-
[08:21] <wgrant> lifeless: Well, it changed from a directory to a symlink.
[08:21] <wgrant> And dpkg doesn't seem to always handle that nicely.
[08:21] <wgrant> But sometimes it does.
[08:21] <wgrant> And blah.
[08:21] <wgrant> rm -rf, apt-get install --reinstall, problem solved.
[08:22] <wgrant> But it is way down the queue :(
[08:23] <poolie> i have a process-one-mail.py
[08:23] <poolie> that's so much better
[08:24] <nigelb> This reminds me.
[08:24] <StevenK> wgrant: Do I still need to delete that slave?
[08:24] <nigelb> How does one do QA of something email related on qastaging?
[08:24] <wgrant> StevenK: Please.
[08:25] <StevenK> nigelb: One panics.
[08:25] <wgrant> Although it seems to be gone.
[08:25] <wgrant> If you haven't already killed it.
[08:25] <nigelb> StevenK: Test in production? :P
[08:25] <StevenK> wgrant: If the slave sits unused for 30 minutes, it gets killed
[08:26] <nigelb> We tested summit in production /at/ UDS :D
[08:26] <bigjools> nigelb: ask someone with access to the staging inbox to look at it for you
[08:26] <wgrant> StevenK: Ah, that would do it. Thanks.
[08:26] <nigelb> bigjools: Great, that means anyone from LP team right?
[08:27] <bigjools> nigelb: not everyone IIRC, all squad leads and assorted others
[08:27] <wgrant> I think it's meant to be squad leads. But most people seem to know the password.
[08:27] <nigelb> Anyway, I need to get that branch merged.
[08:27] <nigelb> Testfix mode was blocking it.
[08:28] <nigelb> I don't know if that's done yet.
[08:28] <wgrant> I think I've fixed it all. But the current build is missing the last two revisions. If we are lucky we will be green in 3 hours. If we're not, it will be 8 hours.
[08:28] <wgrant> And I may of course be wrong, and we will still be red in 8 hours.
[08:28] <nigelb> Oh good, so by the time I'm home.
[08:28] <wgrant> In which case I will be sad.
[08:28] <wgrant> And hope that gary fixes it before I awake.
[08:30] <nigelb> BUt you won't sleep anyway.
[08:30] <nigelb> :D
[08:36] <poolie> woo, locally accepted second-address signed mail
[08:36] <poolie> that feels like a much better test
[08:38] <lifeless> bigjools: inbox is accessible to all canonical I think :)
[08:41] <G> bigjools: yeah, just tested again, the <distro>.current_series.architectures method against devel definately returns no entries, when authenticated, but this change allows it to happen, but are you saying it should be a seperate bug?
[08:42] <G> (Just a tad confused, because this seems to be the last bit a puzzle (even if the majority of the work was done prior) to get what the description etc lays out
[08:43] <StevenK> stub: O hai!
[08:48] <poolie> wgrant, well, that script is added in https://code.launchpad.net/~mbp/launchpad/643223-dkim-aliases/+merge/74061 plus some other stuff
[08:48] <poolie> i'm happy to hold off landing it until things are safe enough
[08:48] <StevenK> stub: Can you please look at the SQL in bug 834384 and comment in the bug about it?
[08:48] <_mup_> Bug #834384: StaticDiff is unused and needs to be shot <Launchpad itself:Confirmed> < https://launchpad.net/bugs/834384 >
[08:59] <stub> StevenK: commented
[09:01] <bigjools> G: that is a separate bug because it seems the main one was fixed some time ago
[09:01] <bigjools> not sure why it's not returning anything even when authenticated
[09:02] <bigjools> I'd hope there was a test for this somewhere!
[09:02] <G> bigjools: couldn't find any, which is why my branch includes a test for this logic
[09:02] <G> bigjools: so what, I close the current bug, open a new one tagged regression?
[09:03] <bigjools> G: yes, I think so.  I'd check to see if there's a passing test for DS.architectures as well.
[09:03] <G> bigjools: yes, that is what the test, is testing
[09:04] <bigjools> apart from you new one I meant :)
[09:04] <bigjools> your*
[09:04] <G> that the number of entries returned via the API is the same as in the DB
[09:04] <G> oh, I couldn't find any other tests that seemed to be testing it
[09:14] <G> bigjools: oh funny. I tried again, and it worked for an authenticated launchpadlib setup this time
[09:14] <G> so obviously it was just never enabled for anonymous
[09:14] <bigjools> hah
[09:15] <G> I was going through everything again unauthed, then authed to make double sure, and for clean output for the bug
[09:15] <G> so I guess less regression, more oversight/accident?
[09:15] <bigjools> could be an out of date wadl
[09:15] <bigjools> cache
[09:16] <G> this dev environment has only existed for half a week
[09:17] <bigjools> oh you were testing against a .dev instance?
[09:17] <poolie> does anyone know how i could get a gpg key into a dev instance?
[09:17] <G> bigjools: yeah
[09:17] <poolie> maybe poking it into the db
[09:17] <wgrant> poolie: utilities/start-dev-soyuz.sh, push key to keyserver.launchpad.dev, add key as normal.
[09:18] <G> bigjools: maybe I originally took too long to validate the OAuth session or something the first time round :)
[09:18] <poolie> thansk
[09:18] <poolie> *thanks
[09:18] <poolie> and apparently i'm going to need to do a mail roundtrip to confirm it?
[09:18] <wgrant> Or just grab the token from logintoken;
[09:19] <wgrant> Then go to /token/$WHATEVER, and confirm.
[09:19] <G> bigjools: alright, so I'll close 674854, and the new bug will be that it doesn't allow anonymous access :)
[09:20] <bigjools> G: great!#
[09:20] <poolie> just manually testing i haven't broken that
[09:20] <poolie> s// gpg signing
[09:26] <G> bigjools: okay, done, I'll propose the branch for merging now under the new bug
[09:37] <bigjools> today's Google Doodle is the best ever
[09:39] <poolie> is there going to be an ocr today?
[09:39] <poolie> o/ bigjools
[09:40] <nigelb> bigjools: I agree :)
[09:41] <bigjools> o/ poolie
[09:41] <bigjools> nigelb: is your 'tache in honour of Fred? :)
[09:41] <nigelb> haha, no
[09:41] <nigelb> hrm, but I can claim so now :P
[09:42] <G> bigjools: I guess I'm too young to know much about Freddie Mercury the doodle is fancy, but yeah...
[09:42] <nigelb> Also, pep8 and pyflakes are awesome. I just started using it on all my python code after getting used to make lint :D
[09:42] <bigjools> nigelb: use vim?
[09:42] <G> any yeah, no OCR today?
[09:43] <nigelb> yeah, I'm guessing there's a plugin as well?
[09:43] <bigjools> G: let's just say he was the frontman for the biggest rock band of all time :)
[09:43] <bigjools> nigelb: there is and it's teh awsum
[09:43] <nigelb> Oh good. let me find and isntall.
[09:44] <G> bigjools: whats a good song of his band's then, I might look at it on YouTube or something
[09:44] <StevenK> G: Are you serious?!
[09:44] <bigjools> !
[09:44] <bigjools> G: you've heard of Queen, right?
[09:44] <G> yeah... Queen
[09:45] <nigelb> G has successfully made everyone feel old I believe.
[09:45] <G> Another One Bites the Dust, We Will, We Will, Rock You...
[09:45] <StevenK> Bohemian Rhapsody?
[09:45] <bigjools> nigelb: you're a whippersnapper yourself
[09:45] <G> yeah...
[09:45] <StevenK> I don't feel old, just surprised that someone doesn't recognise Freddie
[09:45] <G> ohhhh he was one of the main guys?
[09:45] <G> haha, never knew
[09:45] <nigelb> bigjools: Hey, but I've heard of Freddie.
[09:46] <StevenK> G: He was the vocalist until his death
[09:46] <G> I'd heard of Freddie, but never knew the Freddie<->Queen connection
[09:46] <StevenK> G: For David Bowie's 50th birthday concert, they had to get a women to sing Freddie's part of Under Pressure
[09:46] <G> hmmm according to Wikipedia, I was born before he died
[09:46] <G> so I guess that is something
[09:47] <nigelb> G: How old are you again?
[09:47] <G> learn something new every day eh
[09:47] <G> nigelb: > 21
[09:48] <nigelb> Hrm, we may be around the same age :D
[09:49] <G> heck, I'd never seen the Matrix films and the Back to the Future films until a few years ago
[09:49] <nigelb> ....
[09:50] <nigelb> G: Star War / Star Trek?
[09:50] <nigelb> *Star Wars
[09:51] <G> nigelb: I've seen a couple of Star Wars films, didn't really like them that much, and only really seen the latest Star Trek movie (which I thought was okay), but I'm not generally a big fan of the whole Sci-Fi genre
[09:51] <StevenK> G: Heathan.
[09:51] <StevenK> Sigh. Heathen, even.
[09:52] <G> StevenK: it's just not my thing
[09:52] <bigjools> if it was two from episodes 1-3 them I am not surprised, I didn't like them either :)
[09:52] <G> The Phantom Menace was one of the ones I know I've seen
[09:53] <bigjools> awful movie
[09:53] <G> but anyway, don't hold it against me :)
[09:54] <bigjools> go and dig out the original 1977 Star Wars, it's great.
[09:54] <nigelb> Amen
[09:55] <nigelb> I watched the entire Star Wars series in one day from 8 am to 8pm (I was sick and had nothing else to do)
[09:55] <G> bigjools: is http://www.fatso.co.nz/Movies/Star_Wars_-_Episode_IV_A_New_Hope_1977/5552 the one you were talking about?
[09:55] <bigjools> G: yes
[09:56] <G> okay, it's in my queue, might even get it Wednesday
[09:56] <bigjools> rock on
[09:56] <StevenK> nigelb: More impressive if you do it with LOTR
[09:56] <G> meh, Demand: High :(
[09:56] <bigjools> "PG - Contains medium level violence" - heh
[09:56] <nigelb> StevenK: I didn't like LOTR for some reason.
[09:56] <StevenK> nigelb: *Out*
[09:56] <G> bigjools: gotta love the NZ ratings :)
[09:56] <nigelb> I tried multiple times to get myself to watch it
[09:57] <nigelb> Or read the books
[09:57] <G> I'm a Kiwi, I watched all 3 LOTR movies, and didn't really like it, much prefer the books :)
[09:57] <bigjools> I faltered half way through the second LOTR book.  But then I was 13 when I tried to read it
[09:57] <StevenK> I managed all 3 when I was 15
[09:58] <G> yeah, thats around when I managed to get them all ready
[09:58] <G> *read
[09:59] <StevenK> G: Out of interest, North or South?
[09:59] <G> StevenK: Auckland
[10:00] <G> ahh I see, you are Sydney
[10:01] <bigjools> jafa!
[10:01] <G> yep, born & raised
[10:01] <G> although I spent a bit of time in Brisbane
[10:01] <nigelb> jafa?
[10:02] <nigelb> bigjools: Wait, your Australian as well?
[10:02] <G> nigelb: Just Anotherl F'ing Aucklander
[10:02] <nigelb> haha
[10:02] <bigjools> nigelb: no, Brit.
[10:02] <G> Jaffa on the other hand, is a nice lolly
[10:03] <nigelb> bigjools: That's what I thought as well :)
[10:05] <gmb> bigjools: The comments on https://bugs.launchpad.net/launchpad-buildd/+bug/840055 seem to suggest it's not actually a bug but is user error of some sort. Can you cast your weather eye over it and tell me if that's the case please?
[10:05] <_mup_> Bug #840055: strange build loop with binutils 2.21.53 <Launchpad Auto Build System:New for dns> < https://launchpad.net/bugs/840055 >
[10:05] <bigjools> gmb: sure
[10:05] <gmb> Ta
[10:06] <bigjools> gmb: it's a buildd-manager bug
[10:06] <bigjools> and a dupe
[10:06] <gmb> bigjools: Ah, thanks.
[10:07]  * bigjools plays hunt the dupe
[10:08] <bigjools> gmb: I suspect it's bug 717969
[10:08] <_mup_> Bug #717969: storeBuildInfo is sometimes ineffective <boobytrap> <buildd-manager> <Launchpad itself:Triaged> < https://launchpad.net/bugs/717969 >
[10:09] <gmb> bigjools: Thanks, I'll mark it as such and include this commentary.
[10:09] <bigjools> ta
[10:23] <wgrant> bigjools: A generic "audit" sounds a lot too much like "karma"
[10:23] <StevenK> gmb: O hai, we seem to be Henning-less today. Would you mind reviewing https://code.launchpad.net/~stevenk/launchpad/destroy-openidrp/+merge/74007 ?
[10:23] <bigjools> completely different
[10:23] <wgrant> bigjools: Similarly FK-less and prone to weak keys.
[10:23] <gmb> StevenK: I can't look right now, but I'll try to take a look this pm.
[10:23] <wgrant> Unless treated very carefully.
[10:24] <StevenK> gmb: Just to sweeten the deal, the line count is +2 / -283
[10:25] <G> oh can someone comment on my suggestions for Bug 306569?
[10:25] <_mup_> Bug #306569: Link to https://help.launchpad.net/Code from project branch listing page <lp-code> <trivial> <ui> <Launchpad itself:In Progress by dev-nigelj> < https://launchpad.net/bugs/306569 >
[10:29] <stub> Somewhere buried in our wiki there will be a document documenting how to rollback revision and revert rollbacks. But I know not where.
[10:32] <wgrant> stub: You've fixed pgbouncer?
[10:32] <stub> That will be part of this branch, yes
[10:33] <wgrant> stub: You're hacking PATH? :/
[10:33] <stub> No, I'll install a new .egg
[10:33] <wgrant> Oh.
[10:33] <wgrant> Right.
[10:33] <wgrant> So, just do a cherrypick of the original revision, or an inverse cherrypick of my rollback.
[10:33] <wgrant> bzr merge -r1233..1234 .
[10:35] <stub> Any magic when it needs to go to pqm?
[10:37] <stub> Why did we decide a hard coded path was best btw? That is making the module Ubuntu/Debian specific.
[10:37] <lifeless> just add /usr/sbin to PATH in the subprocess call
[10:37] <lifeless> subprocess has glue for that
[10:38] <lifeless> and it won't make it so specific
[10:38] <stub> k
[11:00] <nigelb> gmb: Thanks! (re:spamming user)
[11:01] <gmb> nigelb: No worries. I enjoy taking the loving mallet of correction (as John Scalzi would call it) to spammers.
[11:01]  * gmb -> lunch
[11:01] <nigelb> hehe
[12:00] <wgrant> jtv: Hm, that's upsetting.
[12:00] <jtv> wgrant: what is?
[12:00] <wgrant> jtv: That the rabbitfixture fix didn't fix/
[12:00] <jtv> Ah that.  Yes.
[12:00] <wgrant> Can you pastebin 'sudo rabbitmqctl status'?
[12:06] <jtv> wgrant: http://paste.ubuntu.com/682518/
[12:07] <wgrant> Baaaah.
[12:07] <wgrant> It's like postgres.
[12:07] <wgrant> It only quotes when necessary.
[12:07] <wgrant> Rip the quotes out of the first line of the regex. Should work then.
[12:07] <wgrant> Or ? them, as desired.
[12:09] <jtv> wgrant: quotes?  I only see one quote on that line.  And a few more in other lines.
[12:10] <jtv> Making what look to be the opening and closing quotes optional…
[12:10] <jtv> —and that does the trick.  Thanks wgrant!
[12:10] <wgrant> The first line of the original uncommented version of the regex that I never pastebinned, come on, read my mind.
[12:11] <jtv> Not on Mondays, you know that.
[12:12] <jtv> Review needed!  Any takers?  https://code.launchpad.net/~jtv/launchpad/pre-832647/+merge/74087
[12:14] <G> good point, I've got two MPs for review when there is an OCR around :)
[12:17] <StevenK> I have one as well
[12:17] <StevenK> G: I bet my MP removes more code than yours. :-P
[12:18] <G> StevenK: you can be sure of it :)
[12:18] <G> hold on. is this a "My MP is bigger than yours" thing? :)
[12:18] <jtv> StevenK: how about we trade reviews and then take one of G's each?
[12:19] <nigelb> bigjools: I think there's something ajax that changes the links from blue to green.
[12:20] <wgrant> nigelb: You can't actually see the links he's talking about.
[12:20] <wgrant> nigelb: They are indeed blue and un-spinnered.
[12:20] <nigelb> wgrant: Why can't I see those?
[12:20] <nigelb>  :(
[12:20] <wgrant> They were mostly hacked up during the thunderdome, and are enabled only for ~launchpad right now AFAIK.
[12:20] <nigelb> Also GAH for these things.
[12:20] <nigelb> Can I see it on my dev instance?
[12:21] <wgrant> Sure, if you push up a branch, go to https://launchpad.net/+feature-rules, add 'code.ajax_revision_diffs.enabled default 0 on', and view the MP page.
[12:21] <wgrant> s/net/dev/
[12:21] <nigelb> okay :)
[12:22] <wgrant> https://launchpad.dev/+feature-info describes all the current feature flags.
[12:24] <nigelb> QAing this will be fun.
[12:27] <jtv> StevenK: I reviewed yours.  Will you review mine?
[12:27] <wgrant> StevenK: Looks like Jenkins will succeed in 10 minutes.
[12:28] <wgrant> With a little luck buildbot will follow.
[12:41] <cjwatson> RT#47040 fixed - does that mean I can have another go at landing multiarch-translations?
[12:41] <_mup_> Bug #47040: while booting initial kernel, progress bar looks like Win98 <gfxboot-theme-ubuntu (Ubuntu):Confirmed> < https://launchpad.net/bugs/47040 >
[12:41] <cjwatson> or do I still need to wait for bug 809123 to actually go fix-released?
[12:41] <_mup_> Bug #809123: we cannot deploy DB schema changes live <fastdowntime> <qa-ok> <Launchpad itself:Fix Committed by stub> < https://launchpad.net/bugs/809123 >
[12:42] <wgrant> As of a few hours ago we are, roughly, fastdowntime-capable. But we haven't actually done one yet.
[12:43] <cjwatson> I suppose that multiarch-translations-schema (landed on db-devel a while back) is the thing that needs to be fastdowntimed
[12:43] <wgrant> There is an as-yet undeployed Storm reconnection fix that we probably want to wait for, but if all goes to plan (as it hasn't for the last week) we'll be deploying that tomorrow.
[12:44] <wgrant> Right. Let's see what the fastdowntime queue looks like.
[12:44] <wgrant> I guess we could do more frequent episodes to clear it out.
[12:44] <wgrant> But the first run will hopefully be on Wednesday, and it will be a no-op.
[12:45] <wgrant> stub: Anything to wait for but the Storm fix?
[12:45] <cjwatson> just bounce all the connections to see if anything falls over?
[12:45] <wgrant> Basically.
[12:45] <wgrant> The full update script asks the DB cluster to sync, kills all connections, works out patches to apply, applies any patches, reconfigures security, syncs the cluster again, turns connections back on.
[12:46] <wgrant> All the clients just see no DB for a while. Which could have amusing effects.
[12:55] <jtv> G: I reviewed one of yours in hopes that StevenK would perform the other half of the deal I proposed, but he seems to have bugged out.
[12:57] <G> jtv: hold on, let me load it up on LP itself
[12:58] <G> jtv: I'm not too sure on transaction.commit() I'll test it now
[13:04] <stub> wgrant: Nothing required but the storm fix. The branch-revision fix would be nice, but that is an understood problem we can work around by bouncing that service immediately after the update.
[13:05] <stub> Same for the storm fix really, but I'd like to minimize the amount of bouncing that needs to happen.
[13:05] <wgrant> stub: We should be able to deploy all the way tomorrow, unless buildbot still wants me dead.
[13:05] <stub> Yes, agreed.
[13:06] <stub> mthaddon: Can we do updates tomorrow?
[13:06] <LPCIBot> Yippie, build fixed!
[13:06] <LPCIBot> Project devel build #1,035: FIXED in 4 hr 36 min: https://lpci.wedontsleep.org/job/devel/1035/
[13:06] <wgrant> !!!!
[13:06]  * stub looks into which particular db patch cjwatson needs
[13:06] <mthaddon> stub: what kind of updates?
[13:06] <stub> mthaddon: fastdowntime db update
[13:06] <mthaddon> stub: have we done a test run?
[13:07] <stub> mthaddon: This is the test run. Staging has been doing it for a while.
[13:07] <mthaddon> stub: okey doke
[13:07] <wgrant> We're starting with a no-op, right? :)
[13:07] <cjwatson> stub: https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations-schema
[13:08] <stub> wgrant: I can argue it either way, but will go with lifeless' suggestion of a no-op first up.
[13:09] <wgrant> stub: We've not had wondrous success syncing the cluster recently, so it is tempting to eliminate unnecessary risk.
[13:09] <wgrant> Even though we're not dropping a slave, so it should all be fine.
[13:09] <wgrant> I still don't trust SSO to not end the world.
[13:09] <wgrant> Should we provisionally announce the 0800 window for tomorrow?
[13:09] <stub> cjwatson: I'll make sure getting that patch up is a priority, even if we can't get all the patches applied
[13:09] <G> oh is LP.net getting updated
[13:09] <cjwatson> great, thanks very much
[13:09] <wgrant> G: We normally update launchpad.net roughly daily.
[13:10] <wgrant> G: But database patches were previously applied only monthly, as they required 60-90 minutes of downtime.
[13:10] <wgrant> G: But the new process allows us to it in hopefully less than a minute.
[13:10] <stub> Announcements are not necessary for 5 minute outages apparently, so we just need to announce to identi.ca for the window.
[13:10] <wgrant> The preparations are finally complete.
[13:10] <G> wgrant: ahhh right
[13:11] <wgrant> stub: Once we have them down below a couple of minutes, a couple of hours is probably enough. But we might want to give a bit more this time, as it's the first and not exactly unrisky.
[13:11] <stub> We can make a blog post if we want I guess since it is the trial by fire of the new process.
[13:11] <wgrant> Hmm, no mrevell.
[13:11] <wgrant> identi.ca two hours before, then :)
[13:12] <wgrant> Heh. Only 1.5 months after the blog post announcing the start of fastdowntime.
[13:12] <stub> Not really much risk apart from OOPSes and more extended outages for the bits that need to be manually bounced.
[13:12] <wgrant> stub: So we say until the cluster falls apart.
[13:12] <wgrant> And takes an hour to recover.
[13:13] <stub> wgrant: No risk of that with a no-op update
[13:13] <wgrant> Hahaha.
[13:13] <wgrant> Although I guess the last few bad rollouts have been because we didn't want to abort.
[13:13] <wgrant> Because the window was huge and expensive.
[13:13] <wgrant> If stuff breaks with fastdowntime we can just abort.
[13:14] <stub> yup. everything that has caused failures in the past is checked for up front and the process doesn't kick off.
[13:14] <wgrant> And even if it does break we don't have to recover into a deployable state immediately.
[13:14] <wgrant> But hopefully SSO won't desync any more.
[13:16] <stub> And even if there is some internal slony failure, we can switch to master-only fairly quickly with pgbouncer. We have covered everything we can, and at some point we need to push the button.
[13:16] <wgrant> Yup.
[13:16] <wgrant> OK, where is the fastdowntime process page these days? The LEP is unhelpful.
[13:17] <stub> mthaddon: I'm going to need a fresh db-stable tree on wildcherry
[13:18] <mthaddon> stub: fresh means which revno?
[13:18] <stub> mthaddon: Not sure if we thought about that with your tree pushing scripts
[13:18] <mthaddon> stub: we pull from wildcherry, but yep, we have thought of it :)
[13:18] <stub> mthaddon: current head of launchpad/db-stable
[13:18] <mthaddon> er, db-stable?
[13:18] <stub> mthaddon: I will need to trim out unwanted patches for the no-op run
[13:18] <wgrant> stub: Can't we just merge the new scripts to devel?
[13:19] <mthaddon> yeah, my thoughts exactly
[13:19] <wgrant> Most of that should come across harmlessly.
[13:19] <stub> If we want to test something other than the scripts we have been testing...
[13:19] <wgrant> I was actually looking at that this morning, when I cherrypicked some other stuff across.
[13:19] <wgrant> stub: Hm? Can't we just merge all non-DB-patch-depending changes into devel?
[13:20] <stub> Yes, and hope we get it right and we didn't miss any dependencies
[13:20] <wgrant> True.
[13:20] <stub> Or we can just use db-stable
[13:20] <stub> I just noticed as I was going to check what db patches need to be pruned for the no-op run and if we need to do multiple real runs or a single 'real' update.
[13:20] <wgrant> Hm. Did we get timings from staging yesterday?
[13:20] <wgrant> I'm guessing not.
[13:21] <wgrant> Because it likes to break just a day or two before the restore.
[13:21] <wgrant> It broke again.
[13:22] <stub> I think we have all the timings we need from all the previous runs.
[13:22] <wgrant> Lots of permission denied (publickey) :(
[13:22] <stub> 2011-09-02 14:44:00 INFO    2208-78-1 applied 2011-08-28 in 9.5 seconds
[13:22] <stub> 2011-09-02 14:44:00 INFO    2208-79-0 applied 2011-08-28 in 1.3 seconds
[13:22] <stub> 2011-09-02 14:44:00 INFO    2208-79-1 applied 2011-08-28 in 0.5 seconds
[13:22] <stub> 2011-09-02 14:44:00 INFO    2208-80-1 applied 2011-08-28 in 0.1 seconds
[13:22] <stub> 2011-09-02 14:44:00 INFO    2208-81-1 applied 2011-08-28 in 0.0 seconds
[13:23] <wgrant> Ah, it was hwdb stuff breaking the update. The SSH errors were unrelated, apparently :/
[13:24] <wgrant> stub: Do a no-op run tomorrow, and if all goes well do another one also tomorrow with everything?
[13:25] <stub> yup, or even later in the same day or same 5 minute window ;)
[13:25] <stub> I guess same 5 minute window is a bit ambitious given we need to confirm everything survived :)
[13:26] <wgrant> Right, I meant later in the same day, as in the same 5 minute window is a little adventurous.
[13:26] <stub> But the db patches I'm looking at all look really, really boring.
[13:26] <wgrant> They are pretty bad, yeah.
[13:26] <wgrant> A good start.
[13:26] <wgrant> Apart from the 10s one.
[13:26] <stub> Its still boring.
[13:26] <wgrant> It is.
[13:26] <wgrant> But slow boring.
[13:26] <stub> 10s is not slow. We have had patches that took hours in the past ;)
[13:27] <wgrant> Oh yes, I recall.
[13:27] <wgrant> The good old days of 12-hour downtime to open a new Ubuntu release, and then the final 10-hour downtime to migrate to the new translations schema.
[13:30] <stub> yer, and get that down to 10s and people still are not happy. I wonder why I bother sometimes :-D
[13:31] <stub> Looks like just those 5 patches.
[13:31] <wgrant> stub: What about 76-3 and 76-4?
[13:31] <wgrant> Oh. Those are bugsummaryrollupbreakage, aren't they.
[13:31] <stub> They are both live
[13:31] <wgrant> How could I forget.
[13:32] <stub> http://paste.ubuntu.com/682567/ will have to do since there is no report yet
[13:32] <wgrant> Indeed, just those 5.
[13:33] <stub> And upgrade.py --dry-run if I'm feeling hungover tomorrow.
[13:33] <wgrant> Do you happen to feel like checking BugMessage.owner's NULLness, just in case?
[13:33] <wgrant> That's the main reason i wanted the restore.
[13:34] <stub> Still 0
[13:34] <wgrant> The appservers aren't going to choke on 79-0? I can't remember if that restriction was relaxed.
[13:35] <stub> I can't see why the appservers would complain.
[13:35] <wgrant> A month ago they would refuse to start if there was a -0 patch applied but not in the tree.
[13:35] <stub> That code has been turned off completely.
[13:36] <stub> Guess we should check that revision is live
[13:36] <wgrant> It seems to have been deleted.
[13:36] <wgrant> I can't find it anywhere nor remember the rev it was deleted in, so it's deployed everywhere.
[13:36] <wgrant> revno: 13638 [merge]
[13:36] <wgrant> timestamp: Tue 2011-08-09 15:28:37 +0000
[13:37] <wgrant>   [r=gmb][bug=809636,
[13:37] <wgrant>         815706] Remove runtime checks of LaunchpadDatabaseRevision
[13:37] <stub> ta
[13:37] <wgrant> librarian1 would have been the last holdout.
[13:38] <wgrant> So we're clean now.
[13:38] <stub> only the appservers checked anyway
[13:38] <wgrant> Yeah.
[13:39] <G> henninge: as OCR, can you at some point take a look at https://code.launchpad.net/~dev-nigelj/launchpad/bug-674854/+merge/74064 please
[13:39] <henninge> G: sure but otp right now
[13:41] <StevenK> OMG, devel built on Jenkins
[13:41] <wgrant> StevenK: Yup. jelmer fixed the four main failures on db-devel a week ago. I cherrypicked it into devel tonight, and disabled the other failing tests, which cascaded into everything else.
[13:41] <wgrant> So we should be green for a while now.
[13:42] <stub> Long enough to take buildbot out the back and shoot it?
[13:42] <wgrant> The new bzr seems to have made everything more reliable.
[13:43] <bigjools> sigh, is the poppy bug STILL not fixed? :/
[13:43] <wgrant> The dozens of varied failures over the last week or two can all be explained by fallout from the three tests.
[13:43] <StevenK> stub: Is that a +1 for Jenkins from you, then? :-)
[13:43] <wgrant> bigjools: Nope.
[13:43] <wgrant> :)
[13:43] <wgrant> See /topc.
[13:43] <wgrant>  /topic
[13:43]  * bigjools hunts in vain
[13:44] <stub> StevenK: I'm for whatever has the least maintenance burden, and whatever I don't have to be involved with.
[13:44] <StevenK> Heh
[13:44] <jml> probably jenkins
[13:44] <StevenK> Death to buildbot and buildbot-poller
[13:45] <G> must say, as an observer, jenkins looks kinda neat
[13:45] <stub> I would like a bot I can tell 'Go test lp:~stub/launchpad/foo and give me the results', but we need the basics first.
[13:45] <StevenK> stub: bin/ec2 test? :-)
[13:45] <jml> that's easy to do w/ jenkins, but breaks polling ime
[13:45] <StevenK> jml: Do it as a seperate job
[13:46] <StevenK> But yes, fairly easy
[13:46] <wgrant> Ideally we do that with Jenkins in front of a couple of machines that run a couple of 4-way test suites or so.
[13:46] <stub> StevenK: I hate the 5 minutes it takes and the risk of leaving hung instances chewing through cash.
[13:47] <stub> StevenK: I also hate having too many environments that tests have to pass on. We currently need everything to pass on 'local dev environment', ec2 image, buildbot image, staging environment, production environment. With the wonders of modern VM tech, there is no reason we can't get that down to one environment that is identical everywhere.
[13:48] <StevenK> stub: +1,000,000,000
[13:52] <G> yeah, I was kind of surprised when I kept seeing EC2 been mentioned
[13:52] <StevenK> G: Whyfor?
[13:54] <stub> Because while the test suite is solid, there are so many dependencies involved the environment ends up fragile. Time consuming to put together an alternative that is as automated and reliable.
[13:54] <G> StevenK: great for testing, but it was a "I thought they'd be using an inhouse cloud or something using UEC"
[13:54] <soren> I'm curious.. How many instances are spawned to run the tests?
[13:54] <G> StevenK: put it this was, I was surprised, but understood
[13:55] <StevenK> soren: One per branch
[13:55] <soren> StevenK: Oh, just one?
[13:55] <soren> Hm.
[13:55] <soren> Ok.
[13:55] <StevenK> soren: Yup.
[13:55] <soren> How long does a test run take?
[13:55] <stub> People are working on parallel testing, but it is still science fixtion
[13:55] <StevenK> soren: 4 and a bit hours.
[13:56] <soren> Ok.
[13:56] <wgrant> stub: LXC-based parallel testing is pretty much doable now.
[13:56] <wgrant> It has been done.
[13:56] <StevenK> soren: It's not okay, it's awful. :-P
[13:56] <wgrant> It's not completely reliable yet.
[13:56] <stub> Yer, just the second 95% to do now.
[13:57] <soren> StevenK: Well, yes :)
[13:57] <G> hmmm so 4 & a bit hours at $0.68/hr?
[13:58] <stub> Yer, the coffee you drink while waiting costs more.
[13:59] <G> I guess it could be much worse
[14:00] <stub> Being able to rent out servers by the hour for < $1 is pretty awesome.
[14:00] <stub> Someone is going to notice this Cloud thing one day.
[14:01] <G> stub: you mean they haven't?
[14:02] <stub> Dunno. I stopped watching TV when I moved into exile.
[14:02] <StevenK> wgrant: Are you happy for me to toss destroy-openidrp at ec2?
[14:03] <wgrant> StevenK: Let me glance over it first.
[14:04] <wgrant> StevenK: Hm, you now always warn?
[14:05] <wgrant> I guess that's OK.
[14:05] <StevenK> It's still dangerous
[14:06] <wgrant> StevenK: To ec2 with it!
[14:07] <StevenK> wgrant: Happy with no-qa, or shall I file a bug just so we can try it out?
[14:07] <wgrant> Bug would be nice.
[14:08] <bac> n
[14:10] <StevenK> wgrant: You and I name too many branches beginning with 'destroy-'
[14:12] <jml> s/too many/not enough/
[14:12] <jml> functionality is an asset. code is a liability.
[14:13] <nigelb> StevenK: Start using nuke- then ;)
[14:13] <StevenK> icbm-
[14:14] <nigelb> +1
[14:14] <G> diediedie-
[14:14] <nigelb> G wins.
[14:14] <G> oh actually, why not go all James Bond... 007- :)
[14:14] <StevenK> I think both wgrant and I have used 'diediedie' in a branch name before
[14:14] <nigelb> lol
[14:14] <G> or from the movie... RED- (Retired & Extremely Dangerous) :)
[14:17] <G> tbh, I like 007- better
[14:17] <nigelb> I'll go with diediedie
[14:18] <StevenK> I don't like the idea of 007-. If he likes the look of the code, he'll seduce it and sleep with it
[14:19] <G> StevenK: in that case, there is far bigger issues to worry about than the bug you are trying to fix :)
[14:20] <bigjools> StevenK: the code is already fucked though
[14:21] <nigelb> bigjools++
[14:56] <wgrant> jelmer: Hi. Can you please QA the bzr upgrade? It's getting reasonably urgent.
[14:57] <wgrant> aaaergwejfwf
[14:57] <wgrant> test_import_bzr failed this time.
[14:57]  * wgrant kills them all.
[14:58] <jelmer> wgrant: I've QA'd it as much as I can so happy to mark it qa-ok
[14:58] <wgrant> jelmer: Great, thanks.
[15:00] <jelmer> wgrant: that's test_import_bzr for the workermonitor?
[15:01] <wgrant> jelmer: Yes.
[15:01] <wgrant> jelmer: I've already disabled the three that you reenabled a couple of weeks back.
[15:01] <wgrant> But now test_import_bzr is failing the same way.
[15:01] <wgrant> I think script startup is just too slow now.
[15:01] <wgrant> So it hits the 20s timeout.
[15:01] <wgrant> Bug #841556
[15:01] <_mup_> Bug #841556: test_import_bzr, test_import_cvs, test_import_bzrsvn and test_import_subversion disabled <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/841556 >
[15:04]  * wgrant sleeps.
[15:04] <jelmer> blergh
[15:04] <jelmer> wgrant: thanks for handling that
[15:09] <henninge> G: reviewed. Sorry for taking so long. Please have a look.
[15:12] <G> henninge: yeah, I couldn't find any tests that related to the distroarchseries webservice
[15:13] <G> henninge: oh, so you are saying I should replace the whole setUp w/ a _makeDistroandArch() type thing?
[15:14] <henninge> G: yup
[15:14] <G> in lib/tests/factory (or the correct path) or in the new file?
[15:14] <henninge> G: Much more obvious when you read it
[15:14] <henninge> G: no, directly in your test case.
[15:14] <G> oh right
[15:15] <G> I'll be back in a few then :)
[15:15] <G> henninge: everything else good okay though?
[15:16] <henninge> yes, looks fine
[15:17] <G> henninge: that said, as multiple tests can benefit from the one-time setup for the data, isn't that what setUp is for?
[15:17] <G> (just curious)
[15:19] <G> (i.e. if I was to do an authenticated test as well, surely the one time setup, instead of multiple setups, would be faster) or am I on the wrong thought track?
[15:20] <henninge> G: No, it is not faster
[15:20] <henninge> G: setUp is executed before each test.
[15:20] <G> oh it is, okay then
[15:33] <G> Okay, giving this a whirl
[15:36] <nigelb> G: Don't you ever sleep? :)
[15:36] <nigelb> I see you *all* the time :D
[15:36] <nigelb> g32
[15:36] <G> nigelb: I have a weird sleep pattern
[15:36] <bigjools> I might also say that about nigelb!
[15:37] <nigelb> bigjools: :D
[15:37] <nigelb> Kettle, pot, black. I know!
[15:37] <nigelb> I need to pull an all-nighter. I'll throw that frustration away by writing some patches.
[15:37] <G> nigelb: I do some of my best thinking late at night
[15:37] <nigelb> Someone needs to write "The therapeutic effects of coding"
[15:38] <G> (not just coding, but thinking in general)
[15:38] <nigelb> G: Join the club.
[15:38] <nigelb> I sleep at about 2 am local time.
[15:38] <G> Tue Sep  6 03:38:30 NZST 2011
[15:38] <G> although, most python applications would have be believe it's currently +1300 for some reason :)
[15:39] <nigelb> Is your tzdata package up-to-date?
[15:39] <G> yep
[15:40] <G> but I don't know when the last time NZ has been +1300 this early, ever
[15:42] <nigelb> Try using pytz to figure out the date. That should work correctly.
[15:43] <nigelb> If it doesn't, you can always crib to its author right here ;)
[15:43] <nigelb> s/date/date\/time/g
[15:45] <nigelb> PyCharm people just need to hire bigjools as their community manager :P
[15:45] <bigjools> they need to write it in Python first :)
[15:45] <nigelb> PyCharm isn't Python/
[15:45] <nigelb> ?
[15:45] <bigjools> Java
[15:45]  * bigjools throws up a little
[15:45] <nigelb> Ewww.
[15:46] <G> a Python IDE written in Java, heh
[15:46] <bigjools> and it uses Swing
[15:46]  * G vomits a little
[15:46] <bigjools> but I forgive it so far
[15:46] <nigelb> Is it as slow as Eclipse?
[15:46] <bigjools> pretty snappy for the most part
[15:46] <bigjools> but then I have 4 cores
[15:51] <nigelb> jml: ping
[15:51] <jml> nigelb: yes?
[15:51] <nigelb> jml: Could you join #ubuntu-classroom, #ubuntu-classroom-backstage, and #ubuntu-classroom-chat?
[15:51] <jml> nigelb: done
[15:51] <nigelb> The bot's complaining that you aren't around :)
[15:56] <G> henninge: I think I might want to take another look at this again tomorrow
[16:05] <G> I think my anonymous test, may not be logging into the API so anonymously afterall
[16:08] <nigelb> I'm liking jono's session :)
[16:09] <henninge> G: np at all, get some sleep first ... ;)
[16:09] <henninge> How do I update sourcedeps.cache?
[16:10] <jelmer> henninge: run ./utilities/update-sourcecode
[16:10] <henninge> jeblair: next question: when would I do that?
[16:10]  * henninge never had to do it.
[16:10] <henninge> jelmer: ^
[16:10] <henninge> sorry jeblair
[16:11] <nigelb> I think either rf-branch or make run already did that
[16:11] <jelmer> henninge: After you change utilities/sourcedeps.conf
[16:11] <henninge> aha
[16:11] <jelmer> henninge: (or after a merge changes it)
[16:11] <henninge> ok
[16:11] <henninge> jelmer: thanks
[16:12] <jelmer> henninge: it would indeed be neat to add some deterministic ordering to it to prevent spurious conflicts and the like
[16:12] <henninge> jelmer: I already found the place where to do that, I just wanted to find out how to try/test it.
[16:16] <jelmer> henninge: cool
[16:19] <G> henninge: actually, I worked out the problem, should be an updated branch coming to you in a moment :)
[16:19] <henninge> G: cool
[16:19] <G> two last bin/test's then I might be able to sleep easier :)
[16:21] <G> okay, it works :)
[16:29] <G> henninge: okay, I pushed the updated fix to my branch, if it's okay please feel free to land it, if there are other issues, I'll tackle them tomorrow :)
[16:30] <henninge> G: I'll have a look in a minute
[16:30] <G> yep, no hurry, I'm off to bed :)
[20:25] <nigelb> sigh, buildbot
[22:34]  * wgrant crushes buildbot into the ground.
[22:52] <wallyworld_> wgrant: i forced a buildbot build but the compile failed - looks like a simple timeout. you seen this sort of thing before?
[22:54] <wgrant> Yes, we are fucked.
[22:54] <wgrant> I'm trying to track down when this failure first showed up.
[22:54] <wgrant> But buildbot doesn't actually keep logs for more than a week or so.
[22:54] <wgrant> Although I guess they should be in emails...
[22:55] <wgrant> Or not.
[22:55] <wgrant> It just emails links.
[22:55] <wgrant> Sigh.
[22:57] <wallyworld_> wgrant: so, it's not an actual compile error right?
[22:57] <wgrant> There is something real there.
[22:57] <wgrant> Oh.
[22:57] <wgrant> That last failure is not real, no. That was hloeung killing the slave.
[22:57] <wgrant> Thought you were talking about the previous ne.
[22:58] <wgrant> But you were hopefully too asleep to force that.
[22:58] <wallyworld_> yes, just saw that one issue when i checked earlier and thought i'd try a force
[22:58] <wallyworld_> since it appeared to be a timeout
[22:59] <wgrant> Yeah, we've forced that failure about 20 times in the last two weeks.
[22:59] <wallyworld_> hopefully one of the maintenace squads is looking
[22:59] <wgrant> lol
[22:59] <wgrant> US and Canada were away yesterday.
[22:59] <wallyworld_> ah, labour day
[23:00] <wallyworld_> wgrant: you tried to install nautilus elementary in oneiric? i really need to be able to customise the toolbars
[23:00] <wallyworld_> natty packages don't work :-(
[23:04] <wgrant> I've not. I hardly use nautilus, but when I do the new one is nice and clean.
[23:05] <wallyworld_> clean yes, but you can't do simple things like add a "up to Parent" button to the toolbar. very annoying when you need to navigate up one directory
[23:06] <wgrant> Ah.
[23:10] <wallyworld_> nautilus elementary also has a bunch of other useful features. it really should be what we ship imho
[23:10] <jelmer> has anybody seen this error on oneiric: "psycopg2.OperationalError: fe_sendauth: no password supplied" ?
[23:14] <wgrant> jelmer: Running the test suite locally?
[23:14] <jelmer> wgrant: yeah - it worked fine a couple of days ago
[23:14] <wgrant> jelmer: We connect over TCP now (as of the weekend, then it was reverted, and now as of a couple of hours ago)... perhaps your postgres config doesn't permit ident/hba over TCP?
[23:15] <jelmer> wgrant: that might be it, I'll have a look at that
[23:15] <wgrant> host    all         all         127.0.0.1/32      trust
[23:15] <wgrant> That line should be in your pg_hba.conf
[23:15] <wgrant> Odd that it's not /8
[23:15] <wgrant> Perhaps your localhost resolves to something else in 127/8 instead?
[23:16] <jelmer> maybe it's ipv6
[23:17] <wgrant> Ah, or that.
[23:17] <wgrant> But that should be ip6-localhost, normally.
[23:17] <wgrant> Most of my home network is IPv6, and it all still works for me.
[23:20] <jelmer> I can't find it right now and it's getting quite late, I'll have a look tomorrow.
[23:21] <wgrant> Night!
[23:30] <nigelb> :\
[23:30] <nigelb> Are the critical bugs breeding :(
[23:31] <wgrant> Yup!
[23:31] <wgrant> wallyworld_: Are you running LP on oneiric still, or back in LXC?
[23:31] <wallyworld_> wgrant: still on oneiric
[23:32] <wgrant> wallyworld_: You've just been dealing with JS, then?
[23:32] <wgrant> Since you haven't wanted my rabbitfixture fix.
[23:33] <wallyworld_> wgrant: yes. but i'm in the middle of a python branch so that will change
[23:33] <wgrant> wallyworld_: See the list when you run into RabbitMQLayer startup trouble. I have a properish fix now.
[23:33] <wallyworld_> wgrant: awesome. thanks
[23:34] <wgrant> pygtk seems somewhat broken on oneiric, so the YUI testrunner doesn't work.
[23:34] <wgrant> But there are only 8 other oneiric failures.
[23:41] <poolie> wgrant, if you have a chance can you read my dkim patch?
[23:41] <poolie> i see one nit in it at least ,though
[23:44] <wallyworld_> wgrant: i can run yui tests ok
[23:48] <wgrant> wallyworld_: Odd. Maybe you upgraded so things work better?
[23:48] <wallyworld_> wgrant: i did upgrade from natty but it hosed my system. so i installed oneiric from scratch
[23:49] <wallyworld_> just to clarify, bin/test --layer=YUITest works, as does running individual test cases from the browser by loading the html page. what error do you get?
[23:50] <wgrant> wallyworld_: No module named gtk. Then I installed python-gtk2, and now it says it can't initialise Gtk or something.
[23:50] <wgrant>   File "/usr/lib/python2.6/dist-packages/gi/overrides/Gtk.py", line 1406, in <module>
[23:50] <wgrant>     raise RuntimeError("Gtk couldn't be initialized")
[23:50] <wallyworld_> hmmm. i never saw anything like that
[23:50] <wgrant> RuntimeError: Gtk couldn't be initialized
[23:51] <wallyworld_> i don't think i installed everything gtk though
[23:51] <wgrant> There's probably some dep I'm missing, since this is in a minimal LXC, while you're presumably running with a full desktop environment.
[23:52] <wgrant> Ah, umask changes are breaking some of the other tests :(
[23:52] <wgrant> 002 ftl :(