[00:05] <lifeless> spm: hi
[00:06] <lifeless> spm: I'm sure you have a bunch of firefighting to do [ubunet cron, *cough*]
[00:06] <lifeless> spm: but I wonder, as its 1am here, if you can do me a solid first
[00:06] <spm> lifeless: yo; sup?
[00:06] <lifeless> spm: which is, to deploy lp:~lifeless/launchpad/malone so I can see if it does make dup checking faster
[00:06] <lifeless> on staging
[00:06] <spm> oh sure, app server I assume?
[00:07] <lifeless> yeah
[00:10] <spm> yoikes that's a big patch. it seemed to have a merge fail; but it scrolled out of my buffer. trying again to a file
[00:10] <lifeless> spm: it should be tiny
[00:10] <lifeless> spm: https://code.edge.launchpad.net/~lifeless/launchpad/malone/+merge/30904,
[00:10] <spm> bzr merge --preview  lp:~lifeless/launchpad/malone ?
[00:11] <spm> just take the latest version then of it?
[00:11] <lifeless> spm: https://code.edge.launchpad.net/~lifeless/launchpad/malone/+merge/30904/+preview-diff/+files/preview.diff perhaps
[00:11] <spm> that works
[00:11] <lifeless> spm: its probably other devel stuff naffing staging up, or something
[00:11] <spm> nod
[00:12] <spm> damn. 4/4 hunks failed
[00:12] <lifeless> which ones
[00:12] <lifeless> oh, all
[00:12] <spm> :-)
[00:12] <lifeless> erm
[00:12] <spm> 1am, you said earlier. noted. :-)
[00:12] <lifeless> on which files
[00:12] <lifeless> there is much more than 4 hunks in there
[00:13] <spm> http://paste.ubuntu.com/469077/
[00:13] <spm> yeah - that was just the most obvious
[00:13] <lifeless> spm: can you revert; then pastebin bzr missing lp:~lifeless/launchpad/malone ?
[00:14] <spm> preview - i always preview/dry-run first
[00:14] <spm> sure sure
[00:15] <spm> lifeless: http://paste.ubuntu.com/469078/
[00:16] <lifeless> odd
[00:16] <lifeless> 11189 should be included
[00:16] <lifeless> can you grep slow_ lib/canonical/database/nl_search.py ?
[00:16] <spm> 0 results found
[00:17] <lifeless> ok
[00:17] <lifeless> merge -c 11199 lp:~lifeless/launchpad/malone ;
[00:18] <lifeless> merge --force -c -1 lp:~lifeless/launchpad/malone
[00:18] <lifeless> should behave better
[00:21] <spm> oki
[00:21] <lifeless> its in place ?
[00:21] <spm> is that bzr, svn or git merge?
[00:21] <spm> not yet
[00:21] <lifeless> bzr
[00:21] <spm> 1am. trolling is too easy. sigh.
[00:21] <lifeless> 121
[00:22] <spm> qed
[00:22] <spm> 1st went in easy
[00:22] <lifeless> vidi vici veni
[00:22] <spm> on a preview; live doing now...
[00:23] <spm> Civil Engingeering Students Association (@ Qld Uni) CESA: We Came. We Swore. We Concreted.
[00:24] <spm> coolio, both applied; restarting
[00:25] <spm> lifeless: restarted with new shiny ponies
[00:26] <lifeless> hmm, you sure ?
[00:27] <spm> yup :-)
[00:28] <spm> this is the app server right? not code*?
[00:28] <spm> start time: Mon Jul 26 00:24:52 2010 (BST natch)
[00:30] <lifeless> booyah
[00:30] <lifeless> wgrant: try this page https://bugs.staging.launchpad.net/ubuntu/+filebug
[00:30] <lifeless> with e.g. eclipse get unmet dependency error
[00:31] <lifeless> its hitting disk a lot still
[00:31] <lifeless> but once off disk its hell snappy for me
[00:32] <lifeless> mwhudson: ^
[00:32] <lifeless> spm: what do you think
[00:33] <wgrant> Hm, not too bad.
[00:33] <spm> whimper. can't login to staging I think...
[00:33] <wgrant> But the results... I don't know.
[00:33] <spm> No I lie. I'm in. wooo!
[00:33] <lifeless> wgrant: they are missing the prefilter step that was doing millions of comparisons to filter out terms; will reinstate that in a bit
[00:34] <lifeless> wgrant: other than that its ranked precisely the same way
[00:34] <lifeless> [terribly]
[00:34] <spm> seemed snappy to me
[00:34] <lifeless> k
[00:34] <lifeless> bombs away, I say
[00:34] <wgrant> lifeless: Isn't it returning a subset of the original results, though?
[00:35] <lifeless> yes
[00:35] <lifeless> but ranking is strictly additive given the same terms as the rank function
[00:36] <lifeless> so 3 terms >> 1 term anyhow
[00:36] <lifeless> and we're not [yet] using heat or anything
[00:37] <lifeless> night
[00:38] <wgrant> Right.
[00:38] <wgrant> I guess you try the one-missing query, and then jump to two-missing, three-missing etc. queries if there aren't enough results.
[00:38] <wgrant> s/you/you could/
[00:52] <wgrant> spm: Some more corrupt builds appear to have sprung up :(
[00:52] <wgrant> If you have a moment, could you run 'SELECT buildqueue.id, builder, lastscore, buildqueue.job, job_type, processor, build FROM buildqueue JOIN job ON job.id = buildqueue.job JOIN buildpackagejob ON buildpackagejob.job = job.id WHERE buildqueue.virtualized = false AND buildqueue.processor = 1 AND job.status = 0' for me?
[00:53] <wgrant> There are somehow 13 corrupt builds in the i386 non-virt queue, for a total of 40 seconds of buildd time, and they don't appear to actually be in the queue.
[00:54] <spm> *blink*. 0 rows?
[00:54] <wgrant> Bah.
[00:55] <wgrant> SELECT buildqueue.id, builder, lastscore, buildqueue.job, job_type, processor FROM buildqueue WHERE buildqueue.virtualized = false AND buildqueue.processor = 1;
[00:55] <wgrant> They must be even more broken than the last lot.
[00:56] <spm> we don't break by halves round here
[00:56] <spm>    id    | builder | lastscore |   job   | job_type | processor
[00:56] <spm> ---------+---------+-----------+---------+----------+-----------
[00:56] <spm>  3233211 |         |     12510 | 1941857 |        1 |         1
[00:56] <spm> (1 row)
[00:56] <spm> wgrant: ^^ that last score seems... curious
[00:56] <wgrant> Just means it's in a private archive.
[00:56] <spm> is that like the 'score' we'd override a build? ahh kk.
[00:56] <wgrant> So there's something odd going on here, and that's not it.
[00:58] <wgrant> (https://edge.launchpad.net/builders -- see the i386 non-virt build queue. that query should have returned everything in it. :()
[00:59] <spm> looks
[00:59] <wgrant> Hmm. What if you say 'virtualised IS NULL' instead of 'virtualized = false'?
[00:59] <spm> non-virt the offical builders yes/no?
[00:59] <wgrant> Official, yes.
[00:59] <spm> non-virt *is* the offical builders yes/no? yes, ta
[01:00] <spm> 13 rows :-)
[01:00] <spm> jobtype=4, processor=1, lastscore=1000
[01:01] <spm> so curious that the = false gets something at all.
[01:01] <spm> recognising that sql has 3 binary states - on, off, null :-/
[01:02] <wgrant> It is somewhat odd, since it's not appearing in the queue, and is not building. But that's not the main issue.
[01:02] <wgrant> Thanks for your assistance.
[01:03] <spm> np
[01:04] <wgrant> Odd that they're not being dispatched. But they're translations jobs, so who knows what evil they are up to.
[01:06] <spm> I know people who would know what evil they're up to.....
[01:10] <wgrant> There are another 18 properly virtual jobs floating around inappropriately, but it's just about impossible to track them down until the queue is empty.
[01:10] <wgrant> Filed bug #609904, but that doesn't explain why the builds exist in the first place.
[01:10] <_mup_> Bug #609904: BuilderSet.getBuildQueueSizes needs to consider virtualized=NULL as false <Soyuz:New> <https://launchpad.net/bugs/609904>
[01:12] <wgrant> jtv: Hi. There are 13 pending translation template build farm jobs, and they're not being dispatched. Do you know why that would be?
[01:27] <jtv> wgrant: hi.  No idea...  it's only TTBJs?
[01:30] <wgrant> jtv: There are some other non-dispatching builds around. But they're corrupt -- I'm not sure if the TTBJs are.
[01:31] <jtv> wgrant: corrupt in what way?
[01:31] <wgrant> jtv: The status of the PackageBuild is wrong.
[01:32] <wgrant> But you only have a single status, so that's not the case.
[01:32] <jtv> wgrant: it's not one of the recent changes and us not having a Build?
[01:32] <jtv> Maybe the bad build status is a result of the same thing that's causing these jobs not to be dispatched?
[01:32] <wgrant> jtv: Unlikely. I saw some TTBJs building not even 24 hours ago, so it's not a general problem.
[01:32] <wgrant> I wonder if the branchjob is missing or something.
[01:33] <jtv> A branchjob and a job are all we have, no?
[01:33] <wgrant> And a buildqueue.
[01:33] <jtv> Ah right
[01:33] <jtv> that too
[01:33] <wgrant> The buildqueue and job are present.
[01:33] <wgrant> Not sure about the branchjob.
[01:33] <wgrant> I don't trust the recent branch deletion changes, so I'll blame them for now :)
[01:33] <jtv> Deleting a branch might cause this...
[01:34] <jtv> The recent branch changes would cause both the BJ and the BQ to be cleaned up.
[01:34] <jtv> How old are these broken jobs?
[01:36] <jtv> The old behaviour (still running on production afaik) would cause an incomplete cleanup.
[01:36] <wgrant> SELECT buildqueue.id, builder, lastscore, buildqueue.job, buildqueue.job_type, processor, job.date_created, job.status, branch, branchjob.job_type, json_data FROM buildqueue JOIN job ON job.id = buildqueue.job LEFT JOIN branchjob ON branchjob.job = job.id WHERE buildqueue.virtualized IS NULL AND buildqueue.processor = 1;
[01:36] <wgrant> spm: ^^
[01:36] <spm> yup
[01:37] <jtv> wgrant: no rows
[01:37] <spm> hrm that json data for those is interesting; all over the place.
[01:37] <wgrant> Those two statements conflict....
[01:37] <jtv> I ran mine on staging.
[01:37] <wgrant> Ah.
[01:38] <wgrant> staging has buildqueue cleared.
[01:38] <wgrant> Because it has a buildfarm now.
[01:38] <spm> jtv: https://pastebin.canonical.com/35029/ I think I can post that publicly, but can you confirm or deny?
[01:38] <wgrant> There are no private TTBJs yet, so it /should/ be safe... but...
[01:39] <spm> yeah, juts wanted to be doubly sure
[01:39] <spm> the sus ones are all public
[01:40] <spm> wgrant: http://paste.ubuntu.com/469098/
[01:41] <wgrant> Hmm.
[01:41] <wgrant> So they all have BranchJobs, are not deleted, and all very fresh.
[01:43] <wgrant> I don't know what's going on, unless it's trying to dispatch them but failing.
[01:45] <jtv> And maybe re-dispatching them?
[01:51] <jtv> wgrant, spm: it's suspicious that these are all so recent...  might it be a dispatch priority issue?
[01:52] <spm> jtv: recent as in the last 24ish hours?
[01:52] <jtv> spm: yes
[01:53] <jtv> Did we CP anything that could have done this?
[01:53] <spm> jtv: and btw, good morning! you're up early? :-)
[01:53] <jtv> spm: amsterdam
[01:53] <spm> 'up early' still rings true :-)
[01:53] <jtv> up late.  :)
[01:53] <spm> heh
[01:54] <jtv> met up with team members & an ex team member today... a good portion of the Prague attendance went to Holland next it seems.
[01:54] <spm> jtv: no CP's to the build side since the 9th, so unlikely?
[01:54] <spm> ahh cool!
[01:55] <jtv> Some for holidays, others for the GNU meeting, yet more for Guadec.
[01:56] <jtv> No CPs...  then either it's operational or it's something weird on edge.
[01:56] <jtv> The branches I checked out weren't deleted, so I doubt it's related to branch deletion.
[01:58] <wgrant> jtv: Yeah, it's a bit odd. I guess someone needs to check buildd-manager logs at some point :/
[01:58] <spm> edge? edge gets updated regularly - so CP?
[01:58] <jtv> :(
[01:59] <spm> try again; so CP isn't as unlikely an option- it's just the normal state of affairs.
[01:59] <wgrant> Hmm. I guess it's possible it's priority.
[01:59] <jtv> spm: yes, it's just that edge is less likely to have an effect on the build farm
[01:59] <wgrant> But that would imply that the queue hasn't been empty for 24 hours.
[01:59] <jtv> ...and yet we have builders sitting idle.
[02:00] <wgrant> I guess I'll just hope they disappear over the day.
[02:02] <jtv> Maybe something's bottlenecking on uploads?
[02:04] <jtv> wgrant: is the builder marked as idle while its uploads are being processed?
[02:04] <wgrant> jtv: No.
[02:04] <jtv> it'd be nice to know what dispatch queries are going out right now...
[02:05] <wgrant> It's normal at the moment for lots of builders to be idle, because buildd-manager sucks.
[02:05] <wgrant> but... buildd-manager is far worse than normal.
[02:05] <wgrant> It's redispatching the same set over and over.
[02:05] <wgrant> Every minute or so.
[02:05] <wgrant> lamont alluded to this yesterday, saying there was a rogue build causing it to abort most cycles.
[02:06] <wgrant> I'll talk to Julian tonight, I suppose.
[02:06] <wgrant> (if you click on a few of the current virtual builds, you'll see they started <30s ago.
[02:06] <wgrant> Then if you refresh in another 30 seconds, they'll still be <30s ago.
[02:06] <wgrant> Anyway, uni.
[02:06] <wgrant> Probably not much we can do now.
[02:10] <jtv> If the builds keep getting re-dispatched, that's a familiar symptom of something...
[02:10] <jtv> ...slaves breaking protocol?
[02:10] <jtv> as you say, uni.
[02:10] <rockstar> build farm busy
[02:12] <jtv> rockstar: would a busy build farm do that?  That'd be a pretty major and obvious bug.
[02:12] <jtv> (IIRC a job gets marked as in-progress _and committed_ before actual processing starts)
[02:13] <rockstar> jtv, well, that was supposed to go to another channel, but yes, it seems the build farm is busy right now.
[02:13] <jtv> rockstar: from what wgrant has been finding out, it appears that some jobs are being eternally re-dispatched
[02:13] <rockstar> jtv, oh wait, you thought I was following your context.
[02:13] <jtv> Yes, I'm weird that way
[02:13] <jtv> :-)
[02:14] <rockstar> jtv, no, I haven't been reading the backchat.
[02:14] <thumper> rockstar: hey
[02:14] <thumper> rockstar: you home again?
[02:14] <thumper> rockstar: and back to normal?
[02:14] <thumper> jtv... weird... surely not
[02:14] <jtv> preposterous
[02:15] <rockstar> thumper, well, "normal" is objective, but yeah, I'm home.
[02:15] <thumper> rockstar: objective or subjective?
[02:15] <thumper> pedantry rules
[02:16] <jtv> thumper: he means that he's normally objective.
[02:23] <spm> heya rockstar
[04:30] <wgrant> buildd-manager is still screwed :(
[04:30] <wgrant> spm: No obvious errors in the log?
[04:30] <spm> orsum. looking.
[04:31] <wgrant> It should try to dispatch lots of builds, then encounter some error. Every <1min, probably.
[04:32] <spm> exceptions.AssertionError: amd64 build of eclipse 3.4.1-0ubuntu2 in ubuntu karmic RELEASE (1213758) can not be built for pocket RELEASE: invalid pocket due to the series status of karmic. ??
[04:33] <spm> not every minute, but surely not far off.
[04:33] <wgrant> That would do it.
[04:34] <wgrant> That's... interesting.
[04:34] <spm> it's a rather impressive stack trace too
[04:35] <spm> been going for ages too
[04:35] <wgrant> Bug #575165
[04:35] <_mup_> Bug #575165: Buildd-manager erroneously checks COPY archives for release pocket upload permissions when dispatching <buildd-manager> <soyuz-upload> <trivial> <Soyuz:Triaged by julian-edwards> <https://launchpad.net/bugs/575165>
[04:36] <wgrant> But that archive is disabled.
[04:36] <wgrant> So the build shouldn't be active :/
[04:38] <thumper> :(
[04:42] <wgrant> I guess we could just manually suspend it.
[04:42] <wgrant> Why it is not already suspended I do not know.
[04:48] <wgrant> thumper: Is the porting of SPRBs from BuildBase to PackageBuild going to happen this cycle?
[04:48] <thumper> wgrant: I believe so
[04:48] <wgrant> Excellent.
[04:48] <thumper> wgrant: abentley has it mostly done I think
[04:48] <thumper> he was stuck though on where the builds get set to fully built
[04:49] <thumper> at least that is where it was on my Friday
[04:49] <thumper> he may have solved that now
[04:49] <wgrant> Yeah, there's the wonderfully-named 'findBuild' method which actually sets it to fully built...
[04:50] <thumper> WTF?
[04:51] <thumper> that does not sound like a mutating function
[04:51] <wgrant> No.
[04:51] <thumper> fix plz?
[04:51] <wgrant> Heh. Maybe once the build farm code has settled down and we've removed the four excess tables and several additional classes...
[05:04] <mwhudson> wow, recipe builds really don't have much priority do they?
[05:06] <wgrant> What is their priority?
[05:06] <wgrant> I forget.
[05:06] <wgrant> Ah, 900.
[05:06] <wgrant> Yeah, normal PPA builds are 2510 or so.
[05:07] <wgrant> I need to convince everyone that build scores are stupid and need to die.
[05:07] <mwhudson> i'm just experiencing the "been expected to start in 1 hour for several hours" thing
[05:08] <wgrant> Well, that's not due to the score.
[05:08] <wgrant> That's due to everything being fucked.
[05:08] <mwhudson> i see
[05:08] <wgrant> The ETA takes the score into account.
[05:17] <thumper> wgrant: you have a +1 from me to kill scores
[05:17] <thumper> but I don't hold much weight there
[05:17] <thumper> what we need is a way to prioritise some builds
[05:17] <thumper> and scores are an approximation for that
[05:19] <spm> if team==losa then do-losa-builds-first elseif team==ubuntu-security then do security-builds else hahahaha fi ?
[05:21] <thumper> spm: something like that
[05:21] <wgrant> thumper: A pretty ineffective approximation.
[05:21] <thumper> mwhudson: I'm deleting most of doc/branch-merge-proposals.txt
[05:21] <thumper> mwhudson: I'm rewriting it to not use sample data
[05:21] <thumper> mwhudson: and in my rewrite I realised it is a big pile of turd
[05:21] <wgrant> There seems to be a bit of that going around lately.
[05:22] <thumper> sample data should be avoided in all tests
[05:22] <thumper> whereever possible
[05:22] <thumper> which is almost everywhere
[05:22] <wgrant> Yeah.
[05:24] <mwhudson> thumper: hooray?
[05:24] <thumper> mwhudson: I'm actually writing documentation
[05:24] <thumper> the level of executability in the doc is very small
[05:25] <thumper> I'm trying to focus on the documentation being meaningful
[05:36] <spm> thumper: !?!? bit excessive isn't it? *meaningful*!?!?! srsly?
[05:36] <thumper> yeah
[05:36] <thumper> what was I thinking
[05:37] <spm> haha
[05:37]  * thumper replaces the doctest with "Read the freaking code bitch"
[05:38] <spm> damn. that's almost too easy to take out of context. I shall pass. this time.
[05:38] <thumper> spm: heh
[05:48] <thumper> lifeless: hi, back in NZ?
[05:49] <lifeless> nope
[05:49] <thumper> lifeless: EU still??
[05:49] <lifeless> I fly out from here in 12 hours
[05:49] <lifeless> GNU Hackers Meeting was on
[05:49] <thumper> how'd that go?
[05:52] <lifeless> pretty good
[05:52] <lifeless> I need to write it up but <so tired>
[05:52] <lifeless> thumper: did you try the staging search perf ?
[05:52] <thumper> no
[05:52] <thumper> but I read your writeup
[05:52] <thumper> and commented on the merge proposal
[05:53] <lifeless> yeah
[05:53] <lifeless> its because of conflation
[05:53] <lifeless> thumper: https://bugs.staging.launchpad.net/ubuntu/+filebug
[05:53] <lifeless> sadly it failed an obscure teset
[05:53] <lifeless> so I need to update some more tests with the same 'it sucks, sorry, fast-first' dialogue.
[05:56]  * thumper nods
[05:56] <lifeless> thumper: I'm interested in perf for you, on that url.
[05:57] <lifeless> 'folk in nz are like folk in china'
[05:57] <thumper> what do you want me to search on?
[05:57] <lifeless> whatever you like
[05:58] <thumper> lifeless: "bzr transfers too much data" gave me a response in about 5 seconds, but none of them really relevant
[05:59] <lifeless> thumper: item 0 for me is bug 82305
[05:59] <_mup_> Bug #82305: push and pull on bound branches use too much network <Bazaar:Confirmed> <bzr (Ubuntu):Confirmed> <https://launchpad.net/bugs/82305>
[05:59] <lifeless> which is pretty relevant
[05:59] <thumper> well... yes
[05:59] <lifeless> remember that this is in-ubuntu rather than in-bzr context - its not searching the bzr bug db
[05:59] <thumper> I suppose
[06:00] <lifeless> https://bugs.edge.launchpad.net/ubuntu/+filebug
[06:00] <lifeless> same search
[06:00] <lifeless> doesn't return any bzr bugs
[06:00]  * thumper has to go collect people
[06:01] <lifeless> ciao
[06:04] <lifeless> and well production simply won't answer
[08:15] <adeuring> good morning
[08:23] <lifeless> stub: hi
[08:28] <stub> Happy Asaha Bucha Day.
[08:29] <lifeless> oh nice
[08:29] <lifeless> you're on leave for it ?
[08:30] <stub> Yer. Just ticking off a few things from my todo list first though.
[08:30] <lifeless> cool
[08:30] <lifeless> well I'm travelling tonight
[08:30] <lifeless> I've done an audit of the timeout bugs in malone
[08:31] <lifeless> and there are a few which would really benefit from you casting an eye over some query<->index stuff
[08:31] <lifeless> if you had time tomorrow to just have an eyeball - just the malone stuff tagged 'timeout'
[08:31] <stub> Ok.
[08:32] <lifeless> theres things like a union plan which takes 11 minutes to run !
[09:01] <mrevell> Morning
[09:58]  * flacoste is away: Gone away for now
[10:09]  * thumper takes a look at the merge conflict
[10:11] <jml> thumper, thanks
[10:12] <jml> thumper, I was about to, since I last touched branch.txt
[10:12] <thumper> jml: it seems to be that we just need to delete that block again
[10:12] <thumper> sound right?
[10:12] <jml> thumper, haven't got that far yet
[10:12] <jml> thumper, still pulling db-devel :)
[10:12] <thumper> jml: but in the branch that you merged into devel you just deleted chunks of branch.txt?
[10:13] <jml> thumper, pretty much.
[10:13] <jml> (note that it doesn't use branch sampledata any more)
[10:13] <jml> thumper, yeah, delete it.
[10:14] <jml> thumper, all of that has been moved to unit tests
[10:14] <thumper> jml: the difference on db-devel is that someone reformatted the lines to the right length
[10:15]  * thumper deletes
[10:18] <jml> thumper, thanks.
[10:18] <jml> thumper, btw, I'm still progressively working through the code tests getting rid of their dependency on branch sampledata
[10:18] <thumper> jml: you should see my merge proposal that is up
[10:18] <jml> thumper, I did!
[10:18] <thumper> I tried to focus on useful documentation
[10:19] <thumper> rather than a test
[10:19] <jml> thumper, cool.
[10:19] <jml> thumper, I'll review that in a moment.
[10:19]  * thumper leaves the office to drink a cuppa
[10:20] <lifeless> adeuring: what does having sec proxies returned from the lpfactory do for us ?
[10:21] <adeuring> lifeless: I think the tests should work with the same objects as we use in real code.
[10:21] <adeuring> and we should be able to detect for example permission problems
[10:21] <lifeless> adeuring: thanks
[10:24] <thumper> jml: yes it is still running, just not under a special layer
[10:24] <jml> thumper, ahh, thanks.
[10:24] <thumper> gah, my email client is getting my emails :(
[10:33] <bigjools> jml: Hope you're feeling better this week because I added you to the MP for the buildd-manager branch last week, it might make you feel sick again.
[10:35] <jml> bigjools, thanks :)
[10:35] <bigjools> jml: I do feel bad and apologise profusely.  However, not *too* bad since I had to suffer when changing the damn thing :)
[10:36] <bigjools> some of the tests are bloody awful, so I hope you have suggestions on how to make them look better.
[10:46] <lifeless> bigjools: can you describe the awfulness ?
[10:47] <lifeless> bigjools: like - overly coupled, too sensitive, not sensitive enough, visually long, hidden setup, ?
[10:47] <bigjools> lifeless: hideous setup
[10:47] <bigjools> see lib/lp/buildmaster/tests/test_manager.py, look for TestSlaveScanner.setUp()
[10:48] <bigjools> it stubs out half the code its testing
[10:48] <bigjools> and then selectively re-enables it in some tests!
[10:48] <bigjools> and it probably falls foul of all those other things you listed
[10:50] <lifeless> so, if different tests need different amounts stubbed
[10:50] <lifeless> I think I'd tackle that by:
[10:50] <lifeless> moving the setup to a helper function
[10:51] <lifeless> delete the setUp() on the test class
[10:51] <lifeless> at the top of each test call the helper with parameters appropriate to the test being run
[10:51] <bigjools> the tests need re-writing, they're rubbish
[10:51] <lifeless> it will still be hideous, but less so.
[10:51] <lifeless> and it will be a cheap change to make
[10:51] <bigjools> sort of - I'd need to work out exactly what each test is depending on
[10:52] <bigjools> I would not be surprised if it's masking bugs
[11:08] <wgrant> bigjools: Evening.
[11:08] <wgrant> Are you aware of the rogue karmic rebuild which is killing everything?
[11:09] <wgrant> (or was over the weekend; not sure if it's been fixed in the last few hours)
[11:14] <bigjools> wgrant: sigh, thought it had disappeared
[11:14] <bigjools> how the *hell* is it even there
[11:18] <wgrant> bigjools: Yeah, it should surely have been suspended...
[11:21] <bigjools> I wonder if someone pocket-copied it
[11:21] <wgrant> bigjools: Within a copy archive?
[11:21] <wgrant> Unlikely.
[11:21] <lifeless> -> stuff
[11:21] <bigjools> how do you know it's a copy archive build?
[11:22] <wgrant> I thought it was build 1213758
[11:22] <bigjools> how do you know that?
[11:22] <bigjools> I can see it in the logs, but...you?
[11:22] <wgrant> spm checked the logs.
[11:22] <bigjools> ah
[11:22] <wgrant> I'm not that good, sadly.
[11:23] <bigjools> this is a test archive from 20090909
[11:23] <bigjools> I know the bug for this
[11:23] <wgrant> Yep.
[11:23] <bigjools> but the archive is disabled...
[11:23] <bigjools> wtf
[11:24] <wgrant> Bug #575165
[11:24] <wgrant> Right.
[11:24] <_mup_> Bug #575165: Buildd-manager erroneously checks COPY archives for release pocket upload permissions when dispatching <buildd-manager> <soyuz-upload> <trivial> <Soyuz:Triaged by julian-edwards> <https://launchpad.net/bugs/575165>
[11:24] <wgrant> But was it disabled before build suspension was implemented?
[11:24] <wgrant> Probably.
[11:24] <bigjools> grar
[11:24] <bigjools> why has it appeared now though?
[11:24] <wgrant> I don't quite know.
[11:24] <wgrant> Maybe it depwaited.
[11:24] <wgrant> So wasn't suspended.
[11:24] <wgrant> And then somehow got retried.
[11:27] <bigjools> ok, I'm going to re-purge disabled copy archive builds
[11:27] <bigjools> FWIW http://pastebin.ubuntu.com/469264/
[11:27] <wgrant> 'Purge' meaning suspend, or really delete?
[11:31] <wgrant> bigjools: Any progress with PPA log parsing?
[11:31] <bigjools> supersede
[11:31] <bigjools> no - but good reminder :)
[11:32] <wgrant> Heh. That query looks good, anyway.
[11:32] <bigjools> that query has been used too many times already :/
[11:35] <wgrant> bigjools: How I look forward to the great purge next month...
[11:36] <bigjools> wgrant: que?
[11:36] <wgrant> bigjools: Once SPRB and TTBJ are ported to the new BuildFarmJob infrastructure, we get to delete lots of classes and DB tables.
[11:36] <bigjools> ah
[11:36] <wgrant> And remove the opportunities for inconsistencies.
[11:36] <bigjools> yay
[12:23] <wgrant> bigjools: https://code.edge.launchpad.net/~wgrant/launchpad/ddeb-domination
[12:24] <wgrant> How does that approach seem?
[12:24] <bigjools> looking ...
[12:26] <bigjools> wgrant: will it look like a deb dominates a dedeb?
[12:26] <bigjools> ddeb, even
[12:28] <bigjools> wgrant: and will the ddeb publisher notice that a ddeb was superseded by a different publisher?
[12:29] <bigjools> BTW, a WIP MP would be useful to see a diff :)
[12:31] <bigjools> wgrant: how is performance affected on the query changed in domination.py
[12:31] <bigjools> (these are all questions that you don't need to answer right now, I am just checking your approach)
[12:35] <wgrant> bigjools: Remember that BPPH.supersededby is a BinaryPackageBuild. For this sort of reason.
[12:35] <wgrant> Hmm, fair point on that query. Not sure about that.
[12:35] <wgrant> As for the publisher noticing the superseded stuff, it works fine apart from the bug I filed yesterday.
[12:36] <wgrant> And that's a restricted case that we can probably ignore, but I wanted to record.
[12:36] <bigjools> wgrant: ok - I was going by one of your commit comments where you implied that ddebs live in the same archive - they don't.
[12:37] <wgrant> Ah, right, yes. That's for the non-primary case, where they do live in the same archive.
[12:43] <wgrant> Do we know when Lucid is happening yet?
[12:49] <bigjools> RSN
[14:41] <rockstar> bigjools, is the build farm overloaded right now?
[14:44] <bigjools> I can say yes without even looking :)
[14:44] <wgrant> It seems to have just been hit with a wave of dh7 retries.
[14:44] <wgrant> But that should clear soon.
[14:44] <bigjools> rockstar: https://edge.launchpad.net/builders <-- self service
[14:56] <rockstar> bigjools, I knew there was a URL, I just never remember what that URL is.  :)
[14:56] <bigjools> rockstar: unfortunately I don't have the same amnesia issue
[14:56] <rockstar> bigjools, yup.  That's why I asked you.  :)
[15:42]  * flacoste is back.
[15:53] <lifeless> \o/ my librarian patch has landed
[15:53] <lifeless> mthaddon: I'm hopping on a plane now, but I'll try to get the configs branch up for you
[15:53] <mthaddon> cool
[15:53] <lifeless> fast search will have to wait for me to get to NZ I think
[15:54] <lifeless> mthaddon: the appservers - please tell me they are not using a single oops dir, with the same oops prefix, across different processes
[15:54] <mthaddon> lifeless: I think you should be able to tell me that :) but I'm pretty sure they use different OOPS prefixes
[15:54] <lifeless> in which case its ok
[15:55] <lifeless> mthaddon: do you have enough data to move forward on rt 40480 ?
[15:57] <lifeless> mthaddon: tell me a bout librarian and librarian2 ?
[15:57] <mthaddon> lifeless: erm, so are you saying that X frontends doesn't expose us to bzr locking any more than 1 frontend?
[15:57] <lifeless> mthaddon: right
[15:58] <mthaddon> lifeless: I don't think we're blocked on you in that case (although testing this theory would be nice, and we'll certainly need to do this on staging first)
[15:59] <lifeless> mthaddon: right
[16:00] <lifeless> mthaddon: lp:~lifeless/lp-production-configs/librarian - don't merge till we do a deployment to the librarian of whats currently out on edge, cause the schema stuff is lock-step
[16:00] <lifeless> mthaddon: I only see a config for 1 librarian though - public and internal instances-of, but only one process for each role. Is that right ?
[16:01] <mthaddon> lifeless: you didn't see the librarian2 folder?
[16:01] <lifeless> I did
[16:01] <lifeless> that appears to be the private librarian
[16:01] <mthaddon> nope
[16:01] <lifeless> not a redundant public librarian
[16:02] <lifeless> ok, cool
[16:02] <lifeless> anyhow, I've put the change into both parts
[16:02] <lifeless> I must run and get security checked etc now.
[16:02] <mthaddon> since it's not proposed for merging yet, we'll leave it til that happens
[16:03] <lifeless> please do grab spiv/mwhudson/jml if the codehosting thing gets hairy - I know all of them have looked at the entire vertical stack
[16:03]  * lifeless fades into the sunset 
[16:03] <jml> sinzui, hi
[16:04] <sinzui> hi jml
[16:05] <jml> sinzui, want to mumble?
[16:17]  * lifeless is in da queue
[16:33] <leonardr> thumper, are you around to talk about your web service question?
[16:33] <leonardr> otherwise i will send an email
[16:37] <lifeless> leonardr: its EEEEEARLY for him - I suggest a mail :)
[16:37] <lifeless> leonardr: 3:30am specifically.
[16:37] <leonardr> lifeless, sure, just checking
[16:45] <rockstar> abentley, we seem to be having some sort of issue with the scanner not setting branch merge proposals to "Merged"
[16:46] <abentley> rockstar, okay, can you be more specific?
[16:47] <rockstar> abentley, no.  I'm trying to find more details now.
[16:47] <abentley> rockstar, you don't have an example where this didn't happen or anything?
[16:48] <rockstar> abentley, actually, I can be a bit more specific.  The folks in #tarmac are complaining that Tarmac keeps trying to land things after its merged them already.
[16:49] <abentley> rockstar, could it be a race condition?
[16:49] <rockstar> abentley, I'm not prepared to rule anything out until I see an example proposal.
[16:50] <mtaylor> https://code.edge.launchpad.net/~mordred/swift/fix-pep8/+merge/30894
[16:50] <rockstar> abentley, https://code.edge.launchpad.net/~mordred/swift/fix-pep8/+merge/30894
[16:50] <rockstar> mtaylor, :)
[16:50] <mtaylor> beat you to it
[16:50] <mtaylor> :)
[16:50] <abentley> mtaylor, that branch hasn't been fully merged.
[16:50] <mtaylor> no?
[16:50] <rockstar> mtaylor, oh, I know what is.
[16:51] <abentley> mtaylor, see the "unmerged revisions" heading?
[16:51] <rockstar> mtaylor, you pushed new revisions AFTER it was approved, didn't you?
[16:51] <mtaylor> rockstar: yes. it does seem so
[16:52] <rockstar> mtaylor, yup, so the approved revision id isn't the tip of the branch.
[16:52] <rockstar> abentley, thanks for your help.
[16:52] <abentley> rockstar, np.
[16:52] <mtaylor> rockstar: ok. but the merge proposal should still be marked as merged, no? the thing that was approved happened
[16:52] <mtaylor> I mean- something should certainly be in some state here
[16:53] <abentley> mtaylor, I don't know.  Certainly I didn't think so when I wrote it.
[16:54] <abentley> mtaylor, the fact that there are outstanding changes definitely seems like something we should represent.
[16:55] <abentley> mtaylor, because we wouldn't want those changes to get lost.
[16:55] <mtaylor> yeah - it's in a weird limbo state right now ... it's still approved -so I can't really approve the new changes easily
[16:55] <abentley> mtaylor, commonly in LP code reviews, we'll say "approved, if you make these changes", and so a branch in this state isn't really merged for us.
[16:56] <abentley> mtaylor, we don't really consider what state the proposal is in when we mark it merged.  It could be rejected, approved, but when tip goes into target, we consider it merged.
[16:57] <rockstar> mtaylor, also, we don't want the proposal to disappear in cases where not all the revisions are merged.  That would lead the user to think something incorrect.
[16:58]  * rockstar sees abentley has already said this
[16:58] <mtaylor> yeah - I understand all of what you're both saying...
[16:58] <rockstar> mtaylor, it helps when we're saying the same thing.  :)
[16:58] <abentley> mtaylor, it's hard in the Launchpad workflow to get into this state, because our landing tool always lands the tip.
[16:58] <mtaylor> I'm just saying that the system does not currently lead me to think any action needs to be taken, or what action I should take
[16:58] <rockstar> abentley, with PQM you mean?
[16:58] <abentley> rockstar, yes.
[16:59] <rockstar> mtaylor, yeah, that's a Tarmac bug more than a Launchpad bug.  I think Launchpad is acting as it should.
[16:59] <abentley> mtaylor, It seems to have gotten your attention, though.
[16:59] <mtaylor> abentley: only because of the failed tarmac job
[16:59] <abentley> mtaylor, you don't use +activereviews?
[17:00] <mtaylor> but no, I don't think it's just a tarmac bug - because I think tarmac is doing the right thing here
[17:00] <mtaylor> I think just merging the tip rather than merging the approved revision is a terrible idea
[17:00] <mtaylor> as there is no workflow support to say that the new revisions are actually good
[17:00] <abentley> mtaylor, you're welcome to your opinion, but I don't share it.  See above.
[17:00] <mtaylor> abentley: you don't think new revisions need code review?
[17:01] <abentley> mtaylor, not always.
[17:01] <rockstar> mtaylor, new revisions need code review, yes, but I think that's a user's responsibility.  Launchpad shouldn't have to know about everyone's different workflows.
[17:01] <abentley> mtaylor, we trust our contributors to not make gross changes.
[17:02] <abentley> mtaylor, and we would find it a pain to have to do another roundtrip for the trivial changes we sometimes request in a review.
[17:02] <mtaylor> ok. that's all fine, we can differ in our opinion here - just tell me, in this state, how do I approve new revisions
[17:02] <rockstar> mtaylor, just unset Approved and reset it.
[17:02] <rockstar> mtaylor, we could probably do better in that regard though.
how about not setting the status to Approved until the changes have been made?</troll>
[17:03] <mtaylor> rockstar: ok. I'll just do that. seems clunky though
[17:03] <rockstar> mtaylor, in my case, that's what I do.  If I ask for changes, I'll vote "approve" but not set it to Approved.
[17:03] <abentley> mtaylor, "We would find it a pain to have to do another roundtrip for the trivial changes we sometimes request in a review".
[17:03] <mtaylor> hehe
[17:03] <rockstar> mtaylor, if you file a bug about the "clunkiness" of re-approved, I'm sure we can triage it.
[17:04] <mtaylor> rockstar: have I mentioned that we need merge queues
[17:04] <rockstar> mtaylor, yeah, I think I was the one that put that earworm in your head.  :)
[17:05] <rockstar> mtaylor, see #tarmac for a tarmac solution.
[17:05] <mtaylor> the _real_ problem here of course being that we are encoding two different pieces of information with the same status - and abentley and I choose to overload that system in different ways
[17:05] <abentley> rockstar, you wrote a song about merge queues?
[17:05] <mtaylor> zomg. I so want to hear rockstar's song about merge queues
[17:05] <rockstar> abentley, earworms don't have to be songs.  :)
[17:05] <rockstar> Although I guess now I need to write a song about merge queues...  :)
[17:06]  * rockstar buckles under peer pressure
[17:07] <abentley> rockstar, http://www.wordspy.com/words/earworm.asp
[17:09] <abentley> rockstar, send me your song about merge queues, and I will lay down some flute, cowbell, tambourine or vox-- your choice.
[17:09] <mtaylor> jml: see... we're writing songs about it now..
[17:11] <rockstar> abentley, all of the above, but always more cowbell.
[17:17] <rockstar> abentley, I regret to inform you that the build estimation code that you worked so hard on is being trampled on by builders being overwhelmed at the last second...  :)
[17:57] <bdmurray> leonardr: how could I address robert's concerns in https://code.edge.launchpad.net/~brian-murray/launchpad/bug-320596/+merge/30690?
[18:02] <leonardr> bdmurray, checking
[18:26] <jml> ciao
[18:30] <leonardr> bdmurray, ok, long story short, to maintain backwards compatibility you need to have two calls to @operation_parameters
[18:30] <leonardr> this should work:
[18:30] <leonardr> @operation_parameters(parameters_for_devel)
[18:30] <leonardr> @operation_for_version('devel')
[18:30] <leonardr> @operation_parameters(parameters_for_old_versions)
[18:31] <leonardr> you should define a list containing the parameters, then copy it and change only the parameter you want to change, so that you don't have two copies of that huge list
[18:32] <leonardr> add a doctest to the stories/webservice directory of malone, illustrating that in 1.0 the default value is True, but that in devel the default value is False
[18:33] <leonardr> does this make sense?
[18:33] <leonardr> (be sure to put @operation_for_version('devel') on top of all of your current annotations--it's the dividing line between 1.0 and devel
[18:34] <bdmurray> the parameters for devel would be very search parameter?
[18:35] <bdmurray> every that is
[18:43] <leonardr> bdmurray: yes, you would pass in slightly different lists for each @operation_parameters call
[18:54] <bdmurray> leonardr: okay thanks
[20:29] <gary_poster> bac: hey.  didn't you have a "run the tests that failed on ec2" command?  If so, does it still work?  If so, what is it? :-)
[20:30] <gary_poster> ah!  bin/retest looks promising...
[20:32] <gary_poster> except that the ec2 output doesn't have the kind of information we need, anymore, it seems :-/
[20:32] <bac> gary_poster: :(
[20:32] <bac> gary_poster: but you can use testr to do it
[20:32] <gary_poster> bac, ?
[20:32] <bac> gary_poster: read robert's email from last week
[20:32] <gary_poster> ok
[20:33] <gary_poster> ("oh," Gary thinks, "Robert's one email from last week!  Right!" :-) )
[20:39] <gary_poster> bac, AFAICT that only helps if you have run tests locally, not if you ran test on ec2 that you want to rerun now
[20:39]  * gary_poster found the email
[20:39] <benji> the person that develops a working knowledge management system that gathers data from email will make a mint
[20:40] <bac> gary_poster: i thought you could unzip the attachment that came in the failure message and pipe it into testr load, or something similar
[20:40]  * gary_poster looks at docs again
[20:41] <bac> gary_poster: i admit i have not tried it personally
[20:41] <bac> gary_poster: perhaps as a non-expert you should write down what actually works and how to do it...assuming you are successful
[20:43] <gary_poster> bac, https://dev.launchpad.net/Testing has what has been written so far.  I see ``testr load < testrun`` in testr --help but the relationship between that and the ec2 email output is less than obvious.  I'll play around for a second, and, yes, write something down if it seems valuable
[20:44] <gary_poster> LOL, "testr load < ~/apidoc.log.gz" didn't seem useful :-)  Maybe I'm supposed to uncompress it...
[20:52] <gary_poster> For those following along at home, uncompressing did in fact seem to allow testr to load the subunit file.  The resulting output was a bit concerning:  id: 0 tests: 8956 failures: 5 skips: 23
[20:52] <gary_poster> i.e., what about the skips?
[20:54] <gary_poster> Then there's the concern that Gavin raised on the list
[20:57] <gary_poster> abentley: hey.  If I get the output "id: 0 tests: 8956 failures: 5 skips: 23" after loading in a subunit ile to testr, what is the source of the "skipped" line?  How do I see what the skipped tests are?  How do I run them, if I need to?
[20:58] <abentley> gary_poster, I believe there are no skips in the Launchpad test suite.  testresources counts failure to tear down layers as skips.
[20:59] <gary_poster> abentley: ah, great, so, for the short term at least that means that the information about skipped stuff is ignorable?
[21:00] <abentley> gary_poster, yes.
[21:00] <gary_poster> cool thanks
[21:13] <mtaylor> rockstar: here's a UI thought for you
[21:13]  * rockstar listens
[21:13] <mtaylor> rockstar: I just had someone be confused as to why merge requests and bug reports are separate things
[21:14] <rockstar> mtaylor, yes, I've heard this joke before.
[21:14] <rockstar> :)
[21:14] <mtaylor> rockstar: so it just got my brain trying to wrap its head a) around the urge and b) around whether or not there is something useful in the request
[21:15] <rockstar> mtaylor, so, this is why we have branch links.
[21:15] <mtaylor> rockstar: the reference was that in JIRA, a patch is attached to a bug and then discussion ensues
[21:15] <mtaylor> I think the main thing here had to do with discussion around a thing
[21:15] <rockstar> mtaylor, I do know the landscape team was doing their code reviews on the bug report.
[21:15] <rockstar> (I think that is crazy)
[21:15] <mtaylor> me too... but I'm wondering if there isn't some way to express some sort of tighter coupling or some sort of information sharing that isn't something we think is on crack
[21:17] <mtaylor> rockstar: the specific initial comment was "It's just annoying that e.g. bug reports are separate from merge requests, and the blueprints don't have a good discussion platform built in... Just feels like I have to repeat everything two or three times to keep launchpad happy..."
[21:17] <rockstar> mtaylor, well, yeah, I've been thinking about ways to display more bug content in the merge proposal page, but I'm currently more focused on the branch page redesign currently.
[21:18] <mtaylor> rockstar: I believe I had a thought a while back about being able to pre-populate a merge request description from a blueprint,
[21:18] <mtaylor> rockstar: essentially thinking that all branches should either be related to a bug or a blueprint -but that in many cases something could be generated automatically in one directoin or another
[21:19] <rockstar> mtaylor, that would require people other than Drizzle, Ubuntu, and Wikkid (because thumper is a karma whore) use blueprints.
[21:19]  * thumper likes karma
[21:20] <mtaylor> rockstar: well, once I fix them, I hope that they will
[21:20] <thumper> at least I don't game the system though
[21:20] <thumper> mtaylor: are you going to work on blueprints soon then?
[21:20] <mtaylor> thumper: yes
[21:20] <rockstar> mtaylor, my hat's off to you.
[21:20] <thumper> mtaylor: how soon?
[21:20] <mtaylor> rockstar: you can add openstack to the blueprint using list :)_
[21:20] <mtaylor> thumper: soon
[21:21] <thumper> real soon now?
[21:21] <rockstar> mtaylor, I might have recruited someone to help you with that.
[21:21] <mtaylor> rockstar: excellent
[21:21] <rockstar> mtaylor, the downside is that he's 15.  :)
[21:21] <thumper> mtaylor: I'd happily have pre-impl calls about blueprints
[21:21] <thumper> mtaylor: and review changes
[21:21] <thumper> mtaylor: I'm sure sinzui would too
[21:22] <mtaylor> thumper: great. well, I _really_ need to write my list of things down
[21:23] <thumper> jelmer: could I get you to send me an email with the known good revisions of the bzr plugins?
[21:23] <thumper> jelmer: also specifically if they'll work with 2.1 and 2.2 at the same time
[21:24] <jelmer> thumper: what sort of timeframe do you have in mind for landing these?
[21:25] <thumper> jelmer: if they are simple, then we could just drop them in ASAP
[21:25] <rockstar> mtaylor, I know this awesome new site that allows your to write a todo list of things.  The call the items "bugs" and it really helps me do my work...  :)
[21:25] <thumper> jelmer: all we need to do is merge them into the launchpad branches, and update the config file
[21:25] <thumper> rockstar: but you can't order them :P
[21:25] <mtaylor> rockstar: hrm. ... relative cost of opening a bug in launchpad vs. writing a list in a file in vi...
[21:25] <rockstar> thumper, sure you can.  It's called "importance"
[21:26]  * thumper coughs 
[21:26] <thumper> that is pretty rough ordering
[21:26] <mtaylor> rockstar: have I mentioned that the bug opening cost in lp is still pretty high?
[21:26] <rockstar> mtaylor, blame lifeless.  That's his job.  :)
[21:27] <mtaylor> rockstar: oh, I will
[21:28] <jelmer> thumper: can I perhaps just pqm-submit to the various lp:~launchpad-pqm/ branches and then have you review the update to sourcedeps.conf ?
[21:34] <james_w> as an alternative to workitems in blueprints, we were pondering last week using bugs
[21:34] <james_w> and providing an API client that allows you to just write a list of things and it will open all the bugs for you
[21:42] <jelmer> thumper: still there?
[21:43] <thumper> jelmer: yep
[21:44] <jelmer> thumper: Did you see my question earlier?
[21:44] <thumper> jelmer: sorry, rebooted due to updates
[21:44] <thumper> jelmer: yes, and yes
[21:44] <jelmer> thumper: ok, I'll that then - thanks
[21:44]  * thumper searches for more coffee
[21:45] <jelmer> thumper: Any plans yet to have bzr-svn/bzr-git in the PPA and built as daily builds from lp:~launchpad-pqm/bzr-{git,svn}/devel rather than checkout out manually in sourcecode.conf ? :-)
[21:55] <rockstar> jelmer, that would be awesome, and I would like that.
[21:57] <thumper> jelmer: we need a way to control the plugins, and add them as we need them
[21:57] <thumper> jelmer: so as long as we can work out how to do that...
[21:57] <thumper> jelmer: should be fine
[22:03] <rockstar> Argh, timeouts on staging make rockstar angry.
[22:03] <rockstar> You wouldn't like me when I'm angry.
[22:05] <jelmer> thumper: what do you mean by add exactly? when to load them, when to update their version?
[22:05] <thumper> jelmer: we don't want them on all the time
[22:06] <jelmer> thumper, isn't that an issue now as well though?
[22:06] <thumper> jelmer: no
[22:32] <james_w> OOPS-1668ED4702
[22:33] <james_w> no bot?
[22:34] <benji> james_w, you might like this: I added a quick search to my Firefox so I could type/paste "OOPS 1668ED4702" in the address bar and be taken to it
[22:35] <james_w> good idea
[22:35] <james_w> could someone confirm that https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1668ED4702 looks to be https://bugs.edge.launchpad.net/launchpad-code/+bug/518337 ?
[22:35] <_mup_> Bug #518337: timeout oopses on person page <oops> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/518337>
[22:35] <benji> made one for "bug LP_BUG_NUMBER" too
[22:36] <james_w> I have that one, plus a bunch of others, but never thought of doing it for oopses
[22:45]  * flacoste is away: Gone away for now