[00:31] <wgrant> Are we out of testfix yet?
[00:33] <jelmer> we were for a bit but I missed the window
[00:33] <spm> wgrant: possibly. I've kicked/forced a bunch of builds, so we remain optimistic.
[00:33] <jelmer> not sure if we're out of textfix again
[00:34] <spm> hey jelmer!
[00:34] <jelmer> 'morning spm!
[00:34] <wgrant> Morning spm, jelmer.
[00:35] <wgrant> Thanks.
[00:36] <spm> hey wgrant
[00:40]  * thumper is getting sad writing the current email
[00:41] <thumper> I'm realising how far behind we are on bzr bits in LP
[00:46] <jelmer> thumper, behind in terms of the version you mean?
[00:47] <thumper> jelmer: yeah
[00:47] <thumper> we are very behing in bzr, dulwich, bzr-git, bzr-svn and bzr-hg
[00:47] <thumper> prolly bzr-loom too
[00:48] <jelmer> thumper: well, the last three have pretty much stalled for the last couple of months
[00:48] <thumper> there have still been a number of commits though
[00:52] <mtaylor> rockstar: hey-ho
[00:52] <thumper> mtaylor: I think rockstar is travelling
[00:52] <mtaylor> thumper: dammit
[00:52] <mtaylor> thumper: I need him to fix things in tarmac for me
[00:53] <mtaylor> thumper: it's discarding tag info, even though we thought the other day that it shouldn't
[02:31] <wgrant> We're out of testfix, then?
[02:32] <thumper> wgrant: I'm trying to land
[02:32] <thumper> I think the force builds take us out of testfix
[02:33] <wgrant> Well, mwhudson landed something a few minutes ago.
[02:33] <wgrant> So it seems to work.
[02:36] <mwhudson> ah cool, that landed
[02:38] <wgrant> Hm.
[02:38] <wgrant> When did bin/test start spitting out per-test times?
[02:38] <mwhudson> it's when you have -vvv
[02:38] <wgrant> Maybe I just added more -v than usual.
[02:38] <wgrant> Ah.
[03:17] <wgrant> Can someone please land lp:~wgrant/launchpad/rework-dominator-tests?
[03:26] <thumper> mwhudson: did the email make reasonable sense?
[03:26] <mwhudson> thumper: yes
[03:26] <thumper> cool
[03:27] <mwhudson> thumper: although i guess i had a head start on understanding :-)
[03:28] <thumper> :)
[03:30]  * thumper afk to pickup
[03:59] <wgrant> Some of the queries in lib/lp/archivepublisher/domination.py are extended versions of some on ArchiveSet. In a normal application, I'd call the ArchiveSet method and then .find() on the ResultSet to do further filtering.
[03:59] <wgrant> But IResultSet doesn't contain .find(), so I can't do that in LP.
[03:59] <wgrant> Help?
[04:23] <thumper> wgrant: I think perhaps an updated storm has it...
[04:24] <thumper> not sure though
[04:24] <thumper> could ask on #storm
[04:29] <wgrant> thumper: Thanks, I've asked there.
[04:29] <wgrant> thumper: Could you be convinced to ec2 https://code.edge.launchpad.net/~wgrant/launchpad/rework-dominator-tests/+merge/29668 and https://code.edge.launchpad.net/~wgrant/launchpad/link-uploaded-ddebs/+merge/29669?
[04:29] <thumper> wgrant: yep
[04:30] <wgrant> Thanks.
[04:56] <lifeless> moin
[04:58] <wgrant> Morning lifeless.
[04:59]  * wgrant wishes that the 3.0 heading guidelines were complete.
[05:00] <lifeless> ...?
[05:02] <wgrant> lifeless: Page titles were redesigned for 3.0.
[05:03] <wgrant> But they did not consider how it should work when on a page with a non-root context.
[05:03] <wgrant> The root context is always shown at the top of the page.
[05:03] <wgrant> But say I'm on a page relating to a PPA. It has the owner's name at the top, and no built-in indicator of which PPA it's operating on.
[05:03] <wgrant> What should the h1 be?
[05:03] <wgrant> 'Administer', 'Administer archive', 'Administer PPA for William Grant'?
[05:04] <lifeless> oh
[05:04] <lifeless> is this a widely known issue ?
[05:04] <lifeless> who was doing the work, can we ask them to finish it off? Or is there a good answer ppl are just doing ?
[05:04] <wgrant> I've raised it a couple of times in the last 12 months.
[05:04] <wgrant> There's no consistency at the moment.
[05:05] <lifeless> when you raised it, what happened; was there some sort of consensus ?
[05:05] <wgrant> Nothing really came of it.
[05:05] <wgrant> There's also the whole mess around how the tabs work for non-root contexts.
[05:05] <wgrant> The issue was identified somewhat before 3.0, but never resolved.
[05:05] <wgrant> Resulting in a confusing UI for non-projects.
[05:06] <lifeless> is there something you'd like to do about it ?
[05:07] <lifeless> actually
[05:07] <lifeless> a better question
[05:07] <lifeless> do you have a proposal that would improve consistency and be, on the whole, more useful for users.
[05:07] <lifeless> and not be terrible to calculate perf wise.
[05:07] <wgrant> I don't know what would be best.
[05:07] <lifeless> I'm not talking best.
[05:07] <lifeless> best is freaking hard man.
[05:08] <lifeless> I think some consistency is good - useless consistency isn't. We need to do each page really well, but a sane default can help a great deal.
[05:14] <wgrant> Even some pages on root contexts are pretty bad.
[05:14] <wgrant> https://code.edge.launchpad.net/launchpad/+activereviews, for example.
[05:14] <lifeless> is there some difficulty in fixing them?
[05:14] <wgrant> They're trivial to fix, once we define what they should be like.
[05:14] <lifeless> uh
[05:14] <lifeless> why do we need a pre-fix definition here ?
[05:14] <lifeless> *anything* is trivial if you can define it :P
[05:14] <wgrant> If we are changing titles to suck less and be more consistent, we need to define what they are going to be consistent with.
[05:14] <wgrant> 3.0 tried to do that, but failed.
[05:14] <lifeless> perhaps I'm over simplifying
[05:14] <lifeless> but - a) suck less, b) be more consistent : seems we could make them suck less by /doing/ and make them more consistent by picking the nicest pattern that emerges and using it as a sane default ?
[05:14] <wgrant> Possibly.
[05:14] <lifeless> wgrant: if the only consistent thing at the moment is that they suck, surely we should fix that :)
[05:16] <lifeless> we have a ui review process, so mistakes should be caught - thats a decent safety net
[05:17] <wgrant> You can't make a mistake at the moment.
[05:17] <wgrant> Because there are no guidelines to break.
[05:18] <wgrant> I think we just need to work out the undefined general cases, and define them.
[05:19] <wgrant> This may require a change to the structure of the header.
[05:34] <lifeless> wgrant: You seem to be saying folk can't improve things here without a lot of work, which surprises me,
[05:34] <lifeless> is that right?
[05:35] <wgrant> lifeless: It can be mostly fixed by someone just declaring how much context information should go in titles.
[05:35] <wgrant> A complete fix (which fixes tabs for non-root contexts) is harder.
[05:36] <lifeless> so if I say 'the right amount to look good on a page should go there', is that useful, or am I *still* missing the point
[05:38] <thumper> wgrant: both your branches playing through ec2
[05:40] <wgrant> lifeless: How much looks good on a page?
[05:40] <wgrant> And is that amount enough?
[05:40] <wgrant> thumper: Thanks.
[05:47] <lifeless> wgrant: that seems to be a matter of judgment and testing
[05:47] <lifeless> wgrant: You don't seem to have a hesitation doing that to lp internals; I'm unclear why you have a hesitation doing the same to page templates :)
[05:47] <wgrant> lifeless: Precisely.
[05:48] <wgrant> lifeless: I am no UI designer.
[05:48] <wgrant> LP has a severe lack of UI design
[05:48] <wgrant> And it shows.
[05:49] <lifeless> are you saying we lack the knowledge to improve at all?
[05:50] <wgrant> No. I'm saying that somebody with significant user experience should be consulted before we go and fix all the titles to conform to a new standard that might still suck.
[05:51] <lifeless> wgrant: so - +1 on getting more import. -1 on doing /nothing at all/
[05:51] <lifeless> s/import/input
[05:51] <wgrant> We have no UI designer. How do we get more input?
[05:52] <lifeless> we have a whole design team on call, and they are extremely happy to do small things on-request.
[05:53] <lifeless> I'm not at all convinced that this is a problem we lack the knowledge for though. However, I'll grab mpt today and show him this chat and ask his opinion.
[05:53] <wgrant> An attempt was made a year ago.
[05:53] <wgrant> It did not work.
[05:53] <lifeless> wgrant: so try something different
[05:53] <wgrant> We can certainly make large improvements easily.
[05:53] <wgrant> But it would be nice to get it right.
[05:53] <lifeless> great!
[05:54] <lifeless> the thing about right.
[05:54] <lifeless> is that it lasts for, oh, 2-3 days.
[05:54] <wgrant> OK, as close to right as we can easily get.
[05:54] <lifeless> :)
[05:55]  * bryceh goes close to left
[05:55] <lifeless> bryceh: any progress on the attachments thing?
[05:56] <bryceh> hmm?
[05:56] <lifeless> the api call
[05:56] <lifeless> in bugs
[05:56] <lifeless> is there a partial branch or something like that to make it faster? If not I'll have a stab at same today.
[05:58] <bryceh> nope, I'm trying to get another branch finished up before I go on vacation
[06:03] <lifeless> ok
[06:17] <lifeless> losa hi
[06:23] <lifeless> spm: hi ?
[06:29] <spm> lifeless: hola
[06:38] <lifeless> how long till edge rolls out ?
[06:38]  * lifeless is excited
[06:39] <thumper> lifeless: excited about what?
[06:40] <spm> lifeless: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus "Edge Updates" 0800 BST. we strikethru if they're not for some reason or another
[06:40] <lifeless> thumper: rev 11199 on stable
[06:41] <lifeless> spm: kk
[06:41] <thumper> lifeless: which has what?
[06:41] <lifeless> a /massive/ search overhead reduction
[06:41] <thumper> ah
[06:41]  * thumper EOWs
[06:43] <lifeless> have a good weekend
[07:07] <lifeless> spm: so, another hour right ? :)
[07:07]  * lifeless bounces
[07:10] <bilalakhtar> lifeless: Are you a core-dev in ubuntu?
[07:11] <spm> lifeless: pretty much exactly, yeah
[07:11] <lifeless> bilalakhtar: no, why?
[07:12]  * bilalakhtar is searching for core-devs to sponsor his bugfixes
[07:12] <bilalakhtar> lifeless: ^^
[07:12] <lifeless> put it in the sponsor queue please
[07:56] <lifeless> spm: whats the url for monitoring edge rollout progress again ? I didnae bookmark it
[07:57] <spm> lifeless: we don't have one
[07:57] <lifeless> oh
[07:57] <lifeless> thats right :)
[07:57] <lifeless> we have one for staging or so or something
[07:58] <spm> yup
[07:58] <spm> https://staging.launchpad.net/successful-updates.txt <== tho is more "worked/not" vs progress
[08:00] <lifeless> so how is edge going ? :)
[08:00] <lifeless> <- not keen at all
[08:01] <danilos> OOPSes while trying to log in, a known problem? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1665ED841
[08:01] <spm> from memory, it'll be around 5-10 mins before it gets beyond simply building/syncing the tree
[08:02] <lifeless> danilos: ugh
[08:02] <lifeless> spm: we may have to abort - benji's code may be bust
[08:02] <lifeless> spm: don't stop it yet
[08:03] <lifeless> danilos: does your username have utf8 in it ?
[08:03] <danilos> spm, did we get a new openid rollout by ISD guys as well? (just to confirm we only have to look at our code for the problem)
[08:03] <spm> lifeless: not really; if it's a hard fail; edge1 will break and the rollout will stop
[08:03] <danilos> lifeless, username doesn't, but my name does
[08:03] <spm> if it's more subtle than that; rolling back is trivial
[08:03] <lifeless> spm: actually, thats already on edge. I was thinking a prod stop-point
[08:03] <lifeless> <- EUNCAFFEINATED
[08:04]  * danilos disables edge redirection and tries again
[08:04] <wgrant> lifeless: Is it complaining about the XSRF token sent by SSO?
[08:04] <wgrant> I can't see the OOPS, but there's already a branch to fix that.
[08:04] <lifeless> wgrant: no, unicode
[08:04] <wgrant> Ah.
[08:07] <lifeless> danilos: file a bug on foundations
[08:08] <danilos> lifeless, right, my name is the culprit, I've changed it and can log in
[08:08] <danilos> lifeless, prod also works, so it's edge code
[08:08] <lifeless> its a classic 'this code wasn't QA'd' scenario :-
[08:09] <lifeless> which the new deployment process should address
[08:11] <danilos> lifeless, I'd also lean on marking this one as critical, but will leave that to whoever gets a chance to take care of it
[08:11] <danilos> anyway, bug 609029
[08:11] <_mup_> Bug #609029: UnicodeDecodeError OOPS on login on edge <oops> <Launchpad Foundations:New> <https://launchpad.net/bugs/609029>
[08:12] <lifeless> danilos: I agree
[08:18] <lifeless> spm: hows edge, and how is the db load looking
[08:20] <spm> lifeless the-impatient ;-) fwiw, your email will probably know before I do, unless something breaks hard. error reports: Subject: 	Cron <pqm@praseodymium> /home/pqm/bin/rollout_edge.sh will have the details
[08:21] <lifeless> spm: yes, its live
[08:22] <spm> load looks normal, as a one off snapshot ish.
[08:24] <spm> looks like edge4 is/has just been updated
[08:24] <spm> has
[08:26] <lifeless> ok, https://bugs.edge.launchpad.net/ubuntu/+filebug does seem better on edge now
[08:27] <mrevell> Howdy
[08:33] <lifeless> spm: ok, still looks ok ?
[08:34] <spm> lifeless: I assume so, not getting any alerts; which is the usual 1st line
[08:35] <lifeless> k, thanks
[09:35] <wgrant> Does PQM really take half an hour?
[09:45] <jkakar> FYI, I'm getting 'Internal Server Error' here: http://bazaar.launchpad.net/~jkakar/storm/resultselect/files
[09:54] <spiv> jkakar: that branch is 1.9 format, but is stacked on trunk which is 2a
[09:54] <spiv> jkakar: upgrade it and it'll be fine
[09:55] <jkakar> spiv: Yeah, I figured as much...
[09:55] <jkakar> spiv: Regardless, I shouldn't see "Internal Server Error".
[09:55] <spiv> Oh, certainly.
[09:55] <spiv> I *think* there's a bug about that already, feel free to click "affects me too" on it.
[09:56] <jkakar> Cool.
[10:37] <wgrant> bigjools: Are there really no tests of NascentUpload that don't use a real upload?
[10:38] <wgrant> I can't see any :(
[10:38] <wgrant> I guess I'll work it out.
[10:40] <bigjools> wgrant: probably not :(  I know there's some unit tests for dscfile
[10:40] <bigjools> that's about it
[10:41] <bigjools> but it's the direction we need to go so that soyuz tests don't take 35-40 minutes. :/
[10:41] <wgrant> Yeah, I've started shuffling things around so it's more possible.
[10:41] <wgrant> archiveuploader is really ugly.
[10:43] <bigjools> ugly with boils
[10:58] <lifeless> wgrant: ok, mpt review done, checklist being fixed.
[10:59] <lifeless> its like the title but less so :P
[11:06] <wgrant> lifeless: Ooh, thanks.
[11:15] <mpt> lifeless, https://dev.launchpad.net/UI/Reviews?action=diff&rev2=34&rev1=33
[11:17] <wgrant> mpt: Thanks.
[11:17] <wgrant> So we're to include context in the h1, even if it's the root context?
[11:17] <wgrant> (which is already displayed at the top of the page)
[11:17] <lifeless> mpt: thanks!
[11:18] <lifeless> wgrant: mpt said verbally that folk ignore the global nav stuff, universally
[11:18] <lifeless> perhaps thats worth adding
[11:19] <wgrant> In the root context case, it's in the breadcrumbs and header already.
[11:20] <mpt> wgrant, remind me what "the root context" means?
[11:21] <wgrant> mpt: The pillar or person containing this page.
[11:21] <wgrant> The context may be a source package, but the root context is the distribution.
[11:22] <wgrant> It's displayed at the top of every page.
[11:22] <wgrant> (I believe the concept and term were introduced with 3.0)
[11:22] <lifeless> wgrant: the point is people don't read that stuff
[11:23] <lifeless> wgrant: mpt: anyhow: if we can transfer the info so that wgrant is unblocked, I will be ecstatic. I think the page improvement so far is very useful.
[11:24] <wgrant> It looks like a good idea.
[11:25] <mpt> Ah, I should have mentioned another thing
[11:25] <mpt> More specific stuff should go earlier in the heading and the title
[11:25] <mpt> less specific stuff later
[11:26] <wgrant> That's ensured in the title, since it's reversed breadcrumbs.
[11:26] <wgrant> Although that does result in oddities like "10.04 : Ubuntu", which I'm not really a fan of.
[11:27] <mpt> yeah, reversed breadcrumbs are an unfortunate case of automation over sensibility
[11:27] <wgrant> I think they are better than what came before them.
[11:27] <wgrant> But not better than a fully redesigned set of custom titles.
[11:28] <mpt> They may be better technically, in that pagetitles.py was hilarious
[11:28] <wgrant> There is at least some amount of consistency now.
[11:28] <wgrant> pagetitles.py is almost gone.
[11:28] <mpt> But at least pagetitles.py let me eyeball inconsistencies.
[11:28] <stub> lifeless: Have you looked at garbo.py? It is a framework suitable for many of our hourly and daily tasks - the ones that don't need external resources besides the database.
[11:30] <stub> lifeless: So for instance I'll be suggesting to jtv that the cachesuggestivetranslations.py cronjob should be a task in the garbo, avoiding yet-another-cronscript-to-manage as well as bonuses such as aborts if it takes too long and fewer notifications for people to monitor.
[11:31] <stub> lifeless: It needs better plugin architecture, and it isn't a replacement for a general job handler, but I think a step in the right direction.
[11:38] <lifeless> stub: nice
[11:38] <lifeless> stub: +1
[11:39] <lifeless> stub: I do think jtv's particular case would be better served by events, but the existing event queuing things are not nice enough to draw in adopters yet
[11:40] <stub> I don't know enough of the task system or the suggestions to know if that is a suitable fit.
[11:41] <lifeless> stub: making garbo replace most of the existing cron jobs would help the sysadmins a lot I think
[11:42] <stub> Yup. Also makes scheduling nicer - we don't end up with batch jobs running simultaneously causing load spikes.
[11:46] <lifeless> anyohe that has oopstools working - could you see if https://bugs.edge.launchpad.net/oops-tools/+bug/608914 has a working patch ?
[11:46] <lifeless> jelmer: wb, did you see my reply?
[11:48] <_mup_> Bug #608914: Bugs not shown in the Timeouts section of oops summaries <OOPS Tools:Triaged> <https://launchpad.net/bugs/608914>
[12:05] <jelmer> lifeless: yep, thanks
[12:12] <poolie> james_w, istm that <https://edge.launchpad.net/ubuntu/+source/bzr> (for example) ought to link to the packaging branches
[12:12] <poolie> if any
[12:12] <poolie> or tell us there are none
[12:13] <james_w> yes
[12:13] <james_w> "Code", but yes
[12:22] <rockstar> Morning folks!
[12:22] <bigjools> hey rockstar
[12:30] <rockstar> bigjools, what's the status of the maverick buildd stuff?
[12:33] <bigjools> rockstar: still buggered as far as I know.  Are you referring to the aptitude thing?
[12:33] <rockstar> bigjools, yeah,
[12:33] <rockstar> bigjools, is there anything I can do to help that along?  I have some other tasks, but that's one of the blockers for jorge getting upstreams to use it.
[12:34] <rockstar> (I'm going to work on the other one today)
[12:34] <bigjools> rockstar: I don't know, I am not dealing with it.  Were you expecting me to be?
[12:35] <bigjools> it's a problem with the recipe stuff in the slave code
[12:35] <rockstar> bigjools, yeah, I thought that you were going to get abentley's patch applied to the buildd to fix it.
[12:35] <bigjools> I didn't know there was a patch :)
[12:35] <bigjools> we're currently having problems getting any recipe build to go through dogfood - they all hang.
[12:36] <rockstar> bigjools, okay, lemme chat with abentley when he's around, and we'll sort it out.  We need to make that a priority.
[12:36] <rockstar> bigjools, :(
[12:36] <bigjools> rockstar: see this for example: https://code.dogfood.launchpad.net/~abentley/+recipe/bazaar/+build/263/+files/buildlog.txt.gz
[12:36] <bigjools> "bzr: ERROR: No previous changelog to take the package name from, and --package not specified"
[12:36] <bigjools> wtf?
[12:37] <bigjools> rockstar: ok when you get know where the patch is, we can ask lamont to deploy it on the DF builders.
[12:37] <rockstar> bigjools, yeah, I was just about to point that out.
[12:37] <rockstar> bigjools, ack.  Thanks.
[12:37] <bigjools> it takes 10 minutes to print that error out as well
[12:38] <rockstar> bigjools, since you said they were hanging, I wondered if that was an artifact of killing the build.
[12:38] <rockstar> james_w, ping
[12:38] <bigjools> no, I didn't kill it
[12:39] <rockstar> bigjools, we also have some sourcecode/ updates that need to be applied to fix some more bugs.
[12:39] <rockstar> bigjools, do you have any comments on fixing https://bugs.edge.launchpad.net/launchpad-code/+bug/583403
[12:39] <_mup_> Bug #583403: Sourcepackage recipe builds need much more logging <recipe> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/583403>
[12:41] <bigjools> rockstar: yeah I think the logging needs to come from bzr
[12:41] <rockstar> bigjools, okay.  Are those logs just capturing stdout then?
[12:41] <bigjools> yes
[12:41] <rockstar> bigjools, great.  I'll get a patch together and have james_w review it.
[12:42] <bigjools> nyshe
[12:42] <bigjools> rockstar: we're not going to get far until we figure out that long hang + the error above though :(
[12:42] <rockstar> bigjools, yeah, what resources do you need in order to do that?
[12:43] <bigjools> rockstar: james_w :)
[12:43] <bigjools> it's running stuff I know nothing about unfortunately
[12:43] <wgrant> bigjools, rockstar: That recipe is buggy.
[12:44] <bigjools> well that's a good start.
[12:44] <wgrant> It should be a merge, not a nest.
[12:44] <rockstar> bigjools, okay, well I'll try and reproduce it here.  james_w is kind of going into maintenance mode on some of these udd tasks, and I offered to pick up maintenance for them.
[12:44] <bigjools> it should fail a bit quicker though
[12:44] <rockstar> (Of course, that means I need to learn them first)
[12:44] <wgrant> How long does it hang?
[12:44] <bigjools> 10 minutes
[12:44] <wgrant> Interesting.
[12:44]  * bigjools fixes the recipe
[12:45] <wgrant> bigjools: Just s/nest/merge/, and drop the 'debian' from the end.
[12:45] <rockstar> That might be a bzr stupidity.
[12:46] <rockstar> Like bzr builds a whole working tree in memory just to get the single folder, or something.
[12:47] <bigjools> dispatching the new recipe now
[12:47] <bigjools> wgrant: btw I have a new buildd-manager on DF.
[12:47] <bigjools> should be fun to see if it breaks production :)
[12:47] <wgrant> Heh.
[12:47] <wgrant> As in the new properly parallel one?
[12:47] <bigjools> testing so far shows no issue
[12:48] <bigjools> yes
[12:48] <wgrant> Now we just need to convince it to dump uploads in a queue, and the problem is mostly fixed!
[12:48] <bigjools> I do like Twisted.  I don't like the b-m code so much.
[12:48] <bigjools> yes, jelmer is working on that :)
[12:48] <wgrant> Excellent.
[12:51] <wgrant> I've also ported lp-buildd to use a modern system version of sbuild, so we can drop our unmaintained ancient buggy fork once Soyuz does ddebs.
[12:51] <wgrant> So we should have a much healthier, more maintainable build farm soon, on multiple fronts...
[12:52] <lifeless> losa ping
[12:52] <rockstar> wgrant, this, my friend, is most excellent news.
[12:52] <lifeless> http://pastebin.com/e9D6x3hX <- I'd like timing data for that query on each DB host please.
[12:52] <mthaddon> lifeless: hi
[12:52] <bigjools> wgrant: \o/ \o/ \o/
[12:52] <wgrant> Although I had to destroy buildrecipe in the process.
[12:52] <rockstar> wgrant, now, if only we could have test coverage for the build farm.
[12:52] <poolie> james_w, https://bugs.edge.launchpad.net/launchpad-registry/+bug/609116
[12:52] <_mup_> Bug #609116: package page should point to package branches (or not) <udd> <Launchpad Registry:New> <https://launchpad.net/bugs/609116>
[12:53] <wgrant> rockstar: Translations made a good start on testing their part of it.
[12:53] <lifeless> it the literal query running on filebug queries on edge; its timing out a lot - I'm interested in figuring out if:
[12:53] <lifeless>  - there is one bad host
[12:53] <lifeless>  - there are bad query plans being generated
[12:53] <lifeless> mthaddon: oh, and hai
[12:53] <mthaddon> :)
[12:54] <wgrant> bigjools: Um, where did that recipe go?
[12:54] <wgrant> It has disappeared.
[12:54] <wgrant> I wonder if you ran into a form bug and accidentally reassigned it yourself.
[12:54] <wgrant> Heh, yes, there it is.
[12:54] <bigjools> the log was there briefly
[12:55] <bigjools> wtf
[12:55]  * rockstar waits for the expansion of "form bug"
[12:55] <wgrant> The owner <select> won't have been populated with the previous value, since it was abentley, and bigjools is not abentley.
[12:56] <gmb> rockstar, Could you or a suitable one of your code team cohorts take a look at the currently-failing merge between stable and db-devel please? There's a conflict but I'm not sure whether to just accept the merge-source version or not.
[12:56] <wgrant> So it will have defaulted to the first available value.
[12:56] <rockstar> gmb, sure.
[12:56] <gmb> rockstar, Thanks.
[12:56] <rockstar> gmb, oh shit, if this is what I thinks it is...
[12:56] <bigjools> wgrant: https://code.dogfood.launchpad.net/~julian-edwards/+recipe/bazaar/+build/265
[12:57] <bigjools> it's hanging again
[12:57] <gmb> ...
[12:57] <rockstar> bigjools, it might just be that it's not logging anything properly.
[12:57] <wgrant> bigjools: Ew.
[12:58] <wgrant> That is possible.
[12:58] <rockstar> bigjools, when I hear "hang" I think "this will not complete" but it gets back to going in a bit.
[12:58] <bigjools> that is not just possible, it's highly likely
[12:58] <wgrant> bigjools: Can you check the CPU usage?
[12:58] <bigjools> no :(
[12:58] <bigjools> the VMs don't let you ssh in
[12:58] <rockstar> It generates an error message that makes no sense, this is true.
[12:58] <wgrant> The error message made perfect sense.
[12:59] <bigjools> we need to make this run quicker, this is way too slow
[12:59] <wgrant> It probably won't occur this time, though.
[12:59] <rockstar> bigjools, maybe we should find a way to keep metrics on the buildd CPU/mem usage.
[12:59] <rockstar> I'm sure lifeless is all for it.
[12:59] <bigjools> aaaaaaand fail
[12:59] <bigjools> /usr/lib/pbuilder/pbuilder-satisfydepends: line 92: aptitude: command not found
[13:00] <bigjools> as expected for maverick
[13:00] <wgrant> Better fail, though.
[13:00] <rockstar> Yup.
[13:00] <bigjools> can we just bin aptitude
[13:00] <bigjools> it causes no end of problems
[13:01] <rockstar> bigjools, no idea what the ramifications are.  The people who would know are sprinting currently.
[13:01] <bigjools> pbuilder could just use apt-get and this problem would disappear
[13:01] <wgrant> Well.
[13:02] <wgrant> aptitude resolves some things differently.
[13:02] <wgrant> It used to be vastly superior to apt-get.
[13:02] <wgrant> Not so much any more, though.
[13:02] <rockstar> s/differently/stupidly/
[13:02] <bigjools> where differently is wrongly, so I'm told :)
[13:02] <wgrant> Indeed.
[13:02] <wgrant> But there was rationale for using it in the past.
[13:08] <jelmer> wgrant: also, it doesn't have moo powers.
[13:09] <wgrant> jelmer: Heh.
[13:09] <bigjools> "apt-get moo" is right up there with "bzr rocks"
[13:09] <poolie> yeah i think the moment has passed
[13:10] <Ursinha> bigjools, oh crap, I didn't know about that
[13:10] <Ursinha> :)
[13:11] <wgrant> bigjools: aptitude's moo is better...
[13:11] <bigjools> heh
[13:12] <bigjools> wgrant: ok the lucid build is working now
[13:13] <bigjools> I wonder what it's doing that takes so long
[13:13] <wgrant> It took 10 minutes to check out from here.
[13:13] <bigjools> hmmm, I wonder if it needs a --lightweight
[13:14] <bigjools> anyway, food time
[13:15] <wgrant> A lightweight checkout might actually be a reasonable idea here.
[13:15] <wgrant> Hmmmm.
[13:15] <wgrant> We should test that.
[13:17] <lifeless> ok
[13:17] <lifeless> so fti can do the search we need in 4ms
[13:17] <lifeless> but if you rank, you're fucked
[13:18] <lifeless> and if you join out from bug, you're in trouble too
[13:19] <wgrant> So basically, we're doubly in trouble?
[13:20] <james_w> rockstar, bigjools: if you update your copy of bzr-builder you will get more logging
[13:20] <lifeless> just as a noddy example:
[13:20] <lifeless> SELECT bug.private FROM Bug, BugTask WHERE Bug.id = BugTask.bug AND BugTask.distribution = 1 AND Bug.fti @@ ftq('depend|eclips|error|get|instal|unmet')    LIMIT 40;
[13:20] <james_w> plus an improvement to that previous error message
[13:21] <lifeless> Time: 4.552 ms
[13:21] <james_w> and many other bugfixes too
[13:21] <rockstar> james_w, awesome.  I need to find out how to upgrade sourcecode/ then.
[13:21] <lifeless> SELECT bug.private FROM Bug, BugTask WHERE Bug.id = BugTask.bug AND BugTask.distribution = 1 AND Bug.fti @@ ftq('depend|eclips|error|get|instal|unmet')  ORDER BY -rank(Bug.fti, ftq('depend|eclips|error|get|instal|unmet'))  LIMIT 40;
[13:21] <lifeless> Time: 7467.015 ms
[13:21] <rockstar> james_w, a bzr-builder and bzr-builddeb update are both in my TODO list for today.
[13:21] <wgrant> rockstar: sourcecode won't help the builders.
[13:21] <james_w> true
[13:21] <rockstar> wgrant, I understand that, but it gives me a start.
[13:22] <james_w> it will still be useful for other fixes though
[13:22] <james_w> where in the log does the 10 minute hang happen?
[13:22] <rockstar> To deploy the new changes, I can just put a new package in a ppa (location still unknown to me) and then have lamont update.
[13:22] <rockstar> bigjools, see james_w's question. ^^
[13:24] <wgrant> james_w: Immediately after the launchpad-login warnings.
[13:24] <james_w> ok
[13:25] <rockstar> james_w, I think it's actually in the builder part at that point, so if we have better logging, we might not have to worry about it anymore.
[13:28] <james_w> rockstar: it's still in bzr-builder I think, but I believe that it may just be slow bzr access over http
[13:29] <rockstar> james_w, yeah, that's what I was thinking.
[13:29] <james_w> rockstar: the new bzr-builder should help with that some as well
[13:29] <rockstar> james_w, yup.  Looking to update it today.
[13:31] <wgrant> Is there a good reason for buildrecipe to be a shell script?
[13:31] <wgrant> I need to move some of it into the Python part of the slave.
[13:32] <james_w> wgrant: none that I know of
[13:32] <wgrant> And it all ends up simpler (particularly the error handling) if the whole thing is integrated into the slave Python.
[13:32] <lifeless> SELECT bug.private, 1 as therank FROM Bug, BugTask, to_tsquery('depend|eclips|error|get|instal|unmet') query WHERE query @@ bug.fti and Bug.id = BugTask.bug AND BugTask.distribution = 1 ORDER BY therank DESC LIMIT 40 OFFSET 0;
[13:33] <lifeless> Time: 4.171 ms
[13:33] <lifeless> SELECT bug.private, ts_rank(bug.fti, query) as therank FROM Bug, BugTask, to_tsquery('depend|eclips|error|get|instal|unmet') query WHERE query @@ bug.fti and Bug.id = BugTask.bug AND BugTask.distribution = 1 ORDER BY therank DESC LIMIT 40 OFFSET 0;
[13:33] <lifeless> SELECT bug.private, ts_rank(bug.fti, query) as therank FROM Bug, BugTask, to_tsquery('depend|eclips|error|get|instal|unmet') query WHERE query @@ bug.fti and Bug.id = BugTask.bug AND BugTask.distribution = 1 ORDER BY therank DESC LIMIT 40 OFFSET 0;
[13:33] <lifeless> bah
[13:33] <lifeless> Time: 5626.790 ms
[13:33] <lifeless> ok
[13:33] <lifeless> plane time
[13:33] <lifeless> mthaddon: thanks a lot for the stats - its useful
[13:33] <mthaddon> cool
[13:33] <wgrant> I guess that sort of makes sense.
[13:34] <james_w> wgrant: there's a use for process separation I think, but the language doesn't have an impact on that
[13:35] <wgrant> james_w: Given that it pretty much just executes a few subprocesses anyway, I don't see much value in having another layer there.
[13:35] <jtv> danilos, btw: a very simple small improvement I discussed w arne & dpm is to discard blocked ubuntu PO uploads after a year.  If there's ever a new upload they'll probably just get re-blocked anyway, but it cuts down the queue by 40%
[13:36] <danilos> jtv, yeah, sounds good
[13:36] <james_w> wgrant: right, it's a small window of exceptions that would affect that. Making it not run builder as a subprocess and call the python functions instead probably isn't a good idea though
[13:36] <jtv> also, arne came up with a great solution for some other process problems: when they approve an upload, do an instant approval run over other entries it's likely to affect.
[13:37] <wgrant> james_w: Oh, certainly. Those will continue to run as subprocesses. But buildrecipe will be merged into sourcepackagerecipe.py.
[13:37] <danilos> jtv, sounds so simple ;)
[13:38] <james_w> wgrant: +1
[13:38] <danilos> jtv, (ok, it's not too hard if "instant" is "in a few minutes" :)
[13:38] <jtv> danilos: the fun is in the fine print...  this would be approvals, not blocking, and probably only needs-review uploads with the same path and for the same package/productseries
[13:39] <wgrant> james_w: Thanks.
[13:39] <jtv> I think that's probably not too hard.
[13:40] <danilos> jtv, hum, language detection, custom language codes, different layouts? blocked is just about the problem of the same complexity so not really a big deal
[13:40] <jtv> danilos: they don't mind if it doesn't work as well for different layouts.  I wouldn't think the rest would be that costly.
[13:41] <jtv> Or how about just the files with the same path?
[13:55] <james_w> hi gary_poster, I requested a review from you of https://code.edge.launchpad.net/~james-w/launchpad/drop-default-skin/+merge/30763 which was a change that salgado discussed with you the other day. Let me know if you would like someone else to review it.
[13:56] <gary_poster> ack, james_w.  I'll look at it in a few
[14:12] <flacoste> gmb: subscribing to a tag is within scope of the better subscription control story right?
[14:13] <gmb> flacoste Yes.
[14:13] <gmb> flacoste, Sorry, just grabbing lunch; I'll answer any questions when I get back.,
[14:13] <flacoste> gmb: no other questions, bon appétit!
[14:17]  * rockstar fetches breakfast
[16:20] <rockstar> bigjools, ping?
[16:47] <bigjools> rockstar: hi
[16:48] <rockstar> bigjools, we're talking recipes in mumble right now.
[16:49] <rockstar> bigjools, I suppose that was my indirect way of inviting you to talk recipes with us in mumble.
[16:53] <mtaylor> rockstar: morning
[16:53] <rockstar> mtaylor, hello sir
[16:53] <rockstar> mtaylor, are you on the west coast?
[16:54] <mtaylor> rockstar: I am.
[16:54] <mtaylor> rockstar: I have some fun tarmac things to poke you about - and now I'm in the right time zone to do it!
[16:55] <rockstar> mtaylor, ah, yes, I am home now, so we're only an hour difference.
[16:55] <mtaylor> w00t
[16:55] <mtaylor> rockstar: well... the most annoying one is that, as lifeless was hinting might be the case, tags are not propogating
[16:55] <rockstar> mtaylor, hm...
[16:55] <rockstar> mtaylor, is there a bug?
[16:57] <mtaylor> rockstar: not yet
[16:57] <mtaylor> rockstar: I made a quick list of bugs to file, which I will do this morning
[16:57] <rockstar> mtaylor, well how do I fix a bug I don't know about?  :)
[16:58] <mtaylor> rockstar: I know I know
[16:58] <rockstar> mtaylor, also, there is #tarmac, that actually has people in it, if you get stuck and I'm not around.
[17:13] <rockstar> gary_poster, is there a doc in the launchpad that tells me how I can update stuff in sourcecode/
[17:14] <gary_poster> rockstar: not that I'm aware of, unfortunately, but the conf file is pretty obvious.  Lemme find it for you (somewhere in utilities...)
[17:14] <gary_poster> sourcedeps.conf
[17:15] <benji> is there a way I can get a full diff between the code on edge and prod?
[17:15] <gary_poster> rockstar: utilities/sourcedeps.conf .  Take a look and then feel free to ask questions
[17:18] <gary_poster> benji: um, yeah...you could do ``bzr diff --new=(edge) --old=(prod)`` where (edge) == lp:launchpad/stable and (prod) == lp:launchpad/production or something close to that, but that sounds to me like you're going to have an awfully unwieldy diff.  I can see why you'd ask though (the unicode login thing, I suspect)...thinking if there might be another way to approach it...
[17:19] <benji> yeah; it turns out that I can't reproduce it in development and I can't imagine how my recent changes caused it
[17:20] <benji> the tests for this corner of the world are pretty screwy; so much stuff is faked out that very little of the real code runs, and when it does run it often takes different code paths than in actual use
[17:24] <gary_poster> benji, you've duped on edge or (mildly better because then you are not changing the real db) staging?
[17:24] <benji> yep, I can dupe on edge
[17:25] <benji> I just had to change my login.launchpad.net user name slightly, so no DB change worries
[17:26] <gary_poster> the fact that you can't dupe locally (launchpad.dev) is concerning
[17:27] <benji> gary_poster: the problem is that it's not possible to dupe in dev, not that the error doesn't exist; there are no test users with unicode in their names and the prod OpenID provider doesn't entirely work with dev
[17:34] <gary_poster> benji, oh, I see.  So, to dupe, you would need to have a new user (or modified user) that has the state.  ...I *think* we have some helpers now to create users on the fly
[17:36] <gary_poster> benji: utilities/make-lp-user might or might not help
[17:36] <benji> k, thanks
[17:41] <rockstar> gary_poster, okay, so I just submit a merge against the pqm managed branch and then update the sourcecode.conf file with the new revno?
[17:41] <gary_poster> rockstar: right
[17:42] <rockstar> gary_poster, okay, thanks.
[17:42] <gary_poster> np
[17:56] <rockstar> james_w, ping
[17:56] <james_w> hi rockstar
[17:57] <rockstar> james_w, abentley tells me you're making packages for bzr-builder and bzr-builddeb to be used in the buildds (so I don't have to make those).  Is this true?
[17:57] <james_w> I did last time
[17:57] <rockstar> james_w, I'm happy to make them and put them in the right place if you can provide me some details.
[18:08] <james_w> rockstar: bzr-builddeb is a native package, and I usually upload via Debian, it's easy to tweak the version number and build a pre-release package though
[18:09] <rockstar> james_w, I'm more concerned about bzr-builder's status more than anything now.
[18:09] <james_w> rockstar: I'm just checking if there is anything I should fix before doing a bzr-builder release. If not then I will do that and we can get a package in to its PPA.
[18:11] <james_w> I wanted to get the merge-subdirs change in, but it is blocked right now, so I will do a release now, and then one with it when it lands
[18:25] <rockstar> james_w, I think the big bzr-builder bug we need the fix for is the new merge instruction.
[18:32] <statik> hola launchpad hackers
[18:32] <statik> i'm trying to install launchpad-dependencies on maverick, and can't find the python-profiler package that is wanted
[18:33] <statik> is this something i could fix by rebuilding a package for maverick in the PPA?
[18:38] <james_w> rockstar: https://edge.launchpad.net/bzr-builder/trunk/0.3
[18:38] <james_w> statik: you need multiverse enabled, at least on lucid
[18:38] <rockstar> james_w, did you mean to have that file start with .. ?
[18:39] <james_w> rockstar: hah, no, I obviously don't have the script with that bug fixed.
[18:39] <statik> james_w, ah thx, i will check that i have multiverse enabled
[18:39] <rockstar> james_w, does this have the new instruction in it?
[18:39] <james_w> rockstar: no
[18:40] <rockstar> james_w, :(
[18:40] <james_w> that branch isn't ready yet
[18:40] <james_w> yeah
[18:40] <rockstar> james_w, what do we need to do to get it ready?  What can I do to help it along?
[18:40] <james_w> New tarball uploaded
[18:41] <james_w> https://code.edge.launchpad.net/~spiv/bzr-builder/merge-subdirs-479705/+merge/14979
[18:43] <james_w> rockstar: kicking it off to build into the PPA
[18:43] <rockstar> james_w, awesome, thanks.
[18:44] <james_w> now I'm heading out for dinner
[18:45] <james_w> the packages should show up here in a bit: https://edge.launchpad.net/~dailydebs-team/+archive/bzr-builder
[18:49] <rockstar> james_w, the package is basically trunk right now, yes?
[19:03] <maxb> statik: Just testing, or actually going to try LP dev on Maverick? Is it time that we should be looking to properly populate the PPA for Maverick?
[19:24] <mars> abentley, ping, have a moment for a quick question about testrepository and testr?  I was wondering what you have in your .testr.conf
[19:25] <abentley> mars, I have the default value except in one branch where I added xvfb-run to the run command.
[19:26] <abentley> mars, right now the lack of incremental output is a real downside, so I haven't been using it much.
[19:26] <mars> incremental output?
[19:26] <mars> like "It runs everything and spits out the buffer at the end"?
[19:27] <abentley> mars, all the stuff about setting up layers, tearing down layers, and individual test runs.
[19:27] <abentley> mars, pretty much, yeah.
[19:29] <mars> abentley, ok, thanks.  Still sounds like it would work well for my one-off "I need to run all windmill tests for failures in a background process" type of work.
[19:29] <abentley> mars, yes, it should be good for that.
[19:29] <mars> many thanks
[19:33] <mars> abentley, ack, next question: I did the obvious thing, "testr run -t test_me_too", and it says "no such option, -t".  Quoting doesn't seem to help.  Any way to pass options right on through?
[19:33] <mars> just a sec, thought of something
[19:33] <mars> maybe '-- -t foo' will work
[19:33] <abentley> mars, use --
[19:33] <mars> alright, "next to least surprising thing" works :)
[19:35] <abentley> mars, once you've had a test run, you'll probably want "testr run --failing"
[19:36] <abentley> mars, though you can also run individual tests through it to knock them off the failing list.
[19:37] <mars> neat
[19:49] <statik> maxb, yes i'm on maverick fulltime and trying to dip my toe back in the water with launchpad dev
[19:51] <maxb> I shall copy some packages and see if they install in a chroot
[19:55] <abentley> mars, (quoting doesn't work because the shell interprets quotes, so testr can't distinguish it from testr run -t\ test_me_too)
[19:56] <maxb> What on earth? I'm trying to copy pocket-lint from lucid to maverick in the launchpad ppa, and getting "binaries conflicting with the existing ones"
[19:56] <maxb> it's a copy with binaries, how can there be a conflict?!
[19:59] <statik> maxb, thanks for looking after this
[20:43] <rockstar> abentley, so, bzr experts apparently can also rescore/cancel builds.  That's why you were seeing the link.
[20:43] <rockstar> (I had forgotten that thumper suggested it)
[20:44] <abentley> rockstar, ah, okay.
[20:52] <mtaylor> daily ppas should be able to be triggered by push to branch
[20:52] <mtaylor> just saying
[22:15] <lifeless> losa ping
[22:19] <mbarnett> heya lifeless
[22:19] <lifeless> hey
[22:19] <lifeless> I want to generate some stats from staging's fti index
[22:19] <lifeless> this will take hours and hours
[22:20] <mbarnett> heh
[22:20] <lifeless> so I'm wondering if someone can do it in a screen session on the box
[22:21] <mbarnett> lifeless: sure.
[22:21] <lifeless> ok, I'll prep the exact thing to run form devpad then pastebin it for you
[22:26] <lifeless> actually
[22:27] <lifeless> its late, I'll skip it and get it organised over the weekend, might see if someone is around tomorrow to kick it off or some such
[22:27] <lifeless> have a good weekend - gotta log off the wifi here
[23:21] <wgrant> benji: You know you can run canonical-identity-provider locally, right?
[23:21] <benji> no!
[23:21] <wgrant> benji: It's not too difficult, and lets you replicate the production setup almost exactly.
[23:21] <wgrant> bzr get lp:canonical-identity-provider
[23:21] <benji> I need to get that going.  Any directions?
[23:21] <wgrant> Then follow the instructions in the tree.
[23:21] <benji> great!
[23:21] <wgrant> It's even open source now.
[23:22] <benji> thanks, that'll make my life much better
[23:23] <wgrant> Then you just have to tweak the LP vhost config to use that rather than login.launchpad.net, configure c-i-p to send full name and email address to launchpad.dev, and it all should work.
[23:24] <benji> sounds good