[00:01] <lifeless> cool
[00:02] <StevenK> Rargh, the lack of tab completion for bzr mv is really really annoying
[00:03] <wgrant> StevenK: It's pushed, anyway.
[00:03] <StevenK> wgrant: I've already commited and pushed my revision, so like you said to me last night: Way ahead of you.
[00:03] <wgrant> Heh
[00:05] <StevenK> I wish lp-open worked with piped branches
[00:05] <nigelb> fuuuu
[00:05] <nigelb> test failuer.
[00:08] <nigelb> and I don't even know whats going wrong.
[00:09] <lifeless> well, what happened ? :)
[00:09] <nigelb> http://paste.ubuntu.com/687980/
[00:10] <StevenK> nigelb: You missed a bit
[00:10] <StevenK> Effectively
[00:10] <StevenK> column "statusexplanation" does not exist\nLINE 11:         SELECT statusexplanation FROM BugTask\n ...
[00:10] <nigelb> FFFFFFFFFFFFFFFFUUUUUUUUUUUUUUU.
[00:10] <nigelb> My devel landing missed something?
[00:11] <StevenK> Looks like
[00:11] <nigelb> *headdesk*
[00:11] <StevenK> nigelb: How did you search for it?
[00:12] <nigelb> grep inside lib/lp I think.
[00:12] <lifeless> I see it in oops.py and oops_prune.py
[00:12] <lifeless> grep lib, not lib/lp
[00:12] <nigelb> Oooooh.
[00:12] <nigelb> Sigh.
[00:12] <StevenK> And look into bzr grep
[00:12] <nigelb> Now I have to land something to devel
[00:12] <lifeless> or . in fact, for complete coverage.
[00:12] <StevenK> It is AWESOME
[00:12] <nigelb> and then re-land
[00:12] <lifeless> bzr grep ++
[00:13] <StevenK> nigelb: lib/canonical/launchpad/scripts/ftests/test_oops_prune.py, lib/canonical/launchpad/scripts/oops.py and lib/lp/bugs/stories/bugs/xx-bug-activity.txt
[00:13] <nigelb> the last one I think is fine.
[00:13] <StevenK> nigelb: Since your earlier work has landed, this must be a seperate branch
[00:13] <nigelb> another incremental landing for the same bug?
[00:13] <StevenK> Yup
[00:14] <nigelb> k
[00:14] <StevenK> nigelb: You're learning, it happens
[00:14] <nigelb> That doesn't mean I'm happy about it :)
[00:15] <lifeless> StevenK: https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-model/+merge/75093 has db and model code mixed - is that intentional ?
[00:15] <StevenK> I missed the pre-req branch, I've resubmitted it
[00:15] <lifeless> ah cool
[00:15] <StevenK> Damn pipes :-P
[00:16] <nigelb> Hrm, what do I do for this query? http://paste.ubuntu.com/687983/
[00:16] <nigelb> remove the select and where?
[00:16] <StevenK> Drop lines 10-12
[00:16] <StevenK> UNION ALL
[00:16] <StevenK>         SELECT statusexplanation FROM BugTask
[00:16] <StevenK>         WHERE statusexplanation %(posix_oops_match)s
[00:17] <nigelb> aha,okay.
[00:31] <nigelb> Oh, excellent. Reviewer on duty earlier.
[00:31] <nigelb> *earlier than I expected.
[00:33] <LPCIBot> Project devel build #1,062: STILL FAILING in 1 hr 3 min: https://lpci.wedontsleep.org/job/devel/1062/
[00:34] <wgrant> That is suboptimal.
[00:34] <wgrant> StevenK, lifeless: Do we want to revert stub's rev, or expedite the pgbouncer fix and see if it helps?
[00:34] <nigelb> wgrant: Was there thoughts of removing opinion?
[00:35] <nigelb> (I should have asked "Whats your opinion on removing opinion")
[00:35] <wgrant> It should never have existed.
[00:35] <poolie> +1
[00:35] <wgrant> It was meant to be removed once the experiment failed, but that never happened.
[00:36] <nigelb> Excellent. Shall I proceed to destory that as well?
[00:36] <wgrant> Needs discussion.
[00:36] <wgrant> And migration.
[00:36] <wgrant> Which is why it shouldn't have been added.
[00:36] <nigelb> A bug with status "Opinion" perhaps? :)
[00:36] <wgrant> It was pointless, and fixing it is expensive.
[00:36] <nigelb> Expensive in terms of downtime?
[00:36] <wgrant> No.
[00:36] <nigelb> Or in terms of plan effort.
[00:36] <wgrant> That's easy.
[00:37] <wgrant> Plan effort, mainly.
[00:37] <wgrant> plain
[00:37] <nigelb> *plain
[00:37] <nigelb> Hrm. How hard?
[00:37] <wgrant> The Bugs squad no longer exists.
[00:37] <wgrant> So they aren't around to abort the experiment.
[00:37] <wgrant> lifeless: Can we abort the experiment? :)
[00:37] <nigelb> Lets declare Teal as bugs squad and declare it aborted? :P
[00:38] <lifeless> last I heard there was a final signoff needed
[00:38] <lifeless> which jml may or may not have done before he stepped down
[00:38] <nigelb> \o/
[00:38] <nigelb> That kind of effort.
[00:38] <lifeless> I think its currently in limbo, but there isn't a lot of point rushing it when IssueTracker is just around the corner
[00:39] <nigelb> IssueTracker?
[00:39] <wgrant> ... he says as a feature is extended to 10 months
[00:39] <lifeless> wgrant: whats this about pgbouncer?
[00:39] <lifeless> http://ec2-204-236-255-184.compute-1.amazonaws.com/ is running my 0.0.5 upgrade atm
[00:39] <wgrant> lifeless: With stub's librarian<->pgbouncer interaction tests, jenkins no longer works.
[00:39] <wgrant> lifeless: It's unclear if buildbot works.
[00:39] <wgrant> We'll find out soon,
[00:39] <lifeless> wgrant: do they fail locally ?
[00:39] <wgrant> But I now consider Jenkins failures to be fatal.
[00:40] <wgrant> lifeless: https://lpci.wedontsleep.org/job/devel/1062/console
[00:41] <wgrant> It passes locally.
[00:42] <wallyworld_> lifeless: if you are able to +1 my db-devel mp in the next 1/2 hour, i can make the next bb run :-)
[00:42] <wgrant> It needs to be sooner than that.
[00:43] <wgrant> 15 minutes will be pushing it, even.
[00:43] <wgrant> buildbot's estimates sort of suck.
[00:43] <wallyworld_> ah
[00:43] <wgrant> It would not surprise me if it finished in less than 5.
[00:44] <StevenK> Haha
[00:45] <wallyworld_> that's not nice :-(
[00:48] <nigelb> Operation timed out.
[00:49] <wgrant> wallyworld_: And it's done.
[00:49] <wgrant> rofl
[00:49] <wgrant> Finished 00:48:43
[00:49] <wgrant> 10:43:43 < wgrant> It would not surprise me if it finished in less than 5.
[00:49] <nigelb> How the..
[00:50] <wallyworld_> wgrant = nostrodamous
[00:50] <StevenK> buildbot's estimates suck.
[00:50] <StevenK> Jenkins tend to work out the time much better.
[00:50] <wallyworld_> i know that now :-(
[00:50] <StevenK> And buildbot will "forget" the time if a build fails
[00:50] <wallyworld_> i wish we could move to jenkins
[00:51] <lifeless> bah. iwlagn --fail--
[00:51] <lifeless> wgrant: do they fail locally ?
[00:51] <wgrant> lifeless: No.
[00:51] <wgrant> It'll be in buildbot in a couple of hours.
[00:51] <lifeless> I can lp-land the 0.0.5 upgrade if you want
[00:51] <StevenK> lifeless: Can haz db review for two branches so they can hopefully be deployed tonight as part of FDT?
[00:52] <wgrant> lifeless: That might be handy.
[00:52] <wgrant> lifeless: It looks safe to me...
[00:52] <lifeless> StevenK: urls ? [note that we only do one change per fdt. So you can't get in anyhow]
[00:52] <wgrant> lifeless: We have a week of patches queued already.
[00:52] <StevenK> https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-indices/+merge/75094 is mine
[00:52] <wgrant> And we deployed 7 on Friday without trouble.
[00:53] <lifeless> I know we did.
[00:53] <flacoste> wallyworld_: RT #43352
[00:53] <_mup_> Bug #43352: ipp jobs not purged; purging causes 100% cpu usage <cupsys (Ubuntu):Fix Released> <gnome-cups-manager (Ubuntu):Invalid> < https://launchpad.net/bugs/43352 >
[00:53] <wgrant> StevenK: Anyway, you're too late for today.
[00:53] <lifeless> ok, lp-land on its way
[00:53] <wgrant> StevenK: buildbot has started already.
[00:53] <wgrant> Oh!
[00:53] <wgrant> No it hasn't.
[00:53] <lifeless> wgrant: the point of one patch at a time is recoverability and diagnosis. Risk mitigation.
[00:53] <StevenK> Indeed
[00:53] <lifeless> wgrant: its nice that folk are keen.
[00:53] <wgrant> Because lucid_lp was broken, there was no merge.
[00:54] <wgrant> wallyworld_: You can still get your patch in if you want.
[00:54] <lifeless> wgrant: but, doing batching won't make this -better-
[00:54] <StevenK> lifeless: This is half joking -- but we want to go faster and you're trying to slow us down.
[00:54] <lifeless> StevenK: those are not FDT safe.
[00:54] <StevenK> They aren't?
[00:55] <lifeless> StevenK: they are hot changes.
[00:55] <lifeless> StevenK: I want to make change *safe* and *frequent*
[00:55] <StevenK> So I can just ask stub to add the indexes for me?
[00:55] <lifeless> yes
[00:55] <StevenK> Right
[00:55] <wallyworld_> flacoste: ah thanks
[00:55] <lifeless> can't be done during a backup or other long transaction
[00:56] <StevenK> I'll talk to stub when he surfaces
[00:56] <StevenK> Now to see if I can reorder this pipeline
[00:56] <nigelb> StevenK: He surfaced a few hours back.
[00:56] <wgrant> He was probably just still up.
[00:56] <lifeless> StevenK: anyhow, as the guy that got buyin and resourcing for fast downtime, I think I can can say I've made a huge improvement to 'going faster' :)
[00:56] <wgrant> Given he appeared right after FDT last night.
[00:57] <nigelb> He said "Cause I woke up at 3pm of course. What a silly question!"
[00:57] <wgrant> Haha
[00:57] <StevenK> Hm, can't reorder pipes
[00:58] <wgrant> lifeless: Thanks, jenkins is building with your pgbouncer fix.
[00:58] <wgrant> StevenK: You have to do it manually.
[00:58] <wgrant> Edit the branch.conf of each.
[00:58] <StevenK> I'll just delete the pipe
[00:58] <nigelb> wgrant: Can haz review? https://code.launchpad.net/~nigelbabu/launchpad/the-return-of-destory-statusexplanation-88545/+merge/75099
[00:58] <StevenK> Hahahaha
[00:58] <StevenK> destory
[00:59] <nigelb> :D
[00:59] <wgrant> nigelb: Hm, how did it land with those references still there?
[00:59] <wgrant> Oh, right.
[00:59] <wgrant> I understand now.
[00:59] <nigelb> Entirely different places.
[01:00] <nigelb> wgrant: err needs the incremental flag
[01:00] <wgrant> nigelb: I see you fixed some lint... do you feel like fixing the last three?
[01:01] <wgrant> (two long lines, one unused variable)
[01:01] <nigelb> one unused variable is false positive
[01:01] <nigelb> its used in SQL query
[01:02] <wgrant> Ah, and it uses local() to get it in?
[01:02] <wgrant> *locals()
[01:02] <nigelb> vars()
[01:03] <wgrant> Ew
[01:03] <nigelb> want me to change that?
[01:03] <wgrant> Let me see.
[01:03] <wgrant> Since that's the only var, yeah.
[01:03] <wgrant> Might as well fix it.
[01:03] <wgrant> Also fix the two long lines while you're there.
[01:04] <wgrant> One of which is the regex.
[01:04] <wgrant> This stuff doesn't get touched frequently, so it's nice to get rid of the lint when it is.
[01:04] <nigelb> close the quotes, use \, and start quotes in new line?
[01:04] <wgrant> I'd use parens instead of \
[01:04] <wgrant> foo = (
[01:04] <wgrant>     "barfewfwefwQ"
[01:05] <wgrant>     "efwfwefwfw")
[01:05] <nigelb> okay.
[01:05] <StevenK> Perl! s///x
[01:05] <StevenK>  /x allows comments and whitespace *inside* regexs
[01:06] <nigelb> http://xkcd.com/208/
[01:06] <StevenK> Yes, yes.
[01:06] <StevenK> Go way, I know regular expressions.
[01:06] <nigelb> s/way/away/g
[01:06] <nigelb> But,yeah.
[01:06] <StevenK> g not needed, you're only replacing once
[01:07] <StevenK> :-P
[01:07] <nigelb> Dammit.
[01:12] <StevenK> wgrant: If you're free, https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-model/+merge/75100
[01:12] <wgrant> StevenK: Doesn't that depend on the hot index?
[01:13] <wgrant> You say it can land anyway, but doesn't it include the patch?
[01:13] <StevenK> It doesn't any more
[01:13] <wgrant> Ah, no prereq any more.
[01:13] <wgrant> icy.
[01:13] <StevenK> And it doesn't need the index, no, it just adds the columns to the model
[01:13] <wgrant> Yeah, but I thought that was the branch I saw earlier with the DB patch.
[01:13] <wgrant> That you said needed a prereq.
[01:13] <StevenK> Yes, I was fighting with pipes
[01:14] <StevenK> And then lifeless said the patch is unsuitable and so I resubmitted the MP again
[01:14] <StevenK> "This time, for reals"
[01:14] <lifeless> StevenK: erm, its not a FDT patch; it was fine in all other regards.
[01:14] <StevenK> What other kind of patch do we have!?
[01:14] <nigelb> wgrant: suggestions on "description='https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1Foo666'"
[01:15] <lifeless> StevenK: hot db patches and code patches.
[01:15] <nigelb> its inside the sql query
[01:15] <wgrant> nigelb:
[01:15] <wgrant> description=
[01:15] <wgrant>     'https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1Foo666'
[01:15] <wgrant> Is that sufficient?
[01:15] <wgrant> Otherwise use || to concat.
[01:15] <lifeless> StevenK: https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess#Hot_Patches
[01:16] <nigelb> wgrant: Is this fine? http://paste.ubuntu.com/688008/
[01:16] <G> StevenK: I'm not needed? ;)  (really wish awaylog gave context)
[01:16] <wgrant> nigelb: If that's short enough, yep.
[01:16] <lifeless> StevenK: and https://wiki.canonical.com/Launchpad/PolicyandProcess/ProductionChange#Hot_database_patches
[01:18] <nigelb> wgrant: Updated MP.
[01:18] <StevenK> lifeless: That's what I said -- I was going to talk to stub about the indexes when he surfaces
[01:18] <lifeless> right
[01:18] <lifeless> StevenK: you still need to land the patch though
[01:18] <lifeless> StevenK: I guess all I'm saying is that your patch wasn't *bad*, it just *wasn't FDT relevant*
[01:18] <wgrant> nigelb: You are now lint-clean?
[01:19] <nigelb> Yes
[01:19] <nigelb> Pasted on MP description as well.
[01:19] <lifeless> StevenK: and I suspect I gave the wrong impression before
[01:19] <lifeless> StevenK: so I want to fix that up
[01:19] <nigelb> What the... every other lint I read has barry as author/co-author.
[01:20] <nigelb> s/lint/PEP
[01:20] <wgrant> lint, PEP, same thing :P
[01:20] <nigelb> lol
[01:20] <lifeless> he is the flufl
[01:20] <wgrant> But yeah, barry is fairly Python.
[01:20] <nigelb> heh
[01:21] <StevenK> lifeless: I didn't say bad, I said unsuitable
[01:21] <wgrant> nigelb: Looks good, thanks.
[01:21] <lifeless> StevenK: ok, if we're copacetic, cool. Sorry for the sidebar :)
[01:21] <wgrant> nigelb: Shall I land it?
[01:21] <nigelb> wgrant: yes, please :)
[01:21] <StevenK> wgrant: With --incremental
[01:22] <wgrant> Indeed.
[01:22] <nigelb> I only noticed last week that Barry co-authored PEP8. I've read that a zillion times and never noticed!
[01:23] <nigelb> Ok, now I can go sleep without something left to do.
[01:23] <wgrant> Night nigelb.
[01:24] <nigelb> Its 7 am, but yeah. Laters.
[01:24] <lifeless> nigelb: uhm. Sustainable patches please ;)
[01:24] <nigelb> lifeless: Huh?
[01:25] <lifeless> nigelb: You Need To Sleep.
[01:25] <nigelb> What did I write unsustainable?
[01:25] <wgrant> nigelb: If you're not gone yet, care to set a commit message?
[01:25] <nigelb> Setting.
[01:25] <wallyworld_> lifeless: sorry to nag - if you are able to +1 my db-devel mp i can land it before something else triggers bb :-)
[01:25] <nigelb> lifeless: I was up till 4 for work, then I got test failure email, and then sat and fixed :)
[01:26] <lifeless> wallyworld_: I prefer to let stub review them all as much as possible
[01:26] <wallyworld_> ok. np
[01:26] <nigelb> wgrant: What's the official word on that? Some reviewers like me setting the commit message, otheres do it themselves.
[01:26] <lifeless> wallyworld_: your patch will be several days away anyway, due to folk dogpiling in so excitedly ;)
[01:26] <lifeless> wallyworld_: which is nice.
[01:26] <wgrant> nigelb: They're just being nice.
[01:27] <wgrant> nigelb: Generally the author sets it.
[01:27] <nigelb> Oh. /me notes for future.
[01:27] <lifeless> wallyworld_: and may make the one-patch-per policy get revisited.
[01:27] <nigelb> I'm in queue for a db patch as well :D
[01:27] <wallyworld_> lifeless: you you are enforcing that atm
[01:27] <wallyworld_> sounds a bit rigid
[01:27] <lifeless> wallyworld_: FTR - https://dev.launchpad.net/Database/LivePatching
[01:27] <nigelb> (they landed 7 patches the other day. wgrant was plotting 3 tomorrow :P)
[01:28] <nigelb> Or was it 2)
[01:28] <StevenK> They can't land in the same run
[01:28] <lifeless> wallyworld_: bah, wrong link
[01:28] <G> nigelb: from reading the ML wasn't the 7 an exception to try and catch up?
[01:28] <wgrant> nigelb: Instance is starting.
[01:28] <wgrant> nigelb: Now go away.
[01:28] <wgrant> :)
[01:28] <lifeless> wallyworld_: on https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess#Making_a_database_patch it has discussion about adding columns and performance
[01:29]  * nigelb goes away.
[01:29] <lifeless> wallyworld_: the reason for one at a time is to avoid multiple zomg's.
[01:29] <lifeless> wallyworld_: db schema changes are -the- most risky routine thing we can do.
[01:29] <wallyworld_> sure, but if there are a few totally orthogonal changes....
[01:30] <wallyworld_> these could be done together
[01:30] <lifeless> they could
[01:30] <lifeless> if we want to risk multiple disparate fallouts all at once.
[01:30] <wgrant> Otherwise people will just conspire to do more things in one patch.
[01:30] <wgrant> eg. I'm dropping 36 tables in one patch instead of 36.
[01:30] <lifeless> which is fine
[01:30] <lifeless> its one conceptual thing.
[01:30] <StevenK> wgrant: Otherwise we'd need to jump to 2209 :-P
[01:30] <lifeless> folk aren't going to be silly.
[01:30] <lifeless> there is precisely one DBA.
[01:31] <wgrant> lifeless: Right, they're going to be sensible and work around a silly restriction :P
[01:31] <lifeless> speaking bluntly, if we have a shitstorm, there is only one person that will be fixing it. I don't want us to -ever- have 2 shitstorms and one person working on them.
[01:31] <wallyworld_> lifeless: btw, i did read that wiki page on adding columns but then saw previous patches adding not null columns and figured it must have been a "soft" policy
[01:32] <lifeless> wallyworld_: it is soft 'Similarly, new columns must default NULL unless the data size is extraordinarily small. '
[01:32] <lifeless> wallyworld_: Branch is not extraordinarily small :)
[01:32] <lifeless> wgrant: if folk batch inappropriately, I will veto their changes.
[01:32] <lifeless> wgrant: and stomp very hard.
[01:32] <wgrant> Bah.
[01:32] <lifeless> wgrant: there is a fundamental cultural change we *will* make.
[01:32] <wallyworld_> fair point. i can't recall the example i saw, but i could have sworn it was for a lrgish table
[01:33] <lifeless> wallyworld_: if it was more than 6 weeks ago it was before FDT.
[01:33] <wallyworld_> ah that makes sense
[01:33] <lifeless> wgrant: the payoff for this change is a more reliable process and that means we can execute more often.
[01:34] <lifeless> wgrant: we have a -long- way to go to prove that we can be trusted to execute changes rapidly. Until we have things that make it more risky, are well, silly risks to take.
[01:35] <lifeless> wgrant: before FDT we were doing *6* schema changes a month.
[01:35] <lifeless> wgrant: even with 2-step changes, 12 a month is well below the capacity we could do if we had to and still be doing one-per.
[01:36] <lifeless> wgrant: so arguing that batching is important right now seems specious and to be ignoring the very real risk tradeoffs that the new process is designed around.
[01:36] <wgrant> Perhaps.
[01:37] <lifeless> can you make an affirmative argument for why we should accept the increased risk that batching brings *right now*
[01:38] <wgrant> I don't believe there is significant credible risk in batching some kinds of patches.
[01:38] <lifeless> how good would you say our risk assessment for schema patches is?
[01:39] <wgrant> In terms of performance? Terrible. In terms of blowing up? Pretty good.
[01:39] <lifeless> like, if we bet 1000 dollars on the patch being sideeffect free, per patch, how often would we payout [have a patch with undesirable side effects] ?
[01:39] <wgrant> When have we had problems recently, apart from bugsummary fun?
[01:40] <lifeless> If we deploy something that causes grief, we need to roll it back at the next slot - or even sooner if its really bad. That means a pipeline stall on *all* db patches [so we're not compounding the error]
[01:41] <lifeless> wgrant: we've blown up a number of collections with indices that caused sideeffects in other queries
[01:43] <wgrant> True.
[01:44] <lifeless> add to this that we have a number of 24 hour cycles
[01:44] <lifeless> oops reports
[01:44] <lifeless> api scripts in cron
[01:44] <lifeless> its hard to tell if we've borked something without giving it some time to settle in.
[01:44] <lifeless> we need to fix all of this stuff
[01:44] <lifeless> drive the intervals down
[01:44] <lifeless> get more responsive.
[01:45] <lifeless> everything we batch gets in the way of driving the lead-time lower
[01:45] <lifeless> so, we're not batching here. Unless a -really really really- compelling argument that we *want* to batch, in everything, is made.
[01:45] <wgrant> How about that we're not going to be driving down intervals for at least 18 months?
[01:46] <wgrant> So we're going to have few opportunities for optimisation until then.
[01:47] <wgrant> (unless you do it all single-handedly)
[01:48] <wgrant> We are at 263 criticals, and no crisis has been declared.
[01:48] <wgrant> We are not going to be improving anything apart from criticals for a *very* *long* *time*.
[01:50] <lifeless> francis is analysing that
[01:50] <lifeless> fdt is one example of driving down intervals
[01:50] <wgrant> Right, but I think it needs more of a declaration of crisis than an analysis.
[01:50] <lifeless> So I wouldn't say single handed driving things down.
[01:51] <wgrant> FDT wasn't single-handed? It was entirely Technical Architecture.
[01:51] <lifeless> But all the SOA stuff - oops with rabbit etc - are all dovetailing into more responsiveness in our systems
[01:51] <wgrant> Single-handed.
[01:51] <lifeless> red are working on rabbit
[01:51] <wgrant> lol
[01:51] <lifeless> thats support for this
[01:52] <lifeless> we have the test suite stuff enqueue for this year
[01:52] <wgrant> hahaahhahha
[01:52] <lifeless> 30m test runs or thereabouts
[01:52] <wgrant> rofl
[01:52] <lifeless> okok, 60m.
[01:52] <wgrant> lol
[02:00] <wgrant> I am impressed with your vision for these improvements.
[02:00] <wgrant> But I present exhibit A.
[02:00] <wgrant> http://webnumbr.com/launchpad-critical-bugs
[02:01] <wgrant> Everyone seems to be ignoring the fact that we are probably never going to exhaust the queue.
[02:01] <wgrant> All progress is predicated on that happening.
[02:02] <wgrant> It's been clear for at least a month, and probably 3.5 months, that a crisis needed to be declared.
[02:02] <poolie> wgrant, so do you think there should be less feature work and more critical bug work, or ...?
[02:02] <wgrant> I don't know.
[02:02] <wgrant> Probably.
[02:02] <wgrant> But the current situation is dire.
[02:02] <wgrant> And it is steadily getting even worse.
[02:03] <wgrant> What we're doing now is not working.
[02:03] <wgrant> At. All.
[02:03] <lifeless> so
[02:03] <lifeless> the trend is a problem
[02:04] <wgrant> Even more amusingly, the squad that was unassigned at exactly the time the downward trend stopped will remain on feature work until probably April.
[02:04] <lifeless> thats why francis is analysing to determine the source of the criticals.
[02:04] <lifeless> Are we breaking things too often?
[02:04] <lifeless> are we still finding debt that meets the policy rules?
[02:04] <lifeless> etc.
[02:04] <lifeless> until we have that breakdown, there isn't any change that is sensible to take - we need data.
[02:04] <lifeless> this is a crucial thing for us.
[02:05] <wgrant> Well, when are we likely to have results?
[02:05] <lifeless> I don't know.
[02:06] <lifeless> I forgot to ask him today.
[02:06] <lifeless> as an example though - if the problem is that we're creating more criticals.
[02:06] <wgrant> We have more than 25% more criticals than when the problem first becams eclear.
[02:06] <lifeless> more folk working on criticals might just increase the rate at which we break things.
[02:08] <G> most of the criticals seem to be timeout/db related right? (just my observation, don't know if I'm right or not)
[02:08] <wgrant> All is lost, so I am going to have lunch.
[02:13] <wgrant> lifeless: your pgbouncer fix worked, btw.
[02:15] <lifeless> heh
[02:15] <lifeless> thats good
[02:16] <lifeless> wgrant: care to note that in one of the bugs? (so we learn from our mistakes ;P)
[02:18] <wgrant> lifeless: Commented on bug #848400
[02:18] <_mup_> Bug #848400: fixture shares fd's with testrunner process, will cause test hangs and worse. <Python PGBouncer:Fix Released by lifeless> < https://launchpad.net/bugs/848400 >
[02:34] <lifeless> jamesh: around ?
[02:34] <jamesh> hi
[02:35] <lifeless> hi
[02:35] <lifeless> got time to talk storm ?
[02:35] <lifeless> jamu and I weren't sure if we needed to rollback my landing
[02:35] <jamesh> sure.
[02:35] <lifeless> or to tweak, etc
[02:36] <jamesh> I don't think it is worth rolling back: it is new code that isn't being used anywhere, so we may as well build on top of it
[02:37] <lifeless> I replied to your review
[02:37] <lifeless> (and thanks for that)
[02:37] <jamesh> so, paramstyle was a reference to the DB-API spec: "pyformat" is the style used by psycopg2 with "%s" as a marker
[02:37] <lifeless> ah
[02:37] <jamesh> and qmark is the style used by sqlite with "?" as a marker
[02:37] <lifeless> I haven't read that spec
[02:38] <lifeless> how is something like "column LIKE '%s' AND column2=%s" handled where the LIKE clause is actually a suffix search, not a substitution
[02:38] <jamesh> the degenerate case that would cause problems would be something like connection.execute(u"SELECT ? = '%s'", (u"foo",))
[02:39] <jamesh> on a backend using the qmark paramstyle, the interpolation would fail in the tracer.  Of course, it would fail earlier on a pyformat paramstyle backend though.
[02:39] <lifeless> one way to handle it would be a try:except: around the interpolation with a fallback of outputting something more like debug does.
[02:40] <lifeless> or just the plain statement.
[02:40] <jamesh> fyi: you might find the startswith(), endswith() and contains_string() sql expression methods in Storm handle most of the LIKE clauses you have pretty well
[02:41] <lifeless> jamesh: I think we use them in various places
[02:42] <lifeless> do they compile to "column LIKE %s" , (u"%suffix",) ?
[02:42] <lifeless> more-or-less
[02:42] <jamesh> for the quoted '%' case for a LIKE expression, the if you're using a pyformat paramstyle backend, you'd usually have to escape the percent sign (one of the reasons some people dislike pyformat)
[02:43] <lifeless> \%s ?
[02:43] <jamesh> %s
[02:43] <jamesh> gar.  "%s"
[02:43] <jamesh> xchat seems to be eating the double percent sign :(
[02:43] <lifeless> hahaha
[02:43] <lifeless>  %%s ?
[02:43] <jamesh> yes.
[02:44] <lifeless> so, what should we test ?
[02:45] <jamesh> Storm's startswith(), etc methods change the glob characters to ones that don't look like a pyformat interpolation sequence
[02:46] <jamesh> If you have an unescaped "%" character, I don't think psycopg will be particularly happy anyway.
[02:47] <lifeless> I'm not aware enough of storms needs to predict the missing tests / changes.
[02:47] <lifeless> what I copied over is working live in LP with the pg backend.
[02:47] <jamesh> >>> cursor.execute("SELECT %s = '%'", ('%',))
[02:47] <jamesh> Traceback (most recent call last):
[02:47] <jamesh>   File "<stdin>", line 1, in <module>
[02:47] <jamesh> IndexError: tuple index out of range
[02:49] <lifeless> if I understand correctly you're saying there isn't an issue
[02:49] <lifeless> because something that would break the tracer will break pscyopg too
[02:49] <jamesh> there probably isn't an issue for psycopg2 and other pyformat backends
[02:50] <jamesh> people could pass queries through that will cause problems on qmark backends
[02:50] <lifeless> because we don't escape existing %s's on qmark lookups ?
[02:51] <jamesh> right.  At the point where the tracer is running, the query will be in the adapter's preferred format, so we wouldn't expect '%' interpolation at that point
[02:52] <lifeless> jamesh: but the quoted_statement re sub should fix that
[02:52] <jamesh> I guess the code should look something like: (a) if connection.param_mark == '?', double all percent signs and convert param marks to '%s'
[02:52] <jamesh> (b) if param_mark is '%s', don't alter the query string at all
[02:53] <jamesh> if param_mark is something else again, panic :)
[02:53] <lifeless> ah, we preserve %s's
[02:53] <lifeless> mmm
[02:53] <lifeless> how about
[02:53] <lifeless> if param_mark is not %s: double all percent signs, otherwise double all non-%s % signs.
[02:54] <lifeless> then convert marks to %s
[02:54] <lifeless> done
[02:54] <StevenK> wgrant: Please take back your "Lies." comment: http://pastebin.ubuntu.com/688053/
[02:54] <jamesh> I don't think you should need to do anything to the '%' signs if param_mark is '%s'
[02:54] <lifeless> ian broke LP with that assumption.
[02:54] <lifeless> it was a fun day of broken buildbot.
[02:55] <jamesh> if the non-interpolated '%' signs haven't been escaped, then they will fail when passed to the database adapter anyway
[02:55] <lifeless> apparently we have something somewhere that manages this
[02:55] <lifeless> ENOIDEAWHERE.
[02:59] <wgrant> StevenK: Oh?
[03:00]  * StevenK waits for wgrant to read the pastebin
[03:00] <wgrant> I read the pastebin.
[03:00] <wgrant> I see a SourcePackageName declared as an integer.
[03:01] <lifeless> jamesh: what about the other thing
[03:01] <lifeless> jamesh: the control api
[03:02] <jamesh> lifeless: the part I cringed at was exposing the threading.local() object as part of the public API to the tracer
[03:02] <StevenK> wgrant: ALTER TABLE SourcePackagePublishingHistory ADD COLUMN sourcepackagename INTEGER REFERENCES SourcePackageName
[03:03] <lifeless> jamesh: yes, I explained the things that led me to doing that.
[03:03] <wgrant> StevenK: I'm not sure what the DB has to do with this.
[03:03] <lifeless> jamesh: and a possible alternative, but it seems rather like yagni to me. I'm willing to do it if needed
[03:03] <wgrant> StevenK: SourcePackagePublishingHistory.sourcepackagename is not an integer, so why is ISourcePackagePublishingHistory['sourcepackagename'] an Int?
[03:04] <StevenK> wgrant: Oh, I've declared it wrong in the model?
[03:04] <wgrant> In the interface.
[03:04] <wgrant> The code I quoted!
[03:04] <StevenK> OH
[03:05] <StevenK> Attribute() and Int() are the wrong way around.
[03:05] <wgrant> You know, the only two types in the code I quoted :)
[03:05] <StevenK> Sorry, I read it as "You have sourcepackagename in BPPH and vice versa"
[03:08] <lifeless> jamesh: http://paste.ubuntu.com/688059/ for the quoting thing.
[03:09] <jamesh> lifeless: the timeline_factory argument or an abstract get_timeline() method method would be my preferred APIs.  Do you have example code of how you'd use the class in Launchpad as it is currently written though?
[03:09] <lifeless> jamesh: adapter.set_request_started
[03:10] <lifeless> jamesh: the version of this tracer in LP knows about the 'current request', so it calls 'get_request_timeline'
[03:10] <lifeless> jamesh: which could drop straight into a timeline_factory parameter.
[03:10] <jamesh> lifeless: re. the paste, I still think you should be able to get by without munging the statement in the connection.param_mark == '%s' case
[03:11] <lifeless> jamesh: you can't because "%%" % () -> "%"
[03:11] <lifeless> jamesh: but we want "%%" to be shown as  "%%"
[03:11] <jamesh> lifeless: why?
[03:11] <lifeless> jamesh: the munging is to *preserve* the %'s.
[03:11] <jamesh> lifeless: they aren't preserved in the statement executed by the DB
[03:12] <lifeless> they match what folk see when they look at the code though, which was a strong consideration.
[03:12] <jamesh> hmm
[03:12] <lifeless> I could go either way.
[03:12] <lifeless> I'm convinced its a presentation thing.
[03:13] <lifeless> it also happens to ensure its safe to format, but thats incidental.
[03:13] <jamesh> my point earlier was that if it isn't safe to format, then it is going to fail when passed to the adapter anyway ...
[03:13] <lifeless> jamesh: so I changed to the thread locals() because it seemed to me that that is the common need
[03:14] <lifeless> jamesh: right, I got that, but wouldn't it be better fo rhte adapter to be the thing raising the error ?
[03:14] <lifeless> jamesh: lets users avoid confusion about what is wrong
[03:15] <jamesh> lifeless: if that is a concern, I would consider including a code path that handles errors from the interpolation step.
[03:15] <lifeless> jamesh: if you prefer a factory, I'll change it to want a factory, that happens to fit closer to LP because it already *has* such a factory; it will mean more glue in the wsgi case though.
[03:16] <lifeless> mm
[03:17] <lifeless> actually it has a factory which needs a guard because of other bugs.
[03:17] <lifeless> but meh, its shallow either way.
[03:17] <jamesh> here's how psycopg2 handles the percent sign:
[03:17] <jamesh> >>> cursor.mogrify("SELECT %s LIKE 'foo%'", [u'foobar'])
[03:17] <jamesh> "SELECT E'foobar' LIKE 'foo%'"
[03:18] <jamesh> gar.  imagine there is a double percent sign in the LIKE clause
[03:18] <lifeless> on the >>> or the " starting lines ?
[03:18] <jamesh> the first line
[03:18] <lifeless> right
[03:19] <jamesh> mogrify() is a psycopg2 specific API for invoking its interpolation code
[03:23] <lifeless> jamesh: ok, so what now? I'm getting lost ;)
[03:26] <jamesh> lifeless: okay.  I think your paste looks like a good improvement, but (1) I'd prefer not to munge the statement in the pyformat backend case and (2) I think the actual interpolation stage should check for TypeError and use some other place holder string in that case.
[03:26] <jtv> wgrant: see anything new in debian domination?  Our challenge for today is to get it to re-publish an SPR.
[03:27] <wgrant> jtv: It looks OK, and we know that it handles the republishing case OK from my archivepublisher testing.
[03:27] <wgrant> We could retest that with gina if we really want to, though.
[03:27] <lifeless> jamesh: do we need to defend against unreprable variables ?
[03:27] <lifeless> jamesh: to-database shold be sane, no ?
[03:28] <jamesh> lifeless: I think we can depend on to_database() being sane: it gets called outside of the tracer in this code path anyway.
[03:29] <jtv> wgrant: hmm…  intra-release domination is the same thing for gina domination and publisher domination, so yes, we can take that as validation.
[03:29] <wgrant> jtv: So, any reason not to deploy it and watch the world burn?
[03:30] <jtv> The world won't burn.  For that, I need to add that commit first.
[03:30] <wgrant> True.
[03:30] <jamesh> not sure about repr().  The existing DebugTracer grabs the repr() of the values and hasn't been a pain point so far (although it isn't using to_database())
[03:30] <jamesh> so it might be worth leaving it as is, and see how it works
[03:31] <wgrant> jtv: Shall we deploy up to 13913, then?
[03:31] <jtv> Let me look that up.
[03:32] <wgrant> Hmm, we should really get past 13917 today.
[03:32] <wgrant> But 13914 is a feature-flagged blocker.
[03:32] <poolie> lifeless, can we have another go at bug 660264 now haproxy is live?
[03:32] <lifeless> jamesh: typeerror - http://paste.ubuntu.com/688065/
[03:32] <_mup_> Bug #660264: bzr+ssh on launchpad should fork, not exec <lp-code> <Launchpad itself:In Progress by jameinel> < https://launchpad.net/bugs/660264 >
[03:32] <lifeless> jamesh: I'd like to leave the %% handling on postgresql alone for now
[03:33] <lifeless> jamesh: get everything migrated and consolidated, then revisit.
[03:33] <jamesh> lifeless: fair enough.  The second paste looks good.
[03:33] <lifeless> poolie: no, we need those two bugs I escalated with you addressed first.
[03:33] <poolie> smooth reconnect etc
[03:33] <poolie> ok
[03:33] <jtv> wgrant: 13913 is doable for me, although obviously I'd like to get that commit in with the rest.  Can we treat cocoplum as if it were NDT now?
[03:33] <lifeless> poolie: haproxy is only half-live, we're not actually in nodowntime mode yet.
[03:33] <lifeless> poolie: yes, those.
[03:34] <wgrant> jtv: gina isn't on cocoplum
[03:35] <lifeless> jamesh: so, you're really against the threadinfo thing ? [/me asks with a vague hope :P] - it just seems like -all- the factories folk will write will be identical.
[03:35] <jtv> wgrant: where then?  And is it NDT?
[03:35] <wgrant> jtv: iron, and yes.
[03:35]  * jtv shuffles cards
[03:35]  * jtv eagerly waits for wgrant's message notification bubble to get out of the crucial spot where he needs to see.
[03:35] <jtv> Bubble bursts.
[03:36] <wgrant> jtv: what was that?
[03:36] <wgrant> jtv: I didn't quite hear you
[03:36] <wgrant> jtv: Could you please repeat?
[03:36] <wgrant> jtv: Please?
[03:36] <jtv> That's okay, I no longer need to see that precise spot.
[03:36] <jtv> But it's gratifying to see you go to all that wasted effort.
[03:36] <wgrant> Curses.
[03:37] <jtv> wgrant: what's that?
[03:37] <jtv> I didn't quite hear you
[03:40] <poolie> what does 'half-live' mean?
[03:40] <poolie> it's in place but it doesn't fail over?
[03:40] <lifeless> the project isn't complete
[03:41] <lifeless> (because we can't deploy to it regularly, which was the goal)
[03:44] <lifeless> meep
[03:44] <lifeless> thats a worry
[03:44] <lifeless>     Hard / Soft  Page ID
[03:44] <lifeless>      902 /  227  ScopedCollection:CollectionResource:#message-page-resource
[03:44] <lifeless>      870 /   21  Product:+bugtarget-portlet-bugfilters-stats
[03:44] <lifeless>      820 /  158  BugTask:+index
[03:44] <lifeless>      766 /  240  Distribution:EntryResource:searchTasks
[03:45] <lifeless>      291 /    0  Distribution:+bugtarget-portlet-bugfilters-stats
[03:45] <lifeless>      288 /    0  Person:EntryResource:searchTasks
[03:45] <lifeless>      211 /   24  MaloneApplication:+bugs
[03:46] <wgrant> lifeless: Indeed. I've wondered if the journal rollup isn't happening or something.
[03:46] <wgrant> lifeless: I saw a new garbo job fly past this morning.
[03:55] <lifeless> jamesh: http://paste.ubuntu.com/688073/
[03:59] <lifeless>  select count(*) from bugsummaryjournal;
[03:59] <lifeless>  count
[03:59] <lifeless> -------
[03:59] <lifeless>    875
[03:59] <lifeless> wgrant: ^
[03:59] <StevenK> wgrant: Have another look at https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-model/+merge/75100 ?
[04:02] <wgrant> lifeless: Well, there goes that :/
[04:02] <wgrant> Hmm.
[04:02] <wgrant> What could it be, then.
[04:04] <wgrant> StevenK: What about the other places that create [SB]PPH?
[04:04] <wgrant> StevenK: I think there are some more.
[04:04] <wgrant> StevenK: Although I tried to unify most of them, some slipped through.
[04:04] <wgrant> Also, readonly=False?
[04:04] <wgrant> While it's not exposed through the API now/yet, that seems like a bad start :)
[04:05] <jamesh> lifeless: that does look like a better API to me.  If we do find everyone is passing the same timeline_factory implementation in later on, perhaps we can revisit it.
[04:07] <lifeless> https://code.launchpad.net/~lifeless/storm/timelinetracer/+merge/75109
[04:07] <StevenK> wgrant: I thought it was only new{Source,Binary}Publication :-(
[04:07] <wgrant> StevenK: That's the aim.
[04:07] <wgrant> StevenK: But it's not there yet.
[04:08] <wgrant> StevenK: Also, at least publishBinaries has a batch interface that doesn't involve instantiating the class.
[04:08] <lifeless> jamesh: ^
[04:08] <jamesh> lifeless: saw it.  I'll review it in a sec.
[04:09] <lifeless> thanks
[04:11] <wallyworld_> lifeless: is there a way in storm to hook into an object lifecycle such that i can access any dirty properties' old and new values when a db update is done?
[04:11] <lifeless> uhm, probably at least two. Why?
[04:13] <wallyworld_> lifeless: i want to trigger a db update if someone changes a particular property and then saves the object
[04:13] <lifeless> wouldn't a regular property be sufficient ?
[04:14] <wallyworld_> i could hook into an IObjectModifiedEvent i suppose, but i'd rather do it with just storm
[04:14] <lifeless> huh?
[04:14] <lifeless> just foo = property(...)
[04:16] <wallyworld_> that would work. i was wanting to not do any db work until the flush. but i could set a flag and hook into the storm flush event
[04:16] <lifeless> perhaps some specifics would help the discussion
[04:16] <lifeless> I'm struggling to understand the constraints
[04:17] <StevenK> wallyworld_: Er?
[04:17] <StevenK> wallyworld_: Isn't that what triggers are for?
[04:17] <wallyworld_> when a branch has it's stacked_on branch changed or it's private property changed, i need to update the branch table to set transitively_private for other branches that may be affected
[04:18] <lifeless> that sounds like a trigger case to me
[04:18] <wallyworld_> ok. for some reason i thought we preferred to avoid trigegrs
[04:18] <lifeless> running sql *just-before-a-flush* is going to be -fuuuun-
[04:19] <lifeless> wallyworld_: I loath and fear triggers, but they are the right solution to this sort of thing while we use pgsql.
[04:19] <wallyworld_> i share your view, hence trying to finf a non-trigger solution
[04:19] <lifeless> wallyworld_: if we were using e.g. cassandra, the whole way the problem is approach would be different
[04:20] <wallyworld_> any pointers to where we define and set up triggers for other things?
[04:20] <lifeless> trusted.sql
[04:20] <lifeless> though that really wants to DIAF
[04:20] <lifeless> I would just do it as a regular patch I think, at least to start with.
[04:21] <G> wgrant: in my efforts to pick off some of the trivial fixes so you guys can focus on the criticals... with reference to bug 763820, would the text "with the Ubuntu keyserver" be better than "with an Ubuntu keyserver" iirc there is only one right?
[04:21] <_mup_> Bug #763820: double "with" on +editpgpkeys <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/763820 >
[04:23] <wgrant> G: Indeed, that sounds better.
[04:23] <wallyworld_> lifeless: the patch needs to be in db-devel? and have it's own patch number and all that?
[04:23] <lifeless> wallyworld_: yes, most definitely.
[04:23] <G> not to mention, it should've been a not an, but thats by the point since you agree with the :)
[04:23] <wallyworld_> cool. thanks
[04:24] <wgrant> G: "an Ubuntu" is correct, not "a Ubuntu"...
[04:24] <G> wgrant: you don't say "an United Nations <something>" you say "a United Nations <something>" because it flows better (just looked it up)
[04:24] <wgrant> G: Sure.
[04:24] <wgrant> But the U is not the same.
[04:24] <wgrant> oo-boon-too
[04:25] <wgrant> Not yoo-boon-too
[04:25] <poolie> i bet some people do say "an United"
[04:25] <poolie> maybe not incorrectly
[04:25]  * wgrant blames the US
[04:25] <wgrant> It is incorrect, I'm pretty sure.
[04:26] <G> yep, lets just blame the US for everything :)
[04:26] <StevenK> RARGH!
[04:26] <StevenK> CONFLICTS
[04:26] <StevenK> HULK SMASH
[04:26] <wgrant> You conflicted with stub's new garbo?
[04:26] <StevenK> No, this is still the model changes
[04:26] <G> wgrant: you may have a point (w/ pronounciation)
[04:26] <StevenK> But I'll probably have to cope with that too
[04:27]  * StevenK gets a rope.
[04:29] <StevenK> for i in $(find lib/canonical/launchpad/mail | tac) ; do bzr resolve $i ; done
[04:29] <StevenK> And that solves one problem
[04:30] <G> wow `make run` is much faster now
[04:31]  * StevenK waits for bzr pump
[04:44] <StevenK> wgrant: And again?
[04:45] <wgrant> Let's see.
[04:49] <G> wgrant: there you go, MR for you https://code.launchpad.net/~dev-nigelj/launchpad/bug-763820/+merge/75111 has to be the most trivial in a while
[04:50] <StevenK> G: *MP*
[04:50] <StevenK> Merge *Proposal*
[04:50] <G> StevenK: I realized that after I said it
[04:50] <G> StevenK: my mind is about 10 miles away from my body atm :)
[04:51] <StevenK> Up, down, north, south, west or east?
[04:51] <wgrant> StevenK: You didn't get publishBinaries?
[04:51] <StevenK> wgrant: I did a grep
[04:51] <G> StevenK: somewhat South East
[04:51] <StevenK> bzr grep "BinaryPackagePublishingHistory\(" | grep -v makeBinaryPackagePublishingHistory
[04:52] <wgrant> StevenK: Like I said, it does a bulk inset.
[04:52] <wgrant> insert
[04:52] <StevenK> Rargh, annoying
[04:52] <wgrant> Rather, but it's the only way to not have terrible performance.
[04:53]  * StevenK waits for sync-pipeline some more
[04:53] <StevenK> G: By the way, if you haven't used bzr grep, you should -- it is love.
[04:54] <G> StevenK: what is bzr grep when you meet it in the street?
[04:54] <G> (still new to bzr, but I like it)
[04:55] <StevenK> G: It runs grep over versioned files
[04:55] <StevenK> G: It's useful for the Launchpad tree, since we have that whole annoying lib/lp versus lib/canonical split
[04:55] <G> ahhh I just normally do "grep -r <something> lib/lp" :)
[04:56] <poolie> bzr grep is faster, ignores non-versioned files, can search history
[04:56] <StevenK> G: If you search just lib/lp, you will miss stuff. As nigelb found out this morning.
[04:56] <StevenK> G: Hence my service announcement. :-)
[04:58] <StevenK> bzr grep doesn't support -c :-(
[05:02] <G> poolie: you got me at faster ;)
[05:02] <G> where do I get it from? ;)
[05:02] <StevenK> lp:bzr-grep
[05:02] <G> should've guessed :)
[05:02] <wgrant> Or apt-get install bzr-grep
[05:02] <wgrant> On >= maverick, IIRC
[05:02] <StevenK> cd .bazaar/plugins && bzr branch lp:bzr-grep grep
[05:03] <G> wgrant: thanks
[05:03] <StevenK> Since bzr can't deal with plugin directories that contain a -
[05:03]  * G prefers packaged stuff :)
[05:03] <poolie> not packaged?
[05:04]  * StevenK prods wgrant for more review love.
[05:04] <G> poolie: it's packaged for Natty, because after wgrant's pointer I just aptitude install'ed it :)
[05:05] <wgrant> StevenK: What about gina for SPPH? Or does it use newSourcePublication?
[05:05] <StevenK> wgrant: It uses newSP
[05:06] <wgrant> Approvalised.
[05:06] <wgrant> G: Also approved. Want me to land it?
[05:06] <G> wgrant: might as well, thanks :)
[05:07] <wgrant> Could you set a commit message on the MP, please?
[05:07] <LPCIBot> Yippie, build fixed!
[05:07] <LPCIBot> Project devel build #1,063: FIXED in 4 hr 10 min: https://lpci.wedontsleep.org/job/devel/1063/
[05:08] <StevenK> steven@liquified:~/launchpad/lp-branches/no-more-lazr-utils% for i in $(bzr grep -l canonical.lazr.utils) ; do sed 's/canonical.lazr.utils/lazr.restful.utils/' < $i | sponge $i ; done
[05:08] <StevenK> steven@liquified:~/launchpad/lp-branches/no-more-lazr-utils% bzr di | wc -l
[05:08] <StevenK> 745
[05:08]  * StevenK hides
[05:08] <G> wgrant: set
[05:09] <wgrant> Thanks.
[05:10] <G> wgrant: at some point can you take a look at https://bugs.launchpad.net/launchpad/+bug/306569/comments/2 as well and give me your thoughts?  (I'm also now wondering if an AJAXy +help/foo page would also do the job instead of links to the wiki etc
[05:10] <_mup_> Bug #306569: Link to https://help.launchpad.net/Code from project branch listing page <lp-code> <trivial> <ui> <Launchpad itself:In Progress by dev-nigelj> < https://launchpad.net/bugs/306569 >
[05:11] <StevenK> wgrant: lazr imports are the same level as zope and so on right?
[05:11] <wgrant> StevenK: Yes.
[05:14] <mwhudson> StevenK: you might be interested in sed -i :-)
[05:14] <StevenK> mwhudson: But then I can't use the awesomeness that is sponge!
[05:14] <mwhudson> StevenK: admittedly, i had to look up what sponge does
[05:27] <StevenK> Bah, MPs needs a link to bug link
[05:30] <StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/no-more-lazr-utils/+merge/75117
[05:37] <wgrant> StevenK: r=me
[05:45] <poolie> psycopg2.OperationalError: fe_sendauth: no password supplied - halp?
[05:46]  * ajmitch saw some mention of needing ipv6 address in pg_hba.conf for that
[05:47] <wgrant> Yeah, see one of my recent posts to launchpad-dev
[05:48] <wgrant> Or rerun launchpad-database-setup
[05:48] <ajmitch>  host    all         all         ::1/128           trust
[05:49] <poolie> thanks
[06:04] <poolie> wgrant, how can i actually get your rabbitfixture fix active?
[06:04] <poolie> just put a checkout of that branch on my path?
[06:04] <wgrant> poolie: It should be merged now.. let me check.
[06:05] <wgrant> 13906 Launchpad Patch Queue Manager	2011-09-09 [merge] [r=gary][bug=845658] Upgrade rabbitfixture to be oneiric-friendly
[06:24] <wgrant> Dapper has too many architectures.
[07:08] <lifeless> jamesh: ok, so now we need to glue this to wski.
[07:11] <nigelb> Morning.
[07:18] <jamesh> lifeless: yeah.  I wonder what the best way to slice things would be?  A WSGI middleware that simply introduced a timeline object into the environment, and another one that configured the storm tracer based on that timeline and updated oops.context might work
[07:18] <jamesh> could do a single one if you'd never want to use the timeline module without oops and storm
[07:22] <lifeless> I think I'd start with a single one and refactor if we change our mind
[07:22] <lifeless> I can't recall if I've pulled the timeline formatting stuff out of LP yet or not
[07:23] <lifeless> (it is what generates the db_statements key in the oops
[07:23] <lifeless> I'll look at that on thursday.
[07:23] <jamesh> yep.
[07:23] <lifeless> where should this wsgi module live
[07:23] <lifeless> its storm + oops + wsgi specific
[07:23] <jamesh> so, do you stick the timeline in the oops context and rely on an on_create hook to format db_statements for the report?
[07:23] <lifeless> yes
[07:24] <lifeless> because the oops itself has to be bson serializable
[07:24] <jamesh> right.  Was just wondering if you did the formatting prior to generating the report
[07:25] <lifeless> nah, its lazy
[07:25] <lifeless> so just-in-time, and no formatting if no report
[07:25] <lifeless> I think the formatter can live in timeline itself
[07:26] <lifeless> as it doesn't need to import oops by default
[07:26] <lifeless> for packaging it can 'enhances: python-oops'
[07:27] <jamesh> I wouldn't be opposed to making it a sub-package of storm, but it could also be its own module
[07:28] <jamesh> [I assume you don't want this in python-oops-wsgi]
[07:28] <lifeless> right, I don't :)
[07:28] <lifeless> (because storm is the extending thing here - there can be many orms that need oops-wsgi glue, but only one oops-wsgi for now)
[07:28] <wgrant> zomg, p-d-r didn't crash on DF
[07:33] <rvba> wgrant: Hi, could you please update DF for me? (And BTW, thanks for the review, I'll address your concerns shortly)
[07:33] <wgrant> rvba: Did you see I QAed your resolved difference thing?
[07:33] <wgrant> (on qastaging)
[07:34] <rvba> wgrant: yes I did.  Thank you for that.
[07:34] <wgrant> I've got p-d-r running on DF now, so it'd be nice if I didn't have to update.
[07:34] <wgrant> But if you do need it, I can.
[07:34] <rvba> wgrant: no worries, I'll do that later today.
[07:34] <wgrant> it should be done in ~1.5 hours.
[07:34] <rvba> Okay.
[07:49] <adeuring> good morning
[08:07] <mrevell> Hello
[08:21] <jml> wgrant: sure
[08:26] <wgrant> jml:
[08:26] <wgrant> Er.
[08:26] <wgrant> Thanks.
[08:47] <StevenK> Now part 2 of demolishment can land?
[09:09] <wgrant> StevenK: Yup.
[09:12] <rvba> StevenK: Hi. The confirmation overlay stuff has landed. I'm here to help if you need me for some dirty Javascript stuff ;)
[09:17] <LPCIBot> Project devel build #1,064: FAILURE in 4 hr 9 min: https://lpci.wedontsleep.org/job/devel/1064/
[09:22] <lifeless> wgrant: oh hai
[09:22] <lifeless> wgrant: still reviewing ?
[09:22] <wgrant> lifeless: If necessary.
[09:22] <lifeless> wgrant: if so, gpgfixtures review request is up
[09:22] <wgrant> With pleasure.
[09:23] <wgrant> Is it?
[09:23] <lifeless> I think I have enough now that on thursday I'll be integrating it with LP, and with gpgverifyd
[09:23] <wgrant> I don't see it.
[09:23] <wgrant> Oh.
[09:23] <wgrant> *For* gpgfixtures?
[09:23] <lifeless> yes
[09:23] <wgrant> Not quite so exciting, but it'll do.
[09:26] <wgrant> Still no MP diff...
[09:26] <wgrant> Worrying.
[09:26] <wgrant> But no scriptactivity whinging yet.
[09:31] <StevenK> rvba: I've pushed my branch up: lp:~stevenk/launchpad/private-bug-unsubscribe-confirm
[09:32] <StevenK> rvba: If you bzr di -r submit: you can see what I've done, and where I've tried to add ConfirmationOverlay -- a helpful hint for that bit would be most welcome.
[09:33] <rvba> StevenK: looking.
[09:33] <jml> .Hi
[09:33] <jml> I'm trying to patch lazr.restfulclient
[09:33] <jml> In the absence of hacking instructions, I'm running bootstrap then buildout
[09:33] <jml> But buildout fails
[09:33] <jml> Error: Couldn't find a distribution for 'lazr.restful==0.16.0'.
[09:34] <wgrant> lifeless: assertNotEqual(None, [...])?
[09:34] <wgrant> lifeless: Shouldn't it be assertIsNot?
[09:34] <stub> allenap: In what circumstances will _loads get a byte string?
[09:34] <stub> allenap: re: your Storm MP
[09:35] <StevenK> rvba: It's 7:35pm here, so I'll be in and out
[09:35] <allenap> stub: Not sure :) Just being cautious because of weirdness in simplejson/json.
[09:35] <poolie> jml, wow good luck
[09:36] <stub> allenap: Right. I suspect you only need to force it it the _dumps(). The _loads() worries me as you are hardcoding UTF-8, which is incorrect.
[09:36] <jml> poolie: thanks.
[09:36] <jml> *someone* has to do it :)
[09:36] <rvba> StevenK: okay. I'll send you an email with comments ...
[09:36] <poolie> i don't know about that, but i did run in to confusion caused by its pth file
[09:36] <poolie> it seems hard to run it from source if you also have the package installed
[09:36] <rvba> StevenK: but I'm surprised that you don't even pass a submit_fn function to ConfirmationOverlay.
[09:37] <allenap> stub: From what I can remember, that's what json.loads() assumes given a byte string containing non-ASCII characters.
[09:37] <lifeless> wgrant: thats a shrug thing
[09:37] <lifeless> wgrant: they are equivalent
[09:37] <allenap> stub: I'm happy to remove the _loads() stuff though; I only need the _dumps() fix.
[09:37] <stub> allenap: If you do find byte strings being passed into _loads(), we need to know where they are coming from to confirm they are indeed UTF-8 (or what encoding, eg. if they are coming from a DB storing stuff in cp1252
[09:38] <lifeless> wgrant: (in this case)
[09:38] <rvba> StevenK: can you just give me an idea about how I can test this?
[09:38] <lifeless> stub: evening
[09:38] <lifeless> stub: Sorry about the shenanigans around pgbouncer
[09:38] <stub> allenap: If the tests pass, I think that would be best. I'll mention this with the rest of the review.
[09:38] <allenap> stub: So, perhaps there should be an assertion in there, that value is unicode?
[09:38] <lifeless> jml: you need a buildout cache
[09:39] <lifeless> jml: or to specify to buildout that it can run online
[09:39] <lifeless> jml: there is a shared cache like the lp one, for lazr libraries.
[09:39] <jml> lifeless: it was downloading all of the other things slowly enough that I thought it was online
[09:39] <stub> allenap: Fine with me.
[09:39] <lifeless> jml: hmm, perhaps it is then.
[09:39] <allenap> stub: Cool, thanks.
[09:39] <lifeless> jml: if so, is 0.16.0 still on pypi
[09:40] <wgrant> jelmer: Anything special about your import-colocated-branches branch?
[09:40] <stub> lifeless: No worries. I'd be interested in what jenkins saw. I doubt the stdout/stderr issue would be at fault, but perhaps the socket open caused problems in that environment?
[09:40] <wgrant> jelmer: It is not making the MP diff generator happy.
[09:40] <wgrant> At all.
[09:40] <jml> lifeless: umm, where exactly?
[09:40] <lifeless> stub: StevenK can hook you up with the hanging that happened
[09:40] <stub> StevenK: You have a pastebin of anything interesting?
[09:41] <lifeless> jml: http://pypi.python.org/pypi/lazr.restful
[09:41] <jml> lifeless: I don't see any download links there
[09:41] <lifeless> there is a link to lp's +download page there
[09:42] <jml> lifeless: and buildout can figure out to download the tarball from there?
[09:42] <lifeless> who knows
[09:42] <lifeless> I don't :)
[09:42] <jml> ok
[09:42] <stub> Can we switch to jenkins yet? Supporting both buildbot and jenkins envs seems a pain as they seem to flake out in different ways.
[09:42] <lifeless> stub: parallel test project may be a good time to switch
[09:42] <stub> Any blockers apart from time?
[09:43] <lifeless> I'm told IS are now actually running jenkins instances for other departments, so that hurdle is past.
[09:43] <jelmer> wgrant: Hmm, not that I'm aware of
[09:43] <lifeless> no blockers that I'm aware of; just need tuits to migrate processes across
[09:43] <lifeless> stub: hey, so great work on fastdowntime
[09:43] <stub> Yup. The second 90%
[09:43] <wgrant> lifeless: It needs to run in the DC, and not on an unsupported cloud.
[09:45] <gmb> wgrant: I'm going to assume that you're no longer OCRing.
[09:45] <wgrant> jelmer: Hm, and now it's recovered.
[09:45] <stub> wgrant: We can cannibalise the buildbot env if we want. Its just time and losa/dev coordination.
[09:45] <gmb> (For the purposes of the /topic)
[09:45] <wgrant> gmb: I was just about to remove myself.
[09:45] <wgrant> Thanks.
[09:45] <gmb> np
[09:46] <stub> lifeless: Ta. Seems to be working as planned.
[09:46] <wgrant> stub: http://www.postgresql.org/about/featuredetail/feature.213 looks handy.
[09:46] <wgrant> stub: Is that what we needed for branchrevision?
[09:46] <stub> wgrant: yes.
[09:46] <StevenK> rvba: The easiest way to test it is to create a local user by using make-lp-user, and then create a private bug in a harness with the owner as the user you created. Then browse to the bug, log in as yourself, hit Edit bug mail, and then the unsubscribe link.
[09:46] <wgrant> stub: You may be interested to know that we had similar lag spikes both times this week.
[09:47] <wgrant> From :25-29 or so.
[09:47] <wgrant> I think it's karma finishing.
[09:47] <StevenK> stub: You wanted to know about the indexes I needed?
[09:47] <rvba> StevenK: okay, thanks.
[09:47] <lifeless> is this week ubuntu beta or next?
[09:47] <StevenK> lifeless: Beta 1 is already out, dude.
[09:47] <stub> wgrant: Yup. I expect it somewhat. Did the last run have the lag high water mark set to 60 seconds?
[09:47] <lifeless> stub: not 1.
[09:47] <lifeless> bah
[09:47] <lifeless> StevenK: not 1.
[09:47] <StevenK> Fail :-P
[09:48] <wgrant> Beta 2 is the 22nd
[09:48] <wgrant> So next week.
[09:48] <StevenK> lifeless: Look at the ReleaseSchedule? :-P
[09:48] <wgrant> stub: I'm not sure; I didn't see numbers.
[09:48] <StevenK> rvba: I'm surprised that made any sense :-P
[09:48] <lifeless> actually https://dev.launchpad.net/DowntimeDeploymentSchedule is what I needed
[09:48] <lifeless> we have a multi-day no-fdt period during ubuntu release weeks
[09:49] <stub> So I'd love if there is a push button make-virtual-lp-dev-environment script for when I switch to Oneric, and it is an opertunity to get everyone into identical dev environments.
[09:49] <rvba> StevenK: ;)
[09:49] <lifeless> stub: the lxc stuff is pretty close to that.
[09:50] <StevenK> stub: Oh, Jenkins -- https://lpci.wedontsleep.org/job/devel/lastFailedBuild/consoleText
[09:50] <cjwatson> wgrant: care to give https://launchpad.net/~mvo/+archive/apt-ftparchive-lucid/+packages a spin?
[09:50] <lifeless> stub: for FDT going forward - I would like us to be very strict one-patch-at-a-time for the next few weeks.
[09:50] <stub> lifeless: agreed
[09:50] <StevenK> stub: I wanted to get the indexes in http://bazaar.launchpad.net/~stevenk/launchpad/denorm-bspph-indices/view/head:/database/schema/patch-2208-87-1.sql applied to prod, what do you think?
[09:50] <lifeless> stub: prove our reliability and get some feeling for how things behave.
[09:51] <lifeless> stub: cool, thanks!
[09:51] <jelmer> wgrant: it has a prerequisite which was already merged, perhaps that's confusing it?
[09:51] <stub> StevenK: Fine.
[09:51] <jelmer> wgrant: I got two "Launchpad internal failure" emails, which I couldn't explain. Would those perhaps be related?
[09:51] <lifeless> ok, gnight everyone - see you thursday.
[09:51] <jml> lifeless: g'night
[09:51] <stub> StevenK: Is there somewhere to log the live patch requests? I forget.
[09:51] <wgrant> jelmer: Do they have OOPS IDs?
[09:51] <wgrant> lifeless: Night!
[09:51]  * jml downloads a likely looking tarball and stuffs it into a cache
[09:51] <stub> lifeless: o7
[09:52] <StevenK> stub: So I should craft a script and prod a friendly LOSA?
[09:52] <wgrant> jml: I wonder if 0.16.0 has migrated off the first +download batch, so buildout can't see it any more.
[09:52] <jml> wgrant: that occurred to me also
[09:52] <StevenK> stub: Which worked out *so* well for me last time :-(
[09:52] <stub> StevenK: Nah, me or the losas do that. I'll get to it later, but if there is somewhere to log them then there is less of a chance of it being lost :)
[09:52] <jelmer> wgrant: yep, OOPS-2082MPJ3
[09:53] <StevenK> stub: I'll give you a friendly prod in a few hours for an update?
[09:53] <lifeless> StevenK: losa explicitly won't do these
[09:53] <lifeless> StevenK: they want it automated before they touch it.
[09:53] <jml> *sigh*
[09:53] <stub> StevenK: This is all new processes so we expect issues.
[09:53] <StevenK> Right.
[09:54] <jml> so, ./bin/test is supposed to work after buildout has run successfully, right?
[09:55] <wgrant> cjwatson: Looks good.
[09:55] <wgrant> cjwatson: Doesn't hang, at least.
[09:55] <StevenK> rvba: You've managed to get into a testable position?
[09:55] <wgrant> Can't really give it a real test until it's on DF.
[09:55] <stub> lifeless: So a minor screwup happened when a series of scripts were put together and losa run, but the first one failed (setting a statement timeout for concurrent index rebuilds is borked). I think I need to explicitly sign off scripts to be run by losas for a while until we work out the edge cases and everyone is comfortable.
[09:56] <rvba> StevenK: I'm just reading the code now.
[09:56] <StevenK> stub: That sounds like a good plan.
[09:57] <rvba> StevenK: I reckon what you want to do is now exactly what ConfirmationOverlay was originally designed for.
[09:59] <rvba> StevenK: the only required parameter right now is a 'button' and AFAIK you won't be providing one.
[09:59] <lifeless> StevenK: StevenK makes sense to me
[10:00] <lifeless> bahhh
[10:00] <lifeless> stub: makes sense to me
[10:01] <rvba> StevenK: I think I can modify the ConfirmationOverlay a little bit more and then all you will have to do is something like this http://paste.ubuntu.com/688208/
[10:04] <StevenK> rvba: Ah, you need to make button optional?
[10:04] <rvba> Exactly.
[10:06] <jml> OK.
[10:06] <jml> So if lazr.uri is in versions.cfg but not importable after buildout, what does that mean?
[10:12] <lifeless> its not in setup.py
[10:12] <lifeless> or in the transitive dependencies setup.py's.
[10:13] <cjwatson> wgrant: can you take care of that or do I need to ask for something else (e.g. copying into the Launchpad PPA)?
[10:15] <wgrant> cjwatson: I can copy it into the Launchpad PPA, but someone needs to get it into CAT.
[10:15] <wgrant> lifeless, jml: Or you are having namespace package issues with the lazr.* stuff in Ubuntu.
[10:16] <wgrant> lifeless: But you aren't here, are you?
[10:16] <jml> wgrant: I am here. I'm *guessing* I am.
[10:16] <jml> lifeless: it's in versions.cfg. Isn't that enough?
[10:16] <stub> StevenK: So your patch number is allocated to something different in the dbpatches branch
[10:16] <wgrant> jml: It needs to be in buildout.cfg
[10:16] <wgrant> Or setup.py.
[10:16] <wgrant> Depending on where you want it.
[10:16] <wgrant> versions.cfg just defines which version will be used if it's needed.
[10:17] <cjwatson> wgrant: I can do that via RT.  Is that needed to get it onto dogfood?
[10:17] <stub> StevenK: You or wallyworld_ need to change, and I suspect it is you since you're still around.
[10:17] <wgrant> cjwatson: Probably.
[10:17] <stub> (2208-87-2)
[10:17] <wgrant> mthaddon: How can we test a new apt on mawson?
[10:17] <lifeless> wgrant: I'm not.
[10:17] <lifeless> jml: versions.cfg says 'when you need X, choose version Y'
[10:17] <lifeless> jml: it does not say 'you need X'
[10:19] <stub> StevenK: Also, your MP seems to have disappeared
[10:19] <mthaddon> wgrant: you can ask for it to be installed there...
[10:19] <StevenK> stub: It has, yes. The branch still there
[10:19] <StevenK> stub: I can resurrect it
[10:19] <wgrant> mthaddon: Before it's in CAT? :)
[10:19] <stub> StevenK: It needs a MP, as it needs to land on db-devel.
[10:20] <jml> Adding it to install_requires and running buildout doesn't seem to change anything
[10:20] <mthaddon> wgrant: that would be part of what you're asking - for it to be in a -cat suite that's only on mawson
[10:20] <StevenK> stub: Let me jump to -87-2
[10:21] <wgrant> mthaddon: :(
[10:21] <wgrant> mthaddon: OK.
[10:21] <mthaddon> wgrant: why sad face?
[10:22] <wgrant> mthaddon: I was hoping for a quick wget and dpkg -i to test the fix. Not unexpected that it would be denied, but inconvenient :)
[10:22] <mthaddon> yeah...
[10:22] <wgrant> cjwatson: Can you file the RT?
[10:22] <wgrant> cjwatson: I'm trying to not be here.
[10:23] <StevenK> stub: https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-indices/+merge/75149
[10:23] <wgrant> Or I could just unpack the deb on mawson and hack LP to use a non-packaged version, but that might be evil.
[10:24] <stub> StevenK: Want me to allocate the patch number or are you doing that?
[10:24] <StevenK> stub: I'll do it now.
[10:25] <cjwatson> wgrant: sure.
[10:25] <StevenK> stub: Done.
[10:25] <wgrant> cjwatson: Thanks.
[10:29]  * StevenK prods ackee
[10:29] <StevenK> Can haz diff
[10:30] <wgrant> Argh.
[10:30] <wgrant> Maintenance people: merge-proposal-jobs is being slow and crashing sometimes.
[10:30] <wgrant> pls fix.
[10:30] <wgrant> gmb: ^^
[10:31] <gmb> wgrant: Is there a bug for it?
[10:32] <wgrant> gmb: Not for the stuff that's shown up tonight, no.
[10:32] <wgrant> gmb: But abentley has been working on related stuff.
[10:32] <wgrant> I would grab him, except he won't be around for 3 hoursish.
[10:32] <gmb> wgrant: Okay. I'll grab him when he appears.
[10:32] <gmb> Thanks for the heads-up
[10:32] <wgrant> gmb: It may be more urgent than that.
[10:33] <gmb> Hrm.
[10:33] <StevenK> stub: Requested a review of you on the MP, when you're ready
[10:33] <wgrant> it needs to be watched. it may recover again in a few minutes.
[10:33] <wgrant> But it took half an hour to work again when it died 2 hours ago.
[10:33] <gmb> wgrant: Okay. I'll keep an eye on it.
[10:34] <gmb> But my knowledge of it is close to 0, so I might be reduced to prodding and going "oh, it's dead"
[10:34] <gmb> But I'll do my best.
[10:34] <jml> Hmm. OK. I think this *is* a namespace issue
[10:35] <jml> my guess is that it's finding lazr.uri of the correct version on my system and thus not installing it.
[10:35] <wgrant> jml: Yeah, that's been a problem sometimes :/
[10:35] <wgrant> jml: Python namespace package support is awesome, see.
[10:35] <jml> wgrant: what's the solution?
[10:35] <wgrant> And lazr does a spectacularly bad job at it.
[10:35] <wgrant> Um.
[10:35] <wgrant> I tended to remove python-lazr.uri.
[10:35] <wgrant> But that's probably not practical any more.
[10:36] <jml> wgrant: that removes a lot of stuff. apport, for example
[10:37] <wgrant> Yeah, not very practical any more.
[10:37] <wgrant> What if you run with python -S?
[10:37] <wgrant> Like LP does.
[10:37] <wgrant> Yes, i know, that's Zope-grade evil, but it might work.
[10:37] <jml> ./bin/test already runs with python -S
[10:37] <jml> I guess ./bin/buildout needs to be run with that...
[10:37] <wgrant> Maybe,.
[10:38] <jml> nope, it is too
[10:39] <bigjools> jml is a lean mean bug filing machine
[10:39] <jtv> gmb: review pretty please w/cherry on top?  https://code.launchpad.net/~jtv/launchpad/bug-844550/+merge/75150
[10:41] <Riddell> dpm: hi
[10:41] <Riddell> what's a good time before a release to do a call for translations (for bzr)?
[10:41] <Riddell> two weeks?
[10:42] <dpm> Riddell, yeah, 2 weeks is a good rule of thumb
[10:45] <stub> StevenK: Can you please lp-land that branch? (no need to shove it through ec2) I'd do it, but lp-land is bitching about missing revisions
[10:45] <gmb> jtv: sure
[10:46] <jtv> thanks—feel free to skip much of the cover letter if it doesn't interest you.
[10:46] <jtv> Mostly just writing it for historical record, and as proof that this has been thought about and discussed.  :)
[10:48]  * jml moves on to launchpadlib
[10:48] <jtv> cjwatson: what ever happened to your multi-arch translations branch?
[10:48] <jtv> Still got a ticket on my board for landing it once I get word that it's been updated.
[10:48] <wgrant> It's landed and deployed.
[10:48] <wgrant> But not active, because we're waiting on an apt-ftparchive fix.
[10:49] <wgrant> jtv: ^^
[10:49] <jtv> OK, then I'll retire the ticket.
[10:52] <Riddell> dpm: and where's the place to announce it?
[10:55] <cjwatson> jtv: right, you can drop that ticket; I'll drive the apt-ftparchive deployment bits (though I'll need wgrant's help to test on dogfood once RT#47856 is resolved) and I'll deal with flicking the API switch at an appropriate time
[10:55] <_mup_> Bug #47856: Screenshot needs to be updated <Ubuntu Website:Fix Released> < https://launchpad.net/bugs/47856 >
[10:55] <cjwatson> jtv: thanks
[10:56] <jtv> Already done, thanks.
[10:56] <jtv> cjwatson: also, we're very close to deploying the final commit that will make all Debian SPPHs Published instead of Pending.
[10:57]  * cjwatson nods
[10:58] <cjwatson> the ubuntu-dev-tools change to just do status=Published is already out
[10:59] <rvba> gmb: Hi, could you please review this mp? https://code.launchpad.net/~rvb/launchpad/confirmationoverlay-button-optional/+merge/75154
[10:59] <gmb> rvba: Sure
[10:59] <rvba> Thank you.
[11:07] <gmb> jtv: I'm not sure whether to be worried or not: You branch is one of the clearest, best explained Soyuz branches I've ever read (though bigjools's branches are also usually equally clear). I must congratulate you, on the assumption that Soyuz hasn't got simpler and I haven't got any smarter - you've done an excellent job.
[11:08] <jtv> gmb: that's easy then—yes, you should be worried.  Thanks.  :)
[11:08] <gmb> :)
[11:09] <dpm> Riddell, I'd suggest launchpad-translators(AT)lists(DOT)launchpad(DOT)net and ubuntu-translators(AT)lists(DOT)ubuntu(DOT)com
[11:13] <gmb> jtv: approved, with one comment about making the test a bit easier to understand (at least for us lackwits)
[11:15] <jtv> gmb: thanks.  Insert obrant about why it's a good thing that you asked for an improvement.  :-)
[11:15] <gmb> :)
[11:19] <gmb> rvba: r=me with one minor tweak requested.
[11:20] <rvba> gmb: thanks.
[11:22]  * gmb lunches
[11:36] <nigelb> wgrant: hi
[11:36] <nigelb> wgrant: Were you able to reproduce the KHTML issue on your machine?
[11:36] <nigelb> I can't see it in Maverick.
[11:36] <StevenK> stub: Absolutely
[11:36] <stub> StevenK: I just landed that branch
[11:37] <StevenK> stub: How did you fix the missing revisions thing?
[11:37] <wgrant> nigelb: I haven't tested.
[11:38] <nigelb> Oh ok. I fear I may have to download debian sid just to test.
[11:38] <nigelb> Hrm, I wonder...
[11:38] <stub> StevenK: By using lp-land properly, as it only designed to land the branch you are in (so branch your branch and run lp-propose in that directory)
[11:38] <nigelb> I could install it into a chroot and ask it to use the right display.
[11:39] <StevenK> stub: Right, right. The indexes on are prod live now too?
[11:40] <stub> StevenK: All except the master, which is blocked until gina finishes up its current transaction
[11:40] <wgrant> Ah
[11:42] <StevenK> stub: Which could take hours
[11:42] <stub> StevenK: The statement should complete fine when it commits.
[11:43] <stub> StevenK: It won't take hours, as the transaction reapers are still enabled...
[11:43] <StevenK> Heh
[11:49] <wgrant> jelmer: Is anything blocking your two approved import branches from landing?
[11:49] <wgrant> import-colocated-branches and http-git-support
[11:49] <jelmer> wgrant: they're in ec2 at the moment
[11:49] <wgrant> jelmer: Excellent, thanks.
[11:49] <jelmer> wgrant: There was a test failure I had to fix and having a local lp setup was blocking me from that.
[11:49] <jelmer> (yay for lxc)
[11:50] <nigelb> Oh, you got it working?
[11:50]  * nigelb is tempted to upgrade to Oneiric.
[11:51] <jelmer> nigelb: oneiric is the reason I'm trying lxc in the first place :)
[11:51] <jtv> Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another
[11:51] <jtv> Upcoming buildbot failure.  :(
[11:51] <nigelb> jtv: I thought wgrant was the nostradamus here. :P
[11:52] <wgrant> I actually noticed it a couple of minutes ago, and was searching through the old builds to find what leaked.
[11:52] <jtv> Oh, there's a whole bunch more.  :( :(
[11:52] <wgrant> But I can't find anything.
[11:52] <wgrant> This is spurious, but it means our diagnosis was wrong.
[11:52] <jtv> AttributeError: 'TestPullerMasterIntegration' object has no attribute '_lock_actions'
[11:52] <wgrant> There's more broken than we thought.
[11:52] <jtv> This is going to hold up the commit for Gina.
[11:52] <stub> Is the lxc setup pretty well scripted or documented, or does it require a clue?
[11:52] <wgrant> stub: It's well-documented.
[11:52] <wgrant> stub: Works best on oneiric, but mostly works on natty.
[11:53] <wgrant> https://dev.launchpad.net/Running/LXC
[11:53] <stub> This will be Oneiric
[11:53] <stub> ta
[11:53] <jtv> wgrant: that AttributeError doesn't strike me as particularly spurious, especially given the number of errors.
[11:53] <wgrant> jtv: You would think.
[11:53] <wgrant> But it is.
[11:54] <wgrant> jtv: This is one of the types of error that's been showing up for 3 weeks now.
[11:54] <wgrant> We'll clean leaked processes from the buildbot slave, force the build, and it will be happy again.
[11:54] <wgrant> And then we'll abandon buildbot in favour of Jenkins, and all will be good.
[12:06] <StevenK> rvba: Any news re: button becoming optional ?
[12:06] <rvba> StevenK: The branch is in ec2.
[12:06] <StevenK> rvba: Nice, thanks!
[12:07] <rvba> StevenK: https://code.launchpad.net/~rvb/launchpad/confirmationoverlay-button-optional/+merge/75154
[12:07] <rvba> StevenK: I've added an example on the MP.
[12:07] <rvba> I mean an example usage*
[12:12] <nigelb> rvba: Love your example :)
[12:12] <rvba> nigelb: ;)
[12:23] <jtv> bigjools, rvba, StevenK, wgrant: mind a dogfood update right now?
[12:24] <rvba> jtv: fine by me.
[12:27] <wgrant> jtv: Go ahead, if you haven't already.
[12:27] <jtv> thx
[12:41] <nigelb> Did I break buildbot? :(
[12:41] <nigelb> jtv: Is this the failure you predicted?
[12:41] <jtv> nigelb: I don't think you broke it.
[12:41] <jtv> You may have broken one of the pieces that you see on the floor, but…
[12:42] <nigelb> heh
[12:42] <jtv> But yes, this is the failure I predicted.  Now bow before your oracle!
[12:42] <wgrant> Ah, gary_poster!
[12:42] <wgrant> gary_poster: buildbot is being friendly again, this time without an unnoticed failure in the previous run :(
[12:43] <gary_poster> wgrant, joyful
[12:43] <wgrant> Rather.
[12:51] <allenap> Is there likely to be any fallout from running bzr upgrade lp:lazr.uri?>
[12:53] <gary_poster> wgrant, did you look at it?  I have other things going on, but ISTM that lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another broke, for unknown fragility reasons, and everything else fell apart afterwards because of it.  I'll simply disable the test and file a bug unless you or someone else has a better idea.
[12:53] <wgrant> gary_poster: I hope so.
[12:53] <wgrant> gary_poster: Sounds like a good plan.
[12:53] <gary_poster> cool
[12:54] <stub> Any objections to me killing staging and qastaging for a few seconds to QA my Librarian outage behaviour revision?
[12:55] <gary_poster> allenap, thinking...yes...
[12:55] <gary_poster> allenap, oh we do that with eggs
[12:55] <gary_poster> so it's probably fine, actually
[12:56] <allenap> gary_poster: Cool. Let the mayhem begin...
[12:56] <gary_poster> (the only thing I know of that we have to worry about with bzr upgrade scenarios is buildbot and ec2 images for stuff we get in sourcecode)
[12:56] <allenap> That didn't last long; I can't write.
[12:56] <gary_poster> heh
[12:57] <gary_poster> LET THE MAYHEM BEG...oh.
[12:57] <gary_poster> odd that you can't write.  Lemme look
[12:57] <allenap> gary_poster: It's PQM managed. Didn't realise before.
[12:57] <gary_poster> ah ok allenap
[12:58] <gary_poster> hm, I didn't think we had our "extras" under PQM control
[12:59] <allenap> Yeah, odd. Oh well. I think I'll gently put that can of worms back on the shelf (i.e. I don't want to break PQM).
[13:00] <gary_poster> k
[13:00] <jml> jkakar: hey, fwiw, I'm trying to resurrect https://code.launchpad.net/~jkakar/launchpadlib/fake-launchpad/+merge/26391
[13:12] <jtv> wgrant: I wonder if I shouldn't just remove Gina's dry-run mode, if it's not even tested anyway.  Far too risky.
[13:12] <wgrant> jtv: It's not tested?
[13:12] <wgrant> It's clearly not tested very well.
[13:12] <wgrant> But it may be tested.
[13:12] <jtv> Well… an unconditional commit did not break any tests.
[13:13] <wgrant> It doesn't somehow abort earlier?
[13:13] <jtv> Nope, nope, nope.
[13:13] <wgrant> Awesome.
[13:13] <jtv> Insert a commit somewhere in an obscure place to reduce transaction durations, and kaboom.
[13:13] <jtv> May already be happening for all we know.
[13:13] <wgrant> That's quite useful.
[13:14] <jtv> bigjools: your guidance wanted.
[13:14] <bigjools> whu what?
[13:15] <jtv> bigjools: Gina has a dry-run mode.
[13:15] <jtv> It seems to run the whole script in one huge transaction, and then not commit.
[13:15] <jtv> Not tested.
[13:16] <jtv> We don't know of it works as advertised.
[13:16] <bigjools> ok
[13:16] <jtv> Want me to test it (script run in test, and who knows, fixes) or remove it?
[13:16] <bigjools> first question, do we need a dry run mode?
[13:17] <bigjools> IOW, how does it help us?
[13:17] <jtv> Well since young Will here had no idea it even existed, I was wondering whether you might know the answer.
[13:17] <bigjools> I don't I'm afraid, the Gina code has been barely touched in years, well before I started working on LP
[13:18] <bigjools> but I can help with some probing questions
[13:18] <bigjools> which might be slighly leading
[13:18] <jtv> I was nowhere near the cottage.
[13:18] <jtv> Not that there was a cottage!
[13:18] <bigjools> I hereby name you Geoff
[13:19] <jtv> Geoff?
[13:21] <nigelb> I fully expected bigjools to give a Yoda-ish answer. bigjools, I am disappoint.
[13:21] <bigjools> it was too risque, so I PMed it :)
[13:22] <nigelb> lol
[13:22] <nigelb> Yeah, channel' logged. Good idea.
[13:22] <nigelb> This reminds me. Is #launchpad-reviewers used at all?
[13:23] <jtv> *I* was of course referring to the opening episode of The Black Adder.
[13:23] <jtv> nigelb: no, that's obsolete.
[13:23] <bigjools> nigelb: no it's not
[13:24] <nigelb> Hrm, okay.
[13:24] <jtv> bigjools: so about these probing questions you promised..?
[13:24] <nigelb> I noticed it in the logs.
[13:24] <bigjools> jtv: I asked the first above, no answer yet!
[13:25] <jtv> Does it help us?  “Not me, not William, clearly not you.  Someone else out there maybe, but who would know but Julian?”
[13:25] <bigjools> jtv: I cannot think of a use for it either
[13:26] <jtv> So… zap it?
[13:26] <bigjools> yup, blow it
[13:26] <bigjools> away
[13:27] <nigelb> More desctruction \o/
[13:27] <nigelb> *destruction
[13:27] <jtv> Well, with the "away" added, yes.
[13:27] <nigelb> BWAHAHA
[13:28] <LPCIBot> Yippie, build fixed!
[13:28] <LPCIBot> Project devel build #1,065: FIXED in 4 hr 10 min: https://lpci.wedontsleep.org/job/devel/1065/
[13:35] <rvba> abentley: Hi.  I'd like your help to make codehosting run locally. I'm failing to push branches locally and see them on my local instance.
[13:35] <rvba> I'm reading  https://dev.launchpad.net/Code/HowToUseCodehostingLocally.
[13:35] <abentley> rvba: on the phone
[13:36] <rvba> abentley: Okay, please ping we when you're available then (http://paste.ubuntu.com/688322/ is what I've done).
[13:36] <StevenK> rvba: Are you using make run_codehosting ?
[13:36] <rvba> I also tried to create the missing directory manually but then nothing seems to be picked up by make sync_branches.
[13:37] <rvba> StevenK: yes.
[13:37] <StevenK> rvba: bzr push lp://dev/~foo/bar/baz should work with the following .ssh/config snippet: http://pastebin.ubuntu.com/688323/
[13:40] <wgrant> Damn, sinzui is quick.
[13:40] <wgrant> And he's not even here.
[13:40] <nigelb> lol
[13:41] <nigelb> wgrant: Did he reply to your email?
[13:41] <wgrant> https://code.launchpad.net/~wgrant/launchpad/delete-more/+merge/75179
[13:41] <nigelb> Neat.
[13:41] <nigelb> Its all a hidden plot to not make you sleep :)
[13:41] <wgrant> He reviewed it within 2 minutes :(
[13:42] <bigjools> wgrant: there's a subtle bug in that MP
[13:42] <bigjools> you need dedent("""\ (with the backslash)
[13:43] <wgrant> I hate that.
[13:43] <bigjools> yep
[13:43] <wgrant> So I called strip() instead.
[13:43] <bigjools> hmm
[13:43] <bigjools> missed that
[13:43] <bigjools> dunno which is more ugly tbh
[13:43] <wgrant> I know. But one doesn't involve a backslash, so I tend to go with that.
[13:43] <bigjools> it's once place where I prefer it
[13:43] <wgrant> I don't know.
[13:44] <bigjools> dedent sucks
[13:44] <wgrant> Derivation is done.
[13:44] <wgrant> Yay
[13:44] <wgrant> Interesting...
[13:44] <bigjools> stalker
[13:45] <wgrant> Well, DSD creation seems to be sort of screwed.
[13:45] <bigjools> ?
[13:45] <wgrant> https://launchpad.net/bilimbitest/angry/+localpackagediffs
[13:45] <wgrant> Ahh, maybe the jobs haven't run yet.
[13:45] <wgrant> So they could all be resolved in a few minutes.
[13:45] <rvba> StevenK: this is what I get now :( http://paste.ubuntu.com/688331/
[13:46] <wgrant> In fact, they're vanishing as I watch.
[13:46] <wgrant> Or appearing.
[13:46] <rvba> And I can see the connection failing in the logs of make run_codehosting
[13:46] <bigjools> it pulls from RELEASE remember
[13:46] <wgrant> bigjools: Ah, and DSDs look at updates too.
[13:46] <bigjools> yes
[13:47] <wgrant> Yeah, the ones I saw first were the same version.
[13:47] <wgrant> But the jobrunner killed those off quickly.
[13:47] <wgrant> Now we are settling down into sensible -updates or -security diffs.
[13:47] <bigjools> there's an argument to pull from -security as well I guess
[13:48] <allenap> gmb: Got time for a ~200 liner? https://code.launchpad.net/~allenap/launchpad/series-init-failure-explanations-bug-835024-db/+merge/75183
[13:48] <gmb> allenap: For you? Sure
[13:48] <StevenK> rvba: I am about to head to bed, but you want to run that against a DB that has had make-lp-user ran across it, and then mkdir foo ; cd foo ; bzr init ; bzr push lp://dev/~rvba/+junk/foo
[13:48] <allenap> gmb: You're very kind :)
[13:48] <rvba> StevenK: that's precisely what I did.
[13:48] <rvba> StevenK: good night :)
[13:48] <gmb> Or misguided :)
[13:48] <StevenK> rvba: Then that's terrible.
[13:49] <rvba> StevenK: I'll ask abentley when he is available.
[13:50] <abentley> rvba: I think you have a stale socket file for the codehosting forker process in /tmp
[13:50] <wgrant> rvba: Check that you don't have a stale forking service pid in /var/tmp
[13:50] <wgrant> yeah, socket, not pid
[13:50] <wgrant> What he said.
[13:50]  * rvba checks
[13:53] <rvba> abentley: wgrant that was it. Branch pushed all right ... let see if make sync_branches works.
[13:57] <rvba> abentley: StevenK wgrant make sync_branches worked! Thanks for your help guys.
[13:57] <abentley> rvba: You're welcome.
[13:57] <wgrant> rvba: Excellent.
[13:58] <wgrant> abentley: merge-proposal-jobs blew up a bit this evening, from around 5 hours ago.
[13:58] <wgrant> abentley: You may want to check the logs to see if it's anything new.
[13:58] <wgrant> As I see you have yet more improvements to this area.
[13:59] <gmb> allenap: Am I right in thinking that the reason for this branch is that it does most of the encode/decode work for us and we're left to just use loads() and dumps() as appropriate?
[13:59] <nigelb> wgrant: Well done driving abently away :P
[13:59] <gmb> Oh...
[14:00] <gmb> allenap: Or does the Storm JSON type avoid us having to use loads() and dumps() at all?
[14:00]  * gmb might be getting confused by the diff
[14:00] <allenap> gmb: We don't have to do the loads() or dumps() either.
[14:00] <gmb> Right
[14:00] <allenap> gmb: There's a bulk job creation bit which still needs to dumps() because it composes SQL directly.
[14:00] <gmb> Okay
[14:01] <abentley> wgrant: none of my improvements have been deployed yet.  I had to roll them back when QA failed.
[14:03] <wgrant> abentley: Ah :( that would explain the failures that I thought were fixed.
[14:03] <gmb> allenap: All looks sane. r=me
[14:03] <allenap> gmb: Thanks!
[14:03] <gmb> np
[14:08] <jkakar> jml: Awesome!  I've basically ditched it... but I'm happy to help if need be.
[14:08] <jml> jkakar: thanks.
[14:08] <jml> jkakar: actually, I do have one question about _create_resource
[14:08] <jml> jkakar: it seems that if you get something back from a method, you have no good way of determining if it's a collection or an entry.
[14:08] <jkakar> jml: Am on a call, but ask here now, I'll take a look at the code again, and answer when I'm off the phone.
[14:09] <jml> jkakar: ok. thanks.
[14:10] <nigelb> When does devel get merged into db-devel? By buildbot?
[14:10] <wgrant> nigelb: buildbot-poller merges stable into db-devel once buildbot passes it.
[14:10] <nigelb> so, I have to wait because of builtbot breakage.
[14:10] <bigjools> I hate browsers
[14:11] <wgrant> nigelb: You can't land your DB patch until we have deployed your stuff, anyway.
[14:11] <nigelb> Argh, ok.
[14:11] <nigelb> Need to just work on js-oopsd then.
[14:16] <nigelb> wgrant: I call conspiracy on that ;P
[14:16] <wgrant> Oh?
[14:17] <wgrant> It is, I admit, rather convenient, since I want the window tomorrow for meeeee. But it's also true!
[14:17] <nigelb> HA!
[14:17] <nigelb> Also, I don't think I'll ever forget about grepping the whole tree next time.
[14:18] <wgrant> bzr grep is your friend.
[14:18] <nigelb> I've lost count of the number of times StevenK and bigjools have already told me that.
[14:35] <jtv> gmb: perhaps you'd like to review this one as well?  Not much to see, really.  https://code.launchpad.net/~jtv/launchpad/bug-848954/+merge/75188
[14:35] <gmb> jtv: Sure.
[14:35] <stub> nigelb: Your db patch dropping BugTask.statusexplanation had test failures. Didn't look like anything taxing but I didn't have a chance to look today.
[14:36] <gmb> jtv: Points for referring to wgrant as "Young Will"
[14:36] <jtv> heheh
[14:40] <gmb> jtv: Approved with one question about a somewhat opaque (to me, anyway) error message.
[14:41] <jtv> :)
[14:41] <jtv> Thanks
[14:42] <jtv> gmb: I suspect nobody's going to know much more about that message, so I might as well try to do something about it.  Hang on.
[14:42] <gmb> Ok
[14:46] <jtv> gmb: if it's any consolation, I've already asked Julian for permission to be left alone in a locked room with Gina and Katie for a while, so I can give them a stern talking to.
[14:46] <gmb> ...
[14:46] <gmb> E_NO_CORRECT_RESPONSE
[14:47] <gmb> jtv: I'm not hung up on the idea of that message changing before it lands, but it seems like an opportune moment.
[14:48] <jtv> Yeah, this is stuff that can do with any drive-by improvement.
[14:48] <jtv> Including the one that this branch is about: “Oh, there may be a feature there.  I think I broke it.  Let's destroy it.”
[14:51] <jtv> gmb: the diff has updated.
[14:51]  * gmb looks
[14:51] <jtv> For some reason the page won't display very legibly on my browser.
[14:52] <jtv> But that's probably just a bad connection.
[14:52] <gmb> jtv: It looks good from here. Nice change, thanks!
[14:53] <jtv> Thank you.
[15:12] <jml> jkakar: also, I think I'm going to kill the testresources stuff
[15:16] <nigelb> Hrm, just missed stub
[15:21] <jkakar> jml: That was a long call... am off now.
[15:22] <jml> jkakar: yay
[15:22] <jml> jkakar: my earlier question still stands...
[15:22] <jkakar> jml: Uhm, if I remember correctly, the reason for not knowing whether something is a collection or not is that you can't (or couldn't, maybe you can now) tell whether something is a collection from the WSDL.
[15:22] <jml> jkakar: right. so leonardr's comment about trying to return a FakeCollection from methods is ... invalid?
[15:22] <jkakar> Maybe there's a convention that you can use to tell... or something?  I rememeber it being pretty confusing (WSDL in general, in addition to the way it's used by Launchpad).
[15:22]  * jkakar looks again
[15:24] <jml> jkakar: I'm ignoring lifeless's suggestion about including a timestamp and doing IMS. Also ignoring suggestions to move to lazr.restfulclient.
[15:24] <jkakar> jml: I don't understand leonardr's comment/question/suggestion.
[15:24] <jml> jkakar: I'm currently preparing a smaller wadl file for testing
[15:24] <jkakar> jml: Awesome!
[15:24] <jml> jkakar: and have replaced the wadl file that you included with the official 1.0 wadl file
[15:24] <jkakar> jml: Why do you want to drop testresources support...?
[15:24] <jml> jkakar: because it makes it harder to use alternate wadls
[15:24] <jkakar> jml: Yeah, flacoste was quite keen to have it be downloaded when tests run, as I recall.
[15:24] <jml> jkakar: not "drop" per se, just not export it
[15:25] <jkakar> jml: You could pass one in, I guess... but anyway, the testresources stuff is sugar, not a big deal.
[15:25] <jkakar> jml: I'm not very happy with a test suite that *needs* net access, but I could live with it.
[15:25] <jml> jkakar: I've also added FakeEntry and FakeRoot
[15:25] <jml> jkakar: I'm not making it need net access
[15:26] <jml> jkakar: just bundling 1.0 and putting a suggestion in the docs that you might like to download your own to test against.
[15:26] <jkakar> jml: Cool... I meant that in reference to the (old) request from flacoste that the WSDL be downloaded automatically.
[15:26] <jml> jkakar: oh right. ugh.
[15:26] <jml> jkakar: also, I've expanded the docstring to http://paste.ubuntu.com/688401/
[15:27] <jkakar> jml: Anyway, I'm really happy you're picking up that branch.  Sorry the code is so crap... it was a bit of a discovery process, since I discovered the WSDL file as I went.
[15:27] <jml> jkakar: I'm going to ignore poolie's suggestions about changing the API
[15:27] <jml> jkakar: and about not doing XML parsing
[15:27] <jml> jkakar: the code is fine. I still haven't discovered WSDL yet, so maybe I'm behind :)
[15:28] <jkakar> jml: I remember being surprised that wadllib didn't actually do much with WADL files.  I expected it to be an API to the WADL file, but it wasn't (maybe that's changed, it's been a long time).
[15:28] <jml> jkakar: oh yeah :)
[15:28] <jkakar> jml: WSDL is, uhm, fantastic! :)
[15:29] <jml> jkakar: I think I'll leave that particular box closed for now.
[15:29] <jkakar> jml: Cool.  Great work on the docstring, thanks.
[15:30] <jml> hmm
[15:30] <jml> does having the docs in the wadl actually *help* the client?
[15:34] <jkakar> Now I'm totally confused about the difference between WADL and WSDL... uhm, I think I've been using them interchangeably...
[15:34] <jkakar> I guess we care about WADL, not WSDL, only because we have a wadllib and not a wsdllib. :)
[15:48] <jml> OK. Done.
[15:48] <jml> gmb: Can you please review: https://code.launchpad.net/~jml/launchpadlib/fake-launchpad/+merge/75211
[15:48] <gmb> jml: Sure.
[15:48] <jml> gmb: thanks.
[15:48] <nigelb> Hi! I'd like to run this by someone for sensibility. Javascript related.
[15:49] <nigelb> Currently, we harvest bug links and send them to an internal API for checking if they're valid/accessible.
[15:49] <jml> gmb: I have to go in 10m, fwiw.
[15:49] <nigelb> Can I add a feature to pull out bug title from that?
[15:49] <gmb> jml: Okay, no woirries. I'll put any questions in the mp.
[15:51] <nigelb> deryck: Hi, around?
[15:52] <deryck> nigelb, hi, I am.  Just about to break for lunch though.  but what's up?
[15:52] <nigelb> deryck: I saw you were marked as javascript person, so I just wanted to run something by you
[15:52] <jkakar> "<jml> Not doing this. If you really care about it, perhaps you could do it." <-- Awesome. :)
[15:53] <deryck> nigelb, ok, cool.  go for it.
[15:54] <adeuring> gmb: could you please review this mp: https://code.launchpad.net/~adeuring/launchpad/cronscript-for-hwdb-report-fixes/+merge/75214 ?
[15:54] <gmb> jml: E_5000_LINE_DIFF
[15:54] <nigelb> deryck: I'd like to add to the javascript that checks if the linked bugs in pages are valid by updating the "title" attribute of the <a> with the bug's title
[15:55] <gmb> adeuring: I shall add it to my queue; not sure I'll get to it by EOD.
[15:55] <jml> gmb: most of that is boring WADL diff.
[15:55] <adeuring> gmb: ok, np
[15:55] <jml> gmb: there's instructions on the linked mp for how to get a sane diff.
[15:55] <gmb> jml: Ah, good, I'm just waiting for that to load...
[15:55] <deryck> nigelb, ok, sounds fine
[15:55]  * gmb wonders if it's LP or his connection that's being odd
[15:55] <nigelb> deryck: cool! I'll file a bug and start doing it :)
[15:56] <deryck> cool
[15:56] <nigelb> deryck: Thanks!
[15:56] <deryck> nigelb, subscribe me to the bug please
[15:56] <deryck> and np!
[15:56] <nigelb> okay :)
[16:04] <gmb> jml: The saner diff is still 979 lines. I'm afraid I won't be able to get through all of this before EOD, and I need to be gone at 6 on the dot tonight.
[16:04] <gmb> You might be better off finding someone at the start of tomorrow.
[16:11] <sinzui> compiz/unity is very unstable today. I am restarting as a last resort to get control of my windows.
[16:22] <nigelb> Is there documenation somwhere of what's returned by IBugTask.search?
[16:23] <nigelb> I see how to call it, parameters, etc.
[16:26] <danilos> matsubara, hi, I can't see OOPSes from ackee-bzrsyncd in OOPS reports (eg OOPS-2075SMS19) that's in /srv/launchpad.net-logs/scripts/ackee-bzrsyncd/branch-scanner/2011-09-06 on devpad; are these directories perhaps ignored?
[16:27] <matsubara> danilos, let me take a look
[16:27] <danilos> matsubara, thanks
[16:31] <danilos> sinzui, hi, do you think you'd have a minute to chat about bug 480123? basically, while it sounds the solution is "simple", I am sure we need to worry about what's going to break, and you probably have a few other ideas of what's to look out for?
[16:31] <_mup_> Bug #480123: Milestone names/version should be unique to series <escalated> <linaro> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/480123 >
[16:32] <sinzui> it is not simple
[16:32] <sinzui> We tried to fix that in 2009 and failed,. we reverted the schema changes after 2 months on no progress
[16:34] <sinzui> danilos, mumble, skype?
[16:35] <danilos> sinzui, skype works better for me, "danilo.segan"
[16:36] <gmb> adeuring: Approved, but you need to change the "# Disabled because of bug ..." comments to XXXs for clarity.
[16:36] <adeuring> gmb: ah, right! thanks!
[16:38] <gmb> np
[16:51] <danilos> abentley, hi, we have an escalated bug which is a (non-DB) timeout in "ScanBranch" jobs, could you perhaps look at bug 808930 and comment if you have any ideas of where one would start (i.e. could we somehow look into speeding these operations up for gcc-linaro project, checking if they are using stacked repos and whatever gives us the best performance)
[16:51] <_mup_> Bug #808930: Timeout running branch scanner job <escalated> <linaro> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/808930 >
[17:06] <abentley> danilos: I think the way to close this bug is to optimize bzr.
[17:07] <abentley> danilos: I would start by downloading the branch locally and profiling the script.
[17:08] <danilos> abentley, right, thanks for the input
[17:10] <abentley> danilos: it is already using the fastest bzrlib operations I know of.  I wrote the script, and I also wrote the Graph operations, several years before that.
[17:11] <abentley> danilos: it's possible the algorithm could be improved.  it's also possible that we have to switch to C or Pyrex to get that performance.
[17:12] <abentley> danilos: Or we may need to implement a greatest-distance-from-origin cache in order to get good performance.
[17:12] <abentley> danilos: jam has also done a lot of work in this area, and may have advice.
[17:14] <james_w> hi, can someone tell me what caching may be involved in producing/serving +temp-meeting-export?
[17:14] <james_w> hitting that URL repeatedly gives back results of varying staleness
[17:14] <nigelb> I was just about to ask :)
[17:14] <james_w> :-)
[17:15] <danilos> abentley, right, at the moment I was just looking for some more info other than "this is slow"; I hope you don't mind me pasting your input on the bug?
[17:16] <abentley> danilos: feel free.
[17:21] <danilos> abentley, thanks again
[17:21] <abentley> danilos: np
[17:24] <sinzui> danilos, lp:~edwin-grubbs/launchpad/bug-174468-milestone-release-overlap should contain chunk of code it its history that made milestones unique to the series. I do not see that in the history though. The branch lived for many weeks and it is very confusing.
[17:26] <sinzui> danilos, I can see from my discussion that changes to uniqueness will has a few themes of work: product-release-finder, project groups and ProjectMilestone, sorting milestones (wants debversion)
[17:27] <danilos> sinzui, right, thanks, I'll add that to my notes
[17:30] <danilos> sinzui, the DB patch in r7278 does change the unique constraint, fwiw
[17:38] <danilos> james_w, hi, do you think you'd have a few mins to talk about what you need out of bug 480123?
[17:38] <_mup_> Bug #480123: Milestone names/version should be unique to series <escalated> <linaro> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/480123 >
[17:38] <james_w> danilos, hi, yes
[17:38] <james_w> but not now
[17:38] <james_w> sorry
[17:38] <james_w> I need to get some food
[17:39] <james_w> danilos, I'll forward you what I sent to flacoste, and then you can let me know if it's sufficient
[17:39] <danilos> james_w, ok, I am not going to be around for much longer today, so we can perhaps talk tomorrow
[17:39] <danilos> james_w, excellent, that's even better
[17:40] <james_w> danilos, sent
[17:40] <james_w> I'll be happy to talk if it's not clear from that
[17:42] <danilos> james_w, cool, thanks, this should give me a better idea of what you need, I'll have to wrap my head around it first :)
[17:48] <nigelb> deryck: Hey, could you give me a quick hand with BugTask.search? How do I find out what it returns?
[17:50] <deryck> nigelb, I believe it returns a result set of bugtasks.  Let me look to confirm....
[17:50] <deryck> nigelb, how I'm checking is to go to the method in lp.bugs.model.bugtasks
[17:50] <nigelb> I'm there as well :)
[17:50] <nigelb> I'm looking at _search()
[17:50] <nigelb> since that's what seems to be called.
[17:50] <nigelb> I think storm is what's confusing me there.
[17:51] <deryck> nigelb, yeah, you're in the right place.  and see the comment about what it returns.
[17:52] <nigelb> hrm, yes
[18:41] <abentley> gmb: I suspect you're no longer OCR.
[18:42] <gmb> abentley: You suspect correctly. I shall remedy this falsehood presently.
[18:43] <abentley> deryck: Since there's no OCR, could you review https://code.launchpad.net/~abentley/launchpad/daily-build-disabled-archive/+merge/75239 ?
[18:44] <deryck> abentley, sure
[18:47] <deryck> abentley, r=me.  nice, succinct, and good change.
[18:47] <abentley> deryck: thanks.
[18:47] <deryck> abentley, np
[19:44] <nigelb> I thought you were supposed to be away.
[19:58] <nigelb> deryck: Still around?
[19:58] <nigelb> I er have an implementation confusion
[20:05] <deryck> nigelb, I am.  just tending to a few things, but ask away.  I'll answer as I can.
[20:21] <nigelb> Hrm. I may have sorted this. Verifying.
[20:22] <nigelb> Hrm. No.
[20:22] <nigelb> Currently, we post all the branch URLs and all the bug urls to +check-links
[20:22] <nigelb> This returns a JSON with invalid_branch_links and invalid_bug_links
[20:23] <nigelb> I'll how to now modify how this works a bit more non-trivially than I thought.
[20:55] <m4n1sh> is there a way to be subscribed to all bugs related to a product, but exclude selected tags. Example I want to free myself of all the apport bugs
[20:55] <m4n1sh> they have the tag "apport-crash"
[20:55] <nigelb> instead of +, use - I think.
[20:56] <cr3> in lazr.restful, is there a way to export a parameter, not an operation, as another name?
[20:56] <m4n1sh> nigelb: where?
[20:57] <nigelb> m4n1sh: URL
[20:57] <nigelb> m4n1sh: what project is this?
[20:57] <m4n1sh> nigelb: for this https://bugs.launchpad.net/ubuntu/+source/zeitgeist
[20:57] <m4n1sh> want to make myself free of apport-crash bugs
[20:58] <m4n1sh> hundreds of them are filed in a week
[20:58] <m4n1sh> duplicates
[20:58] <m4n1sh> due to zeitgeist being installed by default
[20:58] <nigelb> m4n1sh: https://bugs.launchpad.net/ubuntu/+source/zeitgeist/+bugs?field.tag=-apport-crash
[20:58] <m4n1sh> yes then?
[20:59] <m4n1sh> that is just for filtering
[20:59] <m4n1sh> but for subscription
[20:59] <nigelb> AH.
[20:59] <m4n1sh> looks like there is no such right now
[20:59] <m4n1sh> need to file a bug - to be marked as wishlist
[20:59] <m4n1sh> btw against which package should I file?
[20:59] <lifeless> sure there is
[20:59] <nigelb> I wonder if you can do -apport-crash in the subscription.
[21:00] <m4n1sh> launchpad-foundations
[21:00] <lifeless> use an advanced subscription
[21:00] <m4n1sh> lifeless: used that
[21:00] <m4n1sh> that is "include tags"
[21:00] <m4n1sh> lifeless: I want all minus specific tags
[21:00] <lifeless> ah, then perhaps file a bug yes.
[21:00] <m4n1sh> against which module?
[21:00] <lifeless> bugs.launchpad.net/launchpad
[21:00] <m4n1sh> ah. I thought foundations
[21:00] <nigelb> I don't think there are modules anymore.
[21:00] <m4n1sh> I mean like lazr.restful
[21:01] <m4n1sh> launchpad-foundations
[21:01] <m4n1sh> etc etc
[21:01] <nigelb> lifeless: Could you quickly review what I'm doing for the bug autolinkifying?
[21:02] <lifeless> m4n1sh: lazr.restful is a piece of code, launchpad-foundations was a sub-team of the launchpad team, but it no longer exists post restructure
[21:02] <m4n1sh> so right now how many pieces are there?
[21:02] <m4n1sh> only launchpad?
[21:02] <lifeless> m4n1sh: in what sense?
[21:03] <m4n1sh> like lazr.restful, launchpad etc etc
[21:03] <lifeless> organisationally or bits of software?
[21:03] <nigelb> lazr.restful is not launchpad only.
[21:03] <nigelb> things like malone, blueprints, are all launchpad.
[21:03] <m4n1sh> bits of software on which launchpad is dependent on
[21:03] <lifeless> m4n1sh: several hundred
[21:03] <m4n1sh> which are part of launchpad, but developed as seperate project
[21:03] <lifeless> m4n1sh: about 50 of which we are the primary developers of
[21:04] <nigelb> I think I added one to that list :P
[21:04] <m4n1sh> launchpadlib also comes under launchpad.net/launchpad ?
[21:04] <lifeless> https://launchpad.net/launchpad-project
[21:05] <lifeless> m4n1sh: no, only the launchpad source code is managed at /launchpad. Components etc all have their own bugtrackers. malone and so forth are not (currently) components, they are just part of the launchpad source code
[21:05] <nigelb> Yep. JS OOPS Daemon.
[21:05] <m4n1sh> ah yes
[21:05] <m4n1sh> that is what I was asking
[21:05] <m4n1sh> like loggerhead is developed seperate
[21:05] <m4n1sh> lazr.restful
[21:05] <m4n1sh> lazr.enum
[21:06] <m4n1sh> wadllib etc
[21:09] <m4n1sh> done
[21:09] <m4n1sh> https://bugs.launchpad.net/launchpad/+bug/849429
[21:09] <_mup_> Bug #849429: Subscription: Allow the option of selectively blacklisting certain bug tags. In short add tag blacklisting <Launchpad itself:New> < https://launchpad.net/bugs/849429 >
[21:20] <jml> gmb: thanks anyway
[21:21] <nigelb> I think I just murdered some javascript. Gah.
[21:22] <bac> nigelb: as they say in Texas: it needed murdering
[21:22] <nigelb> haha
[21:23] <nigelb> I took some perfectly generic javascript and made it specific.
[21:23] <nigelb> Just realized I can make it generic with my changes as well. So fixing that.
[21:24] <nigelb> How evil am I?
[21:24] <nigelb> We went from http://paste.ubuntu.com/688640 to http://paste.ubuntu.com/688639
[21:25] <nigelb> With changes in Javascript to deal with that.
[21:48] <flacoste> bac: you got my unprompted review :-)
[21:48] <bac> flacoste: great, thanks
[21:49] <nigelb> Now I understand why debugging launchpad js is nightmare.
[21:50] <flacoste> bac: well, you might not thank me after you read it :-)
[21:50] <bac> flacoste: i'd read it.  i was just being nice.
[21:50] <bac> :)
[21:51] <flacoste> bac: do these comments make sense? i might be dreaming
[21:51]  * flacoste does reality cehck
[21:51] <bac> flacoste: i think so.  i'll have to look at the db procedure that updates the date_last_message. shouldn't be a problem
[21:52] <bac> flacoste: i'd assumed (incorrrectly) that the triggers for that were sensible and what we wanted...
[21:53] <flacoste> bac: ok, thanks for working on that btw
[21:53] <bac> flacoste: np.  i'm off tomorrow but will wrap it up thursday
[22:08] <nigelb> Anyone around for some evil javascript review? I don't need a full review, but just help on whether it looks okay.
[22:16] <nigelb> flacoste: Do you have a quick few minutes?
[22:31] <nigelb> I guess not. I'll grab some other javascript-y person tomorrow.
[22:32] <lifeless> benji: re semver: you might also like reading the libtool soname rules
[22:33] <benji> sounds like my kind of reading material
[22:34] <nigelb> benji: Hey, could check something for evilness? :)
[22:35] <benji> nigelb: if it's pretty quick, I have two chilli-cheese-onion-dogs with my name on them
[22:35] <lifeless> benji: http://www.gnu.org/s/libtool/manual/html_node/Updating-version-info.html
[22:36] <lifeless> "try to set the interface numbers so that they correspond to the release number of your package. This is an abuse that only fosters misunderstanding of the purpose of library versions."
[22:36] <nigelb> benji: https://code.launchpad.net/~nigelbabu/launchpad/bug-title-849121/+merge/75267
[22:36] <nigelb> You don't need to actually do a full review
[22:36] <lifeless> bah, 'Never ...'
[22:36] <benji> thanks lifeless, I've put that in a tab in my reading-for-after-the-kids-are-in-bed window
[22:37] <nigelb> benji: Its a WIP. There's some code to be deleted in that. And *lots* of tests to be fixed. And javascript needs testing.
[22:37] <nigelb> But I need help with the sanity of my approach :)
[22:39] <benji> nigelb: it looks good to me, the only questionable thing is hand-building the bug links (line 35 of the diff), there is a mechanism to build those links for you
[22:39] <nigelb> benji: I don't think so. I can't use canonical_url for that.
[22:40] <benji> nigelb: may be, it's not a big sin (but I worry about the day we want to change our URL structure and all of these one-off URL generation sites have to be changed)
[22:40] <benji> I really have to go now.  Later.
[22:40] <nigelb> benji: Thanks!
[22:41] <nigelb> I shall make it better over the week :)
[22:41] <nigelb> Javascript - 0; Nigel - 1.
[22:46] <mwhudson> nigelb: is what you are doing going to make bug titles reappear when i mouse over "bug 1234" in text?
[22:46] <_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
[22:47] <nigelb> mwhudson: Yes
[22:47] <mwhudson> nigelb: hooray
[22:48] <nigelb> heh
[22:48] <mwhudson> nigelb: are you coming to orlando so i can buy you a beer?
[22:48] <nigelb> No, I'm not :)
[22:48] <mwhudson> ah well, i'm sure i'll get the chance eventually
[22:48] <nigelb> But I'll try to come for the next european one :)
[22:50] <nigelb> mwhudson: I took the generic XHR updating framework and extended it to deal with valid and invalid.
[22:50]  * nigelb dreads broken tests