[01:20] <wgrant> I think LP now feels only very slightly slow.
[01:48] <lifeless> its getting there
[01:48] <lifeless> still has that mainframe feel
[01:53] <wgrant> Yeah. But bug pages and listings load in just a couple of seconds now.
[01:53] <wgrant> Three years ago they freqently took >10.
[02:08] <lifeless> we're making progress :)
[02:08] <lifeless> +bug-index is going to be much happier on thursday
[02:09] <wgrant> Yeah.
[02:46] <lifeless> I think we'll need to do constraint based batch offsets soon
[02:47] <lifeless> sadly storm doesn't seem to have editable queries
[02:47] <lifeless> and given the contribution dynamic there, I don't fancy trying to do it upstream
[02:56] <sidnei> lifeless, :D
[02:57] <wgrant> What's the issue with contributions?
[02:58] <lifeless> wgrant: perfect is the enemy of the good, or thats how it feels to me so far
[02:58] <lifeless> wgrant: I think I've put up 4 patches and had none get in
[02:59] <wgrant> Hmm.
[02:59] <wgrant> We should fix that.
[02:59] <lifeless> wgrant: 2 got rewritten, one got bounced [no tests, which is fair enough, though finding tests in the storm layout is idiosyncratic]
[02:59] <lifeless> and one is stalled (the with patch)
[02:59] <lifeless> sidnei: hai :)
[03:00] <lifeless> wgrant: https://code.launchpad.net/storm/+activereviews
[03:01] <sidnei> lifeless, maybe you'll get a suggestion to go look at sqlalchemy with your next patch
[03:01] <lifeless> wgrant: its a lot healthier in absolute terms than LP; but in terms of reviews-per-dev, or reviews-per-loc, I think LP would measure better
[03:01] <lifeless> sidnei: hah!
[03:01] <lifeless> sidnei: so let me tell you about this idea
[03:02] <lifeless> sidnei: you know we have big collections in LP
[03:02] <lifeless> like ubuntu - 400K bugs
[03:02] <lifeless> sidnei: and a batch navigator that lets users page through the collection
[03:02] <sidnei> lifeless, im having dinner, but go on, i will read l8r
[03:02] <lifeless> sidnei: this breaks down spectacularly when the user is paging through (say) page 30
[03:03] <lifeless> or earlier (depends on the cost of calculating the batch)
[03:03] <lifeless> because the way we implement it is to say LIMIT [batch size] OFFSET [page number * batch size]
[03:04] <lifeless> LIMIT + constraint is a *much* better way to do it [plus reversing the sort order when walking backwards]
[03:04] <lifeless> e.g when we show the first page [or the last], we remember the sort order key of the last row.
[03:05] <lifeless> and rather than saying OFFSET [1 * batch size] to go to the next page, we say And(order_column1 > lastrow1, order_column2 > lastrow2, ...) LIMIT [batch size]
[03:06] <lifeless> to walk back, flip the sort order; to skip a page use an OFFSET as well as this technique. To go to the end, flip the sortorder take the first page of results.
[03:06] <lifeless> In LP we pass ResultSet objects to BatchNavigator, which then calculates # of pages, does the slicing etc.
[03:07] <lifeless> so, we need some way for the batchnavigator to:
[03:07] <lifeless>  - ask for the needed last-row-tuple
[03:07] <lifeless>  - pass in a last-row-tuple and say 'after this row please'
[03:08] <lifeless> just wrapping the query (using it as an inner query) is pretty much guaranteed to give poor query performance.
[03:08] <lifeless> (and by pretty much, I mean 'its not even worth trying')
[03:08] <lifeless> challenges I see are that storm doesn't understand the contents of its queries.
[03:10] <lifeless> so we're likely to get redundant constraints and so on - which pgsql can optimise out, but that often results in odd plans even so
[03:35] <lifeless> we may be best off redefining the contract
[03:35] <lifeless> pass BN a factory to create a page result set as well as a resultset for 'entire collection'
[03:39] <lifeless> wgrant: still around ?
[03:40] <wgrant> lifeless: Mostly.
[04:02] <lifeless> wgrant: where did you and wallyworld get to fixing that undefined thing
[04:28] <wgrant> lifeless: It's on qas.
[04:28] <wgrant> r12723
[04:32] <lifeless> cool, so we can deploy?
[04:33] <wgrant> Once we're QA'd past it, yeah.
[04:33] <wgrant> That's blocking on fixing lists.staging though, which I might try to get done tomorrow morning.
[04:34] <lifeless> staging or qastaging ?
[04:34] <wgrant> Either.
[04:34] <wgrant> But qastaging doesn't have one at all, so it's going to be more work.
[04:34] <lifeless> ah
[04:34] <lifeless> whatever you learn.... can you rt that towards a qastaging one in future ?
[04:34] <wgrant> Sure.
[04:35] <lifeless> thanks
[07:22] <lifeless> 275 timeouts
[07:22] <lifeless> hmm
[07:23] <jelmer> moinmoin
[07:23] <wgrant> Morning jelmer.
[07:23] <wgrant> lifeless: Not quite below 500 critical OOPSes yet :(
[07:31] <lifeless> wgrant: indeed
[07:31] <lifeless> 1918HWDB1 will wipe out 150
[07:31] <wgrant> Yeah.
[07:31] <lifeless> and fixing OOPS-1918SMP10 will likely fix real issues (e.g. stale locks)
[07:32] <StevenK> SMP? What is that prefix from?
[07:32] <wgrant> Supermirror Puller
[07:33] <wgrant> Not to be confused with SMPJ, which is the staging merge proposal job runner.
[07:39] <lifeless> OOPS-1918CIW12 seems suspect too
[07:41] <lifeless> OOPS-1918A602...
[07:41] <lifeless> odule zope.publisher.http, line 73, in sane_environment
[07:41] <lifeless>     dict['PATH_INFO'] = dict['PATH_INFO'].decode('utf-8')
[07:41] <lifeless> wtf
[07:41] <lifeless> another 20 a day there trivially fixed (its garbage; give a 400 bad request)
[07:42] <lifeless> or a 404
[07:42] <StevenK> Which could be bad bookmarks or something of that ilk?
[07:42] <lifeless> could be
[07:42] <lifeless> though how you get \x0a into a bookmark, I dunno
[07:44] <wgrant> Not spammers?
[07:48] <StevenK> It could be ...
[07:50] <LPCIBot> Yippie, build fixed!
[07:50] <LPCIBot> Project devel build #605: FIXED in 5 hr 23 min: https://lpci.wedontsleep.org/job/devel/605/
[08:09] <LPCIBot> Project windmill build #137: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/137/
[11:00] <lifeless> is json or simplejson the right one to use
[11:06] <wgrant> It depends. json is the version of simplejson in the standard library, but simplejson is better at some things.
[11:12] <lifeless> json will do then
[11:14] <lifeless> wgrant: I've just pushed a better schema for oops-repository
[12:12] <lifeless> gnight
[22:01] <thumper> morning
[22:10] <lifeless> moin
[22:31] <lifeless> anyone know how we land changes to lazr.batchnavigator ?
[22:46] <lifeless> hmm
[22:46] <lifeless> I think we've broken subscribing to mailing lists via indirect team membership
[23:03] <lifeless> false alarm, but horribly confusing ui
[23:04] <wgrant> Confusing UI in LP? Surely not!
[23:05] <lifeless> indeed
[23:05] <lifeless> not to mention buildbout breaking for me again
[23:05] <lifeless> :(
[23:06] <lifeless> bug 749753
[23:06] <_mup_> Bug #749753: Confusing missing 'subscribe to mailing list' on disabled list with indirect membership <confusing-ui> <easy-ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/749753 >
[23:06] <wgrant> lifeless: buildbot, or buildout?
[23:07] <lifeless> out
[23:07] <lifeless> bug 749739
[23:07] <_mup_> Bug #749739: fails to bootstrap on lucid <lazr.batchnavigator:Triaged> < https://launchpad.net/bugs/749739 >
[23:07] <wgrant> Erm, were you running it in a virtualenv?
[23:07] <wgrant> That eperm suggests not.
[23:07] <lifeless> no
[23:07] <lifeless> why would I ?
[23:08] <lifeless> I have a VM for development for a reason
[23:09] <lifeless> if it won't bootstrap in lucid, building debs would be hiiiiiilarious
[23:15] <lifeless> wgrant: ^
[23:15] <wgrant> Hmm.
[23:15] <lifeless> separately
[23:15] <lifeless> in just over two hours we're going to merge db-stab;e
[23:16] <lifeless> Revision 10382 can not be deployed: needstesting - Adds a method via the API to hide question comments and 10383 are unqaed
[23:16] <lifeless> I suspect both can be landed directly on devel after the merge, if they aren't qaed before
[23:16] <wgrant> That is also my suspicion.
[23:16] <wgrant> But let's see.
[23:17] <lifeless> wgrant: I'm going to be pretty terrible today - was @ 24 hour medical centre till 3am last night w/Lynne
[23:17] <wgrant> Ugh.
[23:17] <wgrant> No chance in hell of QAing 10383.
[23:17] <wgrant> Fortunately there's nothing useful behind it.
[23:17] <wgrant> lifeless: Ugh, that's not ideal. Everything OK?
[23:18] <lifeless> wgrant: yeah, something triggered asthma (she has no history of it), and with pregnancy you take everything seriously
[23:18] <wgrant> Yup.
[23:55]  * thumper is sending away for urgent passport
[23:55] <thumper> bad planning on my part