wgrantI think LP now feels only very slightly slow.01:20
lifelessits getting there01:48
lifelessstill has that mainframe feel01:48
wgrantYeah. But bug pages and listings load in just a couple of seconds now.01:53
wgrantThree years ago they freqently took >10.01:53
lifelesswe're making progress :)02:08
lifeless+bug-index is going to be much happier on thursday02:08
lifelessI think we'll need to do constraint based batch offsets soon02:46
lifelesssadly storm doesn't seem to have editable queries02:47
lifelessand given the contribution dynamic there, I don't fancy trying to do it upstream02:47
sidneilifeless, :D02:56
wgrantWhat's the issue with contributions?02:57
lifelesswgrant: perfect is the enemy of the good, or thats how it feels to me so far02:58
lifelesswgrant: I think I've put up 4 patches and had none get in02:58
wgrantWe should fix that.02:59
lifelesswgrant: 2 got rewritten, one got bounced [no tests, which is fair enough, though finding tests in the storm layout is idiosyncratic]02:59
lifelessand one is stalled (the with patch)02:59
lifelesssidnei: hai :)02:59
lifelesswgrant: https://code.launchpad.net/storm/+activereviews03:00
sidneilifeless, maybe you'll get a suggestion to go look at sqlalchemy with your next patch03:01
lifelesswgrant: its a lot healthier in absolute terms than LP; but in terms of reviews-per-dev, or reviews-per-loc, I think LP would measure better03:01
lifelesssidnei: hah!03:01
lifelesssidnei: so let me tell you about this idea03:01
lifelesssidnei: you know we have big collections in LP03:02
lifelesslike ubuntu - 400K bugs03:02
lifelesssidnei: and a batch navigator that lets users page through the collection03:02
sidneilifeless, im having dinner, but go on, i will read l8r03:02
lifelesssidnei: this breaks down spectacularly when the user is paging through (say) page 3003:02
lifelessor earlier (depends on the cost of calculating the batch)03:03
lifelessbecause the way we implement it is to say LIMIT [batch size] OFFSET [page number * batch size]03:03
lifelessLIMIT + constraint is a *much* better way to do it [plus reversing the sort order when walking backwards]03:04
lifelesse.g when we show the first page [or the last], we remember the sort order key of the last row.03:04
lifelessand rather than saying OFFSET [1 * batch size] to go to the next page, we say And(order_column1 > lastrow1, order_column2 > lastrow2, ...) LIMIT [batch size]03:05
lifelessto walk back, flip the sort order; to skip a page use an OFFSET as well as this technique. To go to the end, flip the sortorder take the first page of results.03:06
lifelessIn LP we pass ResultSet objects to BatchNavigator, which then calculates # of pages, does the slicing etc.03:06
lifelessso, we need some way for the batchnavigator to:03:07
lifeless - ask for the needed last-row-tuple03:07
lifeless - pass in a last-row-tuple and say 'after this row please'03:07
lifelessjust wrapping the query (using it as an inner query) is pretty much guaranteed to give poor query performance.03:08
lifeless(and by pretty much, I mean 'its not even worth trying')03:08
lifelesschallenges I see are that storm doesn't understand the contents of its queries.03:08
lifelessso we're likely to get redundant constraints and so on - which pgsql can optimise out, but that often results in odd plans even so03:10
lifelesswe may be best off redefining the contract03:35
lifelesspass BN a factory to create a page result set as well as a resultset for 'entire collection'03:35
lifelesswgrant: still around ?03:39
wgrantlifeless: Mostly.03:40
lifelesswgrant: where did you and wallyworld get to fixing that undefined thing04:02
wgrantlifeless: It's on qas.04:28
lifelesscool, so we can deploy?04:32
wgrantOnce we're QA'd past it, yeah.04:33
wgrantThat's blocking on fixing lists.staging though, which I might try to get done tomorrow morning.04:33
lifelessstaging or qastaging ?04:34
wgrantBut qastaging doesn't have one at all, so it's going to be more work.04:34
lifelesswhatever you learn.... can you rt that towards a qastaging one in future ?04:34
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
lifeless275 timeouts07:22
wgrantMorning jelmer.07:23
wgrantlifeless: Not quite below 500 critical OOPSes yet :(07:23
lifelesswgrant: indeed07:31
lifeless1918HWDB1 will wipe out 15007:31
lifelessand fixing OOPS-1918SMP10 will likely fix real issues (e.g. stale locks)07:31
StevenKSMP? What is that prefix from?07:32
wgrantSupermirror Puller07:32
wgrantNot to be confused with SMPJ, which is the staging merge proposal job runner.07:33
lifelessOOPS-1918CIW12 seems suspect too07:39
lifelessodule zope.publisher.http, line 73, in sane_environment07:41
lifeless    dict['PATH_INFO'] = dict['PATH_INFO'].decode('utf-8')07:41
lifelessanother 20 a day there trivially fixed (its garbage; give a 400 bad request)07:41
lifelessor a 40407:42
StevenKWhich could be bad bookmarks or something of that ilk?07:42
lifelesscould be07:42
lifelessthough how you get \x0a into a bookmark, I dunno07:42
wgrantNot spammers?07:44
StevenKIt could be ...07:48
LPCIBotYippie, build fixed!07:50
LPCIBotProject devel build #605: FIXED in 5 hr 23 min: https://lpci.wedontsleep.org/job/devel/605/07:50
LPCIBotProject windmill build #137: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/137/08:09
lifelessis json or simplejson the right one to use11:00
wgrantIt depends. json is the version of simplejson in the standard library, but simplejson is better at some things.11:06
lifelessjson will do then11:12
lifelesswgrant: I've just pushed a better schema for oops-repository11:14
lifelessanyone know how we land changes to lazr.batchnavigator ?22:31
lifelessI think we've broken subscribing to mailing lists via indirect team membership22:46
lifelessfalse alarm, but horribly confusing ui23:03
wgrantConfusing UI in LP? Surely not!23:04
lifelessnot to mention buildbout breaking for me again23:05
lifelessbug 74975323:06
_mup_Bug #749753: Confusing missing 'subscribe to mailing list' on disabled list with indirect membership <confusing-ui> <easy-ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/749753 >23:06
wgrantlifeless: buildbot, or buildout?23:06
lifelessbug 74973923:07
_mup_Bug #749739: fails to bootstrap on lucid <lazr.batchnavigator:Triaged> < https://launchpad.net/bugs/749739 >23:07
wgrantErm, were you running it in a virtualenv?23:07
wgrantThat eperm suggests not.23:07
lifelesswhy would I ?23:07
lifelessI have a VM for development for a reason23:08
lifelessif it won't bootstrap in lucid, building debs would be hiiiiiilarious23:09
lifelesswgrant: ^23:15
lifelessin just over two hours we're going to merge db-stab;e23:15
lifelessRevision 10382 can not be deployed: needstesting - Adds a method via the API to hide question comments and 10383 are unqaed23:16
lifelessI suspect both can be landed directly on devel after the merge, if they aren't qaed before23:16
wgrantThat is also my suspicion.23:16
wgrantBut let's see.23:16
lifelesswgrant: I'm going to be pretty terrible today - was @ 24 hour medical centre till 3am last night w/Lynne23:17
wgrantNo chance in hell of QAing 10383.23:17
wgrantFortunately there's nothing useful behind it.23:17
wgrantlifeless: Ugh, that's not ideal. Everything OK?23:17
lifelesswgrant: yeah, something triggered asthma (she has no history of it), and with pregnancy you take everything seriously23:18
* thumper is sending away for urgent passport23:55
thumperbad planning on my part23:55

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!