[01:20] I think LP now feels only very slightly slow. [01:48] its getting there [01:48] still has that mainframe feel [01:53] Yeah. But bug pages and listings load in just a couple of seconds now. [01:53] Three years ago they freqently took >10. [02:08] we're making progress :) [02:08] +bug-index is going to be much happier on thursday [02:09] Yeah. [02:46] I think we'll need to do constraint based batch offsets soon [02:47] sadly storm doesn't seem to have editable queries [02:47] and given the contribution dynamic there, I don't fancy trying to do it upstream [02:56] lifeless, :D [02:57] What's the issue with contributions? [02:58] wgrant: perfect is the enemy of the good, or thats how it feels to me so far [02:58] wgrant: I think I've put up 4 patches and had none get in [02:59] Hmm. [02:59] We should fix that. [02:59] wgrant: 2 got rewritten, one got bounced [no tests, which is fair enough, though finding tests in the storm layout is idiosyncratic] [02:59] and one is stalled (the with patch) [02:59] sidnei: hai :) [03:00] wgrant: https://code.launchpad.net/storm/+activereviews [03:01] lifeless, maybe you'll get a suggestion to go look at sqlalchemy with your next patch [03:01] wgrant: its a lot healthier in absolute terms than LP; but in terms of reviews-per-dev, or reviews-per-loc, I think LP would measure better [03:01] sidnei: hah! [03:01] sidnei: so let me tell you about this idea [03:02] sidnei: you know we have big collections in LP [03:02] like ubuntu - 400K bugs [03:02] sidnei: and a batch navigator that lets users page through the collection [03:02] lifeless, im having dinner, but go on, i will read l8r [03:02] sidnei: this breaks down spectacularly when the user is paging through (say) page 30 [03:03] or earlier (depends on the cost of calculating the batch) [03:03] because the way we implement it is to say LIMIT [batch size] OFFSET [page number * batch size] [03:04] LIMIT + constraint is a *much* better way to do it [plus reversing the sort order when walking backwards] [03:04] e.g when we show the first page [or the last], we remember the sort order key of the last row. [03:05] and rather than saying OFFSET [1 * batch size] to go to the next page, we say And(order_column1 > lastrow1, order_column2 > lastrow2, ...) LIMIT [batch size] [03:06] to walk back, flip the sort order; to skip a page use an OFFSET as well as this technique. To go to the end, flip the sortorder take the first page of results. [03:06] In LP we pass ResultSet objects to BatchNavigator, which then calculates # of pages, does the slicing etc. [03:07] so, we need some way for the batchnavigator to: [03:07] - ask for the needed last-row-tuple [03:07] - pass in a last-row-tuple and say 'after this row please' [03:08] just wrapping the query (using it as an inner query) is pretty much guaranteed to give poor query performance. [03:08] (and by pretty much, I mean 'its not even worth trying') [03:08] challenges I see are that storm doesn't understand the contents of its queries. [03:10] so we're likely to get redundant constraints and so on - which pgsql can optimise out, but that often results in odd plans even so [03:35] we may be best off redefining the contract [03:35] pass BN a factory to create a page result set as well as a resultset for 'entire collection' [03:39] wgrant: still around ? [03:40] lifeless: Mostly. [04:02] wgrant: where did you and wallyworld get to fixing that undefined thing [04:28] lifeless: It's on qas. [04:28] r12723 [04:32] cool, so we can deploy? [04:33] Once we're QA'd past it, yeah. [04:33] That's blocking on fixing lists.staging though, which I might try to get done tomorrow morning. [04:34] staging or qastaging ? [04:34] Either. [04:34] But qastaging doesn't have one at all, so it's going to be more work. [04:34] ah [04:34] whatever you learn.... can you rt that towards a qastaging one in future ? [04:34] Sure. [04:35] thanks === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [07:22] 275 timeouts [07:22] hmm [07:23] moinmoin [07:23] Morning jelmer. [07:23] lifeless: Not quite below 500 critical OOPSes yet :( [07:31] wgrant: indeed [07:31] 1918HWDB1 will wipe out 150 [07:31] Yeah. [07:31] and fixing OOPS-1918SMP10 will likely fix real issues (e.g. stale locks) [07:32] SMP? What is that prefix from? [07:32] Supermirror Puller [07:33] Not to be confused with SMPJ, which is the staging merge proposal job runner. [07:39] OOPS-1918CIW12 seems suspect too [07:41] OOPS-1918A602... [07:41] odule zope.publisher.http, line 73, in sane_environment [07:41] dict['PATH_INFO'] = dict['PATH_INFO'].decode('utf-8') [07:41] wtf [07:41] another 20 a day there trivially fixed (its garbage; give a 400 bad request) [07:42] or a 404 [07:42] Which could be bad bookmarks or something of that ilk? [07:42] could be [07:42] though how you get \x0a into a bookmark, I dunno [07:44] Not spammers? [07:48] It could be ... [07:50] Yippie, build fixed! [07:50] Project devel build #605: FIXED in 5 hr 23 min: https://lpci.wedontsleep.org/job/devel/605/ [08:09] Project windmill build #137: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/137/ [11:00] is json or simplejson the right one to use [11:06] It depends. json is the version of simplejson in the standard library, but simplejson is better at some things. [11:12] json will do then [11:14] wgrant: I've just pushed a better schema for oops-repository [12:12] gnight [22:01] morning [22:10] moin [22:31] anyone know how we land changes to lazr.batchnavigator ? [22:46] hmm [22:46] I think we've broken subscribing to mailing lists via indirect team membership [23:03] false alarm, but horribly confusing ui [23:04] Confusing UI in LP? Surely not! [23:05] indeed [23:05] not to mention buildbout breaking for me again [23:05] :( [23:06] bug 749753 [23:06] <_mup_> Bug #749753: Confusing missing 'subscribe to mailing list' on disabled list with indirect membership

< https://launchpad.net/bugs/749753 > [23:06] lifeless: buildbot, or buildout? [23:07] out [23:07] bug 749739 [23:07] <_mup_> Bug #749739: fails to bootstrap on lucid < https://launchpad.net/bugs/749739 > [23:07] Erm, were you running it in a virtualenv? [23:07] That eperm suggests not. [23:07] no [23:07] why would I ? [23:08] I have a VM for development for a reason [23:09] if it won't bootstrap in lucid, building debs would be hiiiiiilarious [23:15] wgrant: ^ [23:15] Hmm. [23:15] separately [23:15] in just over two hours we're going to merge db-stab;e [23:16] Revision 10382 can not be deployed: needstesting - Adds a method via the API to hide question comments and 10383 are unqaed [23:16] I suspect both can be landed directly on devel after the merge, if they aren't qaed before [23:16] That is also my suspicion. [23:16] But let's see. [23:17] wgrant: I'm going to be pretty terrible today - was @ 24 hour medical centre till 3am last night w/Lynne [23:17] Ugh. [23:17] No chance in hell of QAing 10383. [23:17] Fortunately there's nothing useful behind it. [23:17] lifeless: Ugh, that's not ideal. Everything OK? [23:18] wgrant: yeah, something triggered asthma (she has no history of it), and with pregnancy you take everything seriously [23:18] Yup. [23:55] * thumper is sending away for urgent passport [23:55] bad planning on my part