[00:05] Project devel build (151): STILL FAILING in 3 hr 37 min: https://hudson.wedontsleep.org/job/devel/151/ [00:05] Launchpad Patch Queue Manager: [r=abentley][ui=none][bug=620615, [00:05] 659078] change Launchpad to use Storm 0.18 === matsubara-afk is now known as matsubara [00:25] thumper: they can suck, you just have to know how to tune them :-) i think storm may not be as advanced as say hibernate in this area [00:25] wallyworld: I'm attacking a place in the code where there is simplistic thinking [00:25] hibernate has all sorts of goodies, including support for batch updates and mixing native sql with it's object query language (HQL) [00:25] wallyworld: so does storm [00:26] * wallyworld needs to learn more storm [00:28] * wallyworld wonders, does storm support per query fetching strategies and pre-fetch sizes, or lazy instantiation of collections? [00:37] wallyworld: kinda [00:38] it's stuff like all that which is needed to properly tune one's app of course [00:39] Project db-devel build (98): SUCCESS in 3 hr 37 min: https://hudson.wedontsleep.org/job/db-devel/98/ [00:39] Launchpad Patch Queue Manager: [rs=thumper][ui=none][no-qa] Manual merge of stable into db-devel. [00:40] hey thumper, i'm likely the last to realise - i suppose you know about http://blogs.computerworld.com/17224/ubuntu_changes_its_desktop_from_gnome_to_unity [00:40] no [00:40] someone emailed it to me [00:43] * wallyworld gets out the tissues and blubbers over how unloved KDE is :-( [00:43] wallyworld: if it helps, i know in OS X circled KDE 4 got serious circulation. :-P [00:44] s/circled/circles/ [00:44] it would help more if KDE had more support on Ubuntu :-) [00:46] at least maybe there's a growing realisation that Qt is good. one step forward... [00:47] it helps that Qts license situation got cleaned up. [00:47] (or the FUD around it got cleaned up; whatever your preference might be) [00:49] jcsackett: same thing really, given the impact it had... [00:49] wallyworld: fair point. [00:51] wallyworld: these days i'm a wmii man, if you want to talk no love. :-D [00:51] :-) [00:53] * thumper afk for walk to shops and lunch [01:19] Does anyone have any idea why ~vcs-imports is associated with https://code.edge.launchpad.net/~brian-rogers/gnome-power/report-percentage/+merge/39329 ? [01:19] maxb, it's probably the default reviewer for some reason [02:08] * thumper needs coffee badly === matsubara is now known as matsubara-afk [02:25] * thumper has coffee and is screaming at his screen [02:25] * thumper dodges a tumbleweed [02:26] why is that damn test being run twice? [02:26] lifeless: ping [02:26] * wgrant rolls some more tumbleweed in. [02:26] hi [02:26] It's not one of those tests which is confusingly run with the same name for several object types? [02:27] lifeless: when running a doc test with "bin/test -vvt message-holds-xmlrpc.txt" it runs twice [02:27] lifeless: any idea? [02:27] lifeless: btw, this will fix the mailman xmlrpc call when it makes 5000 database calls [02:28] nice [02:28] what causes that ? [02:28] simplistic iteration over a large result set [02:28] lack of eager loading ? [02:29] and silly logic [02:29] * thumper is enfixorating [02:29] I'm changing it slightly [02:29] so will just do 6 [02:29] I could attempt to make it 2, but I don't see the benefit for that [02:29] the logic would be harder to follow if I did [02:30] 5000 -> 6 is much better [02:30] I think 6 queries is fine ;) [02:30] sometimes less is worse too [02:30] depending on the related data density [02:30] deryck, lp:~rockstar/launchpad/javascript-refresh [02:31] lifeless: this one: OOPS 1758XMLP341 [02:32] nice [02:34] thumper: I see the double run in trunk [02:34] thumper: its being loaded twice by whatever module constructs it [02:34] lifeless: is it all doc tests or just some? [02:34] lp/registry/tests/test_doc.py [02:35] will be a buglet in there [02:35] ignore for this branch IMO [02:36] ok [02:36] lifeless: are you going to file a bug for it? [02:36] lifeless: if it is simple, I could fix in this branch [02:36] its late + jetlag [02:37] I haven't looked [02:38] lifeless: ah, it seems to have two different setup methods [02:38] lifeless: so it is testing things twice [02:39] one with the real mailinglist api and another with a fake [02:51] thumper: you mean it's doing two tests in one via diff setUps? [02:51] kinda [02:52] * jcsackett pulls up code. [02:53] or not; i forgot my dev machine doesn't multitask well when updating. [02:53] and here i thought i might be helpful. this is a test in registry? [02:54] it is, and there isn't a problem [02:54] just my misunderstanding [02:55] thumper: ah, dig. [03:03] * wgrant WTFs at launchpad-users. [03:04] oh? [03:04] The last email to it. [03:05] on oct 15th? [03:05] About 47 minutes ago. [03:06] swiss branded watches? [03:06] chain letter about returning vets, looks like. [03:07] It looks like it. [03:07] that does seem about as Off-topic as you're going to get... [03:07] \o/ edge bye bye [03:07] :( poor edge [03:09] hah [03:10] i had only just met poor edge. [03:10] wgrant: the previous one about getting mugged in KL was mighty off topic too [03:10] * wgrant remembers back when it was 'beta', restricted access, and had the shiny new UI. [03:10] bac: Ah yeah, I forgot about that one. [03:14] bac: that one is however the result of a compromised account. [03:14] my sister had some people get into her yahoo account a few months back and received (nearly) word for word that email. it's apparently a pretty popular scam. [03:23] lifeless: has edge gone? [03:23] Heh. I was about to ask if someone had turned off the redirect. [03:23] * thumper imagines some people getting a little miffed at no recipes ... [03:23] Then I noticed the last rev. [03:41] thumper: ping [03:41] EdwinGrubbs: hi [03:42] Yippie, build fixed! [03:42] Project devel build (152): FIXED in 3 hr 37 min: https://hudson.wedontsleep.org/job/devel/152/ [03:42] * Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Refactor page-performance-report to use [03:42] less memory by using a SQLite3 db to hold the requests and [03:42] generating statistics for only one key at a time. [03:42] * Launchpad Patch Queue Manager: [r=henninge][ui=none][bug=659085] Remove [03:42] getBugNotificationRecipients() [03:42] * Launchpad Patch Queue Manager: [r=henninge][ui=none][bug=628762] Ditch property cache adapters in [03:42] favour of plain old Python. The IPropertyCache adapter is [03:42] replaced by get_property_cache() and the IPropertyCacheManager [03:42] adapter is replaced by clear_property_cache(). [03:42] ! [03:43] now that is a bit of IRC spam [03:43] Does this mean we are green? [03:43] Yes! [03:43] Both are green! [03:43] For the first time! [03:43] thumper: according to bzr annotate, you worked on canonical.launchpad.webapp.sorting.sorted_version_numbers. I'm wondering why you the numbers are sorted in the reverse order as the letters. For example, [3, 2, 1, A, B, C], which looks wrong if a version combines letters and numbers like [5a, 4a, 3a, 3b, 3c]. [03:44] holy crap that is quite a while ago... [03:44] thumper: I'm in the process of moving the sorting into the db so that the results can be batched. It looks like this is just used for the productseries. [03:44] EdwinGrubbs: I think the reasoning was we wanted newer version numbers first [03:45] thumper: ok, so there shouldn't be a problem in making the letters sort descending like the numbers? [03:45] ahh... [03:45] find out where it's used forst [03:45] first [03:45] I really don't remember [03:46] thumper: so far, it looks like it is just used for https://qastaging.launchpad.net/launchpad/+series [03:46] but I'll double check with sinzui [03:46] EdwinGrubbs: in which case it is all yours :) [03:46] who whatever you like [03:46] s/who/do/ [03:46] thanks [04:18] Project db-devel build (99): FAILURE in 3 hr 39 min: https://hudson.wedontsleep.org/job/db-devel/99/ [04:18] * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11791 [04:18] included. [04:18] * Launchpad Patch Queue Manager: [r=henninge][ui=none][no-qa] Fix the models and accompanying tests [04:18] for Branch Merge Queues to use the secured utilities, etc. [04:27] How does launchpad not depend on beautiful soup anymore? [04:53] rockstar: eh? [04:54] thumper, launchpad used to use beautiful soup a lot, but apparently it doesn't anymore. [04:54] ah ... [04:54] really? [04:54] Also, I SHOULD go to bed, but deryck and I is watching teh football. [04:54] thumper, yeah, I just added soupmatchers to Launchpad and it needed BeautifulSoup. [04:55] rockstar: still used by canonical.launchpad.testing.pages [04:55] rockstar: it may be an egg [04:55] thumper, no, it wasn't in versions.cfg or setup.py [04:55] Weird. [04:56] At least, buildout couldn't find it, even when I just installed the package. [04:57] rockstar: it is LP telling you to sleep [04:57] :) [04:57] thumper, probably. [05:49] matsubara-afk: ping me when you arrive [05:49] rockstar: who is playing? === almaisan-away is now known as al-maisan [07:21] Project devel build (153): FAILURE in 3 hr 38 min: https://hudson.wedontsleep.org/job/devel/153/ [07:21] * Launchpad Patch Queue Manager: [r=julian-edwards][ui=none][bug=662912][incr] Add diagnostics for bug [07:21] 662912 [07:21] * Launchpad Patch Queue Manager: [r=henninge][ui=none][no-qa] Remove the beta redirect and is_edge facilities. [07:53] Yippie, build fixed! [07:53] Project db-devel build (100): FIXED in 3 hr 34 min: https://hudson.wedontsleep.org/job/db-devel/100/ [07:53] Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11792 [07:53] included. [08:12] Hello! I cannot push branches to launchpad. Can somebody please look into this? [08:13] Here's the error I get: $ bzr push [08:13] Using saved push location: bzr+ssh://bazaar.launchpad.net/~al-maisan/landscape/edit-computer [08:13] Traceback (most recent call last): [08:13] File "/srv/bazaar.launchpad.net/production/launchpad-rev-9885/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr", line 140, in [08:13] exit_val = bzrlib.commands.main() [08:13] File "/srv/bazaar.launchpad.net/production/launchpad-rev-9885/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/bzrlib/commands.py", line 1191, in main [08:13] _register_builtin_commands() [08:13] File "/srv/bazaar.launchpad.net/production/launchpad-rev-9885/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/bzrlib/commands.py", line 182, in _register_builtin_commands [08:13] import bzrlib.builtins [08:13] ValueError: bad marshal data [08:13] bzr: ERROR: Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist. [08:13] sorry [08:13] meant to paste pastebin url [08:34] good morning [09:07] rockstar: BeautifulSoup is included via the Java Mentality. [09:18] Hi [10:57] Yippie, build fixed! [10:57] Project devel build (154): FIXED in 3 hr 35 min: https://hudson.wedontsleep.org/job/devel/154/ [10:57] * Launchpad Patch Queue Manager: [r=lifeless,thumper][ui=none][no-qa] Slightly fix the IDSJob tests. [10:57] * Launchpad Patch Queue Manager: [r=lifeless][ui=none][bug=664012] Fix the oopses raided by +branch [10:57] navigation when there is no referrer to be 404s. === jtv1 is now known as jtv === al-maisan is now known as almaisan-away === matsubara-afk is now known as matsubara === almaisan-away is now known as al-maisan [13:21] good morning [13:22] Morning. [14:23] jam: staging codehosting seems to be working [14:23] this surprises me :-) [14:23] mwhudson: well, we haven't updated the config for the new code yet [14:23] oh of course [15:17] poolie: bug 666765 is also on my nice-to-have for scopes [15:29] lifeless: Didn't you mention a fix for the "No module named mailman.monkeypatches.defaults" ? I seem to recall it involved wielding rm over a directory [15:31] rm -rf lib/mailman; make [15:31] why is stdout a text stream rather than a bytes stream in Python 3? [15:31] because python three [15:31] jml: hows your session [15:32] lifeless: not exactly riveting [15:35] lifeless: so, running tests locally I get a lot of errors about librarians either running when they should not be or vice versa [15:36] this Is Bad. [15:36] james_w, can we chat at some point about soupmatchers? [15:36] lifeless: looks like some layer issues [15:37] bigjools, yeah, I've had those problems for about three weeks. [15:37] bigjools, did you bin/kill-test-services ? [15:38] yes [15:38] bigjools, where is you? [15:38] rockstar: boner 6 [15:38] bigjools, is there a session in there, or are you loitering? [15:38] rockstar: linaro session [15:38] bigjools, ah. [15:38] rockstar: but I may duck out RSN [15:39] bigjools, okay. I'm at the end of the arterial hallway. [15:39] bigjools: last time I saw that I had a prod librarian running [15:40] I am getting motivated to fix test_on_merge [15:45] rockstar: wtf is the arterial hallway? :) [15:45] antigua [15:45] bigjools, the hallway that acts as artery to all the dead end hallways. [15:46] rockstar: which end? [15:48] lifeless, replied [15:49] lifeless, please tag them so i can find them later [15:50] lifeless: it was a stale pidfile === matsubara is now known as matsubara-lunch === al-maisan is now known as almaisan-away [16:05] jml: :( [16:06] rockstar, sure thing [16:06] james_w, are you going to be in a session after the one that just started? [16:07] rockstar, yes. I think my afternoon is quieter though [16:07] rockstar, or lunch? [16:08] james_w, lunch will work. [16:17] Is it just me or is codehosting down? === Ursinha is now known as Ursinha-lunch [16:40] lifeless: around? [16:41] ys [16:44] fjlacoste: yes [16:44] lifeless: i need your advice, i'm working on a version of the ppr report that computes everything using SQL, that will operate in constant memory [16:44] lifeless: i have all the stats covered except the median [16:44] i know how to compute it, but it is very expensive [16:44] fjlacoste: btw, disk space on sodium - we had to kill your thing [16:45] i know [16:45] that's fine [16:45] we can sort that out later [16:45] it was using 5% of the disk :) [16:45] 2.9G [16:45] yeah [16:45] thats ~ 5% [16:45] it needs a bigger disk :-) [16:45] anyway [16:45] so the way i can compute the median is by sorting on the column and using LIMIT 1 OFFSET n/2 to get the value [16:46] but this seems like very expensive to do given that we are taking about millions of records [16:46] and we are computing 3 medians [16:46] and we can only do that by key [16:46] WHERE Rank = (SELECT (COUNT(*)+1) DIV 2 FROM [16:46] ah [16:47] also [16:47] http://wiki.postgresql.org/wiki/Aggregate_Median [16:47] though I haven't looked into that implementation [16:47] lifeless: i'm using sqlite [16:47] oh [16:47] :P [16:47] i was asking if you mind dropping it :-) [16:48] but your rank thing gave me an idea [16:48] fjlacoste: hmm, so if the column is indexed grabbing the median should be darn cheap [16:49] the question is how do I index the column relatively cheaply === fjlacoste is now known as flacoste [16:49] insert the data [16:49] index the column [16:49] then start querying [16:50] oh [16:50] you'll want to increase the sqlite cache size too [16:50] because its like 2MB or something insane by default [16:50] give it 200MB or so [16:50] create an index on category, time for example? [16:50] ok [16:50] yeah [16:50] remember that the query prefix has to match the index [16:51] right [16:51] category,time won't help time only sorts [16:51] IIRC [16:51] well i need the rank of time within each category [16:51] (or url, or pageid) [16:51] yack [16:52] lifeless: any chance you could help me think through bac's notes on bug 652156 and bug 652149? [16:52] basically, it looks to me like the timeouts there aren't related to either bug/branch. [16:52] qastaging has a 10second timeout [16:52] lifeless: do you mean yak (-shaving), yuck, or some combination? [16:53] I meant 'acl' [16:53] bah [16:53] 'ack' [16:53] jcsackett: so any page that has a timeout bug open (https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=timeout) is almost certain to timeout on [qa]staging too [16:54] jcsackett: I would not stress about incidental timeouts there (but I *would* fix the page to not timeout at all :P separately :) ) [16:54] mtaylor, jam: launchpad/bzr planning over lunch? [16:54] lifeless: that makes sense. so bac and i can mark ours qa-ok if the functionality is working (aside from the branchbug timeout)? [16:55] jcsackett: as long as the page is already timing out with bug, production oops) [16:56] lifeless: actually, there is no Rank expression in sqlite [16:56] and generating the rank seems as costly as getting the median using iterative limit/Offset [16:58] flacoste: how slow is slow [16:58] several hours [16:59] actually, i don't know [16:59] just guessing here [16:59] i'll measure and see [16:59] assess [16:59] lifeless: so, there isn't a bug filed against project/+branches, but person/+branches has one (bug 627945) and the query is the same in both. i'm thinking either add detail to that bug or file a new bug for project linked to it. thoughts? [17:01] mtaylor, jam: which isn't right now, of course [17:01] * mwhudson was just getting hungry and optimistic [17:02] lifeless: http://oreilly.com/catalog/transqlcook/chapter/ch08.html#Calculating%20a%20Median has an actual elegant solution :-) [17:02] mwhudson: sounds good [17:02] I'm pretty sure poolie is interested too [17:04] jcsackett: add the page id for the other case to the title, done. [17:04] lifeless: fantastic. thanks. === benji is now known as benji-lunch [17:11] hmm, that query does look expensive === matsubara-lunch is now known as matsubara [17:12] flacoste: up the cache size as a start [17:12] lifeless: ack [17:12] you could denormalise the counts per category [17:12] given you're starting with a fresh db each run, right ? [17:18] mwhudson: yes [17:20] jam, mtaylor: i'll loiter by the registration desk [17:20] poolie: ^^ [17:55] jml: https://code.edge.launchpad.net/~lifeless/testtools/fixtures/+merge/39388 === benji-lunch is now known as benji [18:14] Project db-devel build (101): SUCCESS in 4 hr 4 min: https://hudson.wedontsleep.org/job/db-devel/101/ === Ursinha-lunch is now known as Ursinha [19:09] lifeless: reviewed [19:13] gary_poster: http://pypi.python.org/pypi/storm - needs 0.18 ? [19:15] the PyPi version is almost a year old [19:15] lifeless, yes, remaining release of storm (per the release instructions) is stalled until Jamu gives me privs to upload docs or does it himself. There are a number of small things remaining (also announcing it on ML, for instance). [19:15] For PyPI, Storm didn't release 0.17 there and it's not on their release checklist, but I got privs to do it anyway, so I'll do it then. [19:16] gary_poster: Jamu here, go ahead and release, I'll deploy docs later. [19:16] Cool Jamu, thanks [19:16] * mars loves UDS :) [19:22] i am inpressed by the fact that people are just registering kernel imports, and they basically work [19:23] it would be much better if we supported stacking for import branches sensibly but well [19:24] mwhudson: +1 [19:24] it shouldn't really be all that hard if I understand correctly [19:24] * mwhudson quickly reaches for his "not a launchpad developer" hat [19:24] * jelmer reaches for his "not a launchpad-code developer" hat :-) [19:25] jelmer: i think the bug report on this describes what to do fairly clearly [19:26] i hope so, becuase i'm fairly sure i knew what to do at one point and certainly don't any more [19:33] lifeless: do you still thing that https://bugs.edge.launchpad.net/testtools/+bug/584824 is still worth doing? [19:33] think [19:36] lifeless: also, https://bugs.edge.launchpad.net/testtools/+bug/666923 wrt previous discussions [19:58] gary_poster: Hi! [19:58] jkakar: hi :-) [20:05] deryck: hey [20:05] hi lifeless [20:05] https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html [20:06] looking.... [20:06] slow openid dance [20:09] deryck: so https://bugs.qastaging.launchpad.net/ubuntu/+source/xorg/+bug/10000 looks ok [20:09] deryck: I'm not entirely sure how to validate it further [20:09] lifeless, yeah, I'm looking at the changes now to see. [20:13] lifeless, I smell a bad rev. https://bugs.qastaging.launchpad.net/bugs/10000/+bug-portlet-subscribers-content [20:14] hmmm, maybe that's not gmb's [20:18] deryck: whats wrong with it ? [20:18] lifeless, that link OOPS for me. [20:19] https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1760QS46 [20:21] oh, that's security.cfg permissions again. Like bug expiry script. We missed something in pg 8.4 migration, I guess. [20:22] lifeless, ok, so gmb's is qa-ok. Visiting https://bugs.qastaging.launchpad.net/ubuntu/+source/xorg/+bug/10000/+subscribe confirms the new code is hit and works fine. [20:23] deryck: will you mark it as such? the two bugs ? [20:23] lifeless, indeed. [20:23] thanks [20:25] lifeless, do you not get an OOPS at the first URL I provided? the portlet one. [20:25] yes, I do, i thought it might be transient but its very quick [20:26] lifeless, transient? You mean timeout? I get ProgrammingError. [20:26] deryck: I meant I thought it might be a slow db or something [20:27] lifeless, ah, ok [20:27] lifeless, we should talk more about this after we get out of sessions, since I saw the same on expiry script test runs. [20:27] THE EAGLE HAS LANDED [20:28] where eagle == new buildd manager [20:28] bigjools, awesome! [20:28] 7k branch with no mechanical changes! [20:29] bigjools: what does it give us? [20:30] asynchronous comms [20:30] and far, far better failure detection and dealing with failures [20:30] lp:~julian-edwards/launchpad/builderslave-resume if you want gory details :) [20:31] hi thumper. Being at UDS makes us overlap for once. :-) [20:31] james_w, are you going to have some bandwidth this afternoon? [20:31] deryck: hey [20:32] rockstar, the next session looks pretty good to me [20:34] james_w, okay, I think I need to make some hard choices here. [20:34] rockstar, 6pm would also work for me [20:34] james_w, yeah, that won't work at all. Next session is good, I'll just chat with aq before the next session. [20:37] rockstar, well, that was the session I was thinking going to as well :-) [20:37] rockstar, are you here all week? [20:37] james_w, indeed I am, but I want to land some work before next week. :) [20:37] rockstar, sure :-) [20:38] rockstar, where are you now? [20:38] james_w, I'm in the game development session, but it's a bust, so I can get out now if you can. [20:38] rockstar, I can [20:38] rockstar, outside the plenary room? [20:40] james_w, I'm in the hall. === fjlacoste is now known as flacoste === Ursinha is now known as Ursinha-brb [21:12] bigjools: grats! [21:13] \o/ BrowserTestCase just got a whole lot better. [21:13] jml: and thanks to you! [21:19] jml: FWIW I hammered it on DF with a load of production builders that were borrowed and it didn't put a foot wrong. [21:19] bigjools: sweet. [21:23] rockstar: what got better about it ? [21:23] jml: of course, I just set myself up for a big fall :) [21:23] bigjools: sure. :) [21:23] bigjools: it's not going to work the first time. never does. [21:23] lifeless, I'm teaching it about james_w's soupmatchers. [21:23] kiko's 1st law of software? [21:23] bigjools: exactly [21:24] lifeless: https://code.launchpad.net/~jml/launchpad/atexit-warning/+merge/39403 [21:24] jml: I saw, i don't understand it. It seems wrong [21:24] lifeless: that's why I wanted to talk about it :) [21:24] why not just remove the atexit call [21:25] and if things go screwy fix the code that isn't using a finally where needed. [21:25] lifeless: I don't know why the atexit call was there. it seems like a belt-and-braces thing [21:26] I think its because zope.testing stops teardown when a NotImplemented is enoucountered except in the last runner instance [21:26] lifeless: I guess I should do some archeology and find out why it was added. === matsubara is now known as matsubara-afk [21:27] lifeless: ok. I'm not sure what to do with that though. [21:27] jml: change zope.testing to not mess about [21:28] if a layer is setUp, it should tear it down [21:28] perhaps we could use Fixture to do that ;) [21:28] lifeless: I don't want to fix zope.testing to silence this warning [21:28] jml: the warning is a symptom of the root cause [21:28] the root cause is that teardown is inconsistently called [21:29] and because of that we have: [21:29] lifeless: the root cause being that layers is a pos [21:29]