[00:00] thumper: if I'm going to propose some work on launchpad and I want to get direction approved, should I file a blueprint or a bug? [00:00] thumper: or should I write a patch containing a user story and submit that :) [00:01] mtaylor: you should have a pre-implementation call :) [00:01] mtaylor: chat with a dev [00:01] mtaylor: everything else is makework. [00:01] mtaylor: you could chat on the lp-dev list, here, or on voice [00:02] lifeless: ok.... just wanted to be sure I was prepping for the right thing [00:02] if you need a persistent record, I'd start with the list, or a bug [perhaps] === Ursinha is now known as Ursinha-afk [00:02] _I_ don't need a persistent record - I just wanted to make sure I was being friendly and all [00:03] whats your ideal, when someone does stuff for dribble? [00:03] lifeless: depends on scope of work - if it's a new feature or something decent sized, we ask them to file a blueprint [00:03] what do you use the blueprint for? [00:04] lifeless: tracking work against the task [00:04] lifeless: they'll then tie their bzr branch to that blueprint, we'll use the blueprint during planning meetings to check progress on it [00:04] etc. [00:04] mtaylor: but not for deciding on direction :) [00:05] lifeless: well, not directly, no... although we'd certainly like to :) [00:05] lifeless: but fair enough [00:33] lifeless: say, do you have any particular opinions about pyunit-compatible test runners? [00:34] why yes, I do [00:34] lifeless: which would you recommend? (he asks, half expecting "bzrlib's") [00:35] testrepository if you can [00:35] failing that testtools.run [00:35] oh, testtools has a runner? [00:35] didn't know that [00:35] and failing that your own glue to bzrlib.tests.$stuff [00:35] my current absolute favourite is testrepository [00:36] which is generally used via subunit.run [00:36] which backs onto testtools stuff [00:36] lifeless: i'm currently writing up a python programming practices document for the $censored work [00:37] so i get to inflict my opinions on how some things should be done [00:37] have I shown you testrepository and all the bits [00:37] i'm aware of it [00:37] I think you should experience before recommended [00:37] s/ed/ing/ [00:37] yeah [00:38] so, apt-get install testrepository [00:39] this doesn't do test discovery yet, it will trivially when someone gets around to testtools + discovery [foords thing] glue [00:39] I should ask, do you have say, 10 minutes for a quick tour ? [00:40] yes [00:40] ah, i was going to ask about discovery [00:40] it would be good to add that [00:40] perhaps you could find a day and JFDI ? [00:40] perhaps [00:41] let me see, how hard would it be [00:41] http://pypi.python.org/pypi/discover/0.3.2 [00:43] * mwhudson looks at unittest2 and finds nothing that looks particularly good, apart from the discovery stuff [00:44] yeah [00:44] I asked if he'd contribute to testtools [00:44] but apparently it was too hard. Or something. [00:44] <- a little bitter [00:45] :( [00:45] i think i told him to look at testtools at pycon 2009 [00:45] anyhow [00:45] shiny shiny stuff [00:45] so, do you have something small with a test_suite method ? [00:45] um [00:45] no [00:46] if not, make something small with a test_suite method (because thats something that stock pyunit, and thus testtools.run, can support) [00:46] just [00:46] from testtools import TestCase [00:46] class foo(TestCase): [00:46] def test_pass(self):pass [00:47] def test_fail(self):self.fail('sad') [00:47] bzr question... I have a branch with an html file in it. I want whenever I commit to the branch that it updates a "last-updated-date" line in the footer. Any way to set up bzr to do this easily? [00:47] def test_suite(): [00:47] bryceh: see the bzr-keywords plugin [00:47] return unittest.TestLoader().loadTestsFromNames('foo') [00:47] in foo.py [00:47] jelmer, thanks [00:48] mwhudson: we can test this works with 'python -m testtools.run foo.test_suite' [00:48] make another file, .testr.conf [00:48] [DEFAULT] [00:49] test_command=python -m subunit.run $IDLIST [00:49] test_id_list_default=foo.test_suite [00:49] EOF [00:49] then, testr init; testr run [00:49] should show you one failure [00:49] testr run --failing [00:49] should run the one failing test [00:49] tour over [00:50] for more detail, keep chatting, and/or read /usr/share/doc/testrepository/MANUAL.txt [00:50] ok [00:50] I totally totally depend on the 'run only failing' stuff now that I have it [00:51] * jelmer admits to also being a total testr fanboy [00:51] hm yeah that looks pretty neat [00:51] py.test had this --looponfailing thing that i remember using a bit, ages ago [00:52] sounds similar [00:52] this is not coupled to test runner or python [00:52] as long as there is an interface, and it can output subunit, it works ;) [00:53] so where should discovery fit? [00:54] as an alternative to test_id_list_default ? [00:55] so [00:55] small bit of glue in subunit.run [00:55] then you'd do [00:55] python -m subunit.run --discover [discovery arg] [00:55] when running it by hand [00:55] for .testr.conf you'd do [00:56] test_id_list_default=--discover [00:56] thats the entire set of changes needed AFAICT [00:56] ah, so subunit needs to change a bit? [00:57] tiny tiny [00:57] not the protocol, just the glue to setup a test list [00:58] rather than calling the unittest loader with the parameters its given [00:58] it needs to call the discover loader [00:58] thats it [00:59] right [00:59] ok well it doesn't sound too hard... [00:59] note that this workflow works with trial --reporter=subunit too [00:59] for huge value of entertaining [01:00] right, and i guess trial already has its own approach to discovery [01:00] yes [01:04] I'd like bzr switch to stash the .testrepository with the branch [01:04] or perhaps I want a push/pop thing in testr [01:06] lifeless: i guess i could find this out myself, but does trial have anything like the load_tests() protocol? [01:06] it honours test_suite [01:06] I would like to teach it load_tests [01:06] because load_tests is much better [01:07] mtaylor: ^ also another reason to use cppunit -or- make gtest talk subunit [01:08] lifeless: why is load_tests 'much' better? [01:08] i can see how it's a bit more convenient [01:09] test multiplication is a lot easier with it [01:09] less duplication [01:10] and no hard-coding of TestSuite class [01:10] and no hard coding of loader [01:10] the whole process becomes more easily mutated, 2-line patches to change things, rather than change-per-source-file [01:12] ah yeah, that makes sense [02:39] lifeless, you wouldn't happen to have any idea why the zope testrunner + subunit would produce a partial traceback like this?: http://pastebin.ubuntu.com/443072/ [02:51] mars: that looks truncated [02:52] lifeless, yes. I thought a buffer in my own code was holding the output, but it turns out to be somewhere in the testrunner itself. [02:52] lifeless, is the subunit protocol written down anywhere? I'm wondering if that "error: ..." line is correct? [02:53] lines that say "failure: " end with "[ multipart " [03:00] lifeless, found the protocol, EBNF in the README. Clever. [03:02] :) [03:02] so you have the simple output there [03:02] [\n [03:02] there should be a trailing ]\n [03:03] and the output of a simple attachment like that is done at once, so it has nearly no chance of not completin [03:03] g [03:03] if its blowing up in stopTest [03:03] one possibility is stdout being closed or something [03:04] but that would fly in the face of actually *getting* that traceback [03:09] yeah :/ [03:09] lifeless, could it be that the parent process (that watching the subunit stream) choked on some child process output? [03:10] silently ate it maybe? [03:11] thread dump in the parent says nothing out of the ordinary: [03:11] Thread 1 [03:11] #0 0x00007f535ef0cdc2 in select () from None [03:11] #1 0x00007f535ec39784 in time_sleep (self=, args=) from /build/buildd/python2.5-2.5.2/Modules/timemodule.c [03:11] /var/launchpad/tmp/eggs/zope.testing-3.9.4-py2.5.egg/zope/testing/testrunner/runner.py (521): resume_tests [03:19] according to the code and thread dump it looks like the parent is running on the while-input loop [03:19] how long does this take to trigger ? [03:19] and is it reliable ? [03:19] if its not too long, you might try running the child under strace [03:19] as an ugly debugging hack [03:19] not reliable, maybe 40% failure rate, takes 2 hours to trigger with a full suite run. [03:19] also is it only on that test, or a general sort of thing? [03:20] it happens anywhere in the windmill portion of the suite [03:20] does it happen if you just run windmill? [03:20] that is why I really really wanted that full traceback :) [03:20] lifeless, not in my testing, no [03:21] and it doesn't happen on buildbot either [03:21] just on ec2 [03:21] and you can trigger it locally running everything ? [03:21] or on ec2 w/windmill only ? [03:21] nope [03:21] only when running the full suite, including the windmill suite, on ec2 [03:23] bugger [03:23] uhm [03:23] buffer limit ? [03:23] hmm [03:23] oh [03:23] you use wait() right [03:24] IIRC that can trigger on anything in your process group [03:24] want to bet that there is a child of a child which is being allowed to zombie until the child goes, and you're getting a signal on that child-of-child, which you then close the pipe on the immediate child and things go boom [03:24] or something [03:24] well, this is without using my code, but I can check the zope testrunner - it may .wait() as well [03:25] I'm not terribly convinced of my explanation there [03:25] lol [03:25] my first thought was "yay IPC is fun" [03:28] hmm [03:28] lifeless, looking at the zope testrunner code, what you say may be possible. I would have to think through the process though [03:29] there is a note in there how reading the child process (the child layer) can be interrupted by EINTR if something sends SIGCHLD [03:32] but it would be really really odd that I have seen the traceback truncated on that stopTest line so many times [03:34] thats true [03:35] OTOH subunit writes in a very predictable pattern === magcius_ is now known as magcius [04:14] how do I tell what version of bzr-loom launchpad has ? [04:22] mwhudson: thumper: ^ [04:24] lifeless: i guess look at utilities/sourcedeps.conf [05:33] probably quite old [05:50] poolie: I was thinking about your comments about high dkim verification failure rates. You might be better off doing dkim verification at the border too and then consuming an authresults header inside LP. [06:22] hi scottk [06:22] do you mean my speculation that many of them may have inadvertently broken headers? [06:22] two more thoughts there: [06:23] 1- it's much easier for me to land code into the bit of launchpad that processes incoming mail than it is to do that _and also_ get sysadmins to set up something in the border MTA [06:23] 2- i'd like to hope that we don't actually damage the mail on the way from our DC border in to the incoming queue [06:23] imbw [06:23] but i thought i'd try verifying them there first and if it turns out 100% are broken, i guess we'll know === almaisan-away is now known as al-maisan [06:49] given that we successfully verify gpg-signed mail, we can't damage it that badly (although i guess that doesn't sign headers?) === jtv is now known as jtv-eat [09:18] jtv-eat: the query looks acutally quite simple, given that I already have the product id and the sourcepackagename(s). [09:18] http://paste.ubuntu.com/443212/ === mthaddon changed the topic of #launchpad-dev to: Launchpad down/read-only from 11:00 - 14:00 UTC for a code update | Launchpad Development Channel | Week 4 of 10.05 | PQM is in release-critical mode | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes [11:02] Morning, all. [11:29] mwhudson: A dkim signature covers a lot more of the message than a gpg signature. The one case I've managed to see real success data of a large enterprise trying to do dkim verification on a second tier MTA and not on the border it had a significantly lower verification reliability than cases where I've seen it done on the border MTA. [11:32] ScottK: that makes sense [11:32] * mwhudson goes to bed === jtv-eat is now known as jtv [12:53] BjornT, ping [13:01] hi deryck === matsubara-afk is now known as matsubara === mthaddon changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.05 | PQM is in release-critical mode | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes === Angel is now known as AngelSpy [13:28] /builders is timing out on both production and edge. [13:28] Yep... saw that too :/ [13:29] How's it dying? [13:29] view/other_builders [13:30] wgrant: I'll paste the oops.. [13:30] Hm. That's a bit odd. [13:31] wgrant: http://pastebin.ubuntu.com/443300/ [13:31] Ah. [13:37] jamesh, i have a storm question [13:38] leonardr: shoot [13:39] is it possible to define a global hook equivalent to __storm_flushed__, or do i need to create a superclass/mixin and make everything inherit the behavior? [13:40] leonardr: there is an internal event for that, but it is not part of the public API [13:41] what would you want to use it for? === henninge_ is now known as henninge === Ursinha-afk is now known as Ursinha [13:44] jamesh: see https://code.launchpad.net/~leonardr/launchpad/test-representation-cache/+merge/26513 [13:44] i'm invalidating a memcached cache [13:48] leonardr: ah. The event I was thinking of is just fired once in response to store.flush() -- not once per object [13:50] ah, ok [13:50] it sounds like a superclass/mixin is the best bet [13:51] leonardr: there are also internal "flushed" events for each object, but no way to globally register for all objects [13:52] leonardr: note that not all data modifying Storm APIs will result in Python objects being flushed. [13:53] jamesh: I'm talking to stub about that now. are you thinking of his "4) Using Storm for bulk updates, inserts or deletions."? [13:54] leonardr: right. The ResultSet set() and remove() methods [13:54] wgrant: temporary patch - can you check pls: http://pastebin.ubuntu.com/443324/ [13:55] there isn't a bulk insert method in Storm, but I doubt you care too much about inserts if you're working with a cache [13:55] noodles775: Looks fine. [13:55] Are we blaming missing indices for the slow query? [13:56] I'm assuming so, but haven't checked yet... want to get the page back up so we can fix it properly in time. [13:56] Yeah. [14:02] jamesh: store.execute(Update({Person.displayname: 'foo'}, Person.name=='stub')) would be how to get Storm to do updates without clearing the cache (not that we do that, but it would be nice to be able to do that). [14:03] stub: you don't even really need to go behind Storm's back like that [14:04] I thought that would be the blessed approach for bulk updates; didn't think that *was* going behind Storm's back ;) [14:04] stub: if I use ResultSet.set() to do a bulk update, you'll only see __storm_flushed__() calls for the objects that are actually live in memory [14:04] Ahh... that would be a better approach for bulk updates ;) [14:05] it is really only intended to help you keep local caches up to date (e.g. storing computed values on the Python object) -- not external ones like I assume you're talking about [14:06] So there is a check to see if the object is live. We could hook in there to clear the cache at that point... although there are still problems we can only solve at the database level. Not sure if they are problems we really need to worry about at this stage. [14:06] Oh... no we couldn't since we won't know the keys :-( === thekorn_ is now known as thekorn [14:47] Does sinzui have a mark-released button feature again? A script? Or too much time on his hands? :-) [14:54] deryck, I used my script from *before* the feature that I had to retracted. I discovered we failed to update a lot of bugs on 10.04, 02 [15:00] reviewers meeting starting now [16:17] deryck, bug 386757 has been fixed for malone, hasn't it? [16:17] Bug #386757: JavaScript for subscribing should be refactored for reuse [16:18] Ah, or am I getting confused about branches that have landed but don't actually fix the bug? [16:19] gmb, not really fixed completely. [16:19] deryck, Ah, okay. [16:20] gmb, I did some work, rockstar did some work, and once we have state on the bug subscriptions, then we can do more work to unify our approaches. [16:20] though I think rockstar went really different from my work [16:20] deryck, Okay, I'll add it to the subs story. [16:20] And we can take a look at it later. [16:20] gmb, yeah, we should look again late in the UI refactor for this story. It will most certainly have to touch the js subscribing code. [16:20] Right. [16:21] deryck, I think my work was the culmination of discussions you and I had at the lazr-js sprint/UDS [16:22] deryck, also, I have a branch that I've been trying to get back to that does the wizard widget, so we should have that at some point as well. [16:23] rockstar, right, I knew we talked about it. I just meant it's more different than alike to what we have now in bugs. but better, definitely. [16:24] deryck, ah yes, because branch subscriptions have state, and you had planned to add the same to bugs. [16:25] rockstar, exactly. [16:25] So I think it will line up nicely with our subscriptions refactor the next couple months. [16:29] deryck, sweet. [16:34] sinzui: I have a question about packaging if you have 2 minutes ;) [16:35] I can help [16:35] sinzui: Is it correct that we are now copying packaging links whenever a new distroseries is created? [16:35] Yes. they were copied. [16:36] sinzui: for maverick or for all series? [16:36] for maverick using the script that bootstraps soyuz [16:37] When we copy the packages from lucid to maverick and started the builds, we also copied the packaging links [16:38] sinzui: and that will happen on each new series? [16:38] in the future? [16:39] yes. [16:39] sinzui: thanks, great help. ;) [16:39] It is only for Ubuntu because only Ubuntu uses soyuz === Ursinha is now known as Ursinha-lunch [16:41] sinzui: well, it looks like only Ubuntu supports packaging anyway, at least UI-wise. [16:41] "+ubuntupkg" ... [16:42] correct [16:45] henninge, yes, if you try and build for anything but Ubuntu, you can really bugger the build farm. abentley knows from experience. [16:47] rockstar: cool, that makes things easier for me, too. === matsubara is now known as matsubara-lunch === salgado is now known as salgado-lunch === al-maisan is now known as almaisan-away [17:24] noodles, why did you turn buildbase.queuebuild into a staticmethod? === deryck is now known as deryck[lunch] === salgado-lunch is now known as salgado === Ursinha-lunch is now known as Ursinha === matsubara-lunch is now known as matsubara === leonardr is now known as leonardr-afk === deryck[lunch] is now known as deryck === leonardr-afk is now known as leonardr === EdwinGrubbs is now known as Edwin-lunch [19:23] jelmer: http://launchpadlibrarian.net/49567000/chicken-chicken-git-mirror.log [19:29] * maxb ponders creating a dev.launchpad.net/FailingBzrSvnImports to gather common failure cases, and wonders if some such thing exists already [19:30] Also, what is the current situation with svn imports requiring a username password? Does that still require losa intervention on the import slaves? [19:30] gary_poster, ping, buildout question, when you have some spare time [19:33] mars, what's up? [19:35] gary_poster, being annoyed by buildout picking the first zope.interface it sees, then barfing when it reads versions.cfg and realizes that it needs an older zope.interface [19:37] gary_poster, just a sec, ran it through -vvvv, and it says there is a version conflict when loading zc.recipe.testrunner [19:38] so it could be that zc.recipe.testrunner needs a newer zope.interface [19:38] ok [19:38] are you specifying the version of zc.recipe.testrunner? [19:39] gary_poster, yep. According to zc.recipe.testrunner, it does not need an explicit zope.interface [19:40] mars, sounds odd. You have a branch for me to look at? [19:40] gary_poster, yes, I'll push the changes [19:41] ok [19:42] gary_poster, bzr branch lp:~mars/lazr-js/1.0 [19:42] k [19:43] gary_poster, also worth noting: the Makefile uses a LAZR_SOURCEDEPS_DIR variable, and mine points to ~/.buildout/ [19:43] k [19:45] gary_poster, zope.interface-3.5.1 is in the global download-cache, but it doesn't want to use it. The global eggs/ directory only has zope.interface-3.5.3 (maybe because buildout refuses to build the 3.5.1 egg) [19:49] mars, the buildout.cfg file is trying to do something that appears to be against your goals. ...wanna talk on mumble? [19:50] gary_poster, sure [19:50] k, I'm there :-) [21:04] mars: there was a import failures wiki page some time back [21:05] maxb, ^ [21:08] bah [21:08] sorry mars [21:09] np :) [21:28] lifeless, gary_poster, by the way, by hacking our testrunner I managed to get it to spit out the full windmill suite hang traceback: http://pastebin.ubuntu.com/443544/ [21:28] grah [21:28] ? [21:29] oh [21:29] "Look," Gary said intelligently, "a ValueError!" Though it does look like something in subunit on the face of it. [21:29] lifeless, ? [21:29] checking the code [21:30] so, error is not None - this is a plain pyunit api call [21:30] rather than a lovely shiny testtools one [21:31] stopTest was called [21:31] and then the zope testrunner formatter is calling addError [21:31] from within stopTest [21:31] thats odd [21:32] the error object is failing in _exc_info_to_string [21:32] so err is not a exc_info tuple [21:32] its something else [21:32] look at zope/testing/testrunner/formatter.py [21:32] whats the easiest way to get that for me? I have a 2 month old checkout of lp [21:33] mars: ^ [21:34] lifeless, looking [21:34] lifeless, check in eggs/zope.testing-3.9.4 [21:34] do you have that version? [21:35] sec, starting the vm [21:35] yes, I do [21:35] well, a py2.5.egg [21:35] should be good enough [21:36] that's the one [21:36] ok so this is subunut glue [21:36] and its calling _get_text_details [21:36] yes, its buggy [21:37] I'll put a patch up [21:38] lifeless, wow, thanks! [21:41] ok, grab lp:~lifeless/zope.testing/subunit [21:42] I don't know if passes tests, because it blows up when I follow the getting started instructions [21:42] ok [21:42] lifeless, it will take a while to run a new round of tests. A few hours :( [21:43] it will probably have to wait for tomorrow. [21:43] https://code.edge.launchpad.net/~lifeless/zope.testing/subunit/+merge/26638 [21:43] mars: so anyhow, its something blowing up in another thread, but the error reporting codepath was broken [21:44] ok [21:44] * mars waits for the diff to update [21:44] so we'll still expect an error [21:44] but it should be relevant to the windmill tests this time [21:44] \o/ [21:45] that would be a big step forward [21:45] Hurray for cascading failures! [21:45] :) [21:45] actualy, hurray for untested code :P [21:49] lifeless, many thanks. I'll let you know how it goes. [21:49] please do [21:49] sidnei: when you get back [21:49] https://code.edge.launchpad.net/~lifeless/zope.testing/subunit/+merge/26638 [21:53] by coincidence, this may have been the source? Exception in thread Thread-3 (most likely raised during interpreter shutdown): [21:53] Traceback (most recent call last): [21:53] File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner [21:53] File "/usr/lib/python2.5/threading.py", line 446, in run [21:53] File "/var/launchpad/tmp/eggs/windmill-1.3beta3_lp_r1440-py2.5.egg/windmill/server/https.py", line 398, in start [21:53] File "/usr/lib/python2.5/SocketServer.py", line 218, in handle_request [21:53] : 'NoneType' object has no attribute 'error' [21:53] very likely [21:53] the check was a threading thingy [21:55] so my 'reprint the exception' code in bin/test actually spat out the entire error [21:56] who knows why it was getting truncated [21:57] wait, spoke too soon. There is nothing in the on-disk capture file, just the console. May have just been a luck break. === salgado is now known as salgado-afk [22:09] Is there a good way to hold a conversation with the requester of a vcs import? [22:09] 'contact this person' in launchpad [22:10] ... and manually keep the whiteboard up to date with the state :-/ [22:10] yes :( [22:12] In a most perplexing move, someone's trying to register a ~lrcshow-x/lrcshow-x/trunk vcs-import when ~vcs-imports/lrcshow-x/trunk already exists [22:12] maxb: hey, at least you can see who registered the import now [22:12] that was one of the more frustrating problems with the ye olde system [22:12] But given the history of their svn repository, they clearly don't understand version control very well [22:17] mwhudson: so what were your testing framework thoughts after looking at stuff yesterday [22:18] lifeless: i admit to getting a little sidetracked [22:19] lifeless: i think discover + subunit + testr run looks pretty nice though [22:29] Could someone look up OOPS-1614M3672 for me? (occurred attempting to rename a vcs-import branch) [22:29] sec [22:29] Module lp.code.model.branchnamespace, line 135, in validateRegistrant [22:30] % (registrant.displayname, owner.displayname)) [22:30] BranchCreatorNotMemberOfOwnerTeam: Max Bowsher is not a member of lrcShow-X team [22:30] zomg 500ms of sql time there [22:30] maxb: arse [22:31] maxb: was that over the api? [22:31] No, web ui [22:31] maxb: what did the form look like for the owner? [22:31] maxb: you should have had a dropdown [22:32] maxb: if you didn't, it is a bug [22:32] oh, I see [22:33] maxb: BTW I think you are doing an awesome job with the imports [22:33]