[00:00] gah. /me carefully follows the instructions on what to write in the email to contributor-agreement@canonical.com.... and then forgets to actually attach the pdf [00:01] thumper: when i was an undergrad, we had a "rashers of bacon consumed" league table in our house for a term [00:01] mwhudson: heh [00:02] it got a bit silly [00:09] * thumper is making a quadruple bacon egg and cheese muffin, well, two muffins [00:10] the bacon had to be used [00:10] it would be a shame to throw it out [00:10] I only realised after it went in the pan that I _could_ have frozen some [00:19] such a thing to regret, having too much bacon [01:07] Bacon Explosion! [01:07] DoIt [01:18] wgrant: your announcement method rename fix is landing now [01:19] thumper: Thanks. [01:34] Is there any good way to produce a summary list of failing tests? [01:40] man concurrency is hard :( [01:41] * mwhudson gives up on it for now and has lunch instead [02:07] thumper: Does PQM hate me, or is it still just really slow? [02:08] PQM is an equal opportunity hater. [02:19] wgrant: you've got a branch waiting to land via pqm? [02:19] yes. [02:19] spm: I presume so. [02:20] wgrant: pqm is currently stopped due to a CP in progress, sorry. please to be waiting ~ 4 hours. [02:20] But it's all still private, along with buildbot. [02:20] spm: Ah, right. Thanks. [02:21] CP ? [02:21] Cherrypick. [02:23] maxb: basically taking a revno that has already landed on one of the trees; merging it into the existing current build; run the full test suite. push that live where appropriate. Is only done for critically broken things. === spiv_ is now known as spiv [02:24] Or forgotten "What's new?" updates. Ehem. [02:37] How can I ask the test system for a list of tests in the order they will actually be executed? I tried --list and it gave me a list, but they don't seem to be executing in that order [03:29] --list-tests [03:29] maxb: they get sorted to reduce churn on expensive test fixtures [03:29] maxb: there isn't a defined order, as such. [03:32] * maxb crosses fingers and hopes this testrun actually manages to finish [03:32] It's not looking too bad actually (Python 2.5) [03:35] maxb: glad to hear that! [03:38] 26 failures 19 errors out of some ~17000 tests so far [03:45] not bad at all. reminds me of my templating branch. which, by the way, i need to bring up again. hopefully tomorrow :) [04:05] Hi folks, everything still running? [04:06] sidnei: Which templating branch? The Chameleon one? [04:19] crap [04:19] ec2test I started before I ran away failued due to a trunk conflict [04:19] grr [05:29] thumper: buy a faster cloud [05:32] jtv: and why would that help? [05:33] thumper: so you can find out before you run away. [05:34] * thumper smacks jtv around for a stupid idea [05:34] * thumper is not in a good mood today [05:34] * jtv tries to look chipper and supportive, but can't keep the hurt look out of his eyes [05:34] jtv: I have no sense of humour today [05:35] * jtv chokes back obvious bad joke about Americans [05:42] mwhudson: the translations-to-bzr script is doing quite nicely... it just exported translations for 18 productseries, skipped 1, and failed 4 because their branches weren't set up, all in 2 minutes 10 seconds. [05:42] I really ought to start skipping unchanged files, for that order-of-magnitude speed boost. [05:42] jtv: hooray [05:43] mwhudson: the dirty little secret there is that ddtp-ubuntu still breaks it because the transaction runs for too long and the database watchdogs kill it. Fix has landed, but we won't CP it unless it becomes unavoidable. [05:45] mwhudson: would it be very hard, or evil, to create branches that give me "not a branch" errors? [05:48] anyone remember the view attribute for the new page titles? [05:48] thumper: not off the top of my head... but wasn't pagetitles.py also supposed to work? [05:49] yes, but... I'm going to add a comment to the top of pagetitles.py saying "stop bloody using this file!!!" [05:49] and I'd like to be nice and tell them what to use instead [05:49] * thumper goes hunting [05:50] hah [05:50] wgrant: xactly [05:50] page_title [05:50] surprise! [05:50] *gasp* [05:51] wgrant: looking for someone to pick that up and push forward (hint hint) === jtv is now known as jtv-afk === sidnei is now known as sidnei-away [05:53] jtv-afk: no, not at all === jtv-afk is now known as jtv [05:53] hard, i mean [05:54] mwhudson: I guess though that the user may still have a branch sitting around locally, waiting to be pushed... [05:55] If I created the branch and populated it, and then the user pushed a different branch altogether, would the user's branch "win"? [05:59] if they use --overwrite yes [06:00] So not an unsafe idea at all to create the branches... How would I do it? Would be nice if DirectBranchCommit could do it. [06:02] jtv: you could create another branch type [06:03] jtv: and have them pulled from some known location [06:03] (a bit like import branches are now) [06:03] mwhudson: I do require that it be Hosted. If it's another type, it won't be referenced in this field. [06:03] and not let anyone write to them on disk [06:03] (don't know if this would make sense) [06:04] * wgrant curses replication lag to death. [06:04] It's particularly bad now that the bug page is AJAXy, as AJAX POSTs and PATCHes don't force further browser requests to the master. [06:05] wgrant: file a bug [06:05] There is a bug. [06:05] jtv: I would hesitate to write to the users branch directl.y [06:05] jtv: writing to your own branch the user can pull/merge from is conceptually clearer. [06:05] Personally, I'd model it as: there is a user ~rosetta [06:05] lifeless: I already write to the user's branch, as per the user's wishes. Question is whether I should create the bzr branch behind the user's "DB branch" if there isn't one. [06:06] ~rosetta can write to any branch that the user has been given access to [06:07] well, as long as they tell you the branch to branch from [06:08] They have already created the DB branch and configured it as the one I should write to. But there isn't necessarily a real branch behind it yet. [06:08] jtv: if there isn't a real branch, you would have to branch from somewhere [06:08] jtv: thus my comment [06:09] lifeless: oic... basically an empty "prototype branch"? [06:09] jtv: that said, I'm strongly inclined to consider a db branch without bzr data in it a configuration error [06:09] jtv: no, nothing empty [06:09] Okay, near-empty? [06:10] Anyway, isn't a db branch without any bzr data what you get when you register a Hosted branch in LP? It seems a bit pointless to require that users populate the branch. [06:10] (In this case) [06:13] lifeless: actually, why would I have to branch from somewhere? [06:14] to get their content [06:15] jtv: registering 'hosted branches in lp' is a bug, it causes lots of questions and confusion [06:15] In this case it's just what I want though, and if I could just create the bzr directory on the fly if needed, that'd be perfect for me. [06:15] so whats the use case. [06:16] I'm not understanding the desire to have a branch the user can't merge from [06:16] I don't understand... Why would the user not be able to merge from it? [06:16] try this [06:16] cd /tmp [06:16] actually [06:16] cd ~lp-repo [06:16] bzr init --2a foo [06:16] cd foo [06:17] bzr commit -m "first post" [06:17] bzr merge ../trunk [06:17] [adjust paths for your environment] [06:17] The commit fails, of course. [06:18] it shouldn't [06:18] There's no data in there yet. [06:18] oh, add --unchanged there [06:19] Anyway, what you're trying to tell me is I can't merge branches without common ancestors? [06:19] yes [06:19] it goes deeper [06:19] you can't merge files unless the files started at a common place [06:19] jtv, hi! :) [06:19] hi Ursinha! [06:20] jtv, do you have a few minutes after your discussion with lifeless? [06:20] Ursinha: very very few. :) [06:20] Ursinha: if it's about the approval script failures, we're on it. [06:20] jtv, :P [06:21] jtv, actually no, it's about some weird timeouts [06:22] lifeless: so it'd be good for the user to choose the branch wisely... but in this case afaics having a branch that lives in its own universe is still a lot better than having one that produces nothing at all. [06:22] jtv: I still don't understand the case [06:22] what is the use case [06:22] lifeless: this is for daily translations exports. "Pick a Launchpad-hosted branch and I'll commit daily snapshots of your translations to it." [06:23] lifeless: minimally, bzr is just used as an asynchronous transport here. [06:24] An alternative to ftp, almost. Merging is slightly more advanced usage, but not everyone will even want that. [06:24] jtv: thats fine; just check when they select the branch that it has been successfully mirrored once [06:24] jtv: if they want an empty branch, they do 'bzr init lp:~person/project/trunk [06:24] lifeless: mirrored? or pulled? this works for Hosted branches only. [06:25] jtv: copied from private hosting to http [06:25] jtv: what I'm saying is: don't special case this in your code. Depend on them making the branch appropriately. [06:25] lifeless: sorry to be thick, but... private to whom? And to what over http? [06:25] it is, at worst, one command. [06:26] jtv: hosted branches get scanned and published inside the server [06:26] there is a flag written when this succeeds. [06:26] I'm saying check that this flag has been set [06:27] as its being set indicate the branch has been real for at least one puller cycle [06:27] lifeless: now I see... That's good, thanks. It's another way of avoiding these quiet failures. I'll file a bug. [06:31] lifeless: I think they call the http-accessible hosting facility the "read-only area." [06:32] hi thumper [06:35] lifeless: bug 409686 [06:35] Bug #409686: Export branch should have real bzr branch attached [06:36] thumper: about those timeouts... we see weirdness all over the place atm and suspect foundational problems. [06:37] sorry, not thumper, Ursinha [06:37] (I wonder how I could've made _that_ mistake...) [06:39] jtv, I've asked flacoste earlier today about those, questioning that that could be due to the storm update, but he said that was unlikely, and that we had changes in messaging sharing [06:40] and if I asked danilo about that... I don't, so I'm asking you :) [06:40] let me find some of the oopses [06:41] jtv, OOPS-1309F1091,OOPS-1312C1554,OOPS-1309F1091 [06:42] the slowest query is select replication_lag(), that according to flacoste should be fast [06:42] * Ursinha curses the bot [06:42] https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1312H1982 [06:42] https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1312C1554 [06:42] https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1309F1091 [06:51] wgrant: fyi. pqm reopened, so your landing should pass shortly [06:51] spm: Thanks. [07:12] spm: It has indeed just landed. [07:12] \o/ [07:22] I need to go make dinner [07:22] TTFN people === Ursinha is now known as Ursinha-afk [08:51] hello [08:59] hai [09:00] jml: http://advogato.org/person/robertc/diary/108.html [09:03] lifeless, \o/ [09:04] hudson is pretty sext [09:18] Howdy [09:19] * maxb rejoices at running the testsuite all the way through for the first time on python 2.5 [09:20] * maxb doesn't rejoice about it taking 5 hours though [09:23] maxb: What are the stats? [09:29] Great stuff maxb ! [09:29] https://dev.launchpad.net/LaunchpadOnKarmic [09:30] wgrant: I've added a test summary ^ [09:30] in brief - there's work to be done, but it could have been a lot worse [09:31] maxb: 1328 tests ran? [09:31] Hmm... It should be closer to 23k? [09:31] * noodles775 checks latest buildbot run. [09:32] buildbot is still in the exclusive realm of Canonical employees :( [09:32] 23741 tests. [09:32] make check will run 23k tests [09:33] if it doesn't, there's a problem with the tests [09:33] That's getting close. [09:34] wgrant: as in, buildbot visibility? [09:34] 23117 if you sum all the categories [09:34] jml: Yes. [09:34] :( [09:34] maxb: That's what I got too. [09:34] So, not too bad. [09:34] I'm very surprised so many passed. [09:34] how much of Zope did you have to poke? [09:35] I cherrypicked some upstream changes for zope.proxy and zope.security, and did an almightly bodge in zope.sendmail [09:35] Ah, I shall check the branch. [09:36] oh, and a cherrypick for zope.app.security too [09:37] maxb: sounds close (the buildbot run that ran 23741 tests was r9053). [09:37] r9053@devel ? [09:37] Yep. [09:38] * maxb curses bzr's weird idea of revnos [09:39] What's weird about it? [09:40] apart from presumably being different to hg's. [09:40] BjornT, you around? [09:40] jml: yes [09:41] The way it bears no meaning whatsoever cross-branch [09:41] BjornT, I was wondering about bug-branch linking appearing in the activity log. [09:42] BjornT, ISTR you or someone on the bugs team fixing something in the scanner to make the linking appear inline with the bug comments, but that doesn't seem to be what's happening. [09:43] Ah right, so by happy coincidence my testrun was based on devel@r9053 too [09:43] jml: no, we didn't fix it to appear inline, we just fixed it to actually appear in the activity log. [09:43] BjornT, ahh ok. [09:43] BjornT, and one doesn't imply the other, I take it. [09:44] jml: right. we don't want all activity to appear inline, for example subscriptions. however, there are some things that definitely should appear inline, but we didn't have time to implement it. [09:44] BjornT, ok, cool. [09:44] jml: you think it would be useful to have branch linking being inline? there's always the question whether it's actually useful, or if it simply adds noise [09:45] maxb: but that's your local branch of devel, you can easily check the log to see where you diverged from devel (or did you mean that you'd merged devel@r9053 before doing running tests?) [09:45] BjornT, I think so. [09:45] BjornT, it doesn't happen often, and means that someone has done some work on the bug. [09:45] (a secondary bug is that the bug should probably be marked as in-progress) [09:46] noodles775: I mean I used bzr log -n0 --show-ids to find my last merged rev and then looked up the revid in devel to map it back to a revno [09:46] maxb: in which case, it's strange that you had a different number of tests. Was that using 'make check'? [09:46] yes, make check [09:46] BjornT, we have a bug somewhere on launchpad-code about making it appear in line. I'd like to add some notes to the bug about how to actually do it. [09:47] maxb: `bzr misssing` not work for you? [09:47] noodles775: It's not incredible that some test modules would fail to import. [09:48] bigjools: that'd tell me what I didn't have - I wanted to know what I didn't have - oh.... so I should have run bzr missing --this in devel and subtracted one from the lowest shown revno [09:48] wgrant: no not at all, I'm just surprised that it didn't cause an error if that's the case. [09:49] Is it possible that some of the errors noted in that summary could abort the entire test module, skipping later tests? [09:49] BjornT, where should I start looking to add bug/branch linking events to the inline view [09:49] noodles775: The tests don't exist if the module failed to import. [09:49] noodles775: There'll just be an exception somewhere back in the log. [09:49] maxb: yes, that can happen [09:50] maxb: good question. [09:50] particularly in doctests [09:51] I forget exactly how doctests count. Is it one test per block? [09:51] jml: you should start at BugTaskView.activity_by_date, and adjust the interesting_changes bit to include branch linking [09:51] Not as in Python syntax block, but doctest block. [09:51] jml: with some luck, that's all that's needed [09:51] BjornT, thanks. [09:51] wgrant: no idea, it seems random to me :/ [09:51] wgrant: Hmm... if that's the case, then I'd say it's a bug - if 'all tests pass' then it lands (at least, I don't scroll back if all tests passed) [09:52] if he altered tests then the number of tests could have changed [09:52] wgrant: so hopefully the difference is due to the failures or altered tests, rather than import failures. [09:52] noodles775: Hopefully. [09:53] BjornT, the string should match the 'whatchanged' key of the activity dict, I assume. [09:54] jml: yeah. if it doesn't look good, you have to change BugActivityItem to do special things for branch links [09:55] jml: http://gorf.tangent.org/hudson/job/Drizzle-subunit/ has a test graph now [09:55] BjornT, thanks. [09:57] zero altered tests [09:58] (so far) [09:58] maxb: Did you collect a list of the failed tests to start attacking? [09:59] I have the entire log courtesy of screen, I need to work my way through it turning it into a list of test names and initial notes of probably causes [09:59] *probable [10:03] When I say zero altered tests.... [10:03] I did have to alter the harness to wait longer for the librarian to start up [10:03] I wish lp output in subunit at this point [10:04] cause then you could do 'subunit-filter log | subunit-ls' and have that list instantly ;) [10:04] Does a newer zope.testing do subunit? [10:04] buildbot seems to parse it ok! [10:04] wgrant: not per se; jml and I did a patch to launchpads test runner use arbitrary TestResult objects [10:05] which permits subunit; jml was landing that work but I dont know if he had [10:05] lifeless, I think I have [10:05] cool [10:05] but it might have rotted a little with recent buildout shenanigans. [10:06] its a shame buildout != dpkg [10:06] Indeed... [10:06] eggs always seemed like a bad direction. [10:06] its a bit of an awkward halfway-house [10:06] wgrant: I loathe them. With a passion. [10:06] Does anyone not loathe eggs? :-) [10:06] lifeless: I noticed! I'm not a fan either. [10:07] anyhoo... internet dragons to kill [10:07] jml: theres an xfail support patch up you might like to review; if you have a minute [10:08] lifeless, sure. I'd like to dive back into actual Launchpad hacking now that my email kneejerks are done [10:08] lifeless, what's the url? [10:09] jml: hello, btw [10:09] https://code.edge.launchpad.net/~lifeless/subunit/xfail/+merge/9748 [10:09] mwhudson, hi [10:09] jml: want to review a branch? [10:09] lifeless, ta [10:09] mwhudson, is this your race condition branch? [10:09] jml: i'm much happier than i was with this stuff than i was yesterday :) [10:09] jml: yes [10:10] jml: the cover letter will include alan perlis' epigram no. 58 [10:11] mwhudson, beautiful. I'd love to look at it. [10:20] jml: have a good day [10:20] * lifeless goes to entertain himself [10:20] lifeless, good evening [10:23] jml: https://code.edge.launchpad.net/~mwhudson/launchpad/more-task-scheduled-bug-408638/+merge/9749 [10:24] mwhudson, thanks. [10:26] I'm wondering if it's worth having a bug tag for "I'll bend over backwards to help you land a patch that fixes this bug" [10:26] jml: thanks in advance [10:26] mwhudson, np [10:27] jml: if you want to ask things about it, sooner is better than later :) [10:27] mwhudson, sure. [10:27] mwhudson, in context switch mode right nowm. [10:28] k [10:36] 'if' should be an expression in Python [10:37] it is in 2.5! [10:37] ish [10:59] Morning. [11:03] mwhudson, review done. [11:04] but I guess it's pretty late [11:07] A couple of days ago I did a branch to export IArchiveDependency on the webservice, but I'm not entirely sure about the security bits. [11:08] At the moment read access is allowed unconditionally, and write access is completely forbidden, but you can't get hold of one with having launchpad.View on the Archive. Should a security adapter be written for IArchiveDependency, or is that sufficient? [11:11] I'm not sure. [11:13] wgrant: I'd guess 'no', but I don't really have enough domain knowledge to give a full answer. [11:13] bigjools, ^^ ? === danilo-afk is now known as danilo [11:14] jml: 'no' to which? [11:14] wgrant: if you're not sure then you haven't written enough tests [11:15] bigjools: I know I can't get hold of an object, but I wonder if I should make it safer in case another exported method leaks them. [11:15] wgrant: "no you don't need to write an adapter" [11:15] jml: OK. [11:15] But I think I probably do. [11:15] ok. [11:15] you'd know better than me, at this point :) [11:16] wgrant: how do you know write access is forbidden? [11:17] bigjools: I haven't a test for that yet, but lib/lp/soyuz/configure.zcml strongly suggests that it is. [11:17] wgrant: if it's not tested then you don't know, by definition :) [11:17] Right. [11:18] Although similar tests (eg. ArchivePermission in stories/webservice/xx-archive.txt) don't seem to test that. [11:19] then they suck [11:19] :( [11:19] which is probably my fault [11:20] jml: thanks [11:20] Can somebody point me at a good example? [11:20] I'm not clear on exactly how thorough the tests should be. [11:20] * bigjools looks [11:22] wgrant: lib/lp/soyuz/stories/webservice/xx-archive.txt "Modifying privacy" [11:23] wgrant: lp.code.tests.test_branch has unit tests for branch security [11:23] jml: I think he explicitly needs webservice tests though [11:23] oh right. [11:24] the existing object is fine as it is, it's not directly exposed until wgrant exports it [11:24] bigjools, about verify_acl [11:24] yep? [11:24] bigjools, it doesn't currently check component privileges for existing packages. [11:24] is this a feature or a bug? [11:25] feature [11:25] deliberate [11:25] I don't know to what depth I must go -- do I just verify that people can't write to a couple of the fields? [11:25] bigjools, what's the reasoning? [11:26] * bigjools is trying to remember [11:26] wgrant: test important fields [11:26] where it would compromise functionality [11:27] because BugTask.canApprove consults component privileges, and IIRC you implied I should check it for the package branch stuff. [11:27] bigjools: OK, sounds good. [11:29] jml: there's a comment in the middle of verify_acl that explains it. I remember putting it there because I kept forgetting why :) [11:29] it's because of overrides, basically [11:30] bigjools, I'm looking at verify_acl & don't see anything that answers my question. [11:30] oh wait the comment talks about new packages. in any case, it's because of overrides [11:30] ok. [11:30] so we make sure that the person has an ACL to at least one component [11:31] then let the overrides work after that [11:32] jml: ummm wait [11:33] * bigjools engages brain [11:33] why do you think that? [11:34] bigjools, why do I think what? [11:34] jml: "bigjools, it doesn't currently check component privileges for existing packages." [11:34] bigjools, it checks it based on the dsc, but not the other thing... gimme a sec to find it. [11:35] yeah it's at the bottom of verify_acl [11:35] latest_published_component [11:36] right. so it checks the dsc rather than latest_published_component. why's that? [11:36] * bigjools thinks [11:39] It's already overridden, isn't it? [11:39] yes, on the files [11:39] find_and_apply_overrides() is called a few lines before verify_acl === sidnei-away is now known as sidnei [11:40] I can think of a situation where it might help [11:40] but I am not sure if it's valid [11:41] if a package goes universe->main, then a motu can continue to upload it if they target universe [11:41] That's not reasonable, and I don't see why that would work? [11:42] because it checks the dsc and not the current component [11:42] But the dsc component is already overridden with the latest ancestry. [11:43] ah so it os [11:43] is [11:43] That's pretty damn strange. [11:43] But it works. [11:43] christ I need more coffee [11:43] jml: ok so it checks based on the override, just in a roundabout way [11:43] I smell a missing comment. [11:44] and AIUI, the override is not the same as the dsc file is not the same as latest_published_component [11:44] is this correct? [11:44] distroseries.getPublishedReleases()[0] is used [11:44] which should be the same as latest_published_component [11:44] danilo, jtv: Help! [11:44] well, its publishing record [11:44] henninge: ? [11:45] (with the archive, too) [11:45] latest_published_component doesn't know about the archive. [11:45] henninge: seems danilo's network conked out [11:45] But I don't see why that would matter in any current situation. [11:45] jtv: oh [11:45] jml: so, it applies the override first, which changes the component on the DSC [11:45] jtv: I thought it was me [11:45] henninge: and me [11:45] and him :) [11:45] jtv: I don't see you on skype either, though. [11:45] henninge: then I'll just restart to make sure [11:46] wgrant: it does matter, that's not good [11:46] jtv: ok, back, but don't see you on skype [11:46] BugNomination takes the latest_published_component and asks archive.canUpload(person, component) [11:47] danilo: I was just restarting it for luck [11:47] is that any different, in effect, from what nascentupload is doing? [11:47] jtv: it's ringing [11:47] jml: eeek that's wrong, because if a PPA has a more recent publication in main it will use that [11:47] bigjools: When does it matter? [11:47] all of these queries need to be archive specific [11:47] bigjools: Doesn't latest_published_component restrict to distribution_archives? [11:47] * wgrant checks. [11:47] I'm fairly sure it did. [11:48] ah it does [11:48] phew [11:48] It uses _getFirstPublishing or something like that. [11:48] ok [11:48] it's kinda gross [11:48] A little. [11:48]