[00:00] <wallyworld> yes, but we're getting back to the topic i raised on the dev list - i disagree with our current workflow because it encourages only superficial qa just get get stuff deployed rather than testing for defects / verification
[00:01] <lifeless> wallyworld: uhm, thats the point though.
[00:01] <wallyworld> not imho :-)
[00:01] <lifeless> wallyworld: accept *some* risk, in return for velocity.
[00:01] <lifeless> wallyworld: higher velocity gives faster TTR
[00:01] <lifeless> wallyworld: and smaller deltas per deploy
[00:01] <wallyworld> sure, but i think we go too far and rework is the enemy of velocity
[00:01] <lifeless> wallyworld: and that gives a gaster TTD
[00:01] <lifeless> wallyworld: we are doing far less rework than in the old system
[00:02] <wgrant> Anyone want to review https://code.launchpad.net/~wgrant/launchpad/bug-728836/+merge/52355?
[00:03] <lifeless> wgrant: I think it may conflict with my fix
[00:03] <lifeless> the distroseries attribute in particular
[00:03] <wallyworld> now, if there were metrics to show the number of bugs that had to be reopened/reworked, that would be interesting
[00:03] <wgrant> lifeless: Which fix?
[00:03] <lifeless> wallyworld: I don't see how the ok to deploy process affects rework in this case
[00:03] <lifeless> wallyworld: 723560 or whatever it is
[00:04] <lifeless> wgrant: bug 727560
[00:04] <_mup_> Bug #727560: Archive:EntryResource:getPublishedSources <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/727560 >
[00:04] <wgrant> Ah, right.
[00:04] <lifeless> wallyworld: the process of closing bugs when the first deploy against them is done is easily tweaked and doesn't affect the deploy discussion at all.
[00:05] <lifeless> wallyworld: I appreciate that we need to improve the way these two processes interact, but its also -extremly- important we don't conflate the way we improve one with the way we improve the other.
[00:05] <StevenK> I think the qatagger needs a patch to give a list of bugs that can be closed by the rev we can deploy to.
[00:05] <wallyworld> lifeless: if affects rework because it encourages only superficial qa and not a "proper" check that the issue supposed to be fixed really is fully fixed. and so increases the chance that the defect will have to be revisited to fix it "properly"
[00:05] <lifeless> wallyworld: once a commit is in trunk, it *must* be deployable or rolledback in a matter of houres
[00:05] <StevenK> Makes it easier to cut-n-paste into LPS
[00:06] <wgrant> lifeless: It appears not to conflict.
[00:06] <lifeless> StevenK: there is a bug open, and when Ursinha rolls out my tweak to the templates it wil lshow Bug:1234 rather than Bug 1234 in the report
[00:06] <_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
[00:06] <lifeless> wgrant: cool
[00:07] <StevenK> Haha, I love that bug
[00:07] <StevenK> Since it STILL is
[00:08] <StevenK> lifeless: I meant a line at the top that has "If we deploy, the following bugs can be closed: 58585 58568 58922 66856 ..."
[00:08] <wgrant> StevenK: That bug's fix is the cause of it being so terrible.
[00:08] <lifeless> StevenK: yes there is a bug open for that
[00:10] <StevenK> gina needs a good hard rewrite
[00:11] <lifeless> wgrant: does get_archive do queries ?
[00:11] <wgrant> lifeless: I made Archive.debug_archive a cachedproperty to prevent that.
[00:12] <lifeless> where does bpr come from ?
[00:13] <lifeless> 278	+        candidates = (And(
[00:13] <lifeless> 279	+                BinaryPackagePublishingHistory.archiveID ==
[00:13] <lifeless> 280	+                    get_archive(archive, bpr).id,
[00:13] <lifeless> oh, I see
[00:13] <lifeless> 8 line generator.
[00:13] <lifeless> wgrant: not the clearest
[00:13] <wgrant> Bah.
[00:14] <wgrant> Don't blame me for generators being backwards :P
[00:14] <lifeless> forward ones are known as 'for loops'
[00:16] <wgrant> Maybe.
[00:16] <wgrant> It wasn't initially quite so big.
[00:16] <wgrant> The archive used to be in the main query.
[00:19]  * StevenK notes SMM is now out of date, given 45 second PQM is real
[00:20] <StevenK> And I think we could go one better and hook tarmac into Jenkins with a paramaterized build
[00:20] <wgrant> Heh.
[00:22] <wgrant> lifeless: Thanks.
[00:23] <wgrant> benji: Not around?
[00:33] <lifeless> wgrant: what do you need ?
[00:33] <wgrant> lifeless: Unfreezing.
[00:33] <lifeless> wgrant: we don't need benji, we need a losa
[00:36] <lifeless> wgrant: IIRC the docs were updated on friday
[00:36] <lifeless> wgrant: so it should be a simple point, ask a losa, and mail benji & the list
[00:36] <wgrant> I haven't read the new ones.
[00:36] <lifeless> wgrant: might be an idea ;)
[00:37] <wgrant> Better to ignore the docs and see what works ;)
[00:39] <lifeless> wgrant: other folk will look to the docs for reference
[00:39] <lifeless> wgrant: so keeping them in sync is important
[00:39] <lifeless> (or delete if not needed)
[00:40]  * thumper is close to pounding the table in frustration...
[00:40] <wgrant> lifeless: Where are these docs?
[00:40] <wgrant> thumper: If you need to pound a table, please choose BranchRevision.
[00:40] <thumper> haha
[00:40] <wgrant> If you can break a few indices off it would be great.
[00:41] <thumper> wgrant: we have a solution, but no time to implement
[00:41] <wgrant> It is like 130GB!
[00:41] <thumper> wgrant: jam and abentley have a workable solution that'll fix loggerhead speed too
[00:41] <thumper> wgrant: they just need the OK to spend some time doing it
[00:41] <wgrant> Ah.
[00:41] <thumper> wgrant: using bzr-historydb
[00:41] <wgrant> Will that work for LP too?
[00:41] <thumper> wgrant: it'll kill the table completely
[00:42] <thumper> wgrant: as far as we are aware
[00:42] <lifeless> wth is getQuery for
[00:42] <wgrant> I thought that was only planned for loggerhead.
[00:42] <wgrant> That's great news.
[00:42] <thumper> wgrant: quite a bit of analysis went into it
[00:42] <thumper> lifeless: on what?
[00:42] <lifeless> thumper: grep for '\<getQuery\>'
[00:43] <thumper> heh
[00:43] <thumper> kill it
[00:43] <wgrant> It's not needed by the Zope vocab stuff?
[00:43] <thumper> ah... maybe
[00:43] <thumper> check the Interface
[00:44] <wgrant> I'm grepping.
[00:44] <wgrant> Doesn't seem to be.
[00:45] <lifeless> Yeah, I had grepped there already
[00:45]  * thumper stabs some tech-debt in the face
[00:46] <wgrant> lifeless: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadRollout says that we can only unfreeze once the stable rev is determined.
[00:46] <wgrant> By the RM.
[00:46] <wgrant> This is why I don't like reading documentation.
[00:47] <wgrant> Because I means I have to knowingly subvert processes instead of just being ignorant of them.
[00:48] <lifeless> wgrant: [00:48] <wgrant> lifeless: Not there... near the bottom of the first section of "Preparatory Tasks"
[00:49] <wgrant> "Once all QA is done (including any RC branches that were landed after the db-stable -> devel merge), the RM will let the LOSAs know the revno of stable that we will use as the release branch. Once this release branch is determined, a LOSA can put devel back into normal mode on pqm and prep the stable branch for rollout"
[00:50] <wallyworld> anyone know if the teamparticipation table is designed reflect valid team memberships taking account of expiry dates, membership status etc? ie can i just query that one teamparticipation table to see if a person validly belongs to a team?
[00:50] <wgrant> wallyworld: Yes, use TeamParticipation.
[00:51] <wallyworld> wgrant: thanks
[00:51] <wgrant> wallyworld: It is unreliable in one case: some code (eg. inTeam) considers the team owner to be an implicit member.
[00:51] <wgrant> But we consider that a bug, and anything that uses TP directly (lots of stuff does) does not see that.
[00:53] <wallyworld> wgrant: ok. so add a teamparticipation clause to a query is considered ok i assume
[00:53] <wallyworld> adding
[00:53] <wgrant> Yup.
[00:53] <wallyworld> cool
[00:53] <wgrant> That's part of the point of it.
[00:53] <wgrant> Nice and easy.
[00:53] <wgrant> And quick.
[00:53] <wallyworld> sure is. so that means there must be code to maintain that table based on expring memberships etc
[00:54] <lifeless> wgrant: fixed
[00:59] <lifeless> thanks doko
[01:00] <wgrant> What about doko?
[01:00] <lifeless> mbf on dh_python
[01:00] <wgrant> Aha.
[01:00] <lifeless> so my inbox just exploded
[01:10]  * wgrant finds food and works out whether to drive +copy-packages down further or attack checkwatches.
[01:11] <lifeless> wgrant: 59 /    0  LanguageSet:CollectionResource:#languages
[01:11] <lifeless> wgrant: has noone attacking it right now
[01:12] <wgrant> lifeless: Maybe, but I want to get +copy-packages down low enough that I can stick a test over it.
[01:12] <wgrant> I basically just need to preload source files now.
[01:12] <StevenK> Didn't we add a CopyPackagesJob or so?
[01:13] <wgrant> There is the package clone job which is used for copy archives.
[01:13] <wgrant> But anyone pushing normal package copying out into a job is insane and likes causing QISEs.
[01:13] <wgrant> (see delayed copies, for example)
[01:14] <StevenK> Yes, you said earlier delayed copies don't need to exist?
[01:14] <wgrant> One reason for the existence (reuploading files to the public librarian) is wrong.
[01:14] <wgrant> Another reason (slowness) will be wrong once this lands.
[01:15] <wgrant> And the other (correctness when copying into the primary archive) is easily fixed by moving the overriding and emailing stuff into the normal copy logic.
[01:16] <lifeless> StevenK: before we move things into backend jobs we should make them efficient.
[01:16] <wgrant> lifeless: It is a dupe, but I cannot remember the summary of the original :/
[01:16] <lifeless> nor I
[01:17] <StevenK> wgrant: Lets kill delayed copies, then? :-)
[01:17] <wgrant> StevenK: When I have a moment I am going to investigate how much work that is.
[01:18] <wgrant> Custom uploads were another reason for delayed copies, but they were never actually implemented.
[01:18] <wgrant> And any future implementation should be done by making custom uploads not INSANE.
[01:19] <StevenK> Right, I have my test written, now comes the lovely "test dies with a permission error, edit security.cfg, make schema", rinse and repeat
[01:22] <wallyworld> StevenK: i did that over the w/e. not much fun :-(
[01:26] <mwhudson> you don't need to run the whole make schema do you?
[01:26] <mwhudson> just python database/schema/security.py or something
[01:27] <StevenK> I investigated last time I had this issue, and the only thing that helped was the full monty of make schema
[01:27] <mwhudson> enjoy then
[01:27] <lifeless> you need to run it against the template test database
[01:27] <lifeless> but yes, you don't need full schema build
[01:29] <StevenK> It was only 3 perms, so the pain was minimal
[01:34] <StevenK> Oh, sigh, every SPPH for Debian is PENDING.
[01:34] <StevenK> *That'll* help.
[01:44] <wgrant> StevenK: We're going to need to solve that eventually.
[01:44] <wgrant> I have considered getting the dominator to run over it.
[01:45] <StevenK> And after it takes 4 days?
[01:45] <wgrant> But we also need to handle deletions somehow.
[01:45]  * thumper grabs a bite to eat, then gets the girls
[01:46] <StevenK> wgrant: Just trying to write a better query to see how much of Debian's SPR.changelog are None
[01:46] <wgrant> StevenK: You should always use IN (1, 2), whenever you are querying for publications.
[01:46] <wgrant> Querying only for 2 is always wrong, unless you are the dominator.
[01:49] <StevenK> Apparently, 12k. I think my query is wrong, anyway
[01:53] <wgrant> StevenK: Maybe just SELECT COUNT(*) FROM sourcepackagerelease WHERE changelog IS NULL and upload_archive=3;
[01:53] <wgrant> Won't only give you current stuff.
[01:53] <wgrant> But meh.
[01:54] <StevenK> wgrant: Can't we craft the query like the populator does and ignore stuff with expired SPRFs?
[01:55] <wgrant> StevenK: You could. But nothing in Debian should be expired.
[01:55] <wgrant> Because our GC is sort of awful awful awful awful.
[02:00] <StevenK> I guess the timing of a test can't really be taken as anything?
[02:01] <wgrant> What do you mean?
[02:02] <StevenK> From devel: Total: 16 tests, 0 failures, 0 errors in 1 minutes 4.196 seconds.
[02:02] <lifeless> what about it
[02:02] <StevenK> My branch: Total: 17 tests, 0 failures, 0 errors in 1 minutes 5.195 seconds.
[02:02] <lifeless> sounds like your branch adds a 1 second test
[02:03] <StevenK> Clearly, my question is that number "reliable"
[02:03] <lifeless> repeat it 5 times
[02:03] <lifeless> take the lowest time for devel and the lowest time for your branch
[02:09] <lifeless> LEAN
[02:09] <wgrant> Heh.
[02:09] <lifeless>   Hard / Soft  Page ID
[02:09] <lifeless>      130 /  223  BugTask:+index
[02:09] <lifeless>      109 /    0  LanguageSet:CollectionResource:#languages
[02:10] <lifeless> wgrant: ^ let me encourage you to stay flexible and responsive
[02:10] <lifeless> wgrant: we will get back to copy packages
[02:10] <wgrant> Indeed.
[02:10]  * wgrant finds a timeout.
[02:11] <wgrant> Hmm. Or I could fix two checkwatches OOPSes and get us below 1000 exceptions by the end of the week.
[02:11] <wgrant> Tempting, tempting.
[02:11] <wgrant> But #languages it is.
[02:12] <wgrant> WTF, why is it querying Karma?
[02:12]  * wgrant deletes #languages as being stupid.
[02:14] <StevenK> Thereby fixing the timeout?
[02:14] <wgrant> Yes.
[02:15] <StevenK> ... maybe people use it?
[02:15] <wgrant> It fixes OOPSes *and* decreases WTF/m.
[02:15] <StevenK> wgrant: Sigh, SELECT count(spr.id) from sourcepackagerelease as spr JOIN sourcepackagereleasefile as sprf ON sprf.sourcepackagerelease = spr.id JOIN libraryfilealias AS lfa ON sprf.libraryfile = lfa.id WHERE lfa.content is NULL and spr.upload_archive = 3; => 0
[02:15] <wgrant> Who cares if peopel use it :P
[02:18] <StevenK> wgrant: The users, I guess.
[02:18] <wgrant> Hmm.
[02:18] <wgrant> I think I will unexport Language.translators_count, and maybe turn it into a method.
[02:19] <StevenK> Maybe people use that too? :-)
[02:19] <wgrant> No widely distributed scripts use it, so I do not care much about the users.
[02:19] <wgrant> They can adapt.
[02:19] <StevenK> What are you, a kernel developer?
[02:20] <wgrant> Probably.
[02:20] <lifeless> wgrant: is it in 1.0 ?
[02:20] <lifeless> wgrant: was it in 1.0 at release ?
[02:21] <wgrant> lifeless: It was added in March 2010. So it was probably just after 1.0 was declared.
[02:21] <StevenK> lifeless: Anyway, aren't you supposed to drop the lowest, the highest and take the average of the remaining?
[02:22] <lifeless> StevenK: huh?
[02:22] <StevenK> lifeless: Rather than run the tests 5 times, and just take the lowest number
[02:23] <wgrant> So, I could preload, or I could temporarily break a couple of scripts that might not exist and make requests a second or so faster.
[02:23] <wgrant> jtv: Heh, I was just looking at that.
[02:24] <lifeless> StevenK: we assume that a noisy environment can't make your test faster
[02:24] <lifeless> StevenK: and thus everything that makes it slower is noise
[02:25] <wgrant> jtv: We could optimise the query, or we could reexport the attribute as a method, breaking all of the very few scripts that use it.
[02:26] <lifeless> wow
[02:26] <lifeless> wgrant: I would default to optimising in the first instance
[02:26] <lifeless> wgrant: its easier (than the coordination to remove something in 1.0)
[02:27] <wgrant> lifeless: We need to coordinate the removal of most of 1.0 soon.
[02:27] <StevenK> Total: 16 tests, 0 failures, 0 errors in 56.713 seconds.
[02:27] <StevenK> Total: 17 tests, 0 failures, 0 errors in 1 minutes 2.982 seconds.
[02:27] <lifeless> wgrant: is that hyperbole or do you know something I don't ?
[02:27] <wgrant> lifeless: We have committed to supporting 1.0 for another 4 years. That is completely stupid.
[02:28] <StevenK> 6 seconds with the two best times, 2 seconds with the two worst times.
[02:28] <wgrant> We need to rescind that commitment.
[02:28] <lifeless> wgrant: I agree that is has a high cost; its why I'm encouraging devs to only put stuff in devel, and am sure that for 1.1 (or $label) we will only want to move some stuff into supported.
[02:29] <lifeless> so
[02:29] <lifeless> I have an idea
[02:29] <lifeless> why don't we write a caching layer for milestones
[02:30] <lifeless> and use it for the hidden edit forms but not for the visible bugtask rows.
[02:32] <jtv> Hi wgrant—whatever works, I guess.  :)  I'm not sure that "it's a method now" will console many people though…
[02:32] <jtv> wgrant: plus, there's already a bug for timeouts on… getAllLanguages IIRC.  So if you want a method that times out rather than a property that times out, well, you've already got one.
[02:32] <jtv> Though I do think what you say has merit in itself!
[02:32] <wgrant> jtv: Hmm? getAllLanguages is exactly the same problem.
[02:33] <wgrant> It gets lots of Languages, and serialising the Language calculates the translators_count.
[02:33] <jtv> Yes, I figured, but ironically the bug page didn't load.
[02:33] <jtv> That definitely shouldn't be a property, no…
[02:33] <wgrant> It is always going to be expensive.
[02:33] <wgrant> It doesn't have to be so expensive.
[02:33] <lifeless> so
[02:34] <wgrant> But it's still going to be expensive.
[02:34] <wgrant> The number of reasonable uses for that property by an API script is 0.
[02:34] <jtv> I was assuming that the API request specifically was deliberately getting contributor counts for all languages.
[02:34] <lifeless> I really suggest a) optimise, b) if we want to change start the discussion to change it
[02:35] <wgrant> Hmm.
[02:35] <wgrant> Maybe I can reexport it on devel, and preload on 1.0.
[02:36] <lifeless> I need a reviewer
[02:36] <lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-724033/+merge/52358
[02:37] <lifeless> wgrant: yes. Which means starting with the preload, and then (probably) file a bug to do something about it in a while in devel.
[02:37] <lifeless> :P
[02:37] <wgrant> Right.
[02:38] <lifeless> hmm, POFile:+translate is still showing up
[02:39] <jtv> Yes, it's difficult and IIRC there was a rewrite that we never completed.
[02:39] <jtv> lifeless: looking at your branch…
[02:41] <wallyworld> jtv: wanna review my fix for 407260 when you are free? https://code.launchpad.net/~wallyworld/launchpad/branch-picker-team-branches/+merge/52356
[02:42] <jtv> wallyworld: I'm just reviewing Robert's branch
[02:42] <wallyworld> jtv: no hurry at all. i just thought you may be interested :-)
[02:42] <jtv> I am, I am… just not free to look at it just yet.  :)
[02:43] <lifeless> wgrant: DistroSeries:EntryResource:getBuildRecords - improved by your branch ?
[02:43] <wgrant> lifeless: no.
[02:45] <wallyworld> thumper: do you have any examples of making a ws api call on one of our domain objects within javascript?
[02:45]  * thumper looks
[02:46] <lifeless> jtv: fixing the silly docstring
[02:47] <thumper> wallyworld: there is something in lp.code.tests.test_branch_webservice
[02:47] <wallyworld> thumper: thanks.
[02:49] <jtv> lifeless: speaking of docstrings, it might be nice to have one on CachedMilestoneFactory._load.  I have a feeling there's more going on there than I understand.
[02:49] <wallyworld> thumper: that's all python. i was after javascript :-)
[02:49] <lifeless> jtv: what do you think is going on ?
[02:49] <wallyworld> thumper:  i know how to do it from python :-)
[02:49] <thumper> wallyworld: ah...
[02:50] <thumper> wallyworld: what do you want to do?
[02:50] <lifeless> jtv: ah, its buggy. Fixed.
[02:50] <jtv> lifeless: the fact that it's based on self.contexts suggests that it's meant to be called only once, but if it were meant to, I get the impression it would've been a one-liner.
[02:50] <wallyworld> thumper: i want to invoke the recently exported pending_builds api on a recipe object from within javascript
[02:51] <lifeless> jtv: pushing the just once guard now.
[02:51] <jtv> The what
[02:51] <jtv> ?
[02:51] <jtv> Ahhhh, _that_ is what the comment meant…
[02:51] <lifeless> jtv: uhm its not a one liner because our vocabularies are based on /a context/ not on /contexts/
[02:51] <jtv> …it meant you only want to call this once?
[02:51] <jtv> It was very, very unclear and helped confuse me.
[02:51] <lifeless> jtv: its called many times
[02:52] <wallyworld> thumper: so i'm on the recipe index page and there'll be a recipe web_link i can use
[02:52] <lifeless> jtv: I want the heavy lifting to take place just once, and all at once
[02:52] <lifeless> jtv: we currently have some bugs where we construct *300* milestone vocabularies.
[02:53] <thumper> wallyworld: you want to do something like... :
[02:53] <thumper> lp = Y.lp.client.Launchpad();
[02:53] <lifeless> jtv: http://bazaar.launchpad.net/~lifeless/launchpad/bug-724033/revision/12538
[02:53] <thumper> lp.get(some_path, config)l;
[02:53] <thumper> and that'll be the recipe
[02:53] <thumper> then call the method on that object
[02:54] <thumper>  var client =  new Y.lp.client.Launchpad();
[02:54] <wallyworld> thumper: oh ok. looks reasonable. i'll give it a go thanks.
[02:56] <thumper> wallyworld: grep for "client.get" and you'll find some stuff
[02:56] <wallyworld> thumper: yeah, just grepping for ".get(" right now in fact :-)
[02:56] <lifeless> looks like I get POFile:+translate again
[03:01] <wallyworld> thumper: i've got enough to go on now but there really don't seem to be very many places at all we use the ws api from our js. unless i only had a boy look :-)
[03:01] <thumper> wallyworld: correct, we don't
[03:02] <wallyworld> thumper: is that something considered bad or alternatively we just haven't gotten around to it yet? i would have thought it would be the "right thing to do" from a soa perspective
[03:05] <thumper> wallyworld: we don't make many api calls over javascript to code objects
[03:07] <wallyworld> thumper: in that case would you prefer that i got the pending builds for a recipe another way? the ws api seems correct though. otherwise i'll need to hack zcml and add zope stuff
[03:07] <thumper> wallyworld: what are you doing?
[03:08] <wallyworld> thumper: may be easier via mumble?
[03:08] <thumper> wallyworld: yep,
[03:08] <thumper> me hooks up
[03:12] <thumper>         client.asserts.assertProperty(
[03:12] <thumper>             jquery=u'("div#edit-lifecycle_status span.value")',
[03:12] <thumper>             validator=u'className|branchstatusEXPERIMENTAL')
[03:18] <jtv> ahh I'm back.  lifeless, the last I got from you was "I want the heavy lifting to take place just once, and all at once."  It makes sense to me now that you've fixed it: the code I read initially re-created all the MilestoneVocabs on every invocation.
[03:18] <jtv> (Unless I'm going completely dyslexic, which I also wouldn't discount as a possibility :)
[03:19] <jtv> wallyworld: posted a comment on your branch…  hard-hitting questions about your approach.  :)
[03:20] <wallyworld> jtv: thanks. otp. will look soon
[03:22] <jtv> lifeless: to explain what I meant with _load being essentially a one-liner: I was thinking if the work is done just once, then ISTM it would amount to self.vocabularies = dict(target, MilestoneVocabularyTarget(target) for target in map(MilestoneVocabulary.getMilestoneTarget, set(self.contexts)))
[03:23] <jtv> Is that correct or am I still misunderstanding something?
[03:28] <lifeless> jtv: you're missing that it instantiates the vocaularies
[03:28] <lifeless> jtv: not the targes
[03:28] <lifeless> targets
[03:28] <lifeless> milestone_vocabulary = MilestoneVocabulary(target)
[03:28] <jtv> I thought MilestoneVocabularyTarget(target) instantiated a vocabulary?
[03:29]  * thumper afk for a while, back later tonight
[03:29] <lifeless> jtv: a product and a product series of that product share a set of milestones
[03:29] <lifeless> jtv: there is no MilestoneVocabularyTarget
[03:29] <jtv> Sorry, I mis-typed that.
[03:29] <jtv> Scratch the Target.
[03:29] <lifeless> jtv: so its a two pass filter: one to map context->targets which is the actual things that we need milestones on
[03:29] <lifeless> jtv: and a second to construct the vocabularies for each target
[03:30] <lifeless> jtv: a later branch will make the second pass a single call to do optimised sql and generate all the vocabularies out of a minimal set of lookups
[03:32] <jtv> lifeless: but without the typo, and modulo icky external references to self.vocabularies, you're saying _load does not amount to "self.vocabularies = dict(target, MilestoneVocabulary(target) for target in map(MilestoneVocabulary.getMilestoneTarget, set(self.contexts)))"?
[03:32] <lifeless> jtv: it won't once the next stage is done
[03:32] <jtv> The set() needs to be outside the map() perhaps?
[03:32] <lifeless> jtv: because it won't loop at that point
[03:32] <jtv> Ah, that'll be moved inside some batched method?
[03:32] <lifeless> yes
[03:33] <lifeless> this is prep work to do an eager load optimisation on bugs that have 150 different products / distro combinations
[03:33] <lifeless> it will reduce the query counts we have today substantially I hope, and make it easier to do the next stage
[03:34] <jtv> Then maybe you'd also like to move the self.cached_milestone_source.contexts.add() outside the loop that calls _getTableRowView?  The loop iterates over a list anyway.
[03:34] <jtv> Whaa I need to be afk for a moment
[03:34] <lifeless> jtv: I wanted to keep it adjacent to the code that hands out the milestone_source
[03:34] <lifeless> jtv: that way they can't get out of sync
[03:41] <jtv> back
[03:41] <lifeless> jtv: so its layering: someone calling _addTableRow or whatever its called *has* to make sure the context is added to the contexts set or the assertion will trip
[03:41] <lifeless> jtv: the easiest way to ensure that is to have addTableRow do it
[03:42] <jtv> lifeless: sure, makes sense
[03:42] <jtv> _getTableRow
[03:42] <jtv> View
[03:42] <lifeless> right
[03:42] <lifeless> I'm in POFile:+translate now, so going from memory
[03:42] <jtv> (Also, I just noticed it's called in another place as well)
[03:42] <jtv> (Still in the same loop, but nevertheless)
[03:43] <jtv> lifeless: if you have a bug open for the future task of batching this, it'd be nice to make that TODO comment a proper XXX so that all you have to do later is grep for the bug number.  Or someone else doing the same job, if it comes to that.
[03:43] <lifeless> jtv: its the same bug
[03:43] <jtv> Oh so you're doing that right after this branch?
[03:43] <lifeless> probably
[03:44] <lifeless> depends on the impact, but I suspect we'll still have timeouts on this once this lands
[03:44] <lifeless> BugTask:+index is our #1 timeout
[03:49] <jtv> lifeless: I'd prefer a note to check up on that (and file a bug if appropriate—whether it's timing out or "still a bit slow") over a TODO comment in the source TBH.
[03:50] <jtv> (Since you're going to Q/A this anyway, I'm assuming that your eyes will pass over the relevant data anyway)
[03:51] <lifeless> jtv: I think that thats waste TBH: the data will tell us
[03:51] <lifeless> I can delete teh comment if you like, but thats also waste - we're probably going to be back here in two days doing more.
[03:52] <lifeless> the comment will tell someone else guided here by profiling an option I've considered; a bug would not be usefully triaged outside of the data that we have driving maintenance
[03:54] <jtv> I don't think the comment will be very helpful to someone else reading it.  It's little more than a reference to something you have in your head that they'll have to reconstruct by reading the same data, or dig up who wrote it so they can ask.
[03:54] <jtv> If you can make it clearer, then great.
[03:54] <jtv> I take that back; it's clearer now.
[03:57] <jtv> Only other thing is test—it'd be good to know that calls for the same target will return the same vocab.
[03:57] <lifeless> I'm really not inclined to - because:
[03:58] <lifeless>  - I've refactored, vs introducing a new layer: whatever tests there are for this either already ensure that, or were judged unnecessary in the past
[03:58] <lifeless>  - the key thing isn't whether the same instance is returned, its whether the query count is flat
[03:58] <lifeless>  - and we can't test the key thing yet because the page is so darn horrible.
[04:00] <jtv> lifeless: so there is a test to ensure that the query count is flat?
[04:00] <lifeless> jtv: no, because its not
[04:00] <lifeless> jtv: 675  OOPS-1891H475   NullBugTask:+index for instance
[04:03] <lifeless> wgrant: want to mentor https://code.launchpad.net/~stevenk/launchpad/populate-spr-changelogs/+merge/52363?
[04:03] <lifeless> sorry, be mentored on
[04:03] <wgrant> lifeless: I probably shouldn't, since it's an adaptation of my code. But I could.
[04:04] <lifeless> wgrant: I'll still give it a full review
[04:04] <wgrant> k
[04:04] <lifeless> jtv: can I ask a question about pofile:+translate
[04:04] <jtv> go ahead
[04:04] <lifeless> in lib/lp/translations/browser/translationmessage.py
[04:04] <jtv> oh God
[04:04] <lifeless> line 1287 says if self.form_is_writeable:
[04:04] <lifeless> explaining we don't nee dsuggestions if the user cannot do translations
[04:04] <lifeless> but
[04:05] <lifeless> after the else clause of that if block
[04:05] <StevenK> lifeless, wgrant: I'm still attempting to get LP to generate a diff
[04:05] <lifeless> jtv: there is another if block starting with if self.sec_lang is None:
[04:05] <wgrant> StevenK: Push again. If that doesn't work, push -2 then push again.
[04:05] <wgrant> I think I know what's going on, but can't check because ENOLOSA.
[04:05] <lifeless> jtv: and it *also* asks for suggestions, but is not guarded by the form_is_writable check
[04:05] <lifeless> jtv: I am thinking that that is a bug, but is it ?
[04:06] <jtv> lifeless: arguably… the secondary language serves two very different use cases.
[04:06] <StevenK> Unable to obtain lock  held by stevenk@bazaar.launchpad.net
[04:06] <StevenK> at crowberry [process #14934], acquired 52 minutes, 15 seconds ago.
[04:06] <_mup_> Bug #14934: [warty] mozilla-browser: JS can access any mozilla memory <mozilla (Ubuntu):Fix Released by thombot> <mozilla (Debian):Fix Released> < https://launchpad.net/bugs/14934 >
[04:06] <StevenK> Whee!
[04:06] <lifeless> jtv: e.g. alt_submissions should be [] if form_is_writable is False
[04:06] <jtv> One is "this language is so similar, I'd like suggestions from it so I can just click them into the translation I'm working on."
[04:07] <jtv> The other is "me speak english no good, me like read translations other language as well"
[04:08] <lifeless> jtv: ok, so its not a bug
[04:08] <jtv> No
[04:08] <lifeless> so, I'll make it more complex there to keep the behaviour
[04:09] <jtv> Sorry to hear it, but I think it's for the best.
[04:09] <lifeless> jtv: its about 80% more efficient to query for the alt language with the primary language than to query seperately
[04:09]  * jtv whistles
[04:09] <lifeless> 350ms query * two, or one 355ms query
[04:10] <jtv> There's something that could be done about that, but not now:
[04:11] <jtv> re-design with the two use-cases in mind as separate goals.
[04:11] <jtv> The second use case doesn't actually need the suggestions if there's an accepted translation.
[04:12] <jtv> lifeless: there's something we did in the pre-sharing model that we threw out ("for the time being") to keep things simple.
[04:12] <jtv> Just fetch all relevant TranslationMessages, and sort out their roles python-side.
[04:13] <lifeless> jtv: yeah, thats what I'm bringing back in here
[04:13] <lifeless> jtv: my patch last week took a solid chunk of time off
[04:13] <lifeless> but not enough
[04:13] <lifeless> so this is the next step
[04:13] <jtv> There's really no point IMO in having separate calls "get me the current translation," "get me the current translation for the other side, and by the way is it the same one," "get me the suggestions on this side," and even "get me the suggestions for other, similar messages."
[04:13] <lifeless> jtv: the query plan for this is very expensive - it expects tonnes of IO
[04:14] <jtv> You're telling me… cost me years of my life
[04:14] <lifeless> but just think about all the sql you know now.
[04:14] <jtv> Done.
[04:14] <jtv> Why?
[04:15] <lifeless> a joke; suggesting that getting this fast had driven some sql knowledge
[04:15] <jtv> Not really—it drove a lot of optimization knowledge, but optimization goals have shifted a lot since the page formed.
[04:16] <lifeless> jtv: it was humour, not a serious statement
[04:17] <jtv> Forgive me—I am not well
[04:17] <lifeless> jtv: :( jetlag?
[04:17] <jtv> Hang on—friend in what sounds like serious trouble
[04:18] <wgrant> Multiple stores make me sad.
[04:36] <huwshimi> Has anyone here tried to set up Launchpad on Natty and had a problem installing python2.6-minimal?
[04:37] <wgrant> That should still work... what's the error?
[04:38] <huwshimi> wgrant: http://paste.ubuntu.com/576805/
[04:39] <wgrant> lifeless: ^^ you are probably keeping better track of modern Python transitions than I.
[04:40] <lifeless> fun
[04:40] <StevenK> bzr-lpreview-body!!
[04:40] <lifeless> huwshimi: what does dpkg -S /usr/lib/python2.6/site-packages show
[04:41] <StevenK> I suspect huwshimi's output to be the same
[04:41] <huwshimi> yeah the same
[04:41] <huwshimi> bzr-lpreview-body: /usr/lib/python2.6/site-packages
[04:41] <lifeless> in which case, remove that package
[04:41] <lifeless> then install python2.6-minimal
[04:42] <StevenK> But fixing bzr-lpreview-body doesn't enter into it?
[04:43] <lifeless> sure it does
[04:43] <lifeless> two separate things though
[04:43] <lifeless> unblock, then root cause
[04:44] <huwshimi> lifeless: That wants to remove python2.6 and some other python stuff, is this going to kill me?
[04:44] <wgrant> huwshimi: Use dpkg -r bzr-lpreview-body
[04:45] <StevenK> As root or with sudo
[04:45] <wgrant> apt-get remove will try to resolve the dependency mess, dpkg -r won't.
[04:45] <huwshimi> StevenK: thanks :P
[04:46] <StevenK> lifeless, wgrant: Diff for https://code.launchpad.net/~stevenk/launchpad/populate-spr-changelogs/+merge/52363 looks fine now
[04:46] <wgrant> StevenK: OK, will look in a sec.
[04:47] <huwshimi> Thanks guys, all working now
[04:51] <lifeless> jtv: so where are we at on this review?
[04:52] <lifeless> jtv: if you're sick, I can get a different reviewer.
[05:00] <wgrant> StevenK: Commented.
[05:01] <StevenK> wgrant: Rargh! (my fault, but your problem now)
[05:02] <wgrant> Heh.
[05:02] <wgrant> I presume I had a good reason for doing it.
[05:02] <wgrant> But I don't recall.
[05:02] <wgrant> It was my first tunableloop, so I may have been wrong.
[05:02] <StevenK> I'm not sure how to record to skip a given SPR, though
[05:03] <wgrant> Neither am I.
[05:03] <wgrant> Which is why i still hold the opinion that having this in garbo is crack.
[05:03]  * StevenK waves his hands furiously
[05:03] <StevenK> You say garbo is crack, lifeless says this is the way
[05:03] <StevenK> Rock, hard place!
[05:04] <wgrant> Yes, but I am right :P
[05:04] <StevenK> lifeless: ^
[05:10] <jtv> lifeless: back... I think my friend must have run out of battery so I can complete the review.  Not that much more I can say to him except as a way to let him distract himself from his problems.  :(
[05:10] <jtv> So I can complete that review.
[05:10] <lifeless> jtv: cool, he is ok ?
[05:11] <lifeless> wgrant: whats the story
[05:11] <lifeless> wgrant: and what does skipping an spr have to do with running it in the garbo
[05:11] <wgrant> lifeless: SPRs may fail to extract or be otherwise unimportable.
[05:11] <wgrant> lifeless: The job will start from the first SPR without expired files and without a changelog.
[05:12] <wgrant> So say we have 1000 SPRs that fail to import.
[05:12] <wgrant> The job will process those 1000 at the start of every run, then run out of time.
[05:12] <jtv> lifeless: No, he's not OK but he's out of immediate danger.  Part of the work every time this happens is to listen to him explain how he's not OK, and then the other part is the uphill struggle to convince him that he's not OK and there must be things he can do about it.
[05:13] <lifeless> wgrant: this seems orthogonal to the execution framework
[05:13] <wgrant> lifeless: It was designed to be started once.
[05:13] <wgrant> Garbo starts it hundreds or thousands of times.
[05:13] <lifeless> wgrant: thats a high risk design
[05:13] <wgrant> Oh?
[05:14] <lifeless> its extremely likely that anything running for more than a few hours will be interrupted
[05:14] <wgrant> Sure.
[05:14] <lifeless> stash the failed sprs in memcache and move on
[05:14] <wgrant> Well, not extremely likely.
[05:14] <wgrant> Back when this was written we were deploying once a week at most.
[05:16] <wgrant> The script was meant to work and be run just a couple of times, not to be the most robust, efficient creature in the universe.
[05:16] <lifeless> there is a pretty big spectrum in the middle
[05:17] <wgrant> I am curious as to why you say anything running for more than a few hours will be interrupted, though.
[05:17] <wgrant> We have some regular scripts that run for much longer than that.
[05:17] <lifeless> sources of interrupts;
[05:17] <lifeless>  - deploys
[05:17] <lifeless>  - long running transaction killing
[05:17] <lifeless>  - machine maintenance
[05:18] <wgrant> Deploys: OK.
[05:18] <lifeless>  - db deploys [forced interrupt]
[05:18] <wgrant> Long running transactions: there are none.
[05:18] <lifeless>  - 'kill X its making things slow'
[05:18] <wgrant> Sure, and it can easily handle being killed once a day or so.
[05:18] <wgrant> But once every 10 minutes?
[05:18] <lifeless> its a 1 second query to find the work to do
[05:19] <lifeless> why are we expecting failurs?
[05:19] <wgrant> Because some don't have changelogs, some don't unpack any more, and I forget other reasons.
[05:19] <lifeless> what will be the impact o fthat on derived distros
[05:20] <wgrant> Most of them are old, and those that are not old will just not have a base version.
[05:20] <lifeless> are you still doing it in increasing id order?
[05:20] <lifeless> just stash the last done id in memcache; 3 line patch
[05:20] <wgrant> Sounds reasonable.
[05:21] <StevenK> lifeless: Uh, I don't know how to do that
[05:21] <wgrant> There's a utility.
[05:22] <StevenK> Which, sadly, is as useful as saying "There's this website, it's called Google."
[05:22] <wgrant> Well, a memcache utility probably starts with IMemcache.
[05:22]  * wgrant greps.
[05:22] <wgrant> IMemcacheClient.
[05:23] <stub> client = getUtility(IMemcacheClient)
[05:23] <stub> result = client.set(key, value)
[05:24] <stub> if not result: raise RuntimeError('Memcached access denied or memcached servers down')
[05:24] <stub> value = client.get(key)
[05:34] <StevenK> wgrant: To answer one of your points, I'm using the master store since everything else in garbo is.
[05:36] <StevenK> stub: That's excellent, thanks.
[05:39] <jtv> lifeless: with apologies for the delay (my friend found a phone charger) I just approved your branch.
[05:40] <jtv> wallyworld: I initially typed my comment for your MP into the wrong window, so that it now defaces an unrelated bug.  Hopefully it's also on your MP now.
[05:40] <jtv> And now, food.
[05:58] <lifeless> jtv-eat: thanks!
[05:58] <lifeless> stub: https://code.launchpad.net/~lifeless/launchpad/bug-730391/+merge/52370 - I can have review?
[06:10] <stub> k
[06:17] <jtv> StevenK: something I find myself in doubt about now…  I've done the code to create a bunch of DSDs in Needs Attention state, but do I need to create DSDJs for those _as well_?
[06:18] <lifeless> wgrant: could you do me a favour?
[06:18] <lifeless> wgrant: with timeout and oops bugs, always put the oops report key - the page id - in the title.
[06:18] <lifeless> wgrant: e.g. bug 728836
[06:18] <_mup_> Bug #728836: copyBinariesTo query count scales by number of binaries <oops> <timeout> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/728836 >
[06:19] <lifeless> looks like a dupe of bug 575450
[06:19] <_mup_> Bug #575450: Archive:+copy-packages nearly unusable due to timeouts <lp-soyuz> <ppa> <timeout> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/575450 >
[06:19] <wgrant> lifeless: But which one? that has three :(
[06:19] <wgrant> lifeless: It is part of 575450
[06:19] <lifeless> wgrant: list em all
[06:20] <wgrant> But 575450 is big.
[06:20] <wgrant> So I wanted a separate one.
[06:21] <wallyworld> jtv: i was picking up the kid from school before but got your comments and am about to push up changes
[06:21] <jtv> cool
[06:50] <stub> lifeless: r=stub with minor bits
[07:26] <wgrant> Could someone review https://code.launchpad.net/~wgrant/launchpad/bug-728174/+merge/52374?
[07:27] <lifeless> wgrant: so, on bugs - I think its fine to do new bugs
[07:27] <wgrant> lifeless: I see that you tend to close the main one when the first fix is done.
[07:27] <lifeless> wgrant: I'd just like it to be a little easier for ursula and diogo and others that don't have the entire callgraph internalised, to see that there are related things.
[07:28] <lifeless> wgrant: What I do - yes. Uhm, I generally do that because the data is changed by the first fix.
[07:29] <lifeless> wgrant: like, for bugtask:+index it will be 7 or 8 or more distinct fixes before we're flat
[07:29] <lifeless> these are different to use case bugs where there is a specific use case with lots of rationale etc
[07:30] <wgrant> True.
[07:30] <wgrant> But I treat them differently if a user has filed and subscribed to the timeout.
[07:30] <wgrant> Because the bug is not just for us.
[07:30] <wgrant> It is for them too.
[07:31] <lifeless> wgrant: I'm happy either way
[07:31] <lifeless> like i say, what I want is better connections between what folk see in the oops reports and the open bugs
[07:31] <lifeless> so that its easier to avoid duplicates and overlappying work
[07:33] <lifeless> wgrant: why are you using ISlaveStore ?
[07:33] <wgrant> lifeless: getAllLanguages already did.
[07:34] <lifeless> so that leads to two Person objects in memory, for instaance.
[07:34] <wgrant> Indeed.
[07:35] <lifeless> don't destabilise things, but generally ISlaveStore should never be used. Ditto IMasterStore
[07:36] <wgrant> We often use ISlaveStore...
[07:36] <lifeless> we've had traumatic timeouts because of that.
[07:36] <wgrant> Oh?
[07:37] <lifeless> yeah
[07:37] <lifeless> eager load from one store, get object from the other, dereference.
[07:37] <wgrant> Ah.
[07:38] <lifeless> we should, I think, change things to only ever have one store connected.
[07:38] <wgrant> That sounds reasonable.
[07:38] <wgrant> I shall change this to use the default store, as there is no clear benefit, you object, and it makes tests easier.
[07:39] <lifeless> wgrant: there is a risk something else will barf at you from ec2. So its up to you whether you change this or not.
[07:39] <wgrant> lifeless: If it does then I will kill it :)
[07:40] <lifeless> :)
[07:41] <lifeless> ok, two timeout fixes playing today, and a third landed in the morning.
[07:41] <lifeless> not too bad.
[07:42] <lifeless> stub: I'd like a brief voice chat about fast db deploys; got some time?
[07:47] <wgrant> I want Python 2.7 :(
[07:48] <StevenK> Why?
[07:48] <wgrant> Set literals, among other things.
[07:49] <StevenK> You can't import it from __future__?
[07:49] <wgrant> No, they are a new backport in 2.7.
[07:53] <jtv> wallyworld: approved with notes
[08:14] <lifeless> jtv: could you comment on and triage bug 729520 - i'm out of my knowledge zone
[08:15] <_mup_> Bug #729520: Translations changed in Ubuntu should be marked as such, not as current <Launchpad itself:New> < https://launchpad.net/bugs/729520 >
[08:20] <lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/690356
[08:20] <_mup_> Bug #690356: The "xubuntu" packageset is missing in the lp.packagesets collection (LP API) <bugjam2010> <lp-soyuz> <packagesets> <qa-needstesting> <Launchpad itself:Fix Committed by wgrant> < https://launchpad.net/bugs/690356 >
[08:35] <wgrant> lifeless: Sorry, was at dinner, qa'd it, noticed you'd already tagged it, then saw your ping.
[08:35] <wgrant> Sort of the wrong order.
[08:35] <lifeless> ;)
[08:35] <stub> lifeless: back
[08:36] <lifeless> stub: hey, cool
[08:36] <lifeless> stub: now good ?
[08:36] <stub> lifeless: irc or skype if you want to put up with the power tool noise
[08:37] <lifeless> I heard you say hello
[08:37] <stub> lifeless: no sound from you - checking audio here
[08:37] <lifeless> kk
[08:46] <wallyworld> jtv: thanks for the review and tip about assertContentEqual (which i didn't know about). i'll make the necessary tweaks
[08:51] <jam> hi wgrant
[08:52] <jam> and lifeless
[08:53] <jtv> lifeless: henninge needs to know about that bug.
[08:54] <wgrant> Morning jam.
[08:54] <wgrant> jam: We may be LOSAless after the deployment :/
[08:59]  * henninge pumps up the volume to hear pings
[09:00] <henninge> jtv: Hi! which bug?
[09:00] <adeuring> good morning
[09:00] <jtv> henninge: hi!  bug 729520
[09:00] <_mup_> Bug #729520: Translations changed in Ubuntu should be marked as such, not as current <Launchpad itself:New> < https://launchpad.net/bugs/729520 >
[09:00] <henninge> Moin abel
[09:00] <jtv> and good morning adeuring :)
[09:01] <jam> wgrant: well that sucks. I was hoping to hear better news. I'm a bit surprised that they trust general rollouts without a LOSA available.
[09:01] <jam> Is it just that they make sure to have one *during* the deploy? And assume they will catch things then?
[09:01] <jml> lifeless: hello
[09:01] <wgrant> jam: There will be one during the deploy.
[09:02] <wgrant> jam: And hopefully spm will be well again by then :)
[09:02] <jam> wgrant: I know you have to have one *during*. My point is why there isn't one *right after*, in case things go wrong. They just trust that the only problems will be the deployment itself?
[09:02] <lifeless> jml: hello
[09:02] <lifeless> jml: skype?
[09:03] <jml> lifeless: sure. just call me.
[09:03] <lifeless> jml: says you re offline
[09:04] <jml> lifeless: lies
[09:04] <jml> actually
[09:04] <jml> no
[09:04] <jml> lifeless: it's having trouble signing in
[09:05] <jtv> henninge: read the bug?
[09:05] <henninge> jtv: yes, the upstream project has a template ...
[09:05] <henninge> so that behavior sound correct
[09:05] <lifeless> jml: should I call your phone or mobile then?
[09:05] <jtv> henninge: Ahhh… I hadn't realized there was upstream translation!
[09:05] <jml> lifeless: ok.
[09:06] <henninge> jtv: but it's a testcase I created two years ago. Disabled it now.
[09:06] <jtv> Oh!
[09:08] <henninge> jtv: and I just realized I that that won't help because the code does not check if a template is deactivated ...
[09:08] <jtv> Oops.
[09:08] <henninge> jtv: you actually pointed me to that but I forgot to implement it.
[09:08]  * henninge goes to move template out of the way completely.
[09:08] <jtv> henninge: are you sure though?  Maybe you used one of those methods that ignore deactivated templates.
[09:09] <henninge> jtv: no, it's a storm query. pretty obvious
[09:09] <jtv> oic
[09:09] <henninge> translations/utilities/translationsharinginfo.py
[09:09] <jtv> henninge: I'm on a feature rotation now, but robert needed someone to comment on & triage that bug.
[09:10] <henninge> jtv: I will leave a comment.
[09:10] <jtv> Great, thanks!  I'm very relieved that this is not a big design problem.
[09:11] <henninge> jtv: what was that obsolete series called again, that we moved template like that to?
[09:11] <jtv> obsolete-junk?
[09:12] <henninge> "too many matches" ...
[09:12] <henninge> jtv: but that's it
[09:13] <jtv> The pyromaniac in me whispers "there's no such thing as too many matches when it comes to obsolete junk"…
[09:13] <henninge> ;)
[09:18] <lifeless> jml: https://launchpad.net/ubuntu/+source/apport
[09:42] <jml> lifeless: lp:principia
[09:45] <jml> lifeless: sound debugging
[09:45] <lifeless> can you hear me?
[09:45] <lifeless> if so, kick twice
[09:49] <jam>  /kick /kick
[10:54] <wgrant> jam: Will you be able to look at bug #726985 soonish, or should someone else? It'll be in the top 10 exceptions on Wed.
[10:54] <_mup_> Bug #726985: codebrowse OOPSes with GeneratorExit when connection closed early <Launchpad itself:Triaged> < https://launchpad.net/bugs/726985 >
[10:56] <jam> wgrant: if that's a request, sure
[10:57] <jam> wgrant: you create it by doing a HEAD on a 404 page, right?
[10:57] <jam> do we know how they are getting OOPS in production?
[10:57] <wgrant> jam: It seems to always be large files in production.
[10:57] <wgrant> Hmm. Always binary URLs, actually.
[10:57] <wgrant> Which shouldn't be rendering...
[10:58] <wgrant> Oh, it's /download, so that's OK.
[10:59] <jam> wgrant: so basically, people who are ^Cing binary downloads in the middle of streaming are causing this OOPS
[10:59] <wgrant> jam: I suspect so.
[10:59] <jam> no big deal, just trying to understand
[10:59] <jam> easy to reproduce a couple of ways, was hoping for something concrete
[10:59] <jam> 90% of loggerhead doesn't stream
[10:59] <jam> it directly calls start_response().write()
[11:00] <jam> rather than 'yield' the content
[11:00] <wgrant> Ah, that makes more sense now.
[11:50] <dpm> hi henninge_, the unity-2d guys are implementing i18n and using LP for translations. Unity 2D is written in Qt but using a conversion of the Qt format to gettext for translations, so they can be done in LP. A question has come up though, and it is whether qt-format-plurals are supported in LP. I think they should be, as they are supported in gettext itself, but I just want to confirm. They look like this:
[11:50] <dpm> http://paste.ubuntu.com/576943/
[11:58] <LPCIBot> Project db-devel build #424: FAILURE in 5 hr 16 min: https://hudson.wedontsleep.org/job/db-devel/424/
[11:59] <henninge> dpm: I take it that that is how it is exported to gettext.
[11:59] <dpm> henninge, yeah
[11:59] <henninge> dpm: because it looks just like gettext
[12:00] <dpm> exactly, the only "difference" is that normally you don't get the %n in translations coming from most gettext packages
[12:01] <dpm> I know we support this in #kde-format IIRC, but I want to double check we support it for qt-format-plural as well, so that I can give the Unity guys a proper answer
[12:01] <henninge> dpm: the %n has nothing to do with plurals.
[12:02] <henninge> that's c-format
[12:03] <henninge> dpm: so to be correct gettext, the flags schould contain the "c-format" flag.
[12:04] <deryck> Morning, all.
[12:04] <dpm> henninge, hm, no, I don't think that's c-format, hence the question. c-format strings contain specifiers like %d or %s, but not %n, I think
[12:04] <dpm> that's why they use a separate format specifier in gettext
[12:04] <henninge> dpm: argh, got me!
[12:04] <dpm> :-)
[12:04] <henninge> yes, you are right
[12:05] <henninge> dpm: But that does not matter very much for the plurals display in launchpad.
[12:05] <henninge> s/very much/at all/
[12:06] <dpm> henninge, yes, that's right, so I guess I should rephrase the basic question as whether qt-format is supported (nevermind about qt-plural-format)
[12:07] <henninge> dpm: we use the standard gettext library to check the format of such strings. I think the only problem could be that bad translation strings would not be detected.
[12:08] <henninge> "bad" meaning a mismatch in the identifiers.
[12:08] <dpm> henninge, ok, gotcha, thanks
[12:08] <henninge> .. if the gettext library does not know qt-format.
[12:09] <dpm> it does, it does, it does: http://www.gnu.org/software/gettext/manual/gettext.html#qt_002dplural_002dformat
[12:09] <dpm> err, copy paste fail, too many "it does" there
[13:04]  * jml -> lunch
[13:12] <gmb> Hi henninge, benji: Can I get a review for https://code.launchpad.net/~gmb/launchpad/make-sub-link-nothing-aware-bug-721410/+merge/52409 please?
[13:13] <henninge> gmb: otp atm, lunch after that, stand-up after that. I can look at in 90 min.
[13:13] <benji> gmb: I'll take it unless henninge is overcome with the need to do so.
[13:13] <benji> looks like I'm the lucky winner
[13:15] <gmb> benji: Cool, thanks.
[13:40] <benji> gmb: review done
[13:41] <gmb> benji: Thanks
[13:41] <benji> np
[13:41] <gmb> benji: Aha, clever irony in review comments; I like it.
[13:42] <benji> :)
[14:29] <sinzui> jcsackett: ping
[14:29] <jcsackett> hello sinzui.
[14:30] <sinzui> Will you be watching #launchpad this morning?
[14:35] <jcsackett> sinzui: i will; did my setting of channel info not take?
[14:35] <jcsackett> i see that it did not.
[14:35] <sinzui> jcsackett: maybe topics are knackered for me again
[14:36] <jcsackett> sinzui: no, it looks like i didn't properly save the change to info.
[14:44] <deryck> henninge-lunch, wanna chat for a couple minutes now?
[14:45] <henninge> deryck: sure
[14:55] <henninge> deryck: https://launchpad.dev/netapplet/+configure-translations
[14:57] <sinzui> leonardr: do you have time to review https://code.launchpad.net/~sinzui/lazr.restful/json-not-xhtml-0/+merge/52141
[14:57] <leonardr> sinzui, sure
[14:59] <bigjools> heh, thanks benji :)
[15:00] <bigjools> didn't get around to asking for the review, but you got to it anyway, fantastic
[15:01] <leonardr> sinzui: and maybe you'll have some insights into https://code.launchpad.net/~leonardr/launchpad/bug-271029/+merge/52423? otherwise i'll give it to ocr
[15:02] <benji> I'm like the genetically engineered kid's immune system in that episode of Star Trek: TNG, I don't wait for the microbes to come to me, I go out and get them. ;)
[15:02] <bigjools> heh
[15:06] <leonardr> sinzui, i don't understand the change you made to test_webservice.py. did you just disable the test?
[15:06] <leonardr> i think you should change it so it tests the actual behavior
[15:06] <leonardr> unless this is something we plan to bring back soon
[15:11] <leonardr> apart from that, it looks right
[15:11] <sinzui> leonardr: I did disable it. if you recall I suggested to change the name to start with disabled instead of test. I think I mis understood you I thought you said to remove test_ only
[15:12] <leonardr> sinzui: no, i meant to change the behavior of the test. unless we're going to go back to the old behavior someday, the test should be changed or removed
[15:12] <sinzui> I will change the behaviour
[15:14] <jml> sinzui: did you want to have a call today?
[15:14] <sinzui> jml: I have nothing to talk about today
[15:14] <jml> sinzui: good good :)
[15:14] <sinzui> :)
[15:17] <leonardr> sinzui, do you want to take a look at my branch or shall i give it to henninge/benji?
[15:17] <sinzui> leonardr: I can look at your branch now
[15:18] <leonardr> great, i think you will have good opinions on the topic
[15:18] <leonardr> argh, my push is still happening? what's the deal?
[15:18] <leonardr> well, i'll tell you when it finishes. it seems almost done
[15:22] <sinzui> leonardr: I updated the test based on the subsequent test in the module: https://code.launchpad.net/~sinzui/lazr.restful/json-not-xhtml-0/+merge/52141
[15:22] <leonardr> ok
[15:22] <leonardr> i seem to be pushing the entire launchpad tree for some reason
[15:23] <leonardr> sinzui: r=me
[15:23] <sinzui> thank you
[15:31] <dobey> leonardr: ping
[15:31] <leonardr> dobey, hi
[15:32] <dobey> leonardr: hey, am having some trouble with launchpadlib 1.9.7
[15:32] <dobey> 2011-03-07 10:30:07 ERROR    An error occurred trying to merge lp:ubuntuone-storage-protocol: 'NoneType' object has no attribute 'landing_candidates'
[15:32] <dobey> leonardr: ^^ tarmac keeps failing with that now. but it works fine with 1.6.2
[15:33] <leonardr> dobey: can you find the launchpadlib code in tarmac and paste it to me?
[15:34] <dobey> so the aprt that's failing is doing "for entry in lp_branch.landing_candidates"
[15:37] <leonardr> dobey: how is lp_branch set?
[15:37] <dobey> leonardr: lp_branch = self.launchpad.branches.getByUrl(url=branch_url)
[15:37] <leonardr> also, my generic advice: run 'import httplib2; httplib2.debuglevel=1' at the beginning of the code and paste me the http requests made
[15:37] <leonardr> and where does branch_url come from?
[15:38] <leonardr> if the versions are mismatched, that'll probably fail
[15:38] <dobey> the config. is "lp:foo"
[15:38] <leonardr> ok, no version there
[15:39] <leonardr> show me the http requests being made. then add a "version='devel'" argument to the launchpad constructor and see if it works. then try the same with version="beta"
[15:39] <leonardr> sinzui: https://code.launchpad.net/~leonardr/launchpad/bug-271029/+merge/52423 is ready to review
[15:45] <leonardr> sinzui: i took out the part i really wanted your opinion on, so i think i can give it to henninge or benji
[15:45] <leonardr> let me know if you'd like that
[15:46] <dobey> leonardr: oh hrmm
[15:46] <dobey> leonardr: it seems that launchpadlib 1.9.7 is trying to connect to staging by default
[15:46] <dobey> Host: api.staging.launchpad.net
[15:46] <leonardr> dobey: i believe that's true of older versions as well
[15:47] <leonardr> but, one thing i mentioned in the upgrade email is that all clients need to start explicitly specifying their host
[15:47] <leonardr> to avoid exactly this problem
[15:47] <dobey> we WERE doing that already
[15:48] <dobey> leonardr: we're using LPNET_SERVICE_ROOT which is https://api.launchpad.net/
[15:49] <leonardr> hm, so you are passing in production and it's being ignored
[15:49] <gary_poster> henninge, benji: I'd approaciate a review of https://code.launchpad.net/~gary/launchpad/bug723999-2a/+merge/52427 when one of you has some time available.  Fair warning: I'm afraid I may be prolific in my review requests today.  I certainly intend to be, at least.
[15:50] <leonardr> how is the Launchpad object being created?
[15:50] <leonardr> dobey -^
[15:50] <gary_poster> oh, well, nm on henninge :-P :-)
[15:50] <leonardr> is it using login_with? get_token_and_login?
[15:50] <dobey> Launchpad(credentials, SERVICE_ROOT, self.config.CACHE_HOME)
[15:50] <benji> gary_poster: I'll take it.
[15:50]  * henninge looks up prolific
[15:50] <henninge> gary_poster: sorry
[15:51] <gary_poster> np henninge! :-)
[15:51] <dobey> leonardr: it uses get_token_and_login() only if there are are no existing credentials
[15:51] <dobey> leonardr: looks like perhaps the API was broken for Launchpad() instantiation
[15:51] <gary_poster> thank you benji.  to state the obvious, I know you are busy, so if you don't get to all the branches I throw your way today, of course I will understand.
[15:51] <benji> sure
[15:52] <dobey> ugh
[15:56] <sinzui> leonardr: lp thinks that you more than just removed the all the changes
[15:56] <leonardr> dammit!
[15:58] <dobey> leonardr: so it's because new non-kwargs were added to the init, and tarmac was relying on position for 2 things that were kwargs. :-/
[15:58] <abentley> leonardr: lifeless has asked me to ensure that the stuff I export is only available on the devel version.  Is there a way to prevent webservice_entries from being exported in older versions?
[15:58] <abentley> leonardr: And for exported attributes, does this mean I need to declare that they are not exported in every other version?
[16:18] <gary_poster> benji, a large but hopefully easy branch (it just moves tests from one file to another): https://code.launchpad.net/~gary/launchpad/bug723999-2b/+merge/52429
[16:18] <benji> gary_poster: k
[16:22] <leonardr> dobey: yes, i'm pretty sure i talked to the tarmac developer about this
[16:23] <leonardr> tarmac should not be storing its own credentials anymore
[16:23] <dobey> rockstar: ^^ ?
[16:24] <dobey> leonardr: the credentials storing isn't the problem afaict. it's the API breakage. :-/
[16:24] <leonardr> dobey: the api breakage happens to apps that try to store their own credentials and instantiate Launchpad objects from them
[16:25] <leonardr> if i had known about this, i wouldn't have changed the constructor arguments, but now that it's broken i'd rather fix the apps, since they need to change anyway
[16:25] <rockstar> leonardr, dobey and I are "the tarmac developer."  :)
[16:26] <rockstar> leonardr, we stored our own credentials because launchpadlib was storing them in a place that kees said was a bad place...
[16:26] <leonardr> rockstar: ah, tarmac isn't in ubuntu. that's why i didn't contact you
[16:26] <rockstar> leonardr, yeah, I missed the boat in Natty.
[16:27] <leonardr> rockstar: ok, the new system is kees-approved, so you can use it without worry
[16:27] <rockstar> leonardr, ack.
[16:27] <rockstar> dobey, I still have leonardr's email with the changes we need to make.
[16:27] <dobey> what does the new system do?
[16:27] <leonardr> abentley: no, there's no way to prevent an entry from being exported. i need to add that
[16:28] <abentley> leonardr: I see.
[16:28] <leonardr> for exported attributes, you need to set exported=False by default, and pass in a version setting, something like [('devel', dict(exported=True)]
[16:29] <leonardr> i also need to add syntactic sugar to make that easier
[16:29] <leonardr> basically, the original design was that almost everything would be available in earlier versions unless it caused a huge problem
[16:29] <leonardr> and the new design is that nothing is available in earlier versions unless it was originally present there
[16:29] <abentley> leonardr: Hmm.  From the docs, I thought that exported=False set the first version, not a default.
[16:29] <leonardr> abentley: it's also a default, because later versions inherit from the first version
[16:30] <leonardr> so exported=False in beta means exported=False in 1.0, and it would mean exported=False in devel except for the override
[16:30] <abentley> leonardr, aha!
[16:30] <leonardr> dobey: the new system stores credentials in the gnome keyring/kde wallet
[16:31] <dobey> leonardr: what if there is no keyring?
[16:31] <leonardr> dobey: then it's stored encrypted on disk
[16:31] <dobey> ok
[16:32] <leonardr> dobey: there is something of an unchallenged assumption that IF you are in a gui environment you have some kind of keyring available
[16:32] <leonardr> since decrypting the encrypted file requires that you type in a passphrase into the current terminal
[16:32] <dobey> leonardr: tarmac is not meant to be run generally in a gui environment. it is a service run by cron.
[16:33] <leonardr> dobey: in that case, you want to use the 'service run by cron' mechanism
[16:33] <leonardr> that's in the email as well
[16:33] <dobey> leonardr: so any user interaction is not acceptable for it
[16:33] <leonardr> there will be none
[16:33] <dobey> ok, i don't have the email
[16:33] <leonardr> you will pass in the location of an unencrypted credentials file, and launchpadlib will read/write it
[16:33] <leonardr> so you can probably use the same file you use today
[16:34] <leonardr> i'll forward oyu the email
[16:34] <dobey> and we will still need to support both methods in tarmac
[16:34] <dobey> i think
[16:35] <leonardr> dobey: what do you mean by both methods? when there are existing credentials and when there aren't?
[16:35] <dobey> leonardr: i mean "the current way tarmac works"
[16:35] <dobey> leonardr: as we often run it on lucid also
[16:36] <leonardr> dobey: if you switch to passing in the unencrypted credentials file it should work on both lucid and natty
[16:44] <leonardr> sinzui: diff is pasted to https://code.launchpad.net/~leonardr/launchpad/bug-271029/+merge/52423
[16:45] <leonardr> dobey: if credentials_file doesn't work for you, let me know
[16:45] <benji> gary_poster: both of your branches are reviewed
[16:46] <gary_poster> awesome, thanks benji.
[16:46] <gary_poster> more coming ;-)
[16:46] <dobey> leonardr: ok
[16:48] <LPCIBot> Yippie, build fixed!
[16:48] <LPCIBot> Project db-devel build #425: FIXED in 4 hr 50 min: https://hudson.wedontsleep.org/job/db-devel/425/
[16:48] <sinzui> thanks leonardr
[16:58] <sinzui> leonardr: r=me, take a look at bug 579602 before you land. I think you fixed it
[16:58] <_mup_> Bug #579602: Trying to getBranches of a source package using the API oopses with ForbiddenAttribute <api> <lp-code> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/579602 >
[17:11] <bigjools> sinzui: we're still doing ui reviews, right?
[17:11] <sinzui> bigjools: voluntarily.
[17:11] <bigjools> ok, thanks.
[17:12] <bigjools> who's a valid reviewer for that nowadays? :)
[17:17] <sinzui> bigjools: um, I think it is still me and henninge. I have been passing UI reviews to huwshimi
[17:17] <sinzui> bigjools: you can always get two engineers to agree that the UI is good
[17:18] <bigjools> ah I will get rvba to push his review to huwshimi
[17:24] <leonardr> sinzui: i think i stopped the oops, but the bug isn't that there's an oops--i think that code should work
[17:26] <sinzui> leonardr: okay
[17:31] <LPCIBot> Project devel build #512: FAILURE in 5 hr 4 min: https://hudson.wedontsleep.org/job/devel/512/
[17:31] <LPCIBot> Project windmill build #21: FAILURE in 1 hr 4 min: https://hudson.wedontsleep.org/job/windmill/21/
[17:31] <LPCIBot> Project db-devel build #426: FAILURE in 43 min: https://hudson.wedontsleep.org/job/db-devel/426/
[18:01] <gary_poster> benji, https://code.launchpad.net/~gary/launchpad/bug723999-2c/+merge/52448 is ready for you (except for the diff, which will presumably arrive anon), but I'm going to go get some lunch.
[18:01] <benji> k
[18:05] <jcsackett> bigjools: are you (or can you point me to someone) to talk to in regards to https://answers.launchpad.net/launchpad/+question/148005 ?
[18:09] <dobey> rockstar: https://code.launchpad.net/~dobey/tarmac/login-with/+merge/52451
[18:28] <leonardr> i need some help from someone who knows launchpad bugs. preparing a pastebin...
[18:29] <leonardr> http://pastebin.ubuntu.com/577101/
[18:29] <leonardr> i changed createTask so that it calls valid_upstreamtask to validate, and that's breaking some test setup
[18:29] <leonardr> is the test setup technically invalid, or did i make a bad change?
[18:29] <leonardr> sinzui, maybe you know, i think you helped me with this in the first place
[18:36]  * sinzui looks
[18:40] <leonardr> sinzui: i changed the code so that the invalid bugtask was created first, and got the same error
[18:40] <leonardr> it seems okay to create another bugtask for a product if the earlier one was marked invalid?
[18:41] <sinzui> leonardr: I suspect you added a validate method, but our ORM needs an exception to set things up. So we need to exit a method early or return True for validate
[18:41] <sinzui> This happened to me recently. What field was I changing?
[18:42] <sinzui> ah description fields cannot contain only white-space
[18:42] <leonardr> sinzui: i added a _call_ to a validate method within createTask
[18:42] <leonardr> i'm not sure what you mean by "exit a method early..."
[18:47] <sinzui> leonardr: when working on the StrippableText field that lots of things descend from, I broken the initialisation of a lot of objects because the ORM create empty objects, then sets the data. The field should only validate or normalise data when when values are set (not inited). I used a check for current state and the new value to be certain I should proceed. I know there are fields that set a marker in init to show nothi
[18:47] <sinzui> ng is set.
[18:48] <leonardr> sinzui: i'm way out of my depth here, but it seems that this validation _should_ be run upon creation
[18:48] <leonardr> it seems like the test is doing something by manipulating the orm that should not be possible
[18:48] <leonardr> if the rules are to be applied consistently
[18:48] <sinzui> leonardr: if you mean creation means all the data is loaded, then yes
[18:49] <sinzui> storm/sqlobject creates an empty object, then puts data in the fields, and they fire in sequence...before all the data is there to validate an invariant state
[18:50] <sinzui> leonardr: can you pastbin your change?
[18:50] <sinzui> pastebun
[18:50] <sinzui> I give up
[18:50] <leonardr> sinzui: sure
[18:51] <abentley> sinzui: Pastbin was a rejected name for The History Channel :-)
[18:51] <leonardr> here's te whole thing: http://pastebin.ubuntu.com/577120/
[18:51] <leonardr> the relevant code is on line 166
[18:51] <leonardr> i think the problem is simpler than you're saying
[18:52] <leonardr> i started calling a certain method from the create method
[18:52] <sinzui> They rejects The Nazi Channel, but decided to run Nazi only programming anyway
[18:52] <leonardr> this method says "you can't have two bugtasks for the same bug for the same product, no exceptions"
[18:52] <leonardr> so the test that creates two bugtasks for the same bug for the same product fails, even though one of the bugtasks is invalid
[18:52] <sinzui> The International History channel's real name is Alien Encounters Channel
[18:54] <sinzui> leonardr: well, I would expect a failure in that case. The DB constraint does not look at status, it only looks at bug, distro, dseries, sourcepackagename, product, and pseries. Maybe that test is totally bugus
[18:54] <sinzui> excellent, I worked totally bogus into a sentence.
[18:55] <leonardr> sinzui: here's just the test that fails: http://pastebin.ubuntu.com/577123/
[18:55] <sinzui> The example i see is distro and then a distro + source package
[18:55] <leonardr> you think that deserves to fail?
[18:56] <leonardr> here's the original, not modified by me:
[18:56] <leonardr> http://pastebin.ubuntu.com/577124/
[18:57] <sinzui> leonardr: these bug tasks are different
[18:58] <leonardr> sinzui: am i calling valid_upstreamtask with the wrong argument?
[18:58] <leonardr>  target = product or productseries or distribution or distroseries
[18:58] <leonardr> should i be going in the other direction?
[18:58] <leonardr> target = distroseries or distribution or productseries or product
[18:58] <sinzui> leonardr: but I can imagine that if validate is called after the distro is set, but before the spn is set, you will get the error
[18:59] <leonardr> sinzui: i don't see what this has to do with validate at all
[18:59] <leonardr> the exception is raised by a method that i call explicitly
[18:59] <leonardr> i think we are talking past each other
[18:59] <sinzui> I have not seen the lines where you changed to cause the error yet
[19:00] <sinzui> where do you call valid_upstreamtask, and what is the context lines before and after it
[19:00] <leonardr>         if not milestone:
[19:00] <leonardr>             milestone = None
[19:00] <leonardr>         # Raise a WidgetError if this product bugtask already exists.
[19:00] <leonardr>         target = distroseries or distribution or productseries or product
[19:00] <leonardr>         valid_upstreamtask(bug, target)
[19:00] <leonardr>         if not bug.private and bug.security_related:
[19:00] <leonardr> the BugTask object is not created until much further down in the method
[19:01] <leonardr> here's the entire method with plus signs by the code i added
[19:01] <leonardr> http://pastebin.ubuntu.com/577131/
[19:01] <lifeless> jml: I forgot two things; paralel test lep & high frequency db deploys that I wanted to speak with you about
[19:02] <sinzui> what no, spn? that definitely will cause this error. The second example in the test is not getting the sourcepackagename=ubuntu_evolution.sourcepackagename that was passed
[19:03] <leonardr> what does spn stand for?
[19:03] <sinzui> source package name
[19:04] <sinzui> A bug in a distro can affect dozens of packages, so will be represented by dozens of bug tasks
[19:04] <sinzui> all ubuntu, may be all the same distro series, but each has a unique source package name
[19:04] <leonardr> sinzui: the test has changed back and forth quite a bit since i started talking to you. let me paste the original version and ask you to walk me through it
[19:05] <leonardr> http://pastebin.ubuntu.com/577132/
[19:05] <leonardr> so, you are saying that the call to createTask(sourcepackagename=ubuntu_evolution.sourcepackagename)
[19:05] <leonardr> at some point the source package name is lost
[19:06] <sinzui> leonardr: I still see sourcepackagename=ubuntu_evolution.sourcepackagename in all examples of the test, but the fragment of code does
[19:06] <sinzui>     target = distroseries or distribution or productseries or product
[19:06] <sinzui>     target = distroseries or distribution or productseries or product
[19:06] <leonardr> so you're saying that sourcepackagename is also an acceptable target
[19:07] <sinzui> clearly target is a composite maybe we need it to be tuples
[19:07] <leonardr> sinzui: it's not clear to me at all. are you saying that the 'target' might be some combination of, eg. sourcepackagename and distroseries?
[19:08] <sinzui> (ubuntu, ), (ubuntu, natty), (ubuntu, natty, evolution), (ubuntu, evolution) (evolution-project) are all valid combinations for bugtasks, but I see target is just one items by order of precedence
[19:09] <sinzui> ^ leonardr that is what the database sees for the unique constraint
[19:10] <leonardr> sinzui: valid_upstreamtask takes a 'product' and invokes searchTasks() on the product
[19:10] <leonardr> i'll paste the code
[19:11] <leonardr> http://pastebin.ubuntu.com/577133/
[19:11] <leonardr> it sounds like this code is too specific to be called from createTask
[19:11] <sinzui> I suspect I know what is a mis before even looking
[19:11]  * sinzui looks
[19:12] <sinzui> yes, we are checking one of many conditions
[19:13] <sinzui> leonardr: I think the issue many years ago was that the four types were assign targets to have enough information to know everything, but that is not true for an SPN. It is just a name, we do not know what distro it is, and we do not know if the distro publishes it
[19:14] <sinzui> leonardr: we can start making sens of this by fixing the signature: valid_upstreamtask(bug, product) => valid_upstreamtask(bug, bug_target)
[19:15] <leonardr> ok, i understood that
[19:16] <sinzui> yuck. we cannot search an SPN's tasks
[19:17] <leonardr> sinzui: do you think it would be possible now to use the website to create multiple bugtasks for a single spn?
[19:17] <leonardr> or would that finally be caught at the database level?
[19:17] <sinzui> leonardr: I think empty() means return early. If this is a distro or distro series, we need to see if any of the bug tasks have the same sourcepackagename as the spn passed
[19:19] <sinzui> leonardr: we will get a DB ConstraintViolationError after the call without a stack trace. We do need to verify the spn is unique
[19:19]  * sinzui looks for SPN bug target
[19:22] <sinzui> leonardr: the signature is a real issue. I *think* we can get past the issue by getting a DSP or SP for the D or DS respectively. Include that in the target = sp or distroseries or distro line
[19:24] <sinzui> So while I *do* think we need to fix the signature to say bug_target, the real fix is to get the correct target in the callsite
[19:24]  * sinzui hacks
[19:28] <sinzui> leonardr: I think this is what createBugTask needs to do: http://pastebin.ubuntu.com/577145/
[19:29] <leonardr> sinzui: ok, i think i have something similar, i'll check
[19:29] <sinzui> I wonder if we have an adapter
[19:31] <leonardr> yeah, i have something equivalent
[19:31] <leonardr> i'll paste what i have
[19:33] <leonardr> sinzui: http://pastebin.ubuntu.com/577149/
[19:35] <sinzui> leonardr: +1
[19:35] <leonardr> that makes _that_ test work, let me see about the others
[19:47] <leonardr> looking good...
[19:55] <leonardr> ok, now down to 2 test failures...
[20:01] <gary_poster> benji, getStructuralSubscriptionsForPerson was not suppsed to be in this branch.  Sorry for the confusion.  It goes in the next (and hopefully last) branch
[20:01] <lifeless> matsubara_: ping
[20:01] <gary_poster> I'm going to remove ans push
[20:02] <lifeless> Ursinha: ping (the Bug:xyz patch didn't seem to be live on qatagger last night)
[20:02] <benji> gary_poster: ok; just about done reviewing that branch
[20:02] <gary_poster> cool thanks benji
[20:08] <leonardr> sinzui, i wrote this code, but i don't remember what it means anymore. can you sanity check it for me?
[20:08] <leonardr> http://pastebin.ubuntu.com/577158/
[20:08] <benji> gary_poster: review done, including a solution to your lint mystery
[20:08] <leonardr> it's not raising an exception where it should
[20:08] <gary_poster> sweet, thanks benji!
[20:08] <leonardr> but i don't know if the 'conjoined_bugtask' actually is a conjoined bugtask, because i don't know what a conjoined bugtask is
[20:10] <sinzui> leonardr: conjoined_bugtask is a temporary state where a task's series (distro or product) happens to be the development series
[20:10] <lifeless> its a series specific task that also has a project scope task
[20:10] <lifeless> so there are two related tasks and its shown as one in the UI
[20:11] <lifeless> its a significant chunk of complexity :(
[20:11] <sinzui> leonardr: This is very esoteric knowledge. For ubuntu, a task is conjoined only for 6 months
[20:12] <sinzui> leonardr: since you use product.development_focus, I expect this test to work if there is also a bug for the product
[20:12] <leonardr> it doesn't looks like there is
[20:12] <leonardr> conjoined_master is None
[20:12] <deryck> abentley, thanks for sending that packaging permissions email.
[20:12] <sinzui> leonardr: I think this subtle change will work: target=bugproduct.development_focus
[20:13] <sinzui> target=bug.product.development_focus
[20:13] <abentley> deryck: np
[20:13] <sinzui> I think the factory created a product for the bug, we want to reuse it
[20:13] <leonardr> ahg
[20:16] <sinzui> leonardr: I think you can make the the reviewer of your branch. I think I can now skip the cover letter and just read the code
[20:18] <leonardr> ok, we just have some failures in patches-view.txt...
[20:21] <lifeless> deryck: had you seen http://www.pivotaltracker.com/features
[20:21] <deryck> lifeless, no, I haven't seen that.  looking now.
[20:29] <deryck> lifeless, ah, interesting.  it's not what I thought on first glance.  It's basically a scrum board, right?
[20:29] <lifeless> looks like
[20:29] <lifeless> bug cast as a bugtracker replacement (which they effectively are)
[20:30] <lifeless> I -love- the infinite resolution on ordering
[20:30]  * lifeless wants this for bug priority.
[20:30] <lifeless> drag n drop, done.
[20:30] <deryck> yeah, that's nice.
[20:30] <lifeless> abentley was suggesting something like this for lp way back
[20:30] <deryck> but their model is a set of cards.  not bugs.
[20:30] <lifeless> deryck: right, it is a different take on the whole problem
[20:31] <jcsackett> my last job briefly used pivotal. it was a nice tool.
[20:31] <deryck> lifeless, right.
[20:31] <deryck> yeah, it looks like a nice tool.
[20:31] <leonardr> sinzui: ok, help me out one more time
[20:31] <deryck> jcsackett, did you use it as a bug tracker?
[20:31] <leonardr> patches-view contains this code:
[20:31] <leonardr>     >>> make_bugtask(
[20:31] <leonardr>     ...     bug=bug_a, target='hoary', target_is_distroseries_name=True)
[20:31] <jcsackett> deryck: eh, sort of. more of an issue tracker, i suppose?
[20:31] <leonardr> that looks up the 'hoary' distro series and calls factory.makeBugTask
[20:32] <jcsackett> if you're doing things in the agile card mode, it's less about bugs and more about "user stories."
[20:32] <deryck> jcsackett, yeah, it looks like project tracking, more than bug tracking to me.
[20:32] <leonardr> but factory.makeBugTask doesn't like being passed a distro series
[20:32] <leonardr>         if IDistroSeries.providedBy(target):
[20:32] <leonardr>             # We can't have a series task without a distribution task.
[20:32] <leonardr>             self.makeBugTask(bug, target.distribution)
[20:32] <jcsackett> deryck: yes.
[20:32] <leonardr> so instead of creating a bugtask for 'hoary', we create one for 'ubuntu', and we trigger the validation error
[20:32] <deryck> like what we use Kanban for.
[20:32] <lifeless> thats the conjoined master bit
[20:32] <jcsackett> deryck: similarly; though we're sort of using bugs as project tracking as well.
[20:33] <deryck> jcsackett, oh we definitely are.
[20:33] <leonardr> lifeless: i think it's supposed to add the conjoined master and then call addTask *again* to add the slave
[20:33] <leonardr> the problem is that adding a conjoined master when there's already a bugtask for 'ubuntu' now raises an exception
[20:34] <lifeless> what did it do before (and what bug are you working on , to give me context)
[20:34] <leonardr> lifeless: the context is my fix to bug 106338: https://code.launchpad.net/~leonardr/launchpad/bug-106338
[20:34] <_mup_> Bug #106338: Editing a bug targeted to a release crashes if you directly edit the untargeted task  <api> <lp-bugs> <oops> <Launchpad itself:Triaged by leonardr> < https://launchpad.net/bugs/106338 >
[20:35] <leonardr> i added to createTask a call to valid_upstreamtask(), so that validation code previously called only in the view would be called everywhere
[20:35] <sinzui> leonardr: I think the issue here is the factory is wrong. It could be smart enough to do the right thing. It onyl has the features that engineers needs at the time
[20:35] <leonardr> sinzui: ok, that's what i was thinking too
[20:36] <leonardr> so we should see if the master already exists?
[20:36] <sinzui> Lots of factory methods were creating objects with impossible series. I fixed them
[20:36] <sinzui> leonardr: yes, the factory should check and make a decision.
[20:36] <lifeless> I concur
[20:37] <leonardr> ok, help me out. can i adapt the code from valid_upstreamtask to see if there's already a bug targeted to a given target, or is there a simpler way?
[20:39] <leonardr> i'll write some cheap code and you can tell me how to fix it
[20:40] <lifeless> leonardr: 'try: ... except:'
[20:41] <lifeless> (though you'll want an actual type in the except clause)
[20:42] <sinzui> I do not this the try except will work because the exception will be a DB IntegrityError contstaint violation. We need to look before we leap with the ORM
[20:42] <leonardr> my cheap code goes through the existing tasks
[20:43] <lifeless> sinzui: oh :)
[20:43] <lifeless> sinzui: ok
[20:44] <lifeless> leonardr: there is a getBugTask(target) method on bug.
[20:44] <leonardr> let's try it...
[20:45] <jcsackett> leonardr: can i bug you for a sec regarding 404s on lplib?
[20:45] <leonardr> jcsackett, sure
[20:47] <sinzui> benji: do you have time to review a small change to address a project review nuance: https://code.launchpad.net/~sinzui/launchpad/hide-license-info/+merge/52467
[20:47] <benji> sinzui: sure
[20:48] <jcsackett> leonardr: so i am looking at bug 701246
[20:48] <_mup_> Bug #701246: missing bug dereference generates OOPS <oops> <Launchpad itself:Triaged by jcsackett> < https://launchpad.net/bugs/701246 >
[20:48] <jcsackett> it looks as though the problem is just that a NotFound that gets raised in lp doesn't go back to lplib.
[20:49] <jcsackett> i have tried https://pastebin.canonical.com/44375/ as a fix.
[20:49] <jcsackett> while that works, leonardr, i'm unconvinced its the best (or correct) solution.
[20:50] <jcsackett> leonardr: that change is in _handle_next_obj, to save you a trip into the code tree.
[20:51] <leonardr> jcsackett: ok, give me am inute to finish what i'm doing
[20:51] <jcsackett> leonardr: sure.
[20:58] <lifeless> jcsackett: +1 AIUI
[20:59] <jcsackett> lifeless: cool. my major concern is i thought we should a) avoid expose() and b) i'm frightened of the publisher. :-P
[20:59] <leonardr> sinzui, lifeless: going to paste my code and look at jcsackett's code. for some reason, valid_upstreamtask is raising an exception for 'hoary' even though bug.getBugTask(hoary) is returning None
[20:59] <benji> sinzui: review done (https://code.launchpad.net/~sinzui/launchpad/hide-license-info/+merge/52467)
[21:01] <sinzui> thanks benji
[21:01] <benji> np
[21:02] <leonardr> lifeless, sinzui: http://pastebin.ubuntu.com/577173/ looks good to me, don't know why it sometimes fails as i mentioned above
[21:03] <leonardr> jcsackett: at the very least, you need to do expose(NotFound(self.context, name), 404)
[21:03] <leonardr> otherwise you will start giving out 400 errors
[21:04] <jcsackett> leonardr: ok.
[21:04] <leonardr> i think it would make more sense to publish NotFound as 404 _in general_
[21:04] <jcsackett> i sort of assumed it was, to be honest. :-P
[21:04] <sinzui> leonardr: I think the code is creating confusion between master and slave. the distro is the slave, the series is the master.
[21:05] <leonardr> jcsackett: maybe it's the fact that the exception is raised during traversal
[21:05] <leonardr> you say that the code you have works? what behavior do you see?
[21:06] <sinzui> We really need to rethink the value for conjoined relationships. They are ephemeral, limited to projects that develop with leading series, and the code is very expensive to work with
[21:06] <jcsackett> leonardr: with make run_all going, if i connect to my dev instance via lplib, and run through the code shown in the bug, i get a 404 http error, and no oops is generated.
[21:06] <leonardr> jcsackett: we do have special views registered for NotFound that should cause them to yield 404 errors, but i bet that code doesn't apply to exceptions raised during traversal
[21:06] <leonardr> that's weird
[21:07] <leonardr> if you put a weird error code like expose(..., 409), does that get passed through?
[21:07] <sinzui> leonardr: the if ISourcePackage.providedBy block talks about series but we can have bugs in SPs without a series
[21:08] <sinzui> maybe the whole paste represents an attempt to solve the case where the target implements ISeries
[21:09] <jcsackett> leonardr: let me check.
[21:09] <leonardr> sinzui: i was trying to solve the case where it tries to create a prerequisite target *of some kind*
[21:09] <leonardr> there were three such cases
[21:10] <leonardr> are you saying these cases are different, that in some cases the target is the master and in some cases the target is the slave?
[21:10] <leonardr> i can change the variable names so i'm not talking about 'master_target'. i don't think i changed the actual meaning of the code
[21:12] <leonardr> sinzui: see http://pastebin.ubuntu.com/577178/. the code is the same, except i check for a bug before creating one
[21:18] <sinzui> leonardr: okay. I agree with what you are doing. Lp does require a PrimaryContext level bug before creating a series bug.
[21:19] <leonardr> sinzui: i am in a situation where bug.getBugTask(bug_target) returns None, but valid_upstreamtask raises an exception
[21:19] <leonardr> because bug_target.searchTasks(BugTaskSearchParams(user, bug=bug)) finds something
[21:20] <leonardr> so i think bug.getBugTask(bug_target) does not tell the whole story
[21:20] <sinzui> leonardr: by "sometimes fails" do you mean not reliably repeatable? maybe we need something like IStore(prerequisite).flush()
[21:21] <lifeless> leonardr: getBugTask is trivial
[21:21] <lifeless> leonardr: but note that it uses cached data.
[21:22] <lifeless> leonardr: you may need to try: del get_propery_cache(bug).bugtasks; except AttributeError: pass
[21:23] <leonardr> sinzui: i mean it fails when i don't think it should
[21:24] <LPCIBot> Yippie, build fixed!
[21:24] <LPCIBot> Project windmill build #22: FIXED in 1 hr 12 min: https://hudson.wedontsleep.org/job/windmill/22/
[21:24] <lifeless> leonardr: at the point where bug tasks are added
[21:25] <leonardr> lifeless:trying that now
[21:25] <lifeless> leonardr: bottom of Bug.addTask would be the place to add it
[21:26] <lifeless> leonardr: a more complex version would be to add the new one to the tasks IFF the tasks attribute is already populated
[21:27] <jcsackett> leonardr: actually, expose is telling me it only accepts on argument, which prevents doing expose(...,409). something trivial i've missed?
[21:27] <jcsackett> s/on argument/one argument/
[21:27] <leonardr> jcsackett: ah, it's because i haven't landed my branch that fixes this, because of these test failures
[21:27] <leonardr> i got ahead of myself
[21:28] <leonardr> but, if expose() is working at all, you should be getting 400 errors, not 404
[21:28] <lifeless> leonardr: normal browsers will still see a 404, right ?
[21:29] <leonardr> lifeless: browsers on the website will see a 404. if you hit the web service with a browser it will go through the lazr.restful publisher and you'll see what a non-browser client would see
[21:30] <lifeless> leonardr: ok, and we map 404 to 400 for some reason here?
[21:30] <leonardr> lifeless: no, i'm saying that jcsackett's change shouldn't be doing what he's seeing
[21:30] <lifeless> ah cool
[21:30] <leonardr> if expose() was working at all, it would be giving him a 400 error, not a 404
[21:31] <lifeless> leonardr: would it be reasonable for NotFound to map to 404 in the webservice by default?
[21:31] <leonardr> still getting the failures in patches-view. again, the problem is that attempting to create a bugtask for Hoary gives an error "there's already a bugtask for Ubuntu"
[21:31] <leonardr> lifeless: yes, and that is what happens normally. i think the problem we're seeing is that the exception is happening in traversal, and lazr.restful doesn't catch those
[21:32] <lifeless> ah
[21:32] <lifeless> yes, that makes sense
[21:34] <leonardr> lifeless: the end of createTask already contains 'del get_property_cache(bug).bugtasks'
[21:35] <lifeless> leonardr: ok, well thats not it then :)
[21:35] <jcsackett> leonardr: so, basically, what i'm seeing isn't possible?
[21:35] <thumper> leonardr: mumble?
[21:36] <leonardr> jcsackett: i don't understand how you're seeing it. i wouldn't say it's impossible, especialyl if you can turn it on and off easily
[21:37] <jcsackett> leonardr: so, either way a 404 is returned; it's just that with expose, no OOPS is created.
[21:38] <leonardr> jcsackett: oh, i see how that could happen
[21:40] <gary_poster> benji, one last one, then the marathon is done.  Maybe you can fit it in, maybe not. :-)  https://code.launchpad.net/~gary/launchpad/bug723999-2d/+merge/52478
[21:40] <benji> gary_poster: good timing, I was just about to resort to real work. ;p
[21:40] <gary_poster> :-)
[21:40] <jcsackett> leonardr: and basically killing the oops is the thing. given that context, this seem better?
[21:41] <leonardr> jcsackett: my gut feeling is that your bug will simply go away once my branch lands
[21:41] <jcsackett> leonardr: really?
[21:41] <leonardr> yeah
[21:41] <jcsackett> that will be twice that you make my work irrelevant, but i'm totally happy with that. :-)
[21:42] <jcsackett> i'm going to move on to something else and revisit this post your branch landing.
[21:42] <leonardr> ok, it'll probably be tomorrow
[21:42] <jcsackett> leonardr: cool. can you link me the branch (mostly for my curiosity)?
[21:43] <leonardr> jcsackett: https://code.launchpad.net/~leonardr/launchpad/bug-106338
[21:43] <jcsackett> thanks
[21:45] <kfogel> From sabdfl's latest blog post: "Nothing hugs quite like dholbach, though, and he's no hairy ape."  I'm sorry, but I just had to repeat that.
[21:45] <gary_poster> :-)
[21:46] <kfogel> We now return you to your regularly scheduled, uh, programming :-).
[21:46] <lifeless> kfogel: hey, how goes it
[21:46] <gary_poster> thanks for saying hi kfogel
[21:46] <kfogel> hey lifeless -- okay.  Busy, but good.  You?
[21:46] <kfogel> gary_poster: miss y'all!
[21:46] <gary_poster> you too :-)
[21:47] <lifeless> kfogel: insanely busy, very well - lp is going great guns, and lynne is 14 weeks now ;)
[21:48] <kfogel> lifeless: congrats!!!
[21:48] <kfogel> lifeless: and re that other thing, launchpad, I've been watching the blog and been seeing the progress with pleasure.
[21:49] <lifeless> kfogel: thanks
[21:52] <lifeless> wow
[21:52] <lifeless> I think we're spending 12 seconds /rendering/ 150 bugtask rows.
[21:52] <lifeless> https://bugs.qastaging.launchpad.net/++profile++show/ubuntu/+source/afflib/+bug/230350
[21:52] <_mup_> Bug #230350: Missing Debian Maintainer field <Ubuntu:Invalid> <afflib (Ubuntu):Fix Released by saivann> <alac-decoder (Ubuntu):Fix Released by warp10> <axiom (Ubuntu):Invalid> <beneath-a-steel-sky (Ubuntu):Fix Released> <bibletime-i18n (Ubuntu):Fix Released by txwikinger> <binkd (Ubuntu):Fix Released> <bzr-builddeb (Ubuntu):Won't Fix> <capisuite (Ubuntu):Invalid> <chinput (Ubuntu):Fix Released by warp10> <chmsee (Ubuntu):Fix Released> <ciso (U
[21:53] <leonardr> sinzui: ok, i'm in pdb right at the point where the error is about to happen
[21:53] <leonardr> i'm about to call self.makeBugTask(bug, a)
[21:53] <leonardr> a is the Distribution 'ubuntu', and the target from which i derived a is the DistroSeries 'hoary'
[21:53] <lifeless> gary_poster: chameleon templates; can they mix n patch with the tal engine ones?
[21:53] <leonardr> bug.getBugTask(a) is None
[21:54] <leonardr> but, if i call makeBugTask, i'll get an error that there's already a task for 'ubuntu'
[21:54] <gary_poster> lifeless, once we switch to chameleon, everything uses the chameleon engine, including current TAL stuff.
[21:54] <lifeless> gary_poster: sure; I'm looking at bug 724033
[21:54] <_mup_> Bug #724033: BugTask:+index timeouts - distributions and milestones being late evaluation loaded - repeatedly - on bug 230350 <qa-ok> <timeout> <Launchpad itself:Fix Committed by lifeless> < https://launchpad.net/bugs/724033 >
[21:54] <gary_poster> and it is supposed to help with rendering nasties like the one you are mentioning
[21:55] <lifeless> gary_poster: I've gotten the sql component down to 2 seconds
[21:55] <gary_poster> 12334: suck
[21:56] <lifeless> gary_poster: I'm wondering if I could migrate just the bug task row & edit forms - which are the bits repeated - to chameleon, assuming further investigation doesn't show it to be something simple like the view initialization being slow
[21:56] <gary_poster> OIC
[21:56] <mtaylor> lifeless: hey - whatever happened to those merge queue things rockstar was working on?
[21:56] <lifeless> mtaylor: deferred
[21:56] <mtaylor> lifeless: k. so it's not just that I missed them somewhere
[21:57] <lifeless> mtaylor: not at all; we had too much work in progress. May do them when we overhaul our landing process
[21:57] <gary_poster> lifeless, you could.  The only way I can think of doing it right now is having the view render the chamelon bits as a string, inserted into the TAL as structure
[21:57] <gary_poster> does that make sense?
[21:57] <lifeless> yes
[21:57] <gary_poster> cool
[21:57] <lifeless> thats how the rows are done already, more or less
[21:57] <gary_poster> oh, ok, cool
[21:58] <lifeless> <div tal:replace="structure context/bug/@@+bugtasks-and-nominations-table" />
[21:58] <gary_poster> that would be a great test to see of chameleon actually buys us what we expect then, particularly if it is apples to apples
[21:58] <leonardr> sinzui: i'm starting to think there's a problem with vaild_upstreamtask
[21:58] <gary_poster> yeah
[21:58] <leonardr> check this out
[21:58] <leonardr> (Pdb) [bug.title for bug in bug_target.searchTasks(params)]
[21:58] <leonardr> [u'Bug #16 in evolution (Ubuntu): "bug_a title"']
[21:58] <_mup_> Bug #16: "Swedish" and "Swedish (Sweden)" should be the same language <lp-translations> <Launchpad itself:Fix Released by daf> < https://launchpad.net/bugs/16 >
[21:58] <lifeless> and then inside it
[21:59] <lifeless> <tal:bugtask-or-nomination  repeat="task_or_nom_view view/getBugTaskAndNominationViews"><tal:block replace="structure task_or_nom_view" />
[21:59] <gary_poster> ew
[21:59] <gary_poster> that may be a problem inherently
[21:59] <lifeless> so it ends up calling __call__() on the view objects its manually created.
[21:59] <gary_poster> though chameleon may make that better too, dunno
[21:59] <lifeless> gary_poster: whats the problem you perceive there?
[22:00] <gary_poster> well, it's a hunch, but I'd be interested to see if doing structure calls and inserting the output of another view within a tight loop is just bad.
[22:00] <lifeless> your hunch is that inserting structure is slow ?
[22:00] <mtaylor> lifeless: well, fwiw, I just wrote a hacky launchpadlib script to make a page showing me which of our brances are in which of our fake-queue-branches ... hopefully it's not causing too much activity :)
[22:01] <gary_poster> I would expect this, from worst to best:
[22:01] <lifeless> mtaylor: heh
[22:01] <lifeless> mtaylor: patch LP!
[22:01] <gary_poster> insert structure of another view, which itself uses a template
[22:01] <gary_poster> use a macro
[22:01] <mtaylor> lifeless: I do not believe you would accept a patch that hardcodes lp:drizzle/build in to it
[22:01] <gary_poster> have code done locally
[22:01] <lifeless> mtaylor: seriously, 50% of our page hits are API now. (and not 'API to make a web page be nice'. Pure launchpadlib style API)
[22:02] <wallyworld> lifeless: code.launchpad.net/+daily-builds - 67 queries/external actions issued in 2.36 seconds \o/
[22:03] <gary_poster> so, lifeless, "structure" itself is fine.  My hunch o' nastiness is that the inner loop, calling a view for each loop, is bad.
[22:03] <lifeless> gary_poster: I'm going to generate some hundreds of rows locally and do some elimination tests to narrow this down. Thanks for the tips - helps me understand this area.
[22:03] <gary_poster> np
[22:03] <mtaylor> lifeless: oh, well in this case, I doubt the launchpadlib hits are gonna kill you (I grab a list of approved brancehs) ... but the "do a checkout of each branch and run run bzr missing a few times to see if it's contained within another branch" might be annoying
[22:03] <gary_poster> I'll look forward to reading the results lifeless!
[22:03] <leonardr> sinzui: we don't have a bugtask for ubuntu, so bug.getTask(ubuntu) returns nothing. but we do have a bugtask for evolution in ubuntu, so the searchTasks thing fails
[22:03] <lifeless> gary_poster: now I have to write em up :)
[22:03] <gary_poster> lifeless: lol :-P
[22:04] <lifeless> leonardr: what searchTasks thing are you referring to ?
[22:04] <leonardr> lifeless: the one run in valid_upstreamtask
[22:04] <leonardr> user = getUtility(ILaunchBag).user
[22:04] <leonardr>     params = BugTaskSearchParams(user, bug=bug)
[22:04] <leonardr>     if not bug_target.searchTasks(params).is_empty():
[22:05] <lifeless> thats buggy
[22:05] <sinzui> leonardr: Ubuntu and evolution in ubuntu are not the same, one being a distro and the other being a package. I do not think any query should confuse the two
[22:05] <lifeless> it will find tasks on related series.
[22:05] <leonardr> ok, now we're getting somewhere
[22:05] <lifeless> I think the validator is broken
[22:05] <lifeless> -or-
[22:05] <lifeless> its not the right thing to use to fix the oops
[22:06] <lifeless> if you have a product with a series A and a series B
[22:06] <lifeless> this validator will raise an error if the product, or series A, or series B have a bugtask.
[22:06] <lifeless> but you are working on distros
[22:07] <lifeless> and distros have another layer in there - the source package
[22:07] <lifeless> you can have a distro task or a distroseries task, or a distro sourcepackage task, or a distroseries sourcepackage task
[22:07] <lifeless> leonardr: what is calling valid_upstreamtask ?
[22:08] <leonardr> lifeless: my code causes it to be called from createTask()
[22:08] <leonardr> formerly it was only called from view code. looking up the view code now
[22:09] <lifeless> leonardr: you need to call validate_distrotask when dealing with distro contexts
[22:09] <lifeless> or validate_new_distrotask etc
[22:09] <lifeless> I don't know offhand which you'll need for createTask
[22:09] <sinzui> wtf: can we have one method that differs to the hidden methods?
[22:10] <lifeless> I need to pop out to the shops
[22:10] <sinzui> There must be duplicate code in three validation methods
[22:10] <lifeless> sinzui: tonnes from the look of it
[22:11] <lifeless> I'm sure you guys will get to the bottom of it soon. BBIAB
[22:11] <lifeless> wallyworld: nice
[22:11] <wgrant> Morning all.
[22:12] <mtaylor> am I dumb - there doesn't seem to be an api member on a merge_proposal to get the web url to see that prop
[22:12] <mtaylor> ?
[22:12] <jcsackett> morning, wgrant.
[22:13] <lifeless> oh grah
[22:13] <lifeless> IOError: [Errno 2] No such file or directory: '/home/pqm/pqm-workdir/home/trunk/launchpad/production-configs/lpnetSLAVE-lazr.conf'
[22:13] <leonardr> mtaylor: there should be one
[22:13] <lifeless> lp-production-config landings are bust
[22:13] <lifeless> testConfig (canonical.config.tests.test_config.../production-configs/lpnet-template/launchpad-lazr.conf)
[22:13]  * lifeless goes really
[22:13] <wallyworld> lifeless: the business queries - the ones to get the data - there's only 4 or so of those. there's an awful many of these: SELECT ValidPersonCache.id FROM ValidPersonCache WHERE ValidPersonCache.id = xxx
[22:13] <leonardr> mtaylor: it should be web_link. what version of the web service are you using?
[22:13] <wgrant> lifeless: How did it land in the first place? :/
[22:14] <mtaylor> leonardr: LPNET_SERVICE_ROOT?
[22:14] <mtaylor> leonardr: (not sure how to answer your question ...)
[22:14] <wgrant> mtaylor: Your cached WADL is probably out of date.
[22:14] <mtaylor> hrm. how do I update my locally cached WADL?
[22:14] <leonardr> wgrant: that's possible, but it should have been updated by now. it's only cached for a week
[22:14] <wgrant> I just blow away the whole launchpadlib cache, but there is probably a better way.
[22:15] <leonardr> mtaylor: show me how you call Launchpad.login_with()
[22:15] <wgrant> Hmm. We were seeing it happening three or four days ago.
[22:15] <mtaylor> launchpad = Launchpad.login_with('Branch Merge Status', LPNET_SERVICE_ROOT, cachedir)
[22:15] <leonardr> mtaylor: add version="devel" and see if that works
[22:16]  * wallyworld has to go and drop tax stuff off at the accountant
[22:18] <mtaylor> leonardr: that just sort of has it hanging there now
[22:19] <leonardr> mtaylor: that's odd. put this code before the login_with call and send me the output
[22:19] <leonardr> import httplib2
[22:19] <leonardr> httplib2.debuglevel = 1
[22:20] <mtaylor> leonardr: well, of course, it didn't hang that time
[22:21] <mtaylor> leonardr: oh - actually, I suck - one more try
[22:21] <leonardr> sinzui: argh, changing the validation code is creating _more_ test failures and taking me further into realms of which i know nothing
[22:21] <leonardr> to start with, can you sanity-check this?
[22:21] <leonardr> http://pastebin.ubuntu.com/577211/
[22:21] <mtaylor> leonardr: it's sitting there doing this:
[22:21] <mtaylor> http://paste.drizzle.org/show/427/
[22:22] <mtaylor> and now it works
[22:23] <leonardr> it hung for a long time and then worked, or you tried again and it worked?
[22:25] <sinzui> leonardr: That block looks good to me
[22:26] <mtaylor> leonardr: tried again - it hung for a while and then worked
[22:26] <leonardr> mtaylor: and is web_link accessible?
[22:26] <leonardr> sinzui: that's giving me a new test failure (which might be legit):
[22:27] <leonardr>     LaunchpadValidationError: u'Package a52dec not published in Ubuntu'
[22:27] <leonardr> except later on there's a reference to ubuntu/+source/a52dec
[22:28] <mtaylor> leonardr: yup. all is now happy and sunny
[22:28] <mtaylor> leonardr: thanks!
[22:29] <leonardr> ok, great. basically, that's a new feature that's not in version 1.0 of the web service
[22:30] <wgrant> leonardr: It's in the 1.0 WADL.
[22:30] <leonardr> hmmm
[22:30] <leonardr> mtaylor: did you end up wiping your launchpadlib directory?
[22:30] <mtaylor> leonardr: I did not - just doing the version='devel' made it happier
[22:31] <leonardr> well, devel would have a _different_ wadl file, that you hadn't retrieved before, so that could have the same effect
[22:32] <SpamapS> lifeless: ping.. trying to wrap my head around adding a new distro to launchpad that is not for built packages...
[22:32] <sinzui> huwshimi: will you be mumbling today?
[22:32] <huwshimi> sinzui: Yes, just getting my other laptop, it doesn't seem to want to connect in Natty
[22:33] <huwshimi> sinzui: two minutes
[22:34] <sinzui> leonardr: I got confused for a moment. That error really could be legitimate. When I started fixing packaging link data 18 months ago, I discovered that 80% of you tests were working with invalid data...no surprising that the real data created by the feature was rubbish
[22:34]  * sinzui looks for test helper to create a published spr
[22:37] <leonardr> sinzui: i'm pushing my changes
[22:38] <leonardr> lp:~leonardr/launchpad/bug-106338
[22:38] <sinzui> leonardr: I am in a meeting. I am also looking for a trick that makes a valid package
[22:38] <leonardr> ok
[22:38] <leonardr> sinzui, i may come back to this tomorrow, because i don't understand this stuff even when my brain is fresh
[22:39] <sinzui> leonardr: I think that is sensible. This is really messed up and timeoff will help us forget what does not work
[22:39] <leonardr> ok
[22:48] <sinzui> leonardr: the failing tests probably need something like this: self.factory.makeSourcePackagePublishingHistory(
[22:48] <sinzui>             sourcepackagename=self.sourcepackagename, distroseries=self.hoary)
[23:05] <wgrant> sinzui: All those style fixes have almost let you catch up :(
[23:07] <sinzui> wgrant: There were some bugs I fixed by accident too.
[23:07] <sinzui> but you are the winner
[23:07] <lifeless> wgrant: mthaddon probably manual landed
[23:07] <lifeless> SpamapS: hi
[23:07] <wgrant> lifeless: Yeah, probably :(
[23:07] <SpamapS> lifeless: I understand you had a chat w/ jml about a distro for ensemble formulae
[23:07] <lifeless> SpamapS: creating a distro is a privileged task; please wait for the ensemble list discussion and LEP before doing it
[23:08] <SpamapS> lifeless: right, I'm not going to do it.. I am trying to understand the challenges.
[23:08] <lifeless> SpamapS: ah, do you mean challenges in using or creating? :)
[23:08] <SpamapS> lifeless: well I don't understand the suggestion that there's no bug tracking if you don't have a built package. How do non-built distros work given that restriction?
[23:09] <lifeless> SpamapS: they don't
[23:09] <lifeless> SpamapS: try and file a bug on openssh in baltic.
[23:09] <lifeless> *baltix*
[23:09] <SpamapS> Oh.
[23:09] <lifeless> I dare you
[23:09] <SpamapS> whats the use of distros then? ;)
[23:09]  * SpamapS is tempted now
[23:09] <lifeless> SpamapS: distros are all about building and shipping packages
[23:10] <SpamapS> Yeah , be that the case in reality.. in my head, they're about tracking collections of software. :-P
[23:11] <lifeless> SpamapS: so we don't permit bugs filed on package names we know about unless the package is in the distro
[23:11] <lifeless> SpamapS: this stops folk filing bugs on e.g. cassandra before its actually in Ubuntu
[23:12] <SpamapS> that makes perfect sense. :)
[23:12] <lifeless> and this matters because all our metadata driving policy - like packagesets etc - depends on source package / binary packages existing.
[23:12] <SpamapS> So the real issue is there's no way, other than building, to create a package.
[23:12] <lifeless> *I* don't know how deep the rabbit hole goes
[23:13] <SpamapS> The way I saw it working was a mapping of a set of directories into source package names...
[23:14] <lifeless> SpamapS: you can map source packages into source package names
[23:14] <SpamapS> But I'm starting to wonder if, for the beginning of the project, we shouldn't just use a flat bzr and let people use tags to distinguish the formula name.
[23:14] <lifeless> SpamapS: that would be a -lot- simpler.
[23:15] <lifeless> SpamapS: jml and I had trouble inferring /requirements/ vs /things that using a distro would let you do/
[23:15] <SpamapS> Yeah I think people got excited about reusing the distro code (myself included)
[23:15] <lifeless> In some ways it makes a lot of sense
[23:15] <lifeless> in other ways we really have two quite different things wedged into one in lp's distro bug support
[23:16] <SpamapS> If the project takes off... we hope to have similar numbers of formulae as we have packages in Ubuntu so it would be very hard to track bugs on such a large body of work.
[23:16] <lifeless> one things is 'lightweight components in a bug tracker' and the other is 'policy driven by uploads of stuff'
[23:16] <lifeless> SpamapS: really? 20K forumulas?
[23:16] <SpamapS> a formula for anything that opens a network port
[23:16] <lifeless> SpamapS: thats more like 1K
[23:16] <SpamapS> so, maybe 2000
[23:17] <lifeless> we can start with one structure and migrate
[23:17] <lifeless> like, it would be great to have lightweight components for e.g. bzr
[23:17] <lifeless> something more than tags and less than separate projects
[23:17] <SpamapS> Yeah.. I think thats probably going to be the thing I recommend in the list discussion.
[23:18] <SpamapS> Tight coupling, once again, proves to make things less usable...
[23:18] <lifeless> indeed
[23:18] <lifeless> I'm a big fan of proving a core facility and letting folk do something cool with it
[23:18] <lifeless> now, on the vcs side
[23:19] <lifeless> jml said we need branch per (formula, release) tuple
[23:19] <lifeless> which the (packagename, series) namespace in distro branches supplies
[23:20] <lifeless> SpamapS: oh, on bugs, you /can/ use product series as a component like thing. same place in the UI
[23:20] <lifeless> but the concept of nominations vs tasks would make it a little ugly
[23:21] <SpamapS> lifeless: the series will be the series of formulae tho.. I'd like to keep that nice and clean.
[23:21] <lifeless> indeed
[23:21] <lifeless> so back to vcs
[23:22] <lifeless> do you need that tuple now, or really just (formula, 'sid') - e.g. you only need trunks for the formulas?
[23:22] <SpamapS> lifeless: I think having one bzr branch for all 2000 formulas would be a little painful. But you just mentioned in a recent discussion that bzr can branch the sub-dirs just fine, right?
[23:22] <lifeless> SpamapS: you can work on just a subdir yes.
[23:22] <lifeless> all the history comes, but honestly, 2K small projects is tiny bzr scale wise
[23:23] <SpamapS> so if its  /principia/sid/formulas/databases/mysql  .. I can bzr branch lp:principia/sid/formulas/databases/mysql and interact with it, yes?
[23:23] <SpamapS> Oh but I still get the whole history for /principia/sid .. :(
[23:24] <SpamapS> A minor concession.. but one that our target market will hate.. because they're all git/ruby fanbois. ;)
[23:24] <lifeless> SpamapS: yes, but I think you're overthinking the size here.
[23:24] <lifeless> when you do this in git you get all the history too
[23:24] <SpamapS> Right its just faster. ;)
[23:24]  * wgrant frowns at Lenovo.
[23:24] <SpamapS> which I *hate* as an argument for any vcs..
[23:24] <SpamapS> but some people think it matters. ;)
[23:24] <lifeless> SpamapS: mmm, not as such per various benchmarks we've done.
[23:25] <lifeless> SpamapS: bzr network fetch is pretty good these days
[23:25] <wgrant> 2a fetch is really slow even locally...
[23:26] <SpamapS> lifeless: Yeah, I'm going to again say that its a very small issue.. the main thing is to start getting those merge proposals cranking.. not achieve intergalactic domination overnight. :)
[23:35] <lifeless> SpamapS: anyhow, i have not particular preference for what we do
[23:36] <lifeless> I'm not trying to guide towards or away from distros
[23:36] <lifeless> but - they are what they are.
[23:37] <lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/730987
[23:37] <_mup_> Bug #730987: cannot merge to lp-production-configs <Launchpad itself:Triaged> < https://launchpad.net/bugs/730987 >
[23:39] <wgrant> lifeless: Will look soon. Currently trying to figure out whether my T400's battery is being malicious or is actually dead.
[23:39] <wgrant> It is 100% charged and still at around 60% design capacity, but this morning refuses to deliver any power.
[23:39] <lifeless> wgrant: freeze it?
[23:39] <wgrant> The battery monitor tool says it has failed due to normal wear.
[23:39] <wgrant> I call BS.
[23:41] <wgrant> Maybe I'll leave it all off and unplugged for a few hours and see what happens.
[23:42] <wgrant> I have never had a Li-ion battery do this before, so I tend to think it is being malicious.
[23:45] <LPCIBot> Yippie, build fixed!
[23:45] <LPCIBot> Project devel build #513: FIXED in 5 hr 17 min: https://hudson.wedontsleep.org/job/devel/513/
[23:50] <lifeless> wallyworld: you need to gall PersonSet's getPrecachedPersonsFromIDs() method and listify the result, or something like that
[23:58] <lifeless> wgrant: thanks!
[23:59] <wgrant> lifeless: The obvious fix is to add a list of excluded configs to the test... but maybe a marker file in the config dir would be better.