[00:04] <thumper> https://code.launchpad.net/~thumper/launchpad/epic-email-bug-description/+merge/51977
[00:04] <thumper> wallyworld: abentley won't still be here
[00:04] <abentley> wallyworld: not really :-)
[00:05] <wallyworld> abentley: i was hoping for a quick +1 on the recipe build perf mp after i fixed the test :-)
[00:05] <lifeless> wgrant: have a look at the same urls in https://bugs.launchpad.net/launchpad/+bug/711064
[00:05] <_mup_> Bug #711064: POFile:+translate timeouts <qa-ok> <timeout> <Launchpad itself:Fix Released by lifeless> < https://launchpad.net/bugs/711064 >
[00:05] <lifeless> wgrant: (click through I mean) - quite a bit snappier :)
[00:06] <wallyworld> abentley:  i take your point about deleted recipes but the page was only designed to show builds with recipes and this mp is a perf tweak. extending what the page is designed to do can be done separately :-)
[00:07] <abentley> wallyworld: It doesn't matter what the page is designed to do, because this is model code.
[00:07] <wallyworld> the page will never show builds without recipes due to how the query is structured
[00:07] <lifeless> abentley: I thought it was view vode
[00:07] <abentley> wallyworld: anyone can see it and reasonably assume that it is safe to call.
[00:07] <wallyworld> ie select stuff from recipe join build .....
[00:07] <abentley> lifeless: no, model.
[00:08] <wallyworld> the recipe will never be None on the page
[00:08] <abentley> wallyworld: It does not matter whether the recipe will ever be None on the page.
[00:08] <wallyworld> so it is always safe to call
[00:08] <wallyworld> sorry, i'm missing something i think
[00:09] <abentley> wallyworld: it is not safe to call if the recipe is None, and there's no reason to believe that it will be non-None for an arbitrary caller.
[00:09] <wallyworld> but it isn't a view off of a recipe
[00:09] <wallyworld> it is a view off ILanchpadRoot which runs a query and displays some data
[00:09] <wgrant> lifeless: Still 4s :(
[00:10] <lifeless> wgrant: down from ... 12
[00:10] <wallyworld> and that data will only ever contain valid recipes and associated build info
[00:10] <abentley> wallyworld: It looks very much like you've added two properties to RecipeBuild.
[00:10] <wallyworld> yes, and the recipe there will never be None
[00:10] <abentley> wallyworld: is that not the case?
[00:10] <abentley> wallyworld: Yes, it will sometimes be none.
[00:10] <wallyworld> how?
[00:11] <wallyworld> the query is guaranteed to return records with recipes
[00:11] <abentley> wallyworld: What query?  The query is not connected to RecipeBuild.
[00:11] <wgrant> lifeless: :(
[00:12] <abentley> wallyworld: It is fully possible for an arbitrary caller to have a RecipeBuild with a None recipe, and to access recipe_name.
[00:12] <wallyworld> abentley: the query which is used to implement findCompletedDailyBuilds()
[00:12] <wallyworld> but i guess if they are paging through the results, the recipe could be deleted after the page was first rendered
[00:12] <abentley> wallyworld: You have no guarantee that the RecipeBuild was returned by that methode.
[00:13] <abentley> wallyworld: Arbitrary code in Launchpad can access RecipeBuilds and then access recipe_name.
[00:13] <wallyworld> abentley: the RecipeBuildRecord used in the collection backing the view is constrcuted directly from the result set
[00:14] <wallyworld> the RecipeBuildRecord is a named tuple
[00:14] <wallyworld> with data needed by the view shoved into it
[00:14] <abentley> wallyworld: so you're saying that even though this code lives in model, it's actually view code?
[00:15] <wallyworld> well, its code which queries the model and returns some data
[00:15] <wallyworld> the data is currently used by a view but it could be used elsewhere too
[00:16] <wallyworld> the main class is RecipeBuildRecordSet
[00:16] <wallyworld> there's also a BugTaskSet for example that is model code
[00:17] <wallyworld> the rendering of the data is done in the view layer
[00:20] <abentley> wallyworld: so my main source of confusion is that I thought we were talking about SourcePackageRecipeBuild, not RecipeBuildRecord, which I don't think I've seen before.
[00:21] <wallyworld> ah ok. the number of times i've be confused between SourcePackageRecipeXXXX, SourcePackageReleaseXXX is too many to mention. so many similar names
[00:21] <wallyworld> etc
[00:22] <abentley> wallyworld: well, it's complicated by the fact that SourcePackageRecipeBuild and RecipeBuildRecord seem to refer to exactly the same thing.
[00:22] <wallyworld> not exactly :-)
[00:24] <abentley> wallyworld: How not?
[00:24] <wallyworld> the later is akin to a denormalised chunk of data, the other is a straight model object
[00:25] <abentley> wallyworld: but they both refer to the same thing, and you can query either to get the same data.
[00:25] <wallyworld> model object in this case = persistent database object perhaps
[00:27] <wallyworld> i would have to take a closer look, but can SourcePackageRecipeBuild directly provide the SourcePackageName, the Archive, the lastest build time etc?
[00:28] <abentley> wallyworld: You can get those from SPRB.source_package_release.
[00:28] <wallyworld> but not without navigating the object model
[00:28] <lifeless> onely one page of getPublishedSources references to update
[00:28] <lifeless> ><
[00:29] <wallyworld> SourcePackageRecipeBuild is designed to package all required data into a readonly single object, a snapshot of the system state if you like, for easy access
[00:29] <wallyworld> to fit the use case
[00:29] <wallyworld> i meant RecipeBuiildRecord
[00:29] <wallyworld> not SourcePackageRecipeBuild
[00:30] <abentley> wallyworld: I get that you have easier, more efficient access to that data.
[00:30] <abentley> wallyworld: I still think the RecipeBuildRecord refers primarily to a SourcePackageRecipeBuild.
[00:31] <abentley> wallyworld: And the rest is just precaching.
[00:31] <wallyworld> and it comes straight off the query so it can be got with fewer object instantiations, so more efficient
[00:31] <lifeless> wgrant: another in _collectLatestPublishedSources
[00:31] <abentley> wallyworld: I wasn't talking about whether it was efficient or direct.
[00:32] <abentley> wallyworld: I was talking about what it referred to.
[00:32] <wallyworld> ah ok
[00:32] <wallyworld> so given the scope of the mp, ie a perf tweak, you're happy to +1? :-)
[00:33] <abentley> wallyworld: I will reluctantly +1 it.  I don't see why you're leaving booby-traps for people who extend it later.
[00:34] <abentley> wallyworld: When the fix is a two-line conditional.
[00:34] <wallyworld> i don't get what the booby traps are. it's not that uncommon not to pass around entire persistent objects / model objects and instead construct objects more suited to a given use case?
[00:35] <wallyworld> but the conditional is not required
[00:35] <wallyworld> i can add it though
[00:36] <lifeless> wallyworld: AIUI you're saying that this memo object is never constructed without a recipe
[00:36] <LPCIBot> Project db-devel build #413: STILL FAILING in 4 hr 50 min: https://hudson.wedontsleep.org/job/db-devel/413/
[00:36] <wallyworld> yes
[00:36] <abentley> wallyworld: the booby-trap is that if someone ever constructs a RecipeBuildRecord whose SourcePackageRecipeBuild.recipe is None, then recipe_name will raise an AttributeError.
[00:36] <lifeless> wallyworld: and that someone constructing it without a recipe would be changing its contract, so there will be other changes they have to make.
[00:36] <wallyworld> since it is constructed in the resultset decorator
[00:36] <wgrant> StevenK: If you see any bug comment spam, you can now hide it yourself.
[00:37] <lifeless> wallyworld: I have no particular opinion here, just trying to make sure I understand
[00:37] <wallyworld> the RecipeBuildRecords are not meant to be constructed with null recipes
[00:37] <abentley> wallyworld: If that is the contract, then you could put it in the constructor.
[00:37] <wallyworld> yes, i could an an assert
[00:38] <wallyworld> add
[00:38] <StevenK> wgrant: I didn't think that rolled out yet
[00:38] <wallyworld> hopefully namedtuples let me do that
[00:38] <abentley> wallyworld: that would make sense to me.  From looking at that object, I see nothing to suggest such a contract.
[00:38] <wallyworld> fair call
[00:38] <wgrant> StevenK: Just in the last hour or so.
[00:39] <wallyworld> i think doing that will remove a lot of the confusion
[00:39] <wallyworld> i guess from my perspective, i never expected one would want or need to construct the object without a recipe
[00:40] <wallyworld> lifeless: you had a question?
[00:40] <abentley> wallyworld: I expect we'll want to construct it with arbitrary SourcePackageRecipeBuilds before long.
[00:40] <abentley> wallyworld: and that would include ones without a recipe.
[00:40] <StevenK> wallyworld: Looking at production data, there are many SourcePackageRecipeBuilds where recipe is None, because the user has deleted the recipe.
[00:41] <wallyworld> abentley: at the moment, it's only constrcuted from the resultset decorator, but that could change
[00:41] <lifeless> wallyworld: no, I asked questions to establish a clear picture of the debate
[00:41] <abentley> wallyworld: right.
[00:41] <wallyworld> StevenK: but those records will never be returned due to how the query is constructed
[00:41] <wallyworld> lifeless: oh, i didn't see your questions
[00:42] <lifeless> wallyworld: you answered them :)
[00:42] <StevenK> wallyworld: I am +0 on adding *another* class for Recipe builds
[00:42] <StevenK> This strikes me as needless duplication
[00:42] <abentley> StevenK: I don't think anyone is suggesting that.
[00:42] <wallyworld> lifeless: ok. hopefully you were satisfied with the answers
[00:43] <StevenK> abentley: Then I apologise, since I've obviously read what wallyworld is doing wrong.
[00:43] <abentley> StevenK: wallyworld is just extending the existing RecipeBuildRecord class.
[00:43] <StevenK> wgrant: Can haz howto?
[00:43] <wallyworld> StevenK: it's not duplication since it's not a model object - it's akin to a DTO to package some data needed for a given use case
[00:43] <wgrant> StevenK: lp.bugs[1234].setCommentVisiblity(comment_number=2, visible=False)
[00:43] <wgrant> With Visibility spelt properly.
[00:43] <StevenK> wgrant: I can't do it via the web UI?
[00:43] <wgrant> StevenK: No UI.
[00:44] <wgrant> But it's a trivial API script.
[00:44] <StevenK> Bah, fail.
[00:44] <StevenK> You'd think the API script would be on the wiki somewhere ...
[00:45] <wallyworld> abentley: thanks for the input. i think we managed to reach an acceptable solution once i add the asserts, or maybe i'll just cater for null recipe if we think that's a future possibility, but that will mean changing the page template too perhaps
[00:45] <wgrant> StevenK: It is two lines...
[00:46] <wgrant> StevenK: I'm not sure that anybody has bothered to publish it.
[00:46] <lifeless> I can has OCR? https://code.launchpad.net/~lifeless/launchpad/bug-727560/+merge/51989
[00:51] <wallyworld> lifeless: seems there's no OCR but i can have a go if you like
[00:52] <wallyworld> or you may prefer a proper review first up
[00:54] <lifeless> wallyworld: feel free; we can tap thumper to mentor ;)
[00:54] <thumper> ?
[00:55] <thumper> lifeless: https://code.launchpad.net/~thumper/launchpad/epic-email-bug-description/+merge/51977
[00:55] <lifeless> thumper: I have a moderate size, 90% mechanical because of stormification branch needing review
[00:55] <lifeless> thumper: on yours now
[00:56] <lifeless> typo
[00:56] <lifeless> lots *of* text to add
[00:57] <wallyworld> thumper: i was going to have a go since no one was around but if you want it....
[00:57] <lifeless> wgrant: want to review thumpers, then I'll mentor?
[00:57] <lifeless> wgrant: I see 2 things.
[00:57] <thumper> wallyworld: no, you take it
[00:57] <wallyworld> thumper: great. i'd like the practice
[00:58] <wgrant> lifeless: Sure, looking.
[00:59]  * wallyworld quickly finishes daily recipe build perf fix first so ec2 can chew on it
[01:01] <thumper> :-(((
[01:01] <StevenK> wgrant: Right, all done. Six comments hiden
[01:01] <StevenK> *hidden
[01:01] <wgrant> StevenK: Thanks.
[01:01] <thumper> time out proposing a review
[01:01] <StevenK> Whee
[01:02] <wgrant> thumper: Was it scanning?
[01:02] <thumper> possibly
[01:06] <thumper> https://code.launchpad.net/~thumper/launchpad/process-email-oops-on-person-adaptation/+merge/51992
[01:09] <wgrant> thumper, lifeless: Reviewed the email size thing.
[01:09] <thumper> wgrant: ta
[01:13] <lifeless> wgrant: thumper: mentored
[01:14] <lifeless> wgrant: want to mentor https://code.launchpad.net/~thumper/launchpad/process-email-oops-on-person-adaptation/+merge/51992 ?
[01:14] <thumper> lifeless: ok
[01:14] <thumper> d'uh
[01:15] <thumper> me didn't read
[01:19] <wgrant> lifeless: Looking.
[01:21] <wgrant> lifeless: Done.
[01:29] <lifeless> thumper: second one reviewed.
[01:31] <wgrant> Thanks.
[01:33]  * thumper relocates
[01:34] <wallyworld> lifeless: wondering why candidates[0] and not candidates.first()
[01:34] <wallyworld> or are they the same thing in terms of efficiency
[01:35] <lifeless> wallyworld: .first() is clearer, they should be equivalent for efficiency. Without reading the code I am not /sure/ that .first() raises IndexError, so I left some [0] around for that case.
[01:35] <lifeless> wallyworld: also some tests were going to be less clear with .first() so I left those alone too
[01:36] <wallyworld> lifeless: ok. i assume that because only Archive.getPublishedSources() was change and not the others, the the 2 resultset types are interchangable?
[01:36] <wallyworld> forgive my storm ignorance
[01:36] <lifeless> wallyworld: which 2 resultset types?
[01:36] <lifeless> wallyworld: (there are, uhm, 4 or so around)
[01:36] <lifeless> wallyworld: and interchangable /where/
[01:36] <wallyworld> SQLObjectResultSet and DecorateResultSet
[01:37] <lifeless> they are not interchangable and not compatible
[01:37] <lifeless> thats why I stormified getPublishedSources so that I could use DRS
[01:40] <wallyworld> ok. i saw a few getPublishedSources() methods and initially thought they were off the one interface but not so
[01:40] <wallyworld> so my question doesn't make sense anymore
[01:42] <lifeless> its entirely possible I have some things in this that ec2 will highlight as faults
[01:42] <lifeless> but I think I identified the types correctly
[01:43] <wallyworld> yeah, i suspect i was seeing an issue that's not there. hard to tell sometimes from looking at diffs
[01:45] <wallyworld> storm sure seems better than what it replaced, looking at the diffs - less string literals and more object based approach
[01:54] <lifeless> how do you make a scanned branch in a test ?
[02:04] <lifeless> \o/
[02:04] <wgrant>  * 3472 Exceptions
[02:04] <lifeless> flat cunt for timeouts
[02:04] <lifeless> *count*
[02:07] <wallyworld> lifeless: keep it clean :-)
[02:08] <lifeless> nice drop in bugtask:+index timeouts
[02:09] <wgrant> We should be below 2000 excepts per day tomorrow.
[02:10] <wallyworld> wgrant: what was the peak value before the tuning work started in earnest?
[02:10] <wgrant> wallyworld: I don't really know. But a few days ago it was 23000.
[02:10] <wallyworld> !!!!
[02:10] <wgrant> 16000 of those were that loggerhead thing, though.
[02:10] <wgrant> So it's sort of cheating.
[02:11] <wallyworld> still 7000 (or anything with that many zeros at the end) is too many
[02:11] <wallyworld> except if its your bank balance - then too many zeros are never enough :-)
[02:12]  * wallyworld needs lunch
[02:13] <lifeless> thumper: hey
[02:14] <StevenK> wgrant: Test failures with my TAL branch, do agree with http://pastebin.ubuntu.com/574779/
[02:16] <wgrant> StevenK: Looks reasonable.
[02:16] <StevenK> And the tests pass, so I'll toss it at ec2 again
[02:16] <wgrant> ec2 is only 3.5 hours now.
[02:16] <wgrant> It's nice.
[02:17] <StevenK> Heh, yes
[02:30] <thumper> lifeless: yus?
[02:31] <lifeless> thumper: I have a problem
[02:31] <thumper> ok
[02:31] <lifeless> thumper: I https://bugs.launchpad.net/launchpad/+bug/722956
[02:31] <_mup_> Bug #722956: BranchSet:CollectionResource#branches timeouts <bad-commit-12489> <bad-commit-12505> <qa-bad> <timeout> <Launchpad itself:In Progress by lifeless> < https://launchpad.net/bugs/722956 >
[02:31] <lifeless> thumper: I had a branch that eager loads that
[02:31] <lifeless> thumper: but it failed on qastaging
[02:31] <lifeless> thumper: i fixed a typo, but then it failed again with a TypeError doing some adaption
[02:31] <lifeless> thumper: IIRC the error properly, it looked like a source package series branch
[02:32] <lifeless> thumper: but I don't know how to set one up in the test suite
[02:32] <lifeless>     
[02:32] <lifeless>     def test_renders_with_source_package_branch(self):
[02:32] <lifeless>         source_package = self.factory.makeSourcePackage()
[02:32] <lifeless>         branch = self.factory.makeBranch(sourcepackage=source_package)
[02:32] <lifeless>         branch.updateScannedDetails(None, 0)
[02:32] <lifeless>         logout()
[02:32] <lifeless>         lp = launchpadlib_for("test")
[02:32] <lifeless>         list(lp.branches)
[02:32] <lifeless> is what I have
[02:32] <lifeless> but I suspect its not a 'source package series branch'
[02:32] <lifeless> because the url is a long one with ~user in it
[02:33] <wgrant>    3 TypeError: ('Could not adapt', <lp.code.model.seriessourcepackagebranch.SeriesSourcePackageBranch object at INSTANCE-ID>, <InterfaceClass lp.code.interfaces.linkedbranch.ICanHasLinkedBranch>)
[02:33] <thumper> lifeless: a package branch is one whose target is a distro series source package
[02:33] <lifeless> thumper: yup, I've got that far
[02:34] <lifeless> thumper: then there is the *SeriesSourcePackageBranch* which is the blessed ones
[02:34] <thumper> lifeless: also, use factory.makePackageBranch
[02:34] <thumper> right
[02:34] <lifeless> how do I get from what I have to the blessed one
[02:35] <thumper> off the top of my head...
[02:35] <thumper> something like
[02:35] <thumper> ICanHasLinkedBranch(branch.target).branch = branch
[02:36] <thumper> ICanHasLinkedBranch(naked_product).setBranch(branch)
[02:36] <thumper> copied from a test
[02:37] <thumper> lifeless: lib/lp/code/model/tests/test_branch.py:803:
[02:38] <thumper> lifeless: is one way
[02:39] <thumper> lib/lp/code/model/tests/test_linkedbranch.py
[02:39] <lifeless> looking, thanks
[02:43] <lifeless> thumper: thanks!
[02:43] <lifeless> thumper: have reproduced the boom
[02:44] <lifeless> oh, no, I haven't
[02:44] <lifeless> I made the test do it instead ><
[02:51] <lifeless> thumper: thanks, *now* I have the failure reproduced.
[02:51] <thumper> cook
[02:51] <thumper> cool
[02:52] <lifeless> its a bit messy - I copied all of lib/lp/code/model/tests/test_branch.py:803: basically, but thats tolerable for now
[03:01] <lifeless> ok, fixed
[03:01] <lifeless> lp-land time.
[03:04] <thumper> wgrant: you were right, I was checking the entire email message
[03:04] <thumper> wgrant: I've fixed it, and pushing now
[03:04] <wgrant> thumper: Great, will look in a sec.
[03:07] <lifeless> thumper: can you mentor https://code.launchpad.net/~lifeless/launchpad/bug-727560/+merge/51989 please?
[03:11] <wgrant> thumper: Why do you need ZopelessAppServerLayer?
[03:11] <wgrant> That actually starts up an appserver process, IIRC.
[03:11] <thumper> wgrant: as opposed to?
[03:11] <thumper> the content gets stored in the librarian
[03:11] <wgrant> Plain LaunchpadZopelessLayer.
[03:11] <thumper> the email bits that is
[03:11] <thumper> the script runs in a zopeless environment
[03:12] <thumper> so the test should mirror that IMO
[03:12] <lifeless> LaunchpadZopelessLayer mirrors that
[03:12] <thumper> to be honest, I copied the layer from the TestCodeHandler test
[03:12] <lifeless> the appserver layers are IMNSHO hideous
[03:12] <thumper> I'm not entirely sure
[03:12] <wgrant> AppServerLayer is a bug.
[03:12] <thumper> I could try it with the LaunchpadZopelessLayer
[03:12] <lifeless> thumper: I suggest you try LaunchpadZopelssLayer
[03:12] <lifeless> jinx
[03:12] <wgrant> LaunchpadZopelessLayer provides a librarian.
[03:15] <thumper> so... what does the ZopelessAppServerLayer provide?
[03:15] <wgrant> An appserver in a separate process.
[03:15] <lifeless> a running zope instance in a subprocess
[03:15] <lifeless> it was added for mailman
[03:16] <lifeless> and should be nuked from orbit along with the way we intgrate mailman
[03:16] <wgrant> Most stuff using it is probably a bug.
[03:16] <thumper> hmm...
[03:16] <thumper> needed for the code handler :(
[03:16] <wgrant> Oh?
[03:16] <thumper> TestCodeHandlerProcessMergeDirective
[03:16] <thumper> don't ask me why
[03:16] <thumper> I'm not sure
[03:17] <wgrant> It doesn't need it.
[03:17] <wgrant> It may use it.
[03:17] <wgrant> But it doesn't need it.
[03:17] <thumper> if fails without it
[03:17] <thumper> and passes with it
[03:17] <wgrant> That may just mean that the test is broken.
[03:18] <thumper> :)
[03:18] <thumper> changed the layer
[03:18] <thumper> and pushed
[03:18] <wgrant> Thanks.
[03:22] <StevenK> lifeless: I spoke to Barry about Mailman 3 at the distro sprint. He said he's close, but there's two things that need fixing.
[03:22] <lifeless> StevenK: I know
[03:22] <lifeless> StevenK: its one of those things that will happen in 2013
[03:22]  * lifeless sadly isn't kidding
[03:24] <StevenK> That makes me sad. :-(
[03:26] <lifeless> its not on our todo list for -ages-
[03:26] <lifeless> and barry has not got time to do a major overhaul in lp space
[03:26] <lifeless> good intentions aside
[03:26] <wallyworld> lifeless: in your mp, why the need for 'distroseriesID = Attribute("DB ID for distroseries.")'. i thought storm did that automagically?
[03:27] <lifeless> wallyworld: thats on the interface.
[03:27] <lifeless> wallyworld: if its not on the interface security proxies kick you out
[03:27] <wallyworld> ah, yes. thanks
[03:28] <StevenK> lifeless: Well, he also said the two things are broken in core, which means they might get fixed soon. :-)
[03:31] <lifeless> thumper: you need to click 'approve' as well
[03:31] <thumper> lifeless: it isn't approved
[03:31] <thumper> lifeless: wallyworld said needs info
[03:32] <lifeless> thumper: ian approved it
[03:32]  * thumper looks again
[03:32] <thumper> I bet he added comments while I had it open
[03:32] <thumper> where's our long-poll
[03:32] <wallyworld> and optimistic locking :-)
[03:35] <lifeless> thumper: thanks!
[03:39] <huwshimi> Is it just me or are a bunch of Canonical's websites down?
[03:40] <StevenK> Such as?
[03:40] <huwshimi> StevenK: Well Everything. I can't access the irc server, ubuntu.com, canonical.com etc.
[03:40] <wgrant> All good for me.
[03:40] <wgrant> What does traceroute say?
[03:41] <StevenK> And me.
[03:43] <huwshimi> Traceroute errored after one of internode's servers, but I seem to be connecting again now
[03:44] <StevenK> I suspect a route failed over or so
[03:46] <huwshimi> oh well as long as it was just me
[03:48] <huwshimi> still flaky though
[03:50] <wgrant> huwshimi: I've just started getting ~50% packet loss in the US.
[03:50] <StevenK> To which part of the US?
[03:50] <StevenK> My route to Fremont is fine.
[03:50] <huwshimi> wgant: which isp are you with?
[03:51] <wgrant> StevenK: Within L3 in San Jose.
[03:51] <wgrant> huwshimi: Optus.
[03:51] <wgrant> StevenK: My route to Fremont is fine too.
[03:51] <wgrant> Doesn't go through L3.
[03:51] <wgrant> Oddly.
[03:52] <StevenK> I hit Exetel, Alter, and then my server
[03:53] <wgrant> I have just one (anonymous) hop between Optus and HE.
[03:56] <StevenK> I have one anonymous hop, but it's Telstra's DSL bridges that they like hiding
[04:08] <lifeless> I'm well and truely sick of hakcing on BugTask:+index
[04:08] <lifeless> won't another page please start timing out massively. I need a change.
[04:10] <lifeless> wallyworld: I thought you were nuking '155 BugWatchUpdateWarning'
[04:10] <lifeless> ExternalBugtracker for BugTrackerType 'GOOGLE_CODE' is not known
[04:10] <wallyworld> lifeless: i've no idea what you are referring to :-)
[04:10] <lifeless> bach
[04:10] <lifeless> wgrant: ^
[04:10] <wgrant> lifeless: That was me.
[04:10] <wgrant> But it requires checkwatches.txt to be redone.
[04:11] <wgrant> It is the hardest OOPS to get rid of.
[04:11] <wgrant> So I did the others first.
[04:11] <lifeless> wgrant: the doctest ?
[04:11] <wallyworld> i can see how you keep getting us confused, but for future reference, i'm the good looking one
[04:11] <wgrant> lifeless: Yes.
[04:11] <lifeless> wallyworld: its all about the tabs
[04:11] <lifeless> wallyworld: if I fumble the g, it tab completes to you
[04:11] <lifeless> wgrant: my sympathies
[04:11] <wallyworld> yeah i know, i was just messing with you
[04:11] <lifeless> wgrant: if you need a tactical nuke, let me know
[04:12] <wgrant> lifeless: We can probably drop that bit of the doctest, but it may be the only test that running the script logs OOPSes properly.
[04:12] <wgrant> I will hopefully get back to that next week.
[04:12] <wgrant> There are what.. 300-400 of them a day?
[04:12] <lifeless> yeah 10% of remaining daily exceptions
[04:13] <wgrant> Note that 1500 of yesterday's exceptions are already fixed.
[04:13] <wgrant> So we are down to ~2000.
[04:13] <lifeless> wgrant: -lovely-
[04:13] <wgrant> So it's more!
[04:19] <lifeless> hmm
[04:19] <lifeless> I find the new style strains my eyes :(
[04:19] <wgrant> I think body and comment text is slightly too small.
[04:19] <lifeless> they are to faint
[04:19] <wgrant> I've had a couple of friends say the same thing.
[04:19] <lifeless> too
[04:21] <lifeless> wgrant: hmm
[04:21] <wgrant> Hmm?
[04:22] <lifeless> wgrant: reckon you could run dogfood with sql backtraces enabled
[04:22] <lifeless> and find where the milestone lookups in https://lp-oops.canonical.com/oops.py/?oopsid=1888C48#repeatedstatements are being triggered ?
[04:22] <wgrant> lifeless: Sure.
[04:22] <lifeless> ditto the distribution one
[04:22] <lifeless> thats another second if we fix both of those.
[04:23] <lifeless> https://bugs.launchpad.net/launchpad/+bug/724033 to record what you find; I'll work up fixes.
[04:23] <_mup_> Bug #724033: BugTask:+index timeouts - distributions and milestones being late evaluation loaded - repeatedly - on bug 230350 <qa-ok> <timeout> <Launchpad itself:In Progress by lifeless> < https://launchpad.net/bugs/724033 >
[04:23] <wgrant> lifeless: Could you run http://paste.ubuntu.com/574807/ on staging?
[04:23] <wgrant> I'm merging the tracebacks branch.
[04:23] <lifeless> qas or s?
[04:24] <lifeless> wgrant: ^
[04:24] <wgrant> lifeless: Whichever has the newer DB.
[04:24] <lifeless> $meh
[04:24] <wgrant> Doesn't really matter.
[04:24] <lifeless> its running
[04:24] <wgrant> Thanks.
[04:25] <wgrant> Shouldn't take too long.
[04:25] <lifeless> you want the output, or timing ?
[04:25] <wgrant> Output please.
[04:25] <wgrant> Shouldn't be much.
[04:25] <wgrant> I hope.
[04:25] <lifeless> will tell you when it finishes
[04:26] <wgrant> It only takes a couple of minutes on DF...
[04:26] <lifeless> https://pastebin.canonical.com/44198/
[04:27] <wgrant> Thanks.
[04:28] <wgrant> lifeless: DF running with backtraced OOPSes.
[04:28] <wgrant> lifeless: It may time out too early, but we can just bump the limit I guess.
[04:28] <lifeless> hit up https://bugs.launchpad.net/ubuntu/+source/afflib/+bug/230350/+index on it
[04:28] <_mup_> Bug #230350: Missing Debian Maintainer field <Ubuntu:Invalid> <afflib (Ubuntu):Fix Released by saivann> <alac-decoder (Ubuntu):Fix Released by warp10> <axiom (Ubuntu):Invalid> <beneath-a-steel-sky (Ubuntu):Fix Released> <bibletime-i18n (Ubuntu):Fix Released by txwikinger> <binkd (Ubuntu):Fix Released> <bzr-builddeb (Ubuntu):Won't Fix> <capisuite (Ubuntu):Invalid> <chinput (Ubuntu):Fix Released by warp10> <chmsee (Ubuntu):Fix Released> <ciso (U
[04:28] <wgrant> Oh, that one, right.
[04:28] <lifeless> (you're logged in)
[04:30] <lifeless> any my show-entire-profile is landed so I can get useful data from bug 1
[04:30] <_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
[04:32] <wgrant> lifeless: It timed out too early the first time. Trying some stuff.
[04:32] <wgrant> Got it.
[04:32] <lifeless> cool
[04:35] <wgrant> The OOPS is large, and bzip2 is slow.
[04:37] <lifeless> wgrant: thanks
[04:37] <lifeless>          738            0      0.6724      0.0146   canonical.launchpad.utilities.celebrities:75(__get__)
[04:37] <lifeless>         +738            0      0.2368      0.0034   +lp.registry.model.person:3369(get)
[04:37] <lifeless> :/
[05:28] <wallyworld> if an attribute is defined to be a CollectionField but there no exported() wrapping the declaration, does that mean it isn't exported to the webservice?
[05:28] <wallyworld> and hence i can delete it?
[05:29] <wgrant> That's right.
[05:29] <wallyworld> \o/
[05:29] <wgrant> Which?
[05:29] <wallyworld> related_bugs on branchmergeproposal
[05:29] <wallyworld> it's being replaced by getRelatedBugTasks
[05:30] <wallyworld> need to pass in a user so that only the correct visible bug tasks are returned
[05:30] <wgrant> Indeed, not exported.
[05:30] <wallyworld> cool. excellent.
[05:30] <lifeless> \o/
[05:33] <lifeless> I am going to hold a freaking party when we have a 0 oops day
[05:33]  * wallyworld has already removed DecoratedBug and linked_bugs and other stuff. need to start new pipe to do the last bit
[05:33] <wgrant> lifeless: Did you find anything else useful in that OOPS?
[05:33] <lifeless> wgrant: I
[05:33] <wallyworld> lifeless: how many oopses today then?
[05:33] <lifeless> 've just got back
[05:33] <lifeless> wallyworld: read the summary :>
[05:33] <lifeless> ~ 15K
[05:33] <wallyworld> oh, didn't see it, wndow too small
[05:33] <wgrant> lifeless: A zero OOPS day is actually not too implausible.
[05:34] <wgrant> We have lots of OOPSes remaining.
[05:34] <wgrant> But most of them are seen very rarely.
[05:34] <wgrant> Zero exceptions could be doable in the next two months.
[05:34] <lifeless> fixing bugs has the impact of fixing bugs.
[05:35] <wgrant> The exception report will be fairly sensible tomorrow.
[05:35] <wgrant> With maybe 10 distinct checkwatches exceptions.
[05:42] <lifeless> curtis is on a rampage
[05:42] <wgrant> Yes.
[05:44] <lifeless> http://webnumbr.com/launchpad-oops-bugs.all
[05:44] <wgrant> Ja, saw that this morning.
[05:44] <wgrant> It's encouraging.
[05:46] <spiv> lifeless: here, let me fix that Y axis for you: http://webnumbr.com/.join%28bzr-pqm-queue-length,launchpad-oops-bugs%29 ;)
[05:46] <poolie> :)
[05:46] <lifeless> spiv: lol
[05:47] <StevenK> Bwahaha
[05:47] <lifeless> spiv: still looks nice.
[05:47] <spiv> lifeless: it does!
[05:52] <lifeless> wgrant: distros are due to the vocabulary too
[05:52] <wgrant> lifeless: Aha.
[05:52] <wgrant> So that's the two big repetitions fixed.
[05:52] <lifeless> its eager loading (something I added)
[05:52] <lifeless> wgrant: identified.
[05:53] <wgrant> Shhhh.
[05:53] <lifeless> Fixing takes a little longer
[05:53] <lifeless> because a new object is invoked per row.
[05:53] <wgrant> Ah.
[05:53] <wgrant> Sad.
[05:53] <lifeless> so I need to:
[05:53] <lifeless>  - eager load from the outer view class
[05:53] <lifeless>  - inject into the bugtasks
[05:53] <lifeless>  - use the resulting attribute
[05:55] <lifeless> wgrant: I'm going to wag that user_can_edit_milestone is the cause of the person lookups
[05:55] <LPCIBot> Yippie, build fixed!
[05:55] <LPCIBot> Project db-devel build #414: FIXED in 5 hr 19 min: https://hudson.wedontsleep.org/job/db-devel/414/
[05:55] <wgrant> That probably means that devel is about to fail.
[05:55] <StevenK> :-(
[05:55] <lifeless> its not
[05:55] <wgrant> lifeless: For authenticated requests it is mostly permission lookups in there, yes.
[05:56] <StevenK> devel just started a build, though
[05:56] <wgrant> Oh.
[05:56] <wgrant> The bad builder must have been culled at some point.
[05:56] <wgrant> I didn't think there was enough idle time.
[05:56] <StevenK> wgrant: Did you see Jenkins last night, building devel, db-devel and windmill at the same time?
[05:56] <wgrant> StevenK: Yes.
[05:56] <wgrant> This morning it did that too.
[05:57] <StevenK> Now if only parellel-test passed
[05:57] <wgrant> Heh.
[05:58] <StevenK> Executing star-merge bzr+ssh://bazaar.launchpad.net/~thumper/launchpad/inline-recipe-name-edit at Thu Mar  3 03:46:22 2011
[05:58] <StevenK> bzr: ERROR: no such option: -0
[05:58] <StevenK> wgrant: ^
[05:58] <wgrant> Ugh. But it's in the check that lifeless wanted to delete anyway.
[05:58] <wgrant> old bzr is old.
[05:59] <StevenK> I'm not used to PQM being this quick!
[05:59] <StevenK> 2 minutes from EC2 shutting down until PQM said sucess
[05:59] <wgrant> Yes.
[05:59] <wgrant> And with ec2 an hour quicker....
[05:59] <StevenK> Remove the doctest layer too
[05:59] <wgrant> And also ec2 and buildbot reliable.
[06:00] <wgrant> StevenK: Why are Hudson's test durations all 0? Does the subunit -> JUnit XML converter not bring that across?
[06:01] <wgrant> s/Hudson/Jenkins, your domain is confusing/
[06:01] <lifeless> probably old subunit
[06:01] <lifeless> it had a bug
[06:01] <StevenK> wgrant: I've been meaning to rename it
[06:01] <StevenK> lpci.w.o or so
[06:02] <StevenK> In fact, my plan for that RT is we grow ci.l.n
[06:02] <StevenK> But I've not told lifeless of that plan, so I may get veto'd
[06:02] <wgrant> Under c.c, maybe.
[06:02] <lifeless> we shouldn't put things under the lp domain
[06:02] <lifeless> unless they are part of the lp offering
[06:02] <wgrant> But anything, even QA environments, under l.n is a bug.
[06:03] <StevenK> So lpci.c.c works?
[06:03] <lifeless> fbm
[06:03] <StevenK> But pqm is under l.n :-P
[06:03] <wgrant> that is a bug.
[06:03] <lifeless> or reuse either the pqm or buildbot sites so we don't have to buy another ssl cert
[06:04] <wgrant> lifeless: We have a *.canonical.com
[06:04] <lifeless> by reuse I mean proxypass a subpart, of course
[06:04] <wgrant> Most stuff uses that.
[06:04] <lifeless> wgrant: ah, I wasn't aware it was wildcarded
[06:04] <StevenK> I also dislike the buildbot name
[06:04] <wgrant> eg lpbuildbot uses it.
[06:05] <StevenK> I'd also like it to be open, but that's a whole other argument
[06:05] <wgrant> If we can tell production-devel to permanently die, it can remain public.
[06:06] <StevenK> production-devel hasn't been used in what, 4 months? 5?
[06:06] <lifeless> thats not a good indicator of need
[06:06] <lifeless> [given the use case for it]
[06:07] <StevenK> I don't think the branch is even updating anymore
[06:07] <lifeless> its not
[06:07] <StevenK> So we can't use it anyway?
[06:07] <lifeless> explictly updated as needed
[06:08] <StevenK> I'd rather it just die
[06:08] <wgrant> Everyone would.
[06:08] <wgrant> But there are still use cases.
[06:08] <StevenK> Which are security and what else?
[06:08] <wgrant> Nothing.
[06:09] <wgrant> Or a possibly super-urgent non-security fix, I guess.
[06:09] <StevenK> So, we cowboy a fix and then request a deploy over the top?
[06:09] <StevenK> (When it's landed and QA'd)
[06:09] <wgrant> That is what would have to replace it.
[06:09] <lifeless> ok, tracked down the person lookups
[06:09] <wgrant> lifeless: In the OOPS?
[06:09] <StevenK> wgrant: Oh, so it doesn't sound like crack? :-)
[06:09] <wgrant> Because the set on dev may be different.
[06:12] <lifeless> wgrant: its clear as daylight
[06:12] <wgrant> lifeless: Oh?
[06:12] <lifeless> and easy to fix too
[06:12] <lifeless> https://bugs.launchpad.net/launchpad/+bug/726370/comments/3
[06:12] <_mup_> Bug #726370: BugTask:+index timeout - late evaluation of Person/ValidPersonCache <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/726370 >
[06:12] <wgrant> Ahh
[06:13] <wgrant> That's not all of them, though.
[06:13] <wgrant> It will be most from that particular bug, I guess.
[06:13] <lifeless> it will be a big chunk
[06:13] <lifeless> up to 2 per row
[06:14] <wgrant> Right.
[06:14] <wgrant> Won't help bug #1 much at all, though.
[06:14] <_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
[06:15]  * StevenK wishes for bugtask deletion/hiding more
[06:15] <StevenK> RARGH, stop rebuilding the WADL!
[06:15] <wgrant> Why?
[06:15] <StevenK> wgrant: Because Bug 1 is drowning in them
[06:15] <_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
[06:15] <StevenK> Damn it
[06:15] <wgrant> Heh.
[06:16] <StevenK> WTB better Makefile, PST
[06:18] <lifeless> right, I think thats fixed
[06:18] <lifeless> quick smoke test
[06:18] <StevenK> In [12]: c.get_versions()
[06:18] <StevenK> Out[12]: []
[06:18] <wgrant> OK, looks like I have killed 1/3 of the copying queries.
[06:19]  * StevenK pouts at debian.changelog
[06:19] <wgrant> Oh?
[06:19] <lifeless> wgrant: have you used the DistroSeriesSet.getCurrentSourceReleases optimisation yet ?
[06:19] <wgrant> lifeless: No.
[06:19] <wgrant> lifeless: Not relevant here, is it?
[06:19] <wgrant> (do you want me to test your fix on DF?)
[06:19] <wgrant> actually, it's probably doing enough at the moment.
[06:20] <wgrant> Half way there.
[06:21] <lifeless> lp:~lifeless/launchpad/bug-726370 rev 12518
[06:24] <lifeless> wgrant: if you wanted to see if that helps,t hat would be great
[06:24] <lifeless> wgrant: the other thing to do is to break into pdb in https://bugs.launchpad.net/launchpad/+bug/726370/comments/4 and get the template details for that event
[06:24] <_mup_> Bug #726370: BugTask:+index timeout - late evaluation of Person/ValidPersonCache <timeout> <Launchpad itself:In Progress by lifeless> < https://launchpad.net/bugs/726370 >
[06:24] <wgrant> lifeless: Can't right now.
[06:24] <wgrant> The librarian is getting a bit of a workout.
[06:25] <lifeless> yeah
[06:25] <lifeless> queuing it up
[06:25] <wgrant> lifeless: That's probably an email address in a message.
[06:26] <wgrant> Yeah, there are a few of them.
[06:26] <wgrant> https://bugs.launchpad.net/ubuntu/+source/afflib/+bug/230350/+text
[06:26] <_mup_> Bug #230350: Missing Debian Maintainer field <Ubuntu:Invalid> <afflib (Ubuntu):Fix Released by saivann> <alac-decoder (Ubuntu):Fix Released by warp10> <axiom (Ubuntu):Invalid> <beneath-a-steel-sky (Ubuntu):Fix Released> <bibletime-i18n (Ubuntu):Fix Released by txwikinger> <binkd (Ubuntu):Fix Released> <bzr-builddeb (Ubuntu):Won't Fix> <capisuite (Ubuntu):Invalid> <chinput (Ubuntu):Fix Released by warp10> <chmsee (Ubuntu):Fix Released> <ciso (U
[06:26] <lifeless> shhh mup, sssh
[06:27] <wgrant> At least it got truncated!
[06:28]  * lifeless gambles that this cannot possibly break tests.
[06:28] <wgrant> lifeless: ec2 is only 3.5 hours now, but go ahead.
[06:28] <lifeless> do as I say, not as I do.
[06:28] <wgrant> We may also need to preload BugActivity actors, but we might already do that.
[06:29] <lifeless> don't think so
[06:29] <lifeless> theres something triggering TP lookups too
[06:29] <wgrant> That's probably permission checking.
[06:29] <lifeless> I suspect another task row thing
[06:29] <lifeless> wgrant: [duh] :P
[06:29] <wgrant> Also most of that here should be archivepermissions.
[06:29] <wgrant> Yet I don't see many of them.
[06:30] <wgrant> Ah, of course.
[06:30] <wgrant> Because I am a component uploader for most of them.
[06:30] <wgrant> Hm, no, no archivepermission queries at all.
[06:30] <wgrant> Huh.
[06:30] <wgrant> Maybe because I'm an admin.
[06:32] <stub> This must be the first good storm of the year. My office roof is leaks :-P
[06:35] <lifeless> oh right, its thursday
[06:35] <StevenK> stub: Your office roof is one big leak?
[06:35] <lifeless> StevenK: he's hosting a wiki
[06:35] <StevenK> Haha
[06:36] <wgrant> lifeless: How aggressively do you tend to test your preloading?
[06:36] <StevenK>  lp.registry.tests.test_distroseriesdifference.DistroSeriesDifferenceTestCase.test_base_version_multiple
[06:36] <StevenK>   Ran 1 tests with 0 failures and 0 errors in 1.268 seconds.
[06:36] <StevenK> SUCCESS!
[06:36] <wgrant> eg. I've adjusted getBuiltBinaries to optionally grab BPFs and LFAs too.
[06:36] <StevenK> wgrant: ^ With changelog parsing goodness
[06:37] <wgrant> StevenK: Excellent!
[06:37] <stub> Great. My roof leaks and I'm talking tinglish.
[06:37] <wgrant> StevenK: Do you want to coerce a LOSA into getting the populator running on prod at some point?
[06:37] <StevenK> Didn't we try that?
[06:37] <wgrant> Never on prod.
[06:37] <wgrant> I discussed it with spm, but nothing ever happened.
[06:37] <StevenK> wgrant: I suspect the best answer is 'file an RT'
[06:38] <wgrant> Possibly, but it's your squad now :P
[06:38] <StevenK> I suspect the script has bitrotted some
[06:38] <wgrant> Shouldn't be much.
[06:38] <wgrant> I'd be surprised if it was more than import locations.
[06:38] <lifeless> wgrant: I tend to not want to overlap unit tests
[06:39] <StevenK> wgrant: And uh, DB user
[06:39] <lifeless> wgrant: e.g. a single smoke test pushing all the scaling buttons and make sure its flat
[06:39]  * StevenK prods at it
[06:39] <wgrant> StevenK: True.
[06:39] <lifeless> wgrant: depends on the layer
[06:39] <wgrant> lifeless: They are all doctests at the moment, which I might replace.
[06:44] <StevenK> lifeless: Thoughts on getting an out-of-tree script run against prod?
[06:44] <lifeless> what sort of script ?
[06:45] <wgrant> One-off to populate SPR.changelog for old rows.
[06:45] <lifeless> lplib? model?
[06:45] <StevenK> Model
[06:45] <lifeless> how long will it take to run ?
[06:45] <lifeless> minutes? hours? days?
[06:45] <wgrant> That's not entirely clear. Possibly a few days.
[06:46] <lifeless> in tree, garbo job.
[06:46] <StevenK> That's a good idea, actually
[06:46] <wgrant> Easy enough to port the existing one to that.
[06:47] <StevenK> wgrant: What are you doing to mawson?
[06:47] <StevenK> Er. Maybe that was me
[06:47] <wgrant> It's me.
[06:47] <StevenK> Fess up
[06:51] <stub> to populate SPR.changelog, do we need to download packages and unpack them? Or pull things from the Librarian?
[06:52] <wgrant> StevenK: Download and unpack packages, then upload a file to the librarian.
[06:52] <StevenK> The former
[06:52] <wgrant> stub: ^^
[06:53] <wgrant> stub: Download packages from the librarian, that is.
[06:54] <stub> So garbo can do that, but need to confirm with the losas that it a) has access to the librarian b) there is suitable disk space in $TMP (if suitable might be large).
[06:55] <lifeless> stub: it runs on loganberry, right ?
[06:55] <lifeless> stub: main processor runs there IIRC - and it stashes raw mails in the librarian
[06:55] <wgrant> loganberry was having disk space issues, but I believe they're resolved now.
[06:55] <wgrant> And the largest package is only a few hundred megabytes.
[06:55] <stub> Not an ideal fit though since this will almost certainly be running with maxchunksize=1
[06:55] <wgrant> stub: What is the target transaction length?
[06:56] <stub> Normally about 4 seconds
[06:56] <StevenK> I'm not happy with the query wgrant wrote for it, either
[06:56] <wgrant> StevenK: Oh?
[06:57] <StevenK> The first query took 65 *minutes*
[06:57] <wgrant> The new one is fine!
[06:57] <stub> But in this case that is likely irrelevant - just set the maximum chunk size to 1 and do one at a time.
[06:57] <lifeless> yup
[06:57] <lifeless> also set a limit on the query of 5
[06:57] <lifeless> or 50 or whatever
[06:57] <StevenK> And make sure it cleans up
[06:58] <StevenK> Or the LOSAs will be unhappy
[06:58] <lifeless> naturally :)
[06:59] <stub> Query to determine work to do takes 65 minutes? better hope that is just a minute or two with a LIMIT or garbo timeouts will kick in and no iterations will ever run.
[06:59] <StevenK> stub: If you can't see any issues with it, I'll start ripping the script apart
[06:59] <StevenK> stub: The query has been fixed since then :-)
[06:59] <stub> ok :)
[06:59] <stub> Garbo hourly jobs are currently stopped if they run more than 10 minutes. Daily I think can run for 1 hour.
[07:00] <stub> A stand alone script may well make more sense in this case.
[07:00] <wgrant> It's already a looptuner.
[07:00] <lifeless> stub: 15 minutes, not 10.
[07:00] <wgrant> aaaa PQM Is fast.
[07:01] <stub> wgrant: It is a few lines to convert that to a stand alone script, which can then run 24x7 until it completes. More work for the losas though.
[07:02] <lifeless> stub: I suggested garbo
[07:02] <wgrant> stub: It is *already* a standalone script.
[07:02] <wgrant> lifeless said garbo.
[07:02] <lifeless> stub: because it will need less losa work, and thats a massive bottleneck for us right now.
[07:02] <stub> Right. If the caveats I mentioned don't raise alarm bells, no problem.
[07:02] <StevenK> I'm happy enough to follow IBug.reindex's lead and do it as a garbo job
[07:09] <StevenK> steven@liquified:~/launchpad/lp-branches/populate-spr-changelogs% pyflakes lib/lp/scripts/garbo.py | wc -l
[07:09] <StevenK> 34
[07:09]  * StevenK sobs
[08:52] <adeuring> good morning
[08:54] <maxb> gosh, it's a novelty to see time-based greetings on IRC which actually match your local timezone :-)
[08:54]  * maxb is on a bus to work
[09:12] <bigjools> morning all
[09:15] <wgrant> Morning bigjools.
[09:20] <lifeless> stub: shall we have a brief call ?
[09:23] <stub> lifeless: Gimme 2 mins
[09:23] <lifeless> thumper: ping - care to see if you've fixed bug 604654 in passing ?
[09:23] <_mup_> Bug #604654: save a roundtrip on POSTs <lp-foundations> <Launchpad itself:Incomplete> < https://launchpad.net/bugs/604654 >
[09:25] <mrevell> Hello
[09:28] <lifeless> stub: sure
[09:29] <stub> lifeless: back
[09:32] <lifeless> stub: skype?
[09:38] <wgrant> allenap: Morning.
[09:38] <allenap> wgrant: Morning :)
[09:38] <wgrant> allenap: Could you please review https://code.launchpad.net/~wgrant/launchpad/bug-728246-copy-expiry-queries/+merge/52020?
[09:38] <allenap> wgrant: With pleasure.
[09:40] <wgrant> allenap: Thanks.
[09:43] <lifeless> stub: https://bugs.launchpad.net/launchpad/+bug/727560 - as an example of a nonbugs query
[09:44] <_mup_> Bug #727560: Archive:EntryResource:getPublishedSources <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/727560 >
[09:44] <lifeless> stub: 5 seconds to grab the first batch *after* removing unnecessary joins
[09:49] <stub> lifeless: You are still broken up
[09:49] <stub> ha
[09:53] <lifeless> https://launchpad.net/oops-repository
[10:29] <wgrant> Are recipes for +junk supported?
[10:29] <wgrant> They broke last week.
[10:56] <gmb> allenap: Morning. I have a branch up for review if you've got the time: https://code.launchpad.net/~gmb/launchpad/prevent-bnl-nothing-oopses-bug-721400/+merge/52036
[10:56] <allenap> gmb: Sure :)
[10:56] <gmb> Ta
[10:56] <allenap> I'm still on wgrant's right now, but I'll try and get a move on.
[10:57] <gmb> No massive rush.
[10:57] <gmb> But puppies die for every half hour that this review takes.
[10:57] <gmb> And I have a finite supply
[11:12] <wgrant> adeuring: Hi.
[11:12] <adeuring> hi wgrant
[11:12] <wgrant> adeuring: Bug #705176 is blocking rollouts at the moment. It's marked [incr], but qa-tagger wants QA tags and I wonder if there is any QA that can be done?
[11:12] <_mup_> Bug #705176: Improve sharing information on translation pages <upstream-translations-sharing> <Launchpad itself:In Progress by henninge> < https://launchpad.net/bugs/705176 >
[11:13] <adeuring> let me check...
[11:55] <wgrant> adeuring: Thanks.
[11:56] <allenap> gmb: Line 43, you start a feature_flags() context, but flags are not changed. Is it needed?
[11:56] <wgrant> allenap: Also thanks.
[11:56] <gmb> allenap: Yes. If you don't do it in the feature flags context it will pay no attention to the subscriptions bug_notification_level (the field just doesn't exist if malone.advanced-subscriptions.enabled isn't True.
[11:57] <allenap> wgrant: Sorry I took so long. I have been very distracted this morning talking unittest with one of the server team :)
[11:57] <wgrant> allenap: No worries.
[11:58] <allenap> gmb: How does the feature_flags() context know to configure malone.advanced-subscriptions.enabled? Honest question, feature flags still baffle me.
[12:00] <gmb> allenap: It's added in the test setUp().
[12:01] <gmb> allenap: The feature_flags() wossname just (re)populates the feature flags stuff; if you don't have that context in place then all feature flags are off.
[12:02] <allenap> gmb: Right, that's bonkers, but okay :)
[12:03] <gmb> allenap: A bit, yes. I think it's because they're tied to the request somehow, but frankly it's all a bit voodoo to me.
[12:05] <gmb> allenap: Ta for the review.
[12:07]  * gmb lunches
[12:21] <jam> I just got a bug post spam message. I thought I saw something about making it easier to hide them.
[12:21] <jam> Can anyone help with: https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/540280/comments/17
[12:21] <_mup_> Bug #540280: openssh (or gvfs) asks for a password everytime I change directory in sftp <lucid> <regression-release> <bzr-gtk (Ubuntu):Triaged> <gvfs (Ubuntu):Triaged> < https://launchpad.net/bugs/540280 >
[12:42] <stub> Should database model objects for the session database tables go in lib/lp/services/session/model.py or lib/lp/services/session/model/session.py ?
[12:43] <stub> or lib/lp/services/model/session.py?
[12:47] <jam> quick process question, if I want to update a sourcecode dependency, do I just update utilities/sourcecode.conf and run utilities/update-sourcecode? Is there anything else that should be done other than submit that 1-line change for review?
[12:51] <stub> jam: Think that is it. Maybe let the mailing list know so people think of it when they see the inevitable weird error messages.
[12:52] <jam> thanks, stub
[13:47] <danilos> mrevell, hi
[13:52] <mrevell> hey danilos
[13:56] <danilos> mrevell, hey-hey
[13:56] <danilos> mrevell, I want you to do stuff for me — will you? :)
[13:56] <mrevell> danilos, I will, will. What do you want?
[13:56] <danilos> mrevell, (I hope you are not going to ask what it is now :)
[13:56] <danilos> I knew it!
[13:56] <mrevell> hah
[13:56] <mrevell> :)
[13:57] <danilos> mrevell, heh, so, I was going through the mockups on https://dev.launchpad.net/LEP/BetterBugSubscriptionsAndNotifications/Testing/EditingRound2 and gary_poster forwarded me your commentary as well
[13:57] <danilos> mrevell, however, I found a few other cases which are not covered in the mock-ups that I want to implement, and I am going to be missing the text for them
[13:57] <danilos> mrevell, I wrote it all down on https://dev.launchpad.net/LEP/BetterBugSubscriptionsAndNotifications/DirectSubscriptionsOnBug
[13:57]  * mrevell looks
[13:58] <mrevell> Okay Daniel, how can I help-?
[13:58] <danilos> mrevell, now, I'd be happy if you can come up with descriptions for every of the "potential actions" in there
[13:58] <danilos> mrevell, well, both ACTION and DESCRIPTIONs
[13:59] <danilos> mrevell, no need to do it for those which are already covered in user testing and existing mockups
[13:59] <danilos> mrevell, (just haven't extracted that yet)
[13:59] <mrevell> danilos, Where ACTION is the link text and DESCRIPTION is some kind of tool-tip?
[14:00] <danilos> mrevell, exactly
[14:00] <mrevell> No problem. When do you need it?
[14:00] <danilos> mrevell, I think stuff that is missing is mostly team-related and supervisor related
[14:00] <danilos> mrevell, by yesterday, but tomorrow morning is fine as well :)
[14:01] <danilos> mrevell, of course, if you are busy, I'll probably come up with stuff myself since I am away next week, and let others fix it :)
[14:01] <mrevell> Heh. Hmm .... okay, I have a couple of other things to do this afternoon but I can fit this in too. I'll have something with you in the next few hours.
[14:01] <cr3> hi folks, congrats on taking a whole second off total time of all launchpad pages!
[14:03] <danilos> cr3, wasn't me, I promise!
[14:05] <cr3> danilos: I hear it was IS this time around, do you happen to know if it would be possible to know more about what they did to the load balancer?
[14:06] <danilos> cr3, I believe lifeless and LOSAs know all about it, and that's where the thanks should go
[14:08] <cr3> danilos: the LOSAs are an elusive bunch though, so I thought of thanking the whole launchpad-dev folks in the hope it would include LOSAs too
[14:09] <danilos> cr3, heh, I think they don't want to be considered part of LP team because I am part of the LP team: who'd want to be grouped together with me?
[14:09] <maxb> Speaking thereof, are the LOSAs back from their sprint yet? I've had a fairly trivial LOSA-needs-to-run-stuff question open for ages :-/
[14:09] <cr3> "I don't care to belong to any club that will have me as a member" -- Groucho Marx
[14:10] <gary_poster> maxb, it doesn't happen to be a SQL-against-staging thing, does it?
[14:10] <maxb> no, it's a branch stacking fixup thingy
[14:10] <maxb> they have a script, they just need to execute it
[14:11] <gary_poster> ack maxb, can't help then
[14:33] <adeuring> allenap, jcsackett: could one of you please review this MP: https://code.launchpad.net/~adeuring/launchpad/bug-681434/+merge/52071 ?
[14:33] <jcsackett> adeuring: i'm looking one MP right now, but it's short. happy to look at that in a moment.
[14:33] <adeuring> jcsackett: thanks!
[14:36] <sinzui> allenap: jcsackett: will either you have time to review https://code.launchpad.net/~sinzui/launchpad/lookup-bug-redirect-0/+merge/51978 today
[14:36] <jcsackett> sinzui: just did it.
[14:37] <jcsackett> it was the short one previously mentioned. :-)
[14:37] <sinzui> I see
[14:37] <sinzui> never mind
[14:37] <sinzui> thanks
[14:37] <jcsackett> your welcome. :-)
[14:37] <deryck> sinzui, hey, I haven't had time to think long about your email due to sprint, but I think your assessment of those rules seems right to me.
[14:38] <deryck> sinzui, maybe gmb or allenap could comment more fully if you need.
[14:38] <sinzui> deryck: thanks for reminding me to follow that up
[14:38] <allenap> deryck, sinzui: Actually, I meant to comment on that already. I'll do that now.
[14:38] <deryck> awesome.  Thanks, allenap!
[14:51] <adeuring> jcsackett: thanks for your review!
[14:52] <jcsackett> adeuring: welcome. sorry i forgot to ping you i was done. :-)
[14:52] <adeuring> jcsackett: no problem :)
[15:17] <m4n1sh> need some clarification related to PPA terms of service
[15:21] <m4n1sh> if I have a library which is not packaged
[15:21] <m4n1sh> but still I want to use it in my application
[15:21] <m4n1sh> I am including it in the source tree
[15:21] <m4n1sh> the source is available
[15:21] <m4n1sh> but the source contains the blob
[15:22] <m4n1sh> if i package it and put it in PPA, with a link to the actual source, would it be against ToS?
[15:25] <leonardr> allenap or jcsackett, i request a review of my awesome branch: https://code.launchpad.net/~leonardr/launchpad/bug-106338/+merge/52085
[15:26] <allenap> leonardr: Sure, I'll look at it in a bit :)
[15:27] <m4n1sh> leonardr: can you answer this ToS question? I am not very sure
[15:27] <leonardr> m4n1sh: sorry, i don't know
[15:27] <mrevell> m4n1sh, Hi, I may be able to help
[15:27] <m4n1sh> anyone can?
[15:28] <m4n1sh> mrevell: yeah, Please
[15:29] <bigjools> what sort of blob?
[15:29] <mrevell> m4n1sh, What's the licence of the blob?
[15:29] <mrevell> And why can't you include it as source?
[15:29] <m4n1sh> I can
[15:29] <bigjools> add it as a depenency in another package
[15:29] <m4n1sh> but again it might be replicated in other appls too
[15:29] <m4n1sh> bigjools: the project newtonsoft-json isn't package
[15:29] <m4n1sh> *packaged in debian
[15:29] <m4n1sh> or anywhere
[15:30] <m4n1sh> and I wanted to use the library quickly without adding the whole gigantic source code in my project
[15:30] <m4n1sh> the license of newtonsoft-json is MIT
[15:30] <bigjools> you must include sources
[15:32] <mrevell> m4n1sh, If you don't want to include the library's source in your package, which is probably a good idea, then you'll need to package it separately.
[15:32] <mrevell> and, as bigjools says, add it as a dependency.
[15:32] <m4n1sh> yeah, which means that increases the work
[15:32] <bigjools> if you provide a binary and no source, it's violating the licence isn't it?  I don't know what MIT says.
[15:33] <m4n1sh> MIT is permissive
[15:33] <m4n1sh> can be closed
[15:33] <bigjools> ok
[15:33] <m4n1sh> but the thing is that including a blob in open source code is looked down upon
[15:33] <bigjools> I'm not entirely sure of the PPA ToS here
[15:33] <m4n1sh> and might be against ToS
[15:34] <m4n1sh> I just wanted to be sure, why do all these stupidity if it can be avoided :)
[15:43] <allenap> leonardr: The mp says "Text conflict in lib/lp/bugs/tests/test_bugs_webservice.py".
[15:43] <leonardr> allenap: ok, i also got some test failures, so i will ping you when i have everything resolved
[15:44] <allenap> leonardr: Cool.
[16:40] <leonardr> allenap, try now
[16:41] <allenap> leonardr: Will do.
[16:49] <leonardr> allenap: now pushing a small additional revision that fixes bug 728507
[16:49] <_mup_> Bug #728507: Expose classes of exceptions with webservice_error() instead of calling export() on each instance. <api> <tech-debt> <Launchpad itself:New> < https://launchpad.net/bugs/728507 >
[16:59] <allenap> leonardr: All looks good.
[16:59] <leonardr> allenap: just to make sure, did you get r12504?
[16:59] <allenap> leonardr: Yep.
[17:00] <leonardr> great
[17:22] <jcsackett> allenap: can you take a look at https://code.launchpad.net/~jcsackett/launchpad/spam-messages-visibility/+merge/52097?
[17:22] <jcsackett> or are you swamped?
[17:23] <allenap> jcsackett: I'm not going to be able to get to it until I return after 2000 UTC.
[17:24] <jcsackett> allenap: that's alright; it needs a db review too, so it won't move out of review till tomorrow.
[17:24] <jcsackett> would you rather i hunt down someone else?
[17:24] <allenap> jcsackett: Okay, I'll look later.
[17:24] <jcsackett> allenap: fantastic. thanks. :-)
[17:28] <rvba> allenap: can you take a look at https://code.launchpad.net/~rvb/launchpad/fix-missing-add-distro-link/+merge/52104?
[17:29] <allenap> rvba: I don't think I'll have time today. jcsackett, could you take that one?
[17:29] <rvba> allenap: nothing urgent really, and I'm off for today too
[17:29] <jcsackett> allenap, rvba: sure, i've got it.
[17:30] <rvba> jcsackett: all right then, I'll see your comments tomorrow morning then ;-)
[17:30] <jcsackett> rvba: indeed. have a good evening. :-)
[17:32] <mtaylor> whatever happenend to merge queues
[17:32] <allenap> rvba: Cool. Fwiw, when asking for a review from the on-call reviewer there is usually an expectation that you'll be around for a while to answer questions. If you can't be around, mention it; most reviewers will take the proposal anyway, but it's worth checking.
[17:32] <rvba> allenap: good to know
[18:45] <lifeless> sinzui: ping
[18:46] <lifeless> bug 728609 - can you guys take that on? its an interrupt from oem
[18:46] <_mup_> Bug #728609: Elements are not closed on bug page source <escalated> <oem-services> <Launchpad itself:Triaged> < https://launchpad.net/bugs/728609 >
[18:50] <sinzui> lifeless:  crack markup?
[18:51] <lifeless> link elements are not closed, xml fail for greasemonkey
[18:51] <sinzui> I will fix this now
[18:51] <lifeless> jamesf in -ops reported it
[18:55] <lifeless> sinzui: how do we hide messages? https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/540280/comments/17
[18:55] <_mup_> Bug #540280: openssh (or gvfs) asks for a password everytime I change directory in sftp <lucid> <regression-release> <bzr-gtk (Ubuntu):Triaged> <gvfs (Ubuntu):Triaged> < https://launchpad.net/bugs/540280 >
[18:57] <jcsackett> leonardr: ping.
[18:57] <leonardr> jcsackett, hi
[18:58] <jcsackett> i just saw you linked a branch to 106338; this is a full fix to the bug?
[18:59] <jcsackett> sinzui asked me to look at that other day, so i assigned myself. i've done a little work but yours looks like it covers the issue.
[18:59] <jcsackett> leonardr ^
[18:59] <leonardr> jcsackett: yes, it's a full fix
[19:00] <jcsackett> leonardr: cool. i'll unassign myself and assign you. thanks.
[19:00] <leonardr> i'm now working on similar bugs like 345904
[19:01] <jcsackett> leonardr: dig.
[19:05] <lifeless> gmb: yo
[19:05] <lifeless> gmb: your subscription form should be loading more snappily now
[19:17] <leonardr> lifeless: i'm considering having lazr.restful treat all IntegrityError exceptions as 400 ("Bad Request"). do you think there are situations in which an OOPS would be appropriate, instead?
[19:20] <lifeless> leonardr: IntegriterError's get retried by the publisher
[19:20] <lifeless> messing with them is almost certainly a mistake
[19:20] <lifeless> they are a routine part of working with our DB
[19:21] <leonardr> lifeless: i'm not proposing messing with them in any way that inteferes with the normal usage of them
[19:21] <leonardr> i'm trying to fix bug 345904
[19:21] <_mup_> Bug #345904: API: IntegrityError when trying to create a newTask to a bug that already has one for that project <api> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/345904 >
[19:21] <leonardr> which is that, once the integrity error has been retried and retried, it's raised to the client
[19:21] <leonardr> and becomes an OOPS
[19:22] <leonardr> in the case of 345904, the integrityerror was caused by the end-user, so an oops is not appropriate
[19:22] <lifeless> in the normal web ui
[19:22] <lifeless> the user would also expect a nice error
[19:22] <jkakar> FYI, I'm getting repeated OOPS emails related to branch that doesn't seem to be rebuilding its diff.  The latest one is: OOPS-1888MPJ140
[19:22] <jkakar> I've been received one every 10 minutes for the last 70 minutes.
[19:23] <lifeless> so my immediate reaction is to ask 'why does the web service not do the same thing'
[19:23] <lifeless> jkakar: thanks
[19:23] <jkakar> lifeless: np
[19:23] <lifeless> those oopses aren't syncing all that often, or something
[19:24] <leonardr> lifeless: perhaps some essential validation code is in the view when it shouldn't be
[19:24] <lifeless> leonardr: yes, thats what I suspect is the case.
[19:25] <lifeless> leonardr: it would be easy for this to be created by a user at the moment. But also
[19:25] <lifeless> leonardr: if two users add a task at the same time, we would get a mid flight collision, and it wouldn't be either users fault.
[19:26] <lifeless> leonardr: for *this specific* integrity error, it seems to me we should be handling this in e.g. model code, raising an appropriate error if the task is already there.
[19:31] <leonardr> ok
[19:38] <lifeless> the idea of a systemic fix is appealing
[19:39] <lifeless> but sadly I don't think the causes for integrity error are homogeneous enough
[19:49] <lifeless> leonardr: curtis has a lot of experience with validators
[19:49] <lifeless> leonardr: I suggest asking him about how to move that call from view to model
[19:49] <leonardr> k
[19:50] <leonardr> sinzui, let me know when you have a minute
[19:56] <abentley> leonardr: is there a less hacky way of getting Launchpad._root_uri ?
[19:57] <leonardr> abentley: what are you trying to do?
[19:58] <abentley> leonardr: I am trying to create a WebServiceTestRequest with the correct root URI: WebServiceTestRequest(SERVER_URL=str(launchpad._root_uri))
[19:59] <leonardr> to answer your immediate question, i think ._root_uri is the simplest way to get the root uri
[19:59] <abentley> leonardr: so that I can get web service url with canonical_url.
[19:59] <leonardr> abentley: you might like the code i'm about to land
[20:00] <leonardr> what do you want to do with the web service url?
[20:00] <abentley> leonardr: so that I can do Launchpad.load() on that URL to get the restfulclient version of that object.
[20:00] <leonardr> yes, you will like this code
[20:00] <leonardr> in the meantime, you can pass force_local_path=True into canonical_url
[20:00] <leonardr> that will give you a relative url, which you can pass into load()
[20:01] <leonardr> my branch adds api_url to lib.lp.testing
[20:03] <abentley> leonardr: Cool.  We may hit conflicts if you've updated ws_object.
[20:03] <abentley> I was updating it, too.
[20:04] <leonardr> abentley: what is ws_object? sounds like something i should be aware of
[20:04] <abentley> leonardr: it accepts a Launchpad model object and returns the corresponding lazr.resfulclient object.
[20:06] <leonardr> abentley: i didn't know about that--we should change it to use api_url
[20:06] <abentley> leonardr: my thoughts exactly.
[20:15] <_thumper_> sinzui: ping
[20:17] <thumper> bac: ping?
[20:17] <bac> thumper: on the phone
[20:18] <thumper> bac: ack
[20:18]  * thumper smirks
[20:18] <thumper> anyone familiar with our release file upload limit of 200 meg?
[20:18] <thumper> my understanding is that it is per file limit only
[20:18] <thumper> and we don't have a per-project limit on uploaded files, is that right?
[20:19] <allenap> jcsackett: I'm not going to get to your review this evening. Would you like me to do it in the morning (i.e. before you come online)?
[20:19] <jcsackett> allenap: that would be fine. thanks for offering. :-)
[20:20] <allenap> jcsackett: Cool :)
[20:20] <sinzui> hi thumper, leonardr
[20:20] <leonardr> sinzui, take a look at https://bugs.launchpad.net/launchpad/+bug/345904
[20:20] <thumper> sinzui: hi, do you know the answer to my question above?
[20:20] <_mup_> Bug #345904: API: IntegrityError when trying to create a newTask to a bug that already has one for that project <api> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/345904 >
[20:23] <bac> thumper: i agree with what you said
[20:23] <sinzui> thumper: released uploaded are not actually limited we observer lp/network/user fails at about 200 meg. Some users have uploaded 640meg, other cannot get 20 megs uploaded
[20:23] <thumper> bac: ok, thanks
[20:23] <thumper> sinzui: ok, thanks too
[20:24] <bac> sinzui: is that true?  i thought we did have a hard limit, even though a lot of time there are time outs before people can push a file of that size
[20:24] <bac> there was a hard limit the last time i worked on it
[20:24] <sinzui> leonardr: the issue here is that the method/factory that tried to create a duplicate bugtask for a product without first checking that it already exists
[20:25] <leonardr> sinzui: right. the validation code that should be in the model is in the view
[20:25] <leonardr> and i don't know enough to move it
[20:27]  * bac reboots. bbiab
[20:29] <sinzui> leonardr: I think this is a simple check to look for the same target among the existing bugtasks. the view is doing this. I think we need to push the check into the model and have the view call it in validate() and addTask() call it.
[20:29] <sinzui> leonardr: I need to get my children from school. I can help after I find the code you are looking at
[20:30] <leonardr> sinzui: the method that does the validation is valid_upstreamtask. so i should explicitly call that in addTask, and raise an exception if it's false?
[20:33] <sinzui> thumper: lifeless: the invalid markup on bug pages comes from the LP.cache js in the page. We cannot place xhtml into a jsblock without <![CDATA[xxx]]>. Recall that the TALES engine goes belly up when we place CDATA instructions in the page
[20:34] <sinzui> leonardr: yes, raise an exception in that method is right.
[20:34] <thumper> sinzui: we aren't placing xhtml into a jsblock are we?
[20:34] <thumper> sinzui: we aren't there...
[20:35] <thumper> sinzui: ah... you mean the bit at the bottom
[20:35] <thumper> sinzui: where we added the xhtml representation
[20:35] <thumper> sinzui: do you recall why TALES barfs?
[20:37] <lifeless> sinzui: thumper: did that change in the last couple of days?
[20:37] <lifeless> this is apparently a regression
[20:37] <thumper> lifeless: it changed at the end of last week I think
[20:37] <lifeless> then yeah, that fits - we had a rough couple of deploys
[20:37] <leonardr> sinzui: ok, i thought it had to go into some special validation code
[20:38] <lifeless> why do we include xhtml in the initial json on the page? Won't it be redundant with the initial page state?
[20:38] <thumper> sinzui: don't we do something like // <![CDATA[
[20:38] <thumper> sinzui: ... // ]>
[20:53] <allenap> sinzui, thumper: simplejson.encoder.JSONEncoderForHTML.
[20:54] <thumper> allenap: which does what?
[20:54] <sinzui> thumper: we do not do that because the tal process died when we tried
[20:54] <allenap> thumper: It also escapes <>& in json.
[20:55] <thumper> allenap: we'd need to then unescape it when we want to use it
[20:55] <thumper> but that may be a sane approach
[20:55] <allenap> thumper: No. It escapes into javascript escapes, i.e. < becomes \\u003c
[20:55] <allenap> So it works out of the box.
[20:56] <thumper> allenap: ah...
[20:56] <thumper> sinzui: since you've claimed the bug, want to try it?
[20:56] <sinzui> allenap: I investigated that lat last year and discovered Lp has an old version
[20:56] <sinzui> maybe I remember incorrectly
[20:57] <allenap> sinzui: I think you're probably right. The code is very simple though, so even if we don't upgrade simplejson, we can use that class.
[20:57] <allenap> Bug 684430.
[20:57] <_mup_> Bug #684430: Use JSONEncoderForHTML when preparing JSON fragments for pages <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/684430 >
[20:57] <sinzui> yes
[20:58]  * jcsackett has very sad internet connection
[20:59] <thumper> I got a bunch of TestPoppy errors from ec2
[20:59] <thumper> wisted.internet.error.CannotListenError: Couldn't listen on any:2121: [Errno 98] Address already in use.
[21:00] <thumper> anyone else getting these?
[21:01] <lifeless> thumper: why do we put xhtml in the initial page render ?
[21:01] <lifeless> thumper: I understand why we optionally send it in api responses
[21:01] <thumper> lifeless: because it is part of the entry code
[21:02] <lifeless> thumper: but does it serve any value in the initial render ?
[21:02] <thumper> lifeless: I'm not entirely sure...
[21:02] <thumper> lifeless: not as far as I'm aware
[21:02] <lifeless> thumper: I thought it needed to be explicity requested ?
[21:02] <thumper> the initial one has it
[21:02] <leonardr> thumper: i don't think it's useful now, but the same code that updates the page when that html data changes can be used to create the _initial_ state of the page
[21:02] <leonardr> if oyu're talking about what i think you are
[21:03] <lifeless> leonardr: but we ship a valid initial state of the page for non-javascript clients anyway
[21:03] <lifeless> (or am I wrong?)
[21:04] <leonardr> lifeless: you're not wrong
[21:04] <leonardr> sinzui: does this make any sense? http://pastebin.ubuntu.com/575180/
[21:06] <wallyworld> leonardr: thumper: standup?
[21:06] <thumper> aye
[21:09] <leonardr> wallyworld: @operation_for_version('devel')
[21:09] <leonardr> at the bottom of the annotations
[21:11] <gary_poster> allenap, jcsackett, I have a large branch up for review if you are willing.  https://code.launchpad.net/~gary/launchpad/bug723999/+merge/52133
[21:12] <jcsackett> gary_poster: i believe allenap is out for the day, so that pretty well leaves me. :-)
[21:13] <gary_poster> ok jcsackett, pretty please then? :-)
[21:13]  * jcsackett looks
[21:13]  * jcsackett whistles
[21:13] <jcsackett> over 1000 loc, eh? you weren't kidding. :-p
[21:14] <gary_poster> :-D
[21:14] <jcsackett> nearly half deletes tho. i like that.
[21:15] <gary_poster> and lots of comments!
[21:15] <sinzui> leonardr: that looks right. I expected the assert to be self.assertWithContent()
[21:17] <leonardr> sinzui: does that require an exact match? the string includes the random product name
[21:18] <sinzui> it does require an exact match. You can interpolate the random name, or just create the product with name='stoat'
[21:19] <sinzui> We do not have enough weasel family members in our tests
[21:20] <sinzui> otter, mink, stoat, badger, skunk, dotson.
[21:24] <LPCIBot> Project devel build #500: FAILURE in 5 hr 11 min: https://hudson.wedontsleep.org/job/devel/500/
[21:25] <lifeless> thumper: sinzui: leonardr: so can we fix this by removing the xhtml in the initial load? smaller initial page, less render overhead
[21:25] <leonardr> lifeless: yes, i think so, since i doubt we have any code depending on that xhtml right now
[21:26] <lifeless> how do we make that happen ?
[21:27] <leonardr> lifeless: i think we undo a small change i made to lazr.restful
[21:27] <lifeless> leonardr: cool
[21:28] <leonardr> lifeless: yes, we change WebLayerAPI.json to ask for JSON_TYPE, not JSON_PLUS_XHTML_TYPE
[21:28] <lifeless> sinzui: ^
[21:29] <leonardr> sinzui: it looks like assertRaisesWithContent doesn't take keyword arguments, the way assertRaises does. that cna't be right, can it?
[21:30] <sinzui> leonardr: I am shocked if it does not
[21:30] <leonardr> sinzui: TypeError: assertRaisesWithContent() got an unexpected keyword argument 'target'
[21:30] <lifeless> you might like the testtools MatchesException matcher
[21:31] <lifeless> also
[21:31] <lifeless> assertRaises returns the exception
[21:31] <lifeless> so you can write
[21:31] <lifeless> self.assertEqual('foo bar', self.assertRaises(...).args[0])
[21:32] <leonardr> that's useful
[21:32] <lifeless> I generally use the matcher based approach now though
[21:32] <lifeless> its pretty cool
[21:33] <thumper> how do you use a matcher with raises?
[21:34] <lifeless> the Raises matcher
[21:34] <lifeless> you the thing being matched needs to be no-arg callable
[21:35] <lifeless> e.g.
[21:36] <sinzui> leonardr: I cannot find WebLayerAPI in the lp tree. can you give me a clue
[21:36] <leonardr> sinzui: it's in lazr.restful
[21:36] <lifeless> self.assertThat(partial(callable, target='bar'), Raises(MatchesException(AssertionError('the target must be foo')))
[21:36] <leonardr> src/lazr/restful/tales.py
[21:36] <lifeless> or
[21:36] <lifeless> self.assertThat(partial(callable, target='bar'), Raises(MatchesException(AssertionError))
[21:36] <lifeless> if you only care about the type
[21:36] <lifeless> or
[21:36] <sinzui> ah out of my depth
[21:36] <lifeless> self.assertThat(partial(callable, target='bar'), Raises())
[21:37] <lifeless> if you only care about it raising /something/
[21:37] <lifeless> there is a helper 'raises' which makes this a little easier to spell - raises(foo ) -> Raises(MatchesException(foo))
[21:37] <leonardr> sinzui: i will make a branch for you to try
[21:38] <lifeless> so the first example is idiomatically spelt as
[21:38] <sinzui> I am pull it now
[21:38] <lifeless> self.assertThat(partial(callable, target='bar'), raises(AssertionError('the target must be foo')))
[21:38] <leonardr> sinzui: ok, just make that change, bump the minor version number, and mention it in the changelog
[21:40] <leonardr> lifeless: thumper and i realized that we meant to talk to you about web service versioning on tuesday and never did
[21:40] <leonardr> here's our proposal
[21:41] <leonardr> instead of numbering versions (which implies that a later version is a certain amount better than any given earlier version)
[21:41] <leonardr> we will name versions, similar to how ubuntu releases are named (which implies only that a later version is a certain number of releases later than an earlier version)
[21:42] <leonardr> so, instead of calling the next release 2.0, we might give it a 'c' name like 'cauliflower'
[21:42] <leonardr> we would require all new web service annotations to reference cauliflower
[21:43] <leonardr> devel would refer to cauliflower, but once we cut a new version ('dandelion'), devel would start pointing to that version
[21:43] <leonardr> this removes the need to think about which version you are really releasing in when you say 'devel'
[21:43] <leonardr> the need to do anything special to the annotations upon release
[21:44] <leonardr> and it lets us pick release names in advance without saying anything about the magnitude of the change in that release
[21:46] <leonardr> i'm going to work on something like this system on monday--we can go back and forth on the details without affecting the basic shape of the system
[21:48] <jcsackett> gary_poster: methods like getSubscriptionForBugTask on the StucturalSubscription mixin fall under the category of "things you left in for tests you think need to be cleaned out later", yes?
[21:49] <gary_poster> jcsackett pretty sure yes; will double check after I get off a call
[21:49] <jcsackett> gary_poster: gotcha.
[21:51] <wgrant> Has anybody looked at the buildbot failure?
[21:51] <lifeless> leonardr: would cauliflower be exposed at all outside our python source tree?
[21:52] <lifeless> leonardr: I don't really get the 'implies that a later version is a certain amount better than any given earlier version' bracket
[21:52] <wgrant> Has anybody looked at the failing script?
[21:53] <leonardr> lifeless: earlier you objected to the idea of saying '2.0' right at the start instead of saying 'devel' up to the moment of release, and then renaming everything to '2.0'
[21:53] <leonardr> i thought you were saying that we might regret calling it '2.0' due to the magnitude (or lack thereof) of the changes present in that version
[21:53] <lifeless> no
[21:53] <lifeless> uhm
[21:53] <lifeless> we don't want everything in devel to be in (1.0)+1
[21:53] <poolie> hi lifeless, wgrant, leonardr, everyone
[21:53] <lifeless> I object to trying to decide that at the start.
[21:54] <wgrant> Morning poolie.
[21:55] <leonardr> lifeless: so, we have two changes in devel, neither of which are in 1.0. we release 1.1, which contains change A, but change B is still only available in devel. is that what you mean?
[21:56] <lifeless> leonardr: yes
[21:56] <lifeless> leonardr: I think what goes in 1.1 needs to be an explicit decision. It's our point for saying 'A is stable, we are willing to commit to 5 years support of that'.
[21:56] <lifeless> leonardr: consider that B might have been in devel for 2 days when 1.1 is created.
[21:58] <leonardr> lifeless: in that case, we should not have a 'devel' at all. those two changes should be in different versions from the start
[21:59] <lifeless> leonardr: why? the question is - at what point does something we have exposed become something we are willing to support long term.
[21:59] <lifeless> leonardr: is it simple duration? is it whether we've signed off the work with jml ? is it whether its highly valued by the distro ?
[22:00] <lifeless> leonardr: I don't think there is a single, simple answer.
[22:00] <leonardr> lifeless: this may be too complex for the existing version system to handle
[22:01] <lifeless> leonardr: I think it handles it fine: we make a 1.1. We move the things we want to support to it. We announce it, and we shift the launchpadlib default version for the next ubuntu to 1.1.
[22:01] <leonardr> lifeless: so, at the point we release 1.1 we go through every change in devel and make the decision to backport or not?
[22:02] <lifeless> yes
[22:02] <lifeless> I think we'd want to do that even if we had a way of flagging them in advance.
[22:02] <lifeless> 5 year commitments are serious business :)
[22:03] <sinzui> leonardr: I am waiting for `setup.py test` to complete before I commit. I see you just landed a change. Do you want to review it. Do I just commit to trunk when the branch is approved
[22:04] <leonardr> sinzui: i recommend running bin/test instead of setup.py test
[22:05] <leonardr> lifeless: what does this mean for our current process? we require that all new annotations be explicitly flagged to devel?
[22:05] <sinzui> leonardr: I do not have a bin/test, but I see 4 failures
[22:05]  * sinzui reverts and runs the tests again
[22:05] <leonardr> sinzui: here's the process i use
[22:05] <leonardr> python bootstrap.py; bin/buildout; bin/test
[22:06] <sinzui> thanks
[22:06] <lifeless> leonardr: yes, I think thats all it means. Making the default devel would make this easier: unflagged would mean unreleased, flagged would mean released.
[22:07] <lifeless> leonardr: what we have is pretty good
[22:07] <leonardr> thumper: are you ok with this?
[22:07] <thumper> leonardr: yeah. If we want to make sure we explicitly choose what to release
[22:07] <thumper> then this seems ok
[22:07] <wgrant> What happened to build failures being a stop-the-line event? :(
[22:08] <jcsackett> wgrant: they're not? or do you mean in terms of people, not pqm/buildbot?
[22:08] <wgrant> jcsackett: In terms of people.
[22:09] <wgrant> lucid_db_lp failed more than 8 hours ago.
[22:09] <wgrant> With an obviously spurious failure.
[22:09] <jcsackett> y'know, i don't get many buildbot emails anymore.
[22:09]  * jcsackett makes not to see if a filter is accidentally grabbing them.
[22:10] <gary_poster> jcsackett, yes lib/lp/bugs/model/structuralsubscription.py getSubscriptionsForBugTask is a prime example of a now-useless function that only exists in that branch to keep test-munging out of the diff.
[22:11] <jcsackett> gary_poster: fantastic.
[22:11] <sinzui> ouch. look like I will be compiling for a while
[22:11] <jcsackett> incidentally, sorry this is taking so long. i'm having to route around a lot of network issues on my end.
[22:11] <gary_poster> np, it was giganto branch anyway jcsackett.  You'll only get thanks from me.
[22:12] <wgrant> lifeless: Is https://hudson.wedontsleep.org/job/devel/500/testReport/junit/lp.registry.browser.tests.test_milestone/TestDistributionMilestoneIndexQueryCount/test_bugtasks_queries/ one of yours?
[22:12] <wgrant> lifeless: The third query (Person, between the TeamParticipations) is intermittent.
[22:13] <leonardr> sinzui: you need a ~/.buildout/default.cfg
[22:13] <leonardr> mine says:
[22:13] <leonardr> [buildout]
[22:13] <leonardr> eggs-directory=/home/leonardr/.buildout/eggs
[22:13] <leonardr> download-cache=/home/leonardr/.buildout/download-cache
[22:13] <leonardr> with that in place you will only go through the huge compile step once
[22:13] <lifeless> wgrant: I think bac wrote it
[22:14] <lifeless> but it interacts with bugtask:+index work obvious
[22:14] <lifeless> ly
[22:15] <wgrant> I'll increase the query count by 1 for now.
[22:15] <sinzui> leonardr: yes I have one of those. I did not have everything that lazr.restful wanted though
[22:16] <wgrant> Hmm, unless I can easily get tracebacks into there.
[22:16] <leonardr> ah
[22:16] <lifeless> wgrant: if you can run it with LP_DEBUG_SQL_EXTRA=1
[22:16] <wgrant> lifeless: What does _EXTRA do?
[22:17] <lifeless> backtraces
[22:17] <wgrant> Ah.
[22:17] <wgrant> Thanks.
[22:17] <lifeless> [22:17] <wgrant> Thatisquitealotofoutput
[22:22] <bac> wgrant: i think abel wrote that test and i have modified it
[22:22] <sinzui> :/ me thinks compiling mozilla might be faster
[22:23] <wgrant> bac: There are no cache invalidations before the query checks :(
[22:23] <lifeless> ok
[22:23]  * lifeless -> sydney
[22:23] <wgrant> bac: So it's sort of not really valid, and very unreliable.
[22:23] <lifeless> if you need me in the next 2 hours, call my mobile
[22:29] <bac> wgrant: sorry, i was thinking of a different test.  the one you're talking about was written by edwin.
[22:31] <sinzui> leonardr: what should I rename test_web_layer_includes_html_with_json_representation => disabled_web_layer_includes_html_with_json_representation
[22:32] <sinzui> leonardr: also, I think lazr.restful may hate python 2.7
[22:32] <leonardr> sinzui: web_layer_includes_json_representation
[22:32] <sinzui> okay
[22:32] <leonardr> ie. test the other thing or get rid of the test
[22:32] <leonardr> sinzui: it's possible. are you getting errors?
[22:33] <sinzui> I see two failures in a pristine tree. I am on natty
[22:38] <sinzui> leonardr: https://code.launchpad.net/~sinzui/lazr.restful/json-not-xhtml-0/+merge/52141 I think you should pull the branch and verify the tests are as you expect
[22:38] <leonardr> ok
[22:47] <wgrant> sinzui: Hi, could you run a query on staging for me please?
[22:51] <sinzui> wgrant: yes
[22:52] <wgrant> sinzui: query 36 (the last one) from https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1888QS70... could you grab an EXPLAIN ANALYSE ?
[22:56] <sinzui> wgrant: http://pastebin.ubuntu.com/575213/
[22:58] <wgrant> sinzui: Erm. Could you try that on qastaging, please? I wonder if there is some difference -- it seems to take 13 seconds there, not 60ms.
[22:59] <sinzui> well this taking time
[23:00] <wgrant> Intriguing.
[23:00] <sinzui> wgrant: http://pastebin.ubuntu.com/575215/
[23:00] <sinzui> wgrant: maybe the difference is hot and cold caching
[23:01]  * sinzui runs again
[23:01] <wgrant> sinzui: I tried that page 20 times yesterday.
[23:01] <wgrant> Got no better.
[23:02] <wgrant> the plan is almost identical.
[23:02] <wgrant> Yet the times are off by two orders of magnitude.
[23:02] <sinzui> wgrant: qastaging run a second time: http://pastebin.ubuntu.com/575218/
[23:02] <wgrant> Huh.
[23:02] <wgrant> Thanks.
[23:56] <wallyworld> anyone up for a review? https://code.launchpad.net/~wallyworld/launchpad/remove-decoratedbug/+merge/52003