[00:00] <mbarnett> rubylicious
[00:00] <wgrant> mbarnett: Which was the old version of libc-bin?
[00:00] <wgrant> That's the only vaguely interesting one, and it looks boring :(
[00:01] <mbarnett> https://pastebin.canonical.com/44062/
[00:01] <wgrant> lifeless: That's what I would have done.
[00:01] <wgrant> mbarnett: Hmmm, that's a bit more interesting.
[00:01] <mbarnett> less rubylicious
[00:02] <wgrant> Yet still very boring.
[00:03] <wgrant> mbarnett: Do you know how to apply pygdb to codebrowse?
[00:04] <mbarnett> i do not
[00:04] <wgrant> mwhudson: Could you assist with interrogating codebrowse2?
[00:05] <wgrant> mbarnett: We tried to do it yesterday, but the process accidentally got killed.
[00:05] <mbarnett> loggerhead history doesn't seem to have naything interesting
[00:07] <mwhudson> wgrant: yeah, but let me grab some lunch so i can concentrate a bit
[00:07] <mbarnett> unfortunately, we are about to go losaless.
[00:07] <wgrant> mbarnett: No spm? :(
[00:07] <mbarnett> spm: is pretending that flying half way around the world is tiring.
[00:07] <wgrant> Heh.
[00:11] <mwhudson> ah
[00:15] <mbarnett> well, i do actually need to go assault my face with some dinner.  Do you want me to leave codebrowse 2 in its degenerate state and try and hop back later?   Or shall i restart it to avoid the potential (at least a bit) of another outage?
[00:16] <wgrant> Personally I'd leave it.
[00:16] <wgrant> They seem to die in tandem now.
[00:17] <mbarnett> yeah
[00:17] <mbarnett> okay
[00:20] <wgrant> lifeless: Your closing-bugs-from-changelogs.txt failure is really confusing.
[00:22] <wgrant> Ah, I see now.
[00:23] <lifeless> wgrant: yes, thus asking for help
[00:24] <wgrant> lifeless: I have a quick fix, but I am looking at ways to actually fix the test properly.
[00:25] <wgrant> Meh, quick fix it is.
[00:26] <wgrant> lifeless: http://pastebin.ubuntu.com/573729/
[00:26] <wgrant> lifeless: It looks for a packageupload with a changesfile in the upload_distroseries.
[00:26] <lifeless> cool
[00:26] <wgrant> So if we set the upload_distroseries to something else, the test can continue to illegally create multiple packageuploads.
[00:26] <lifeless> I'll just wrap this arc and then apply and test
[00:29]  * wgrant foods.
[00:40] <maxb> wgrant: Thanks, I believe I have successfully comprehended the user's corrupted merge directive email now
[00:57] <wgrant> maxb: I can give you more traceback or the email text if you need them.
[00:59] <maxb> The user pasted the email text in the question, and on close inspection it is indeed malformed, so it's ok
[01:00] <wgrant> Great, thanks for dealing with that.
[01:00] <maxb> Its an interesting feature, which I keep thinking I might use
[01:01] <maxb> Apart from gpg-signing mails without involving thunderbird is something I've not figured out yet
[01:01] <wgrant> Has anybody had an ec2 run Windmill-fail this week?
[01:01] <wgrant> maxb: Doesn't bzr do that for you?
[01:03] <StevenK> maxb: gpg | mail -s ... ?
[01:11] <StevenK> wgrant: My evil plan to fix my branch from yesterday using + didn't work. :-(
[01:13] <wgrant> StevenK: :(
[01:13] <wgrant> My evil plan to disable WindmillLayer is getting stronger.
[01:15] <StevenK> wgrant: So I suspect the only option is to stop using structured and escape the arguments before interpolating
[01:17] <wgrant> StevenK: Or interpolate the recipe manually, before you go into structured.
[01:18] <StevenK> Not a bad idea, just struggling how to implement that.
[01:19] <StevenK> And trying not to context switch
[01:19] <StevenK> Switching between branches of devel and db-devel really sucks
[01:28] <maxb> oh, I meant gpg-signing in a way integrated with bzr send, that doesn't involve saving the merge directive off to a file, manually signing, and sending
[01:30] <lifeless> maxb: ask in #bzr, or read the docs ;)
[01:32] <maxb> I read the source, and ended up trying to hack a plugin :-)
[01:32] <lifeless> maxb: -why-?!
[01:38] <lifeless> wgrant: was there a sqlite change perhaps?
[01:38] <wgrant> lifeless: That was my guess.
[01:38] <wgrant> But it doesn't look like it.
[01:38] <wgrant> But the list I obtained may well not be exhaustive.
[01:42] <maxb> Why? Well, I have to confess, because it felt like something that ought to be possible :-)
[01:42] <lifeless> maxb: I mean, it is possible for most mailers I know of, withiut plugins
[01:43] <lifeless> maxb: what mailer do you use?
[01:57] <LPCIBot> Project db-devel build #403: STILL FAILING in 5 hr 35 min: https://hudson.wedontsleep.org/job/db-devel/403/
[02:00] <lifeless> do we still use personlocation for anything?
[02:00] <wgrant> No.
[02:00] <lifeless> sinzui: ^
[02:00] <wgrant> NFI where those queries come from, either.
[02:01] <lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/lp/registry/model/person.py", line 708, in time_zone
[02:01] <lifeless>     if self.location is None:
[02:01] <lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/lp/services/propertycache.py", line 116, in __get__
[02:01] <lifeless>     value = self.populate(instance)
[02:01] <lifeless> time localisation
[02:02] <wgrant> Oh, that's on PersonLocation? :/
[02:02] <wgrant> I always assumed that was just the lat/lon.
[02:02] <lifeless> apparently not
[02:03] <lifeless> guess how many queries all_distro_archive_ids creates
[02:03] <wgrant> That could be a bug.
[02:03] <wgrant> Four? Five?
[02:03] <wgrant> primary, partner, debug.. maybe only three.
[02:03] <lifeless> actually two
[02:03] <wgrant> :(
[02:04] <lifeless> from this call to getCurrentSourceReleases
[02:04] <lifeless> hmm
[02:04] <lifeless> time ot land tihs and get the branch changing that landed
[02:04] <wgrant> Ahh
[02:08] <lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-724033/+merge/51676
[02:08] <wgrant> Has anybody looked at the Windmill SNAFU in depth?
[02:09] <wgrant> I am wondering if the xvfb-run issue is in fact a symptom of the killing, not the hang.
[02:09] <lifeless> xvfb-run issue?
[02:10] <wgrant> lifeless: See the end of https://lpbuildbot.canonical.com/builders/lucid_lp/builds/669/steps/shell_6/logs/stdio
[02:10] <wgrant> We get an X error.
[02:11] <wgrant> I've been presuming that was seen at the start of the hang.
[02:11] <wgrant> But maybe it's because of the mass murder after the hang is detected.
[02:13] <lifeless> I don't see an X error
[02:13] <lifeless> oh
[02:15] <wgrant> There is the occasional thread leak on ec2.
[02:15] <lifeless> yes, I think the X process went before the tes
[02:15] <lifeless> t
[02:15] <wgrant> And locally it just completely fails.
[02:24] <thumper> StevenK: mumble?
[02:25] <StevenK> thumper: I was about to have some lunch, but sure.
[02:25] <thumper> StevenK: it'll be quickish
[02:32] <lifeless> wgrant: thanks, thats off to ec2 now
[02:32] <wgrant> lifeless: Great.
[02:34] <lifeless> who is meant to be OCR today
[02:34] <StevenK> WILLIAM!
[02:35] <lifeless> wgrant: I can haz review please
[02:36] <wgrant> Oh, so I am.
[02:37] <wgrant> lifeless: Ah, I see you are realising the precaching resultset this time :P
[02:38] <lifeless> ftw
[02:38] <lifeless> on the rusty scale, that interface is problematic
[02:38] <wgrant> Yes
[02:51] <thumper> https://code.launchpad.net/~thumper/launchpad/fix-mantis-warnings-bug-218384/+merge/51679 anyone?
[02:51] <wgrant> https://code.launchpad.net/~wgrant/launchpad/squash-cw-protocolerrors/+merge/51680, also anyone?
[02:51] <wgrant> thumper: Looking.
[02:52] <thumper> wgrant: thansk
[02:52] <thumper> wgrant: I'll look at yours
[02:53] <wgrant> Thanks.
[02:56] <thumper> wgrant: do we need the transaction.commit(), or is a store flush enough?
[02:56] <wgrant> thumper: checkwatches checks that no transaction is active.
[02:56] <wgrant> So it has to be a commit.
[02:56] <thumper> ok
[02:57] <lifeless> grr
[02:57] <lifeless> 0 tests run in 4:11:34.318468, 0 failures, 0 errors
[02:57] <lifeless> wgrant: has your fix landed yet ?
[02:57] <wgrant> lifeless: merge devel.
[02:58] <thumper> wgrant: shouldn't the ec2 image start with devel?
[02:58] <thumper> wgrant: or is it client side?
[02:58] <wgrant> thumper: Yes, but ec2test-remote.py is copied from the local branch.
[02:58] <thumper> wgrant: ok
[02:59] <lifeless> wgrant: I had a windmill fail in ec2
[02:59] <wgrant> lifeless: Could you mentor https://code.launchpad.net/~thumper/launchpad/fix-mantis-warnings-bug-218384/+merge/51679?
[02:59] <wgrant> lifeless: When?
[03:00] <lifeless> today
[03:00] <wgrant> Forward pls.
[03:00] <wgrant> I want all the data I can get.
[03:00] <lifeless> sent
[03:00] <huwshimi> How do I actually get a lazr-js branched landed to trunk (after the mp step)?
[03:01] <lifeless> grr
[03:01] <lifeless> wtf:
[03:01] <lifeless>     print test_tales("branch/fmt:bzr-link", branch=branch)
[03:01] <lifeless> Differences (ndiff with -expected +actual):
[03:01] <lifeless>     - <a href=".../~eric/fooix/bar" class="sprite branch">lp://dev/fooix</a>
[03:01] <lifeless>     ?            ^
[03:01] <lifeless>     + <a href="http://code.launchpad.dev/~eric/fooix/bar" class="sprite branch">lp://dev/~eric/fooix/bar</a>
[03:01] <lifeless>     ?          +++++++++++ +++++++++ ^^^                                                 ++++++     ++++
[03:03] <wgrant> lifeless: Dev focus setting fail?
[03:05]  * thumper agrees with wgrant
[03:05] <thumper> huwshimi: commit to trunk
[03:05] <thumper> huwshimi: I don't think it is PQM controleld
[03:06] <huwshimi> thumper: Thanks.
[03:08] <wgrant> thumper: Thanks.
[03:14] <lifeless> thumper: could you eyeball bug 702524?
[03:14] <_mup_> Bug #702524: Merged branches keep 'Development' status when their merge proposal is marked as 'Merged' <Launchpad itself:Confirmed> < https://launchpad.net/bugs/702524 >
[03:14] <thumper> lifeless: yep
[03:14] <thumper> lifeless: although I think I know the answer
[03:14] <lifeless> thumper: can you mentor wgrants review of https://code.launchpad.net/~lifeless/launchpad/bug-724033/+merge/51676 ?
[03:15] <StevenK> wgrant: Can haz review? https://code.launchpad.net/~stevenk/launchpad/db-recipe-description-really-this-time/+merge/51682 ?
[03:16] <wgrant> lifeless: Can you mentor my review of StevenK's?
[03:16] <StevenK> wgrant: :-(
[03:16] <StevenK> wgrant: I know where you live ...
[03:17] <wgrant> :(
[03:18] <lifeless> wgrant: yes
[03:18] <lifeless> just chewing through 3 hours mail ;)
[03:18] <wgrant> Thanks.
[03:19] <StevenK> wgrant: What was your plan with adding recipe first?
[03:20] <thumper> lifeless: yes, I'll look at wgrant's review
[03:21] <wgrant> StevenK: You could put a %%s in the template, format the other args into it using structured(), then format the escaped recipe in using %.
[03:21] <wgrant> *or* you could fix structuredstring to work nicely with other structuredstrings.
[03:23] <wgrant> Does anybody have a valid objection to disabling WindmillLayer until someone works out what is wrong?
[03:24]  * StevenK tries to remember how to do type checking in Python
[03:24] <wgrant> I have never managed to get it to run reliably on any local machine, it is entirely broken on Natty, it leaks threads sometimes in random tests, and sometimes hangs entirely.
[03:25] <wgrant> StevenK: isinstance?
[03:28] <lifeless> wgrant: Revision 12469 can not be deployed: needstesting
[03:29] <lifeless> looks like its half-qaed - were you looking at that ?
[03:29] <wgrant> lifeless: One was a test change.
[03:29] <wgrant> The other needs a LOSA or somebody to decide that we don't care.
[03:30] <james_w> is there a library of matchers for use with storm anywhere?
[03:31] <lifeless> there are some in the lp tree
[03:31] <lifeless> like StormRecorder + HasQueryCount
[03:31] <wgrant> lifeless: Do you have any hints for debugging thread garbage?
[03:31] <james_w> I'm more interesting in query/result set stuff currently
[03:32] <james_w> https://code.launchpad.net/~james-w/launchpad-work-items-tracker/storm/+merge/51684
[03:32] <james_w> (includes unit testing of lplib-using code)
[03:32] <lifeless> I trhink the storm folk have started using matchers
[03:32] <lifeless> so storm specific stuff perhaps land there?
[03:37] <james_w> nothing jumps out from the filenames
[03:39] <thumper> GRRRR
[03:39] <thumper> stupid dist utils
[03:41]  * thumper stabs it in the face
[03:45] <lifeless> oops
[03:46] <lifeless> oh well, hopefully it won't break the build
[03:46] <wgrant> lifeless: Hm?
[03:47] <lifeless> wgrant: I lp-landed rather than ec2 landed
[03:47] <wgrant> Heh.
[03:47] <thumper> lifeless: do you know anything about setup.py, tests and the additional_tests method?
[03:47] <lifeless> thumper: a -little-
[03:47] <lifeless> thumper: lazr-js ?
[03:47] <thumper> lazr.enum
[03:47] <lifeless> ah
[03:47] <thumper> trying to run: ./setup.py test
[03:47] <thumper> should that cause the additional_tests method to be checked?
[03:48] <thumper> because it isn't
[03:48] <thumper> I have a pdb break point in there
[03:48] <thumper> and it isn't breaking
[03:48] <lifeless> http://mail.python.org/pipermail/python-checkins/2006-March/050452.html
[03:48] <thumper> I'm trying to get it to run the actual tests
[03:49] <lifeless> so additional_tests is for non unittest tests
[03:49] <lifeless> what non unittest tests do you have?
[03:49] <thumper> it is to load doctests
[03:49] <lifeless> ah, what it means is 'tests that loadTestsFromName does not find'
[03:50] <lifeless> so, I personally /hate/ setup.py's test support
[03:50] <thumper> I'm starting to too
[03:50] <lifeless> I would always just use a test_suite method
[03:51] <StevenK> lifeless: O hai. Can haz review of wgrant's review of my MP?
[03:52] <lifeless> I did?
[03:52] <StevenK> lifeless: I can't see your scribble on https://code.launchpad.net/~stevenk/launchpad/db-recipe-description-really-this-time/+merge/51682 ?
[03:53] <lifeless> done
[03:53] <lifeless> must have glitched
[03:53] <StevenK> lifeless: Thanks, tossing it at ec2
[03:54] <wgrant> lifeless: Do you know how to get details on threads that are leaked by tests?
[03:54] <lifeless> wgrant: yes
[03:54] <wgrant> I've stuck a pdb in there, but I can't get much useful from the Thread.
[03:55] <lifeless> ok
[03:56] <lifeless> so depending on how it was constructed
[03:56] <lifeless> subclasses have a replaced run() (IIRC), typical uses supplies a callable to the thread which its run() will call
[04:00] <StevenK> wgrant: Can you see any clues on http://pastebin.ubuntu.com/573778/ ?
[04:00] <lifeless> wgrant: you can also attach gdb
[04:00] <lifeless> wgrant: and get pretty good info from thread apply all pybt
[04:00] <wgrant> lifeless: Thanks.
[04:02] <lifeless> I thought we'd changed thread leaks to be non fail though
[04:04] <wgrant> They are a non-fail.
[04:04] <wgrant> But Windmill sometimes leaks them, and Windmill sometimes fails.
[04:04] <wgrant> I am hoping that this is not a coincidence.
[04:09] <lifeless> wtf- librarian.txt fail in my /branches optimisation branch
[04:10] <wgrant> lifeless: I saw one of those a week ago.
[04:10] <wgrant> What is the error?
[04:10] <wgrant> A 200 UploadFailed?
[04:10] <lifeless> yeah
[04:11] <lifeless> passes locally
[04:11] <wgrant> Yeah.
[04:11] <wgrant> And on ec2 >9/10.
[04:16] <StevenK> wgrant: Can haz clue?
[04:16] <wgrant> StevenK: recipe_build_details is probably raising an AttributeError or similar.
[04:17] <StevenK> Hrm. A better traceback would be awesome.
[04:18] <StevenK> TypeError: not all arguments converted during string formatting
[04:18] <StevenK> :-(
[04:19] <StevenK> Ah ha, I get it.
[04:19] <lifeless> \o/ another one bites the dust
[04:22] <lifeless> wgrant: so, bug 688130 - what needed losa answering?
[04:22] <_mup_> Bug #688130: Statistics clean-ups and extra tests <lp-translations> <qa-needstesting> <upstream-translations-sharing> <Launchpad itself:Fix Committed by adeuring> < https://launchpad.net/bugs/688130 >
[04:22] <wgrant> lifeless: It needs the rosetta pofilestats script to be run.
[04:27] <lifeless> I think my branch:+index patch is reasonably successful :)
[04:28] <lifeless> https://code.qastaging.launchpad.net/~software-store-developers/software-center/trunk/+index rendered first hit on qastaging. Which is awesome, if I do say so myself.
[04:30] <LPCIBot> Project devel build #488: STILL FAILING in 5 hr 29 min: https://hudson.wedontsleep.org/job/devel/488/
[04:32] <huwshimi> thumper: Actually it appears that we do use PQM for lazr-js
[04:33] <StevenK> Jenkins, you make me sad.
[04:33] <StevenK> checkwatches.txt passed, but a Windmill test failed? Baaaaah
[04:34] <huwshimi> thumper: Does that mean landing with lp-land or something?
[04:36] <wgrant> huwshimi: pqm-submit
[04:36] <huwshimi> wgrant: Thanks
[04:37] <wgrant> bzr pqm-submit -m "[r=foo] bar" --public-location=bzr+ssh://bazaar.launchpad.net/~path/to/branch --submit-branch=bzr+ssh://bazaar.launchpad.net/~path/to/trunk
[04:39] <StevenK> No fair actually landing with '[r=foo] bar'
[04:39] <huwshimi> wgrant: Thanks, I was just trying to figure all that out :)
[04:40] <huwshimi> StevenK: Oh, cmon
[04:42] <wgrant> lifeless: bug-724033 seems to have an empty commit message.
[04:48] <StevenK> wgrant: Sucess!
[04:49] <wgrant> StevenK: Exccellent.
[04:49] <lifeless> wgrant: see https://pqm.launchpad.net/
[04:50] <wgrant> lifeless: Ah, great.
[04:52] <jtv> StevenK, wgrant: DistroSeries.parent_series seems to have shifted in meaning a bit… should we keep the old "previous series in same distro" meaning or get rid of it?
[04:52] <wgrant> jtv: It hasn't shifted.
[04:53] <wgrant> Any derivation stuff that is using it now is not altering its meaning; it's just using it until we introduce something to replace it.
[04:53] <jtv> Do we expect the replacement to work in the same way?
[04:54] <StevenK> Awwwww. The user on prod with deleted recipes doesn't exist on qastaging
[04:54] <wgrant> That is not known.
[04:55] <jtv> wgrant: okay, do we expect to have DistroSeriesDifferences between e.g. maverick and natty?
[04:56] <wgrant> jtv: I don't believe so.
[04:56] <wgrant> Not in the initial implementation, at least.
[04:56] <StevenK> That isn't the point
[04:58] <StevenK> Publishing details
[04:58] <StevenK>     * Built by deleted recipe for Jelmer Vernooij.
[04:58] <StevenK> \o/
[05:00] <wgrant> Yay.
[05:00]  * wgrant is QAing the greatest OOPS fix of all time.
[05:01] <StevenK> wgrant: Oh, the checkwatches stuff?
[05:02] <wgrant> StevenK: No, the loggerhead thing.
[05:02] <wgrant> Unfortunately it is slightly non-trivial, because both qastaging and staging are borked.
[05:11] <StevenK> Sigh. Can't hear myself think.
[05:13] <StevenK> Some ungrateful person with a leaf blower is outside. Doesn't he know that the BoM forecast high winds for tonight, so it's POINTLESS
[05:13] <wgrant> Heh.
[05:14] <wgrant> Hmm, this is slightly upsetting.
[05:15] <jtv> wgrant: I fervently hope that remark is not connected to SteveK's.
[05:15] <wgrant> No, codebrowse.
[05:15] <jtv> Ah well, talk of upsetting.
[05:16] <jtv> On the bright side, "parallel a-f" has landed on qastaging.
[05:16] <wgrant> And mawson.
[05:16] <wgrant> And it works.
[05:17] <jtv> You tested it?
[05:17] <wgrant> Yes.
[05:17] <jtv> Great!  Thanks.  I'll mark it qa-ok then.
[05:17] <wgrant> I already did.
[05:17] <wgrant> Didn't I?
[05:17] <jtv> Oh, looks like you did.  :)  Pages don't refresh very well here.
[05:21] <StevenK> GAH! He's back!
[05:22] <StevenK> I have somewhere that guy can store the leaf blower when he is finished using it ...
[05:29] <wgrant> D:
[05:29] <wgrant> Codebrowse, you suck.
[05:33] <wgrant> lifeless: So, you know how that loggerhead fix was meant to stop loggerhead from OOPSing when haproxy checked it?
[05:35] <jtv> StevenK: do SPRs ever get deleted?  I'm asking because SPPH.supersededby isn't indexed.
[05:39] <jtv> StevenK: do SPRs ever get deleted?  I'm asking because SPPH.supersededby isn't indexed.
[05:41] <wgrant> jtv: Not at the moment.
[05:41] <wgrant> :(
[06:10] <wgrant> No IS for a week and a half makes me very sad.
[06:14] <lifeless> wgrant: yes?
[06:15] <wgrant> lifeless: Well, my testing showed that it in fact introduced a new OOPS. It does, but in a much rarer case than the one it fixes.
[06:15] <wgrant> Bug #726985
[06:15] <_mup_> Bug #726985: codebrowse OOPSes with GeneratorExit when connection closed early <Launchpad itself:Triaged> < https://launchpad.net/bugs/726985 >
[06:15] <wgrant> I think we should still deploy.
[06:15] <wgrant> Also, feel like waking elmo?
[06:15] <lifeless> wgrant: is that if you abort from a client ?
[06:15] <wgrant> guava has had a heart attack again.
[06:15] <lifeless> *sigh*
[06:15] <lifeless> is s gsa around ?
[06:15] <wgrant> I pinged pjdc/bradm a few minutes back.
[06:15] <wgrant> No response.
[06:16] <lifeless> give them 10
[06:16] <lifeless> then dig up the number and call
[06:16] <lifeless> I'm mid-dinner, sorry
[06:16] <wgrant> k
[06:16] <wgrant> lifeless: I believe the client has to be in the DC, bypassing haproxy.
[06:18] <elmo> bouncing
[06:18] <wgrant> elmo: tiaz might already be.
[06:18] <wgrant> (see #is)
[06:39] <jtv> StevenK: do you have a clear idea of what DSDJ will look like, database-wise?
[06:43] <StevenK> jtv: One row
[06:44] <jtv> StevenK: one row per object?  Ground-breaking.
[06:44] <jtv> Maybe I'm just not aware of the actual design questions you're facing.  :)
[06:44] <StevenK> jtv: A DSD already has an interface and a schema ...
[06:45] <StevenK> Oh, the job itself. Er. Not sure. :-)
[06:45] <jtv> Yes, the part I'm quite interested in right now is whether, for the initial population of DSD, I can create DSDJ rows directly in SQL.
[06:46] <jtv> As tends to happen, a join structure has formed itself in my mind's eye quite early on and it takes ages to figure out any reasons why it might be wrong.  :)/
[06:46] <jtv> I was also wondering about supersededby…  When is a SPPH "superseded" exactly?
[06:47] <jtv> I imagine I can ignore ones that have that column set, but might that also be a sufficient condition?
[06:49] <jtv> In other words, if I selected from SPPH on sourcepackagename and distroseries where superseded is null, would that give me (more or less) just the SPPHs I wanted or would there be a lot of dead wood in there?
[06:49] <wgrant> Selecting on supersededby is not going to give you anything useful.
[06:50] <jtv> Ah
[06:50] <jtv> Conversely, will I be able to ignore ones where supersededby is not null?
[06:50] <wgrant> You should filter on status, if anything.
[06:50] <jtv> (And also, why is supersededby not indexed?  Do SPRs never get deleted?)
[06:50] <wgrant> SPRs do not get deleted at the moment.
[06:51] <wgrant> What are you trying to do, exactly?
[06:51] <wgrant> And why?
[06:51] <jtv> Trying to figure out how to get the "latest" SPPH for a given source package.
[06:51] <lifeless> jtv: due to the bug about not having a sourcepackage object in the db
[06:51] <wgrant> Source package, source package name, or distribution source package?
[06:51] <lifeless> jtv: this is actually hugely expensive
[06:52] <lifeless> getCurrentSourceReleases does it
[06:52] <jtv> wgrant: source package
[06:52] <wgrant> jtv: Are you sure?
[06:52] <jtv> lifeless: ah, thanks
[06:52] <lifeless> well
[06:52] <lifeless> it gets what most people will think of as a source package :)
[06:52] <lifeless> which is not a SourcePackage or  DistroSeriesSourcePackage
[06:52] <wgrant> If you have a SourcePackage, you should be able to call .currentrelease.
[06:52] <jtv> wgrant: I was until you asked that.  :)  A given sourcepackagename for a given distroseries…  I *think* that's what I want!
[06:53] <wgrant> Right, that's an actual SourcePackage.
[06:53] <lifeless> except its versioness
[06:53] <lifeless> which is mondo confusing modelling
[06:53] <jtv> lifeless: I imagine what most people think of as a source package is what we call a distributionsourcepackage… I've never heard of DistroSeriesSourcePackage.
[06:54] <wgrant> Most people would think of an SPR as a source package.
[06:54] <lifeless> jtv: no, most people think of source packages as 'source package release'
[06:54] <jtv> Ah
[06:54] <wgrant> Indeed.
[06:54] <lifeless> jtv: there is a 'most recent source package release'
[06:55] <lifeless> I'd like to really overhaul the soyuz model
[06:55] <lifeless> but not while we're in the hole
[06:55] <wgrant> lifeless: How?
[06:55] <jtv> So should I be looking for the most recent SPR instead of the most recent SPPH when looking for distroseries differences?
[06:56] <jtv> lifeless: for someone who's postponing stuff until we're not in the hole, you sound suspiciously eager to get back into it.  :)
[06:56] <jtv> More power to you, but… sounds like a tall order.
[06:56] <wgrant> SELECT id FROM spph WHERE spph.distroseries = yourfavouritedistroseries AND spph.pocket = 0 AND status IN (1, 2) ORDER BY datecreated DESC LIMIT 1
[06:56] <wgrant> Oh, and spph.archive = yourfavouritearchive
[06:57] <lifeless> wgrant: I don't have a specific schema handy.
[06:57] <jtv> wgrant: would the pocket and archive matter?
[06:57] <StevenK> What are your vague thoughts, then?
[06:57] <lifeless> wgrant: I would start by gathering all the actual use cases the distro *want* - partly what we do today, partly their wishlist.
[06:58] <lifeless> and design to directly match that as a fast schema, with efficient solutions across the board.
[06:58] <lifeless> and finally look at how to migrate.
[06:58] <lifeless> StevenK: I can list some problems
[06:59] <wgrant> jtv: The archive is essential. The pocket is less essential, but you probably still want to restrict it.
[06:59] <lifeless> StevenK: we infer data on every operation;  we have 1:M relationships that our business rules want to be 1:1;  our schema supports invalid states
[07:00] <wgrant> jtv: Since we don't know what it means to have a DSD to a non-Release pocket.
[07:00] <jtv> wgrant: oh do I get all the PPAs if I don't restrict on archive?  Also, another thing that I think matters is the package.  :)
[07:00] <wgrant> jtv: Um, yeah, package would be nice too.
[07:01] <jtv> So that's a join through SPR I guess.
[07:01] <wgrant> Right.
[07:01] <lifeless> StevenK: our schema should naturally constraint the states to be valid; data learnt should not be reinvestigated on every operation
[07:01] <lifeless> StevenK: we should be able to do things like '100 most recent daily builds' in 3-4ms.
[07:02] <lifeless> StevenK: without 'denormalisation'
[07:02] <jtv> wgrant: if I neglected to restrict by archive, would that mean that I would get all the PPAs as well?
[07:02] <wgrant> jtv: And copy archives and partner.
[07:02] <lifeless> StevenK: basically the schema wasn't built to support fast queries, it was built as an object model for the archive/build space: which as a way to design a database leads invariably to performance and maintenance problems.
[07:03] <lifeless> jtv: and other distributions in future ;)
[07:03] <wgrant> lifeless: No, because we constrain by distroseries already.
[07:03] <jtv> lifeless: wouldn't the distroseries restriction take care of unwanted distros?
[07:04] <lifeless> yes, mea culpa
[07:04] <lifeless> as long as we don't try to share distroseries across distributions ;P
[07:04] <jtv> Hmm
[07:04] <jtv> That'd certainly simplify the work I'm doing now.
[07:05] <lifeless> jtv: unless you also make productseries sharable across related project, -no-.
[07:06]  * jtv is not currently well enough to grok that sentence
[07:06] <jtv> anyway
[07:07] <wgrant> Aha. All four hangs today have been the same branch, but some were partly drowned in other requests.
[07:07] <lifeless> jtv: I'm saying we don't want the 'projectseries' and 'distroseries' concepts to drift further from each other.
[07:07] <wgrant> Two different branches were involved yesterday.
[07:07] <wgrant> But it was the same branch at the same time on each instance.
[07:07] <jtv> lock?
[07:10] <wgrant> The requests were 15s apart, though.
[07:11] <wgrant> And they do not match up in all cases.
[07:11] <jtv> oh, lifeless, a quick note: the check for read-only mode goes through the FS.  Might it make sense to log that in the external-action timeline?
[07:11] <lifeless> yes
[07:12] <jtv> I have this vague worry that we don't know how much it's costing us.
[07:12] <lifeless> so, two things
[07:12] <lifeless> a) if its checked outside the request context, we can't put it in the action timeline
[07:13] <lifeless> but b) if its checkout outside the request context, it may not cost anything (e.g. if the main thread does it)
[07:13] <lifeless> and c) we should do that in a thread and maintain it as global state anyhow
[07:13] <jtv> Checks outside the request context are likely to be few (per request).  I'm more worried about calls inside requests—and therefore frequency more than latency.
[07:14] <wgrant> Wiiindmiiiill!!!
[07:14] <jtv> Here, have some cheese, ja?
[07:14] <lifeless> jtv: anyhow, at the moment, its local FS, if its being checked per request it will be hot and approximately free.
[07:15] <jtv> lifeless: I also know that the +translate page used to check it oh, at least dozens of times.
[07:15] <lifeless> meep
[07:16] <jtv> meep indeed.  I fixed that, and then you did your timelines, and now I think it'd be nice to stop ourselves from making the same mistake again.  :)
[07:17] <lifeless> wgrant has made some excellent progress towards us being able to fix the plumbing too
[07:18] <wgrant> ... have I?
[07:18] <lifeless> wgrant: checkwatches
[07:18] <lifeless> wgrant: was the major source of omgwtfness
[07:18] <wgrant> Ah, right.
[07:18] <wgrant> Just pqm-submitted the next branch of that.
[07:18] <jtv> Dear infrastructure.  A connection _can_ have a useful lifetime beyond 5 minutes, even if it does go quiet sometimes.
[07:19] <lifeless> woo 711064 passed ec2
[07:19] <jtv> lifeless: meep indeed.  I fixed that, and then you did your timelines, and now I think it'd be nice to stop ourselves from making the same mistake again.  :)
[07:19] <lifeless> I think I have just 2 outstanding branches now
[07:19] <lifeless> jtv: got that :)
[07:19] <lifeless> 20:16 < jtv> meep indeed.  I fixed that, and then you did your timelines, and now I think it'd be nice to stop ourselves from making the same mistake again.  :)
[07:19] <jtv> Just making sure :)
[07:19] <lifeless> 20:17 < lifeless> wgrant has made some excellent progress towards us being able to fix the plumbing too
[07:19] <wgrant> lifeless: What sort of increase are we looking for there?
[07:19] <lifeless> 20:17 -!- jtv [~jtv@27.130.65.129] has quit [Read error: Connection reset by peer]
[07:19] <wgrant> s/for/at/
[07:20] <lifeless> wgrant: there are 12 queries averaging 1 second each; I've dropped the count to 2/3rds - they were doing a pair for a language, then one for the alt language.
[07:20] <wgrant> Ah.
[07:20] <jtv> lifeless: anyway, I'm not sure we ever consciously made a decision of the sort of timescale a switch to read-only mode should operate at: instantaneous, when-the-GIL-is-passed, per-request…
[07:20] <lifeless> wgrant: so 8 vs 12, so 4 seconds perhaps ?
[07:21] <stub> jtv: Sounds like you are back in TH. TOT doesn't want to hold an SSL connection open for more than a few minutes for me, even if it is busy.
[07:21] <lifeless> jtv: end of request is implied by our contract
[07:21] <jtv> stub: I am, though it's a non-governmental ISP
[07:22] <jtv> lifeless: makes sense… so then we should keep RO-mode in the request.  We've had a small number of oopses from the switchover happening in mid-request—though maybe that was just an inappropriate check.
[07:23] <lifeless> jtv: I'd be happy to have it in a background thread that polls every (say) 1000ms; and new requests read the current state from a global
[07:24] <stub> IIRC, RO mode shouldn't kick in until the next request - there is a thread local that is checked at the start and end of the request.
[07:24] <lifeless> jtv: that would - prepare for us signalling via e.g. rabbit instantly, or using a single centralised ini file. Would eliminate per-requeset overhead.
[07:25] <lifeless> interesting
[07:25] <lifeless> nearly no increase in OOPS with memcache turned off for a day on bugtask+index
[07:25] <lifeless> \o/
[07:26] <lifeless> in fact, a *decrease*
[07:26] <lifeless> 132 /  272  BugTask:+index
[07:26] <lifeless> 187 /  476  BugTask:+index
[07:26] <wgrant> Huh.
[07:26] <wgrant> How expensive were the sets?
[07:27] <lifeless> I saw individual misses at 24ms
[07:27] <wgrant> I'm more interested in the sets.
[07:27] <lifeless> sure
[07:27] <lifeless> go look in the 27th oops report
[07:27] <lifeless> I don't remember the figures offhand
[07:32] <jtv> wgrant: when I look for distroseries differences, I guess the archive restriction should be that the purpose be "primary."  Debug archives don't sound like things that should be shared with derived distros… is that correct?
[07:33] <wgrant> jtv: Debug archives only have binaries, anyway.
[07:33] <wgrant> So yes, primary archive only.
[07:33] <wgrant> If you go near partner, I will delete you.
[07:33] <jtv> Just such a shame to have to join through Archive just to get the purpose.
[07:33] <jtv> wgrant: don't worry, I don't believe in partner swapping.
[07:33] <wgrant> jtv: You could just distribution.main_archive
[07:33] <jtv> Or as I like to say: "you should have kept the receipt then"
[07:33] <wgrant> May be cheaper than joining.
[07:33] <lifeless> wgrant: can a distribution have a currentseries of None ?
[07:33] <jtv> wgrant: ahhh thanks, that's going to be cheaper yes.
[07:34] <wgrant> lifeless: Yes :D
[07:34] <lifeless> wgrant: ><
[07:34] <lifeless> wgrant: I'm looking at the bugtask-target-link-titles_txt failure I forwarded you
[07:35] <wgrant> lifeless: If there is no current series, there is no current version.
[07:35] <jtv> wgrant: Distribution.main_archive happens to be defined as a query on Archive, you unhelpful twit
[07:35] <LPCIBot> Project db-devel build #404: STILL FAILING in 5 hr 35 min: https://hudson.wedontsleep.org/job/db-devel/404/
[07:35] <wgrant> jtv: It is, yes.
[07:35] <wgrant> jtv: May still be faster.
[07:36] <lifeless> wgrant: obvious change made then
[07:36] <wgrant> lifeless: The only case where there isn't a currentseries is when there are no series at all.
[07:37] <jtv> wgrant: could be—more of a "join or fetch separately" tradeoff
[07:37] <wgrant> Right.
[07:38] <lifeless> win
[07:38] <lifeless>         return hash(self.distroseries.id) ^ hash(self.sourcepackagename.id)
[07:38] <lifeless>     AttributeError: 'NoneType' object has no attribute 'id'
[07:38] <jtv> Aigh!  SourcePackage.currentrelease is not cached, and SourcePackage.format evaluates it twice.
[07:39] <wgrant> lifeless: WTF?
[07:39] <wgrant> jtv: What is SourcePackage.format?
[07:39] <jtv> "it's in the model class"
[07:39] <wgrant> Huh.
[07:39] <wgrant> I didn't know that existed.
[07:39] <lifeless> wgrant: dict.get(SourcePackage(packagename, None))
[07:40] <wgrant> jtv: Why are you using that?
[07:40] <jtv> wgrant: I'm not, just noticed it by accident
[07:40] <lifeless> wgrant: SourcePackage hashes them together, and hte None comes from the same currentseries
[07:40] <wgrant> Ah.
[07:40] <wgrant> lifeless: :(
[07:40] <wgrant> jtv: SourcePackageRelease.format was introduced a long long time ago and never used.
[07:41] <lifeless>     - evolution (Published Distro): 2.0
[07:41] <lifeless>     ?                               ^^^
[07:41] <lifeless>     + evolution (Published Distro): No releases
[07:41] <lifeless>     ?                               ^^^^^^^^^^^
[07:41] <lifeless> presumably, again, because the current series *isn't actually published in*
[07:41] <jtv> wgrant: I suppose I should be relieved.  :)
[07:42] <wgrant> jtv: No, you should delete it :P
[07:42] <jtv> wgrant: with all sorts of sadistic pleasure
[07:46] <lifeless> yay dead waiting on librarian ><
[07:47] <lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/canonical/librarian/client.py", line 188, in addFile
[07:47] <lifeless>     response = self.state.f.readline().strip()
[07:47] <lifeless>   File "/usr/lib/python2.6/socket.py", line 397, in readline
[07:47] <lifeless>     data = recv(1)
[07:47] <lifeless> KeyboardInterrupt
[07:56] <lifeless> wgrant: of course, our friend Archive:+packages is back
[08:02] <lifeless> ok this is annoying me
[08:02] <wgrant> lifeless: What is?
[08:02] <lifeless> repeated librarian hang in addFile
[08:02] <wgrant> Hmm.
[08:02] <wgrant> What does your librarian say?
[08:02] <spiv> wgrant: "sssh!"
[08:03] <wgrant> Damn.
[08:20] <lifeless> if we get a deploy done tomorrow morning
[08:20] <lifeless> we can lower the timeout again
[08:20] <lifeless> I think
[08:22] <lifeless> omg
[08:22] <lifeless> cve:+index you suck
[08:22] <lifeless> 100 lookups for distribution
[08:22] <lifeless> 'FROM Distribution
[08:22] <lifeless> WHERE id IN ($INT)
[08:22] <lifeless> '
[08:23] <wgrant> Hah.
[08:23] <lifeless> how to bypass storms cache layer in one easy hit
[08:25] <lifeless> wgrant: bug 727023
[08:25] <_mup_> Bug #727023: Cve:+index timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/727023 >
[08:27] <lifeless> wgrant: data for you on librarian shutdown issues
[08:27] <lifeless> 2011-03-01 13:59:53+0530 [-] Failed to unlink PID file
[08:27] <lifeless>         Traceback (most recent call last):
[08:27] <lifeless>         Failure: exceptions.OSError: [Errno 2] No such file or directory: '/tmp/tmplhlibr/librarian.pid'
[08:27] <lifeless> 2011-03-01 13:59:53+0530 [-] Server Shut Down.
[08:27] <lifeless> wgrant: it shutdown ok, but its a little concerning
[08:27] <wgrant> Hmm.
[08:28] <wgrant> Never seen it hang before :/
[08:37] <rvb> Hi everyone, first day on the job for me today! I hope you lp guys remember the "guy from the future" from Dallas ;-).
[08:37] <rvb> lifeless: Hi
[08:37] <rvb> wgrant: Hi
[08:37] <wgrant> Welcome rvb!
[08:38] <rvb> wgrant: thx, I'm quite happy to be here and ready for action! I'm waiting for bigjools to come online but I still wanted to say hello.
[08:38] <lifeless> rvb: hi, welcome
[08:39] <wgrant> Julian should be around in 20 minutes or so.
[08:39] <rvb> lifeless: thx
[08:39] <StevenK> rvb: Welcome!
[08:39] <rvb> StevenK: Hi
[08:39] <lifeless> wgrant: can you eyeball lib/lp/bugs/browser/tests/bugtask-target-link-titles.txt and tell me why the evo publication in 'Published Distro' is not in the currentseries ?
[08:40] <wgrant> lifeless: Is "because someone thought it was a good idea in 2005, and it was so" a valid answer?
[08:40] <lifeless> wgrant: I think its oversight because getCurrentSourceReleases was bong
[08:40] <lifeless> wgrant: The answer I need is 'change X to make it be in the current series.
[08:40] <lifeless> wgrant: which may simply be 'do X to *set* a current series'.
[08:41] <wgrant> lifeless: Have you tried overriding the series that getPubSource uses?
[08:41] <lifeless> didn't know you could
[08:41] <wgrant> It wouldn't be very useful if you couldn't.
[08:42] <wgrant> ... so I guess that's a reasonable assumption.
[08:42] <lifeless> wgrant: I make no assumptions about usability and soyuz guts.
[08:47] <adeuring> good morning
[08:47] <wgrant> Morning adeuring.
[08:47] <adeuring> hi wgrant!
[08:48] <wgrant> adeuring: Bug 688130 needs QA, and I'm not really clear on how to do it.
[08:48] <_mup_> Bug #688130: Statistics clean-ups and extra tests <lp-translations> <qa-needstesting> <upstream-translations-sharing> <Launchpad itself:Fix Committed by adeuring> < https://launchpad.net/bugs/688130 >
[08:48] <adeuring> wgrant: I'll look at it
[08:48] <wgrant> Thanks.
[08:58] <lifeless> wgrant: this is a disaster
[08:58] <wgrant> lifeless: What is?
[08:58] <wgrant> lifeless: That test?
[08:59] <lifeless> yes
[08:59] <wgrant> Heh.
[09:02] <lifeless> wgrant: tip of lp:~lifeless/launchpad/bug-279513 if you're feeling interested
[09:02] <lifeless> it has a currentseries version of 1.0, not published
[09:05] <lifeless> oh... frell
[09:05] <lifeless> the current series changes when they make a new series midflight
[09:07] <lifeless> so it invalidates out the current series
[09:07] <wgrant> Hah, handy.
[09:11] <rvb> bigjools: Hi
[09:11] <bigjools> rvb: Morning!
[09:13] <lifeless> wgrant: whats the official way to freeze a distro series ?
[09:13] <lifeless> wgrant: just assign the status ?
[09:14] <wgrant> lifeless: Yes.
[09:15]  * bigjools brb
[09:19] <bigjools> rvb: how are you doing?
[09:20] <rvb> bigjools: all right!
[09:20] <rvb> bigjools: ready for action!
[09:20] <bigjools> rvb: I tried to send you a private message, did you get it?
[09:20] <rvb> bigjools: no
[09:21] <mrevell> Hi
[09:21] <rvb> bigjools: all right, I got it now
[09:33] <StevenK> gmb: O hai! Are you up for a review?
[09:33] <gmb> StevenK: Not just yet. If it's in the queue I'll take a look in the next 45 minutes or so.
[09:37] <StevenK> gmb: At some point today is perfectly fine.
[09:56] <LPCIBot> Project devel build #489: STILL FAILING in 5 hr 25 min: https://hudson.wedontsleep.org/job/devel/489/
[10:40] <wgrant> gmb: Hi.
[10:41] <gmb> wgrant: (Mor|Eve)ning
[10:41] <wgrant> gmb: Could you please review https://code.launchpad.net/~wgrant/launchpad/pygments-1.4/+merge/51725?
[10:41] <wgrant> Fixes the codebrowse hangs that have plagued us since the weekend.
[10:43] <gmb> Sure
[10:43] <gmb> Ooh, complex.
[10:43] <gmb> wgrant: Done.
[10:44] <wgrant> gmb: Thanks.
[10:44] <wgrant> Very complex, yes.
[10:46] <bigjools> wgrant: I love the way the decprecation warnings were overridden
[10:47] <wgrant> bigjools: That's the way to do it.
[10:48] <wgrant> adeuring: Any progress on that QA? I hope to be able to request a deploy in the morning to unbreak codebrowse.
[10:48] <wgrant> And I really don't know how to do it myself :/
[10:49] <adeuring> wgrant: sorry, got distracted, but I'm looking now,. I still  need  some time to figure out how to test the branch properly
[10:49] <wgrant> adeuring: Thanks.
[10:51] <bigjools> wgrant: I meant that it's ignoring a valid problem
[10:55] <wgrant> bigjools: Oh, right.
[11:04] <StevenK> gmb: Thank you for the review
[11:05] <gmb> np
[11:43] <Ursinha> morning launchpadders
[11:46] <wgrant> Rolling back thumper's branch.
[11:50] <bigjools> morning Ursinha
[12:01] <wallyworld_> WTF? ec2 land -> 0 tests run in 4:13:40.345985, 0 failures, 0 errors yet log shows no test failures.
[12:01] <wgrant> wallyworld_: You've not merged devel lately?
[12:01] <wgrant> I fixed that yesterday morning.
[12:02] <wallyworld_> wgrant: i did merge a few hours ago
[12:02] <wgrant> wallyworld_: Into the branch that you ran ec2 from?
[12:02] <wgrant> That's the one that matters, not the one that ran it *on*.
[12:03] <wgrant> Ah, crap, this testfix means my codebrowse branch is going to fail while I'm asleep.
[12:03] <wgrant> I might just lp-land it, then.
[12:03] <wallyworld_> wgrant: i'm confused. yes i did merge devel into the branch i submited to ec2 land, but ec2 land just take one's branch and merges into the latest trunk anyway, no?
[12:03] <wgrant> wallyworld_: Sort of.
[12:04] <wgrant> wallyworld_: The bug was in a file that is copied from the local branch.
[12:04] <jml> wallyworld_: may I see the log?
[12:04] <jml> wallyworld_: actually, nm.
[12:04] <wallyworld_> ah, didn't realise that was the case
[12:05] <wallyworld_> so the last 2 lines of the log should say "All tests passed" but instead say Tests failed (exit code 1)
[12:05] <wallyworld_> make: *** [check] Error 1
[12:05] <wgrant> Can you forward it to me or jml?
[12:05] <wallyworld_> i guess ill try merging devel again now and resubmit
[12:05] <wgrant> There's probably a failure in there somewhere.
[12:05] <wgrant> devel is broken at the moment.
[12:05] <wgrant> Fix in PQM.
[12:06] <wallyworld_> wgrant: ok. i'm fairly sure there's no failure but i can forward to log
[12:10] <wgrant> wallyworld_: There is an error.
[12:10] <wgrant> Search for 'error:' in the output.
[12:10] <wgrant> I would paste you details, but unity has decided to curse me.
[12:11] <wallyworld_> wgrant: !@^!^!@%^#! i'll take a look. i would have sworn on my mother's grave i had a proper look
[12:11] <wgrant> lp.code.browser.tests.test_sourcepackagerecipe.TestSourcePackageRecipeView.test_request_daily_builds_action
[12:12] <wallyworld_> wgrant: shit. thanks. i stupidly searched for "eror:" thanks for looking
[12:12] <wgrant> Haha.
[12:12]  * wallyworld_ hands head in shame
[12:13] <wallyworld_> hangs!
[12:13] <wallyworld_> i can't type
[12:13] <wgrant> At least you aren't Unity.
[12:17] <wgrant> Can someone watch buildbot tonight and make sure it stays happy?
[12:20]  * deryck finds it sad wgrant has to ask someone to watch buildbot
[12:20] <wgrant> deryck: I often get up and find that it has been broken for 6 hours.
[12:20] <wgrant> (often because of Windmill *cough*)
[12:21] <deryck> yeah
[12:21] <wgrant> Do you have any ideas on that?
[12:21] <wgrant> it leaks a thread or two in most runs, and occasionally fails. Although I got ten runs of just WindmillLayer through ec2 without a failure :(
[12:22] <deryck> wgrant, yes.  Two things... I *think* this is always related to not using timeouts in asserts.  And doing a layer in a for loop will cause a hang and we could attach a debug then.
[12:22] <deryck> wgrant, I hope to look at it soon, but gladly welcome anyone else checking into it.
[12:23] <wgrant> deryck: I was hoping to be able to reproduce it in a minimal ec2 run, since I have never been able to run it reliably locally.
[12:23] <deryck> by for loop, I mean -- for i in `seq 5`; do test --layer=CodeWindmillLayer; kind of thing.
[12:23] <deryck> wgrant, you can't run windmill locally at all?
[12:23] <wgrant> deryck: Almost every test fails.
[12:23] <deryck> can you increase the timeout constants and get it passing?
[12:23] <wgrant> They also hang occasionally on open()
[12:24] <wgrant> Possibly.
[12:24] <wgrant> I need to investigate.
[12:24] <wgrant> But production keeps exploding.
[12:25] <deryck> wgrant, right.  not saying you used look into it.  Just meant that anyone really should look into this.  But I will as soon as I can.
[12:25] <deryck> s/used/should/
[12:25] <wgrant> Well, nobody looks into test failures :)
[12:25] <wgrant> Or production issues.
[12:26] <wgrant> I am hoping it is as trivial as the librarian one.
[12:26] <deryck> this is true.
[12:26] <wgrant> But somehow I doubt it :(
[12:26] <deryck> sad but true
[12:26] <lifeless> night all
[12:26] <wgrant> Night lifeless.
[12:59] <wgrant> Night all.
[13:06] <LPCIBot> Project db-devel build #405: STILL FAILING in 5 hr 31 min: https://hudson.wedontsleep.org/job/db-devel/405/
[14:37] <sinzui> allenap: how goes you merge person job branch. Do you want me to land it it?
[14:39] <allenap> sinzui: I am trying to land it, but it refuses to go. Database permission errors in ec2 that I cannot replicate locally.
[14:39] <allenap> sinzui: I am looking at it now in fact.
[14:40] <sinzui> okay. I too spent a couple hours with that yesterday
[14:42] <sinzui> allenap: someone will need to add featureflagchangelog. I added that table in db-devel, and that was the one I forgot to give update permission on for merge
[14:43] <allenap> sinzui: Okay, I'll add it now.
[14:43] <sinzui> I do not think you can. It does not exist in devel
[14:44] <sinzui> allenap: or are you planing to land in db-devel instead
[14:44] <allenap> sinzui: I'll land in devel if I can, but if it rejects me I'll go to db-devel.
[14:44] <sinzui> oh duh, security.cfg changed, you do want to land in db-devel
[14:48] <allenap> sinzui: The problem I have had is that I grant only UPDATE on several tables to a new user, person-merge-job. Now this seems to work locally, but in ec2 it fails because an UPDATE with a WHERE clause needs SELECT granted too.
[14:52] <sinzui> interesting
[14:57] <sinzui> losa ping
[15:02] <viciousprimate> sinzui: heya, got dropped by my irc proxy.  i was lead to believe you have an issue worth discussing.
[15:03] <sinzui> The criminalisation of owning Chihuahuas?
[15:05] <benji> Indeed, Chihuahuas can not be "owned" they must run free through the streets like rabbid overgown rats.
[15:06] <viciousprimate> heh
[15:10] <sinzui> viciousprimate: what would you like to discuss?
[15:11] <viciousprimate> sinzui: oops, i didn't realize i didn't re-register as myself (lost my irc proxy this morning), one sec.
[15:12] <sinzui> ooh. just like Scooby Doo when Velma pulls of the villan's mask.
[15:12] <mbarnett> sinzui: sorry, i was trying to respond to a losa ping there, just failing with my technology this morning.
[15:12] <mbarnett> i would have gotten away with it, if it wasn't for those vicious primate.
[15:13] <sinzui> mbarnett: I am testing the the feature flag change log https://staging.launchpad.net/+feature-rules can you change the priority on a few of the items for me to review. maybe change all the 1s to 2s
[15:13] <sinzui> mbarnett: you will need to enter a comment too
[15:16] <mbarnett> so, for instance, change the 1 to 2 in "memcache	pageid:BugTask:+index	1" and "visible_render_time	team:launchpad	1	on" ? 	
[15:17] <sinzui> yep
[15:17] <mbarnett> kk, i'll change and log.
[15:19] <mbarnett> sinzui: done
[15:20] <sinzui> mbarnett: This looks okay to me https://staging.launchpad.net/+feature-changelog
[15:20] <sinzui> thanks mbarnett
[15:20] <mbarnett> welcome
[15:20] <sinzui> jcsackett_: ping
[15:21] <jcsackett_> hi sinzui.
[15:22] <leonardr> gmb, care to review https://code.launchpad.net/~leonardr/launchpad/bug-271010/+merge/51760?
[15:22] <LPCIBot> Project devel build #490: STILL FAILING in 5 hr 26 min: https://hudson.wedontsleep.org/job/devel/490/
[15:23] <sinzui> jcsackett: 1. I realised that giving the flag-expired-memberships a grace period would not work. The message contains a link to a url you cannot access, and it says your membership will expire 2 hours ago.
[15:23] <jcsackett> sinzui: that does sound like a problem.
[15:23] <sinzui> jcsackett: since the user has gotten 6 other emails about this, I updated the docstring to explain such cases will be silently dropped
[15:24] <jcsackett> sinzui: that sounds completely reasonable. if the user has already been warned, there's really no need to email them again in this edge-case.
[15:24] <gmb> leonardr: Sure
[15:24] <sinzui> jcsackett: 2 I am looking at bug 106338. It might interest you if you think you can get to it in a few days.
[15:24] <_mup_> Bug #106338: Editing a bug targeted to a release crashes if you directly edit the untargeted task  <api> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/106338 >
[15:25] <jcsackett> sinzui: oh goody, conjoined bugs.
[15:25] <jcsackett> i think i can get to that tomorrow or the day after, depending on how current message work goes.
[15:25] <jcsackett> sinzui: happy to take a look then, if that's a reasonable time table.
[15:26] <sinzui> jcsackett: okay. I will set it aside. I just looked at the oopses and see that the issue has changed, but I think it still fixable in 400 lines
[15:27] <jcsackett> sinzui: cool. preimp talk about it tomorrow during our usual 1-1?
[15:27] <sinzui> sure
[15:29] <leonardr> danilo, gmb, i will take https://code.launchpad.net/~danilo/launchpad/bug-720826-emails/+merge/51747
[15:29] <gmb> leonardr: I'm already looking at that
[15:29] <gmb> Sorry
[15:29] <gmb> Forgot to claim it.
[15:29] <leonardr> oh, nm
[15:30]  * leonardr assigns it back
[15:30] <gmb> Thanks
[15:31] <gmb> leonardr: Wow, it OOPSed when I tried to claim it right after you had.  That seems a little over the top. I'll file a bug.
[15:32] <danilos> gmb, leonardr: yay, fights over my branches again, I love that :)
[15:32] <danilos> it gets claimed back and forth, even oops fly around
[15:41] <bigjools> sinzui: question for you
[15:41] <bigjools> https://launchpad.dev/ubuntutest versus https://launchpad.dev/gentoo - the latter does not have the "Add series" link and I can't work out why
[15:42]  * sinzui looks
[15:43] <bigjools> the whole <div> has tal:condition=context/series
[15:43] <bigjools> which is repeated for the <ul>, heh
[15:43] <bigjools> so it seems like the addseries link should never be there unless you have existing series
[15:44] <sinzui> bigjools: IDerivativeDistribution will show it if you are a driver. IDistribution does not.
[15:44] <sinzui> oh, I think you are correct.
[15:44] <bigjools> however, ubuntutest does not have any series
[15:44] <bigjools> but it shows the link :)
[15:45] <sinzui> bigjools: I think this is vestigial from when we registered usless distros then realised we had to cripple the ui rather than fix the schema
[15:45] <bigjools> yeah I was guessing it was shitty sample data
[15:45] <bigjools> I need to fix this for derived distros
[15:45] <sinzui> Distros do not automatically get series when created I think
[15:46] <sinzui> bigjools: I think you need to rename the current IDerivativeDistribution to IRemixDistribution. baltix will not be built by soyuz or get translations
[15:48] <bigjools> yeah, we'll have to look at that
[15:49] <sinzui> bigjools: The template is just stupid https://launchpad.dev/gentoo/+series will let you add one
[15:49] <bigjools> sinzui: indeed
[16:13] <danilos> leonardr, gmb: hi, one small branch up for review: https://code.launchpad.net/~danilo/launchpad/devel-bug-720826-clear-level-on-delete/+merge/51768 (diff will show up in a bit)
[16:13] <leonardr> i'll take it
[16:14] <danilos> leonardr, thanks
[16:17] <stub> I've been getting PQM failures on a particular branch: http://paste.ubuntu.com/573990/
[16:17] <stub> Anyone seen similar?
[16:18] <bigjools> never seen that
[16:19] <bigjools> sinzui: is it my imagination or is there no link presented for /distros/+add anywhere?
[16:27] <leonardr> danilos, r=me
[16:28] <danilos> leonardr, thanks
[16:39] <stub> bigjools: Just me attempting to land on the wrong branch + logs of noise warnings
[16:39] <bigjools> stub: pebkac is also my middle name
[16:40] <danilos> leonardr, btw, I don't know of any better way to indicate a base-level default myself; let me check the interface
[16:41] <danilos> leonardr, the interface and model actually use a different value, the model one is correct (not that I'd know how to extract it nicely from the model or the interface)
[16:42] <leonardr> danilos, you mean they use different constants?
[16:42] <danilos> leonardr, yeah, I'll fix that bit
[16:44] <danilos> and I see I can get the default with IBugSubscriptionFilter.getDescriptionFor('bug_notification_level').default
[16:44] <danilos> leonardr, do you think I should use that in the code? (I'd leave the test as-is)
[16:44] <leonardr> yeah, use that in the code
[16:44] <danilos> leonardr, cool, thanks
[17:16] <danilos> gmb, thanks for the review
[17:17] <gmb> np
[17:41] <bigjools> sinzui: so the +addseries was appearing when there was an experimental series that's not actually showing in that portlet
[17:50] <sinzui> bigjools: yes, that is what +series showed. Sorry for not stating that
[17:50] <bigjools> sinzui: no problem - I intend to try and make it all work much nicer
[17:50] <sinzui> bigjools: experimental, obsolete are not manages by the core developers, so we hide them
[17:51] <bigjools> this stuff needs to be easy to use for non-admins when we do the derived distros
[18:03] <lifeless> moin
[18:11] <sinzui> Subclasses enums are not equal to the superclass enum
[18:11] <sinzui> :)
[18:12] <leonardr> gmb, are you still reviewing my branch or should i try to get someone else to do it?
[18:37] <LPCIBot> Project db-devel build #406: STILL FAILING in 5 hr 30 min: https://hudson.wedontsleep.org/job/db-devel/406/
[18:48] <lifeless> bbiab - allergy vaccination time, and the city is still a mess from the quake, so may be longer than desirable :(
[18:49] <sinzui> can someone who cares about storm and enum give me an opinion about bug 727331. I think I want to implement enum equality rather than create a vocab factory
[18:49] <_mup_> Bug #727331: Subclassed DBEnum item is not equal to the super class item <infrastruture> <storm> <Launchpad itself:Triaged> < https://launchpad.net/bugs/727331 >
[18:53]  * sinzui looks for another bug
[18:56] <jam> lifeless: I poked at https://bugs.launchpad.net/launchpad/+bug/727148
[18:56] <_mup_> Bug #727148: Bzr timeouts on SSH connections <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/727148 >
[18:56] <gmb> leonardr: Ark, sorry, I thought I'd voted on it.
[18:56] <jam> And I can confirm routing failure from people.canonical.com
[18:56] <jam> but success from home
[18:56] <gmb> Let me just take another look.
[18:57] <jam> which hints that it is a datacenter firewall/routing issue
[18:57] <gmb> leonardr: Done. Apologies; I'd obviously looked and gone "yeah, that's fine" but the brain->LP interface was down.
[18:58] <leonardr> cool
[19:00] <leonardr> did something happen to lib/canonical/widgets recently? it seems to be gone from my copy of devel, but there's still code that tries to import it
[19:05] <leonardr> that importing code looks bad to me. has no anyone else noticed this?
[19:05] <leonardr> looks like the widgets were moved to lp.apps.widgets.itemswidgets and the shipit code wasn't changed
[19:07] <leonardr> weird... anyway
[19:09] <james_w> leonardr, run update-sourcecode I believe
[20:18]  * thumper is here, coffee in hand
[20:30] <sinzui> leonardr: I moved many of the widgets to lp.app. Other parts of the tree got widgets too. wgrant destroyed the dir after stabbing shipit is a spoon until it yielded.
[20:31] <leonardr> sinzui: thanks, i've figured it out. i didn't know shipit was in sourcecode
[20:32] <sinzui> leonardr: it is symlinked. it is not obvious and you may not know it until ec2 sends you hate mail.
[20:49] <LPCIBot> Yippie, build fixed!
[20:49] <LPCIBot> Project devel build #491: FIXED in 5 hr 26 min: https://hudson.wedontsleep.org/job/devel/491/
[20:54] <thumper> hmm...
[20:54] <thumper> wgrant: ping
[20:59] <maxb> Would someone be able to send me OOPS-1886CMP2 including the full traceback and the text of the triggering email? (re https://answers.launchpad.net/launchpad/+question/147254)
[21:00] <maxb> CMP oopses are apparently stuck locally on crowberry and not being propagated centrally unless something was fixed since yesterday
[21:03] <sinzui> thumper: may I have your opion of bug 727331.
[21:03] <_mup_> Bug #727331: Subclassed DBEnum item is not equal to the super class item <infrastruture> <storm> <Launchpad itself:Triaged> < https://launchpad.net/bugs/727331 >
[21:03]  * thumper looks
[21:04] <wallyworld> thumper: leonardr: standup?
[21:04] <thumper> sinzui: hmm... this was an interesting design decision
[21:05] <thumper> sinzui: if you are around, we could talk after my standup
[21:05] <thumper> wallyworld: yeah
[21:05] <sinzui> thumper: wont fix is an legitimate reply. I could make a vocab factory and test in 50 lines, I just do not want to
[21:23] <lifeless> jam: thanks
[21:23]  * lifeless is back
[21:24] <thumper> sinzui: can you mumble?
[21:31] <wgrant> thumper: Hi.
[21:31] <thumper> wgrant: hi
[21:32] <thumper> wgrant: sorry about my branch yesterday, I was sure I did ec2 land
[21:32] <thumper> bug my bash history tells me different
[21:32] <wgrant> I noticed it landed very quickly, but decided not to query you.
[21:32]  * sinzui starts mumble
[21:33] <wgrant> fuuuu
[21:33] <wgrant> buildbot has failed.
[21:34] <maxb> wgrant: Hi, would you be able (now or later) be able to dig out OOPS-1886CMP2 for me, including traceback & email this time? (Same issue as yesterday)
[21:35] <wgrant> maxb: Looking.
[21:37] <wgrant> maxb: I've requested a log sync.
[21:39] <maxb> thanks
[21:40] <lifeless> bah
[21:40] <lifeless> https://api.qastaging.launchpad.net/1.0/branches
[21:40] <lifeless> I may have  broken that api
[21:40] <lifeless> yeah
[21:41] <lifeless> ForbiddenAttribute: ('branchID', <lp.code.model.seriessourcepackagebranch.SeriesSourcePackageBranch object at 0x2b58d42a97d0>)
[21:41] <wgrant> And we were doing so well for the last couple of weeks.
[21:41] <wgrant> Now we have the chain of pain and suffering again :(
[21:41] <lifeless> well
[21:42] <lifeless> its the result of not deploying daily
[21:42] <lifeless> we can still deploy some stuff
[21:44] <wgrant> We need a qa-reallydontcare
[21:44] <wgrant> For things that are hard to test, largely inconsequential, and remain un-QA'd through two European days.
[21:45] <wgrant> thumper: I think we may have to untestable your Blueprint thing.
[21:45] <wgrant> Or convince a friendly GSA to setup Blueprint mail.
[21:45] <thumper> wgrant: yeah
[21:45] <lifeless> wgrant: qa-untestable it
[21:46] <lifeless> wgrant: untestable covers a multitude of cases
[21:46] <wgrant> lifeless: The worst 12469 can do is break pofilestats, and it was already broken until last week.
[21:46] <wgrant> So yeah, doing so.
[21:52] <wgrant> maxb: BzrCheckError: Internal check failed: Cannot add revision(s) to repository: missing referenced chk root keys: [StaticTuple('sha1:bc9ad731c2af5f4d5b71cbad086b2a0758b8936a',)]
[21:52] <wgrant> Still want more details?
[21:53] <wgrant> lifeless: Do you have a fix in progress for your breakage?
[21:53] <lifeless> wgrant: was just about to start the branch
[21:53] <lifeless> wgrant: if you want to do it that would be awesome
[21:54] <wgrant> lifeless: 'fraid I have enough other time-critical stuff to do.
[21:54] <lifeless> wgrant: its just add the ID (and check for others i may have used that are neither covered by tests nor in the existing interfaces)
[21:54] <lifeless> wgrant: I'm happy to do it
[21:54] <wgrant> lifeless: Sadly you are going to miss the next buildbot run by a few minutes.
[21:54] <wgrant> We might want to hold that or something.
[21:54] <lifeless> wgrant: I'm assessing https://bugs.launchpad.net/launchpad/+bug/724033 first - checking I haven't broken bugs too
[21:54] <_mup_> Bug #724033: BugTask:+index timeouts - distributions and milestones being late evaluation loaded - repeatedly - on bug 230350 <qa-needstesting> <timeout> <Launchpad itself:Fix Committed by lifeless> < https://launchpad.net/bugs/724033 >
[21:56] <wgrant> OK, I am disabling Windmill until someone (maybe me) sorts it out.
[21:56] <wgrant> Unless someone complains quickly.
[21:58] <lifeless> SQL time: 5017 ms
[21:58] <lifeless> Non-sql time: 10126 ms
[21:58] <lifeless> grr
[21:58] <lifeless> https://bugs.qastaging.launchpad.net/ubuntu/+source/afflib/+bug/230350/+index
[21:59] <_mup_> Bug #230350: Missing Debian Maintainer field <Ubuntu:Invalid> <afflib (Ubuntu):Fix Released by saivann> <alac-decoder (Ubuntu):Fix Released by warp10> <axiom (Ubuntu):Invalid> <beneath-a-steel-sky (Ubuntu):Fix Released> <bibletime-i18n (Ubuntu):Fix Released by txwikinger> <binkd (Ubuntu):Fix Released> <bzr-builddeb (Ubuntu):Won't Fix> <capisuite (Ubuntu):Invalid> <chinput (Ubuntu):Fix Released by warp10> <chmsee (Ubuntu):Fix Released> <ciso (U
[21:59] <wgrant> lifeless: Hey, you are making progress on single-threaded appservers.
[21:59] <wgrant> There is hope.
[21:59] <lifeless> this is on qastaging
[21:59] <wgrant> Ah.
[21:59] <wgrant> Point.
[21:59] <lifeless> so I think this particular case is genuine
[21:59] <lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1886QS95
[21:59] <lifeless> things like query 4 make me wonder about pathological stuff
[22:00] <lifeless> (4 by time)
[22:01] <lifeless> holy cow
[22:01] <wgrant> Hm?
[22:02] <lifeless> query 20 in the timeline
[22:02] <lifeless> we have -lots- of tasks on this bug
[22:03] <wgrant> Ohh, one of these, right.
[22:03] <lifeless> after query 36 we waste .7 of a second doing  ???
[22:03] <wgrant> There are only a few.
[22:05] <lifeless> bug one down to 352 queries
[22:05] <wgrant> Unauthenticated?
[22:05] <lifeless> authenticated
[22:05] <wgrant> I don't believe you.
[22:05] <wgrant> Oh.
[22:05] <lifeless> hit it on qa staging
[22:05] <wgrant> No memcache, I guess.
[22:06] <wgrant> That's cheating.
[22:06] <lifeless> more than that
[22:06] <lifeless> bug 724033
[22:06] <_mup_> Bug #724033: BugTask:+index timeouts - distributions and milestones being late evaluation loaded - repeatedly - on bug 230350 <qa-needstesting> <timeout> <Launchpad itself:Fix Committed by lifeless> < https://launchpad.net/bugs/724033 >
[22:06] <wgrant> Huh, confirmed.
[22:08] <lifeless> I wouldn't be so mean as to get your hopes up needlessly
[22:08] <wgrant> buildbot does, so why not you too?
[22:08] <lifeless> because I am a real machine
[22:09] <wallyworld> thumper: if you have time to look, here's the WrongStoreError i've been getting https://pastebin.canonical.com/44117/
[22:11] <lifeless> wgrant: whats wrong with 218384
[22:13] <wgrant> Bug #218384
[22:13] <_mup_> Bug #218384: OOPS processing Mantis bugwatches <bad-commit-12487> <bugwatch> <lp-bugs> <oops> <qa-bad> <story-reliable-bug-syncing> <Launchpad itself:Fix Committed by thumper> < https://launchpad.net/bugs/218384 >
[22:13] <wgrant> lifeless: It exploded the test suite.
[22:13] <lifeless> ass
[22:14] <wgrant> See what I mean?
[22:14] <lifeless> thumper: did you ec2 land it, or direct?
[22:14] <wgrant> Direct.
[22:14] <lifeless> wgrant: you're rolling it back ?
[22:14] <wgrant> lifeless: I rolled it back last night.
[22:14] <lifeless> ok, cool
[22:14] <wgrant> The rollback is through buildbot.
[22:16] <wgrant> lifeless: Still 77 Person and 48 VPC queries :(
[22:16] <wgrant> 37 TP, 21 Milestone.
[22:19] <lifeless> wgrant: yes, step at a time.
[22:23] <wgrant> StevenK: Around?
[22:24] <wgrant> I wonder if we can set up Jenkins to run WindmillLayer in a separate job.
[22:24] <wgrant> So it doesn't bitrot too much and we are still pushed towards fixing.
[22:25] <lifeless> wgrant: we can
[22:25] <lifeless> wgrant: you'll need an entry point to run it is all
[22:26] <wgrant> Great.
[22:26] <wgrant> Thanks.
[22:31] <jcsackett> ah, mumble, how i love your crashing...
[22:39] <lifeless> ok, sigh
[22:40] <lifeless> that branches collection is barely tested :(
[22:42] <lifeless> one liner to fix.
[22:43] <wgrant> lifeless: No other fallout?
[22:43] <lifeless> it gets to the last clause of the eager load method
[22:44] <lifeless> wgrant: check 12505 in lp:~lifeless/launchpad/bug-722956
[22:44] <lifeless> wgrant: are we in testfix?
[22:44] <wgrant> lifeless: No.
[22:44] <maxb> wgrant: thanks for the oops digging. This is sounding like it could be bug 718723, though if so I'm confused why it didn't bite me when I tried it
[22:44] <_mup_> Bug #718723: fetch from merge directive to stacked branch unable to fill in chk pages <Bazaar:Confirmed> < https://launchpad.net/bugs/718723 >
[22:45] <wgrant> lifeless: Aha.
[22:45] <lifeless> wgrant: this is a storm class, the branchID Int column was already defined - I didn't add it in my original patch.
[22:45] <lifeless> wgrant: and I didn't think to check that it was on the interface, given it already existed.
[22:46] <lifeless> its in PQM now.
[22:46] <wgrant> Thanks.
[22:46] <wgrant> I am giving Windmill one last chance to hang on my laptop before I kill it.
[22:46] <lifeless> we are in testfix
[22:47] <wgrant> Then we should have a reliable test suite again.
[22:47] <wgrant> lifeless: Oh, I thought a new build had started.
[22:47] <wgrant> Fail.
[22:48] <lifeless> resubmitted
[22:48] <wgrant> Thanks.
[22:49] <wgrant> lifeless: No [rollback=]?
[22:49] <lifeless> bah
[22:49] <lifeless> ETOOLATE
[22:50] <wgrant> OK, we'll just have to work it out manually.
[22:51] <lifeless> mbarnett: is spm around today?
[22:51] <mbarnett> lifeless: yup,
[22:51] <mbarnett> lifeless: or i should say, that is the plan.
[22:52] <lifeless> wgrant: shall we get 12486 deployed before mbarnett vanishes?
[22:52] <lifeless> s/ed/ing/
[22:52] <mbarnett> need a nodowntime fired off?
[22:52] <lifeless> mbarnett: I'm thinking so
[22:52] <wgrant> I was considering that.
[22:53] <lifeless> wgrant: you're still fighting fires?
[22:53] <wgrant> lifeless: Clubbing Windmill to death.
[22:53] <lifeless> wgrant: I'll prep it
[22:53] <wgrant> Thanks.
[22:53] <lifeless> Ursinha: unless you want to, for more practice? Its a rather bigger one this time
[22:53] <wgrant> Our biggest in a while :(
[22:54] <wgrant> Even the next one is not going to be exactly small.
[22:55] <lifeless> mbarnett: when do you turn into a puff of smoke ?
[22:55] <mbarnett> lifeless: in a little under an hour most likely.
[22:56] <lifeless> mbarnett: ok, I'll have this prepped in 15 - need to shove some food in my gob first
[22:56] <mbarnett> kk
[22:56]  * mbarnett understands the need for feasting. 
[23:00] <lifeless> Ursinha: I'm prepping it
[23:05] <lifeless> mbarnett: its up
[23:06] <lifeless> 26 bugs
[23:06] <mbarnett> lifeless: it shall be done.
[23:06] <lifeless> 2 and a half days
[23:15] <wallyworld> lifeless: the linked_bugs property on BranchMergeProposalView doesn't filter out private bug. do you agree this is incorrect and i should  use check_permission() to stop private bugs being exposed?
[23:16] <lifeless> yes and no
[23:17] <lifeless> better to load them correctly in the first place
[23:17] <lifeless> that way the collection size will be correct, etc et
[23:22] <wallyworld> lifeless: the code to load them is in the model - the linked_bugs attribute on a branch, which is defined an an SQLRelatedJoin
[23:22] <lifeless> yes
[23:22] <lifeless> its a bug
[23:22] <lifeless> on several levels :)
[23:22] <wallyworld> so there's no concept there of what user it trying to do the accessing
[23:22] <lifeless> right
[23:22] <lifeless> fix that
[23:23] <lifeless> give me a sec to do gc on my branches
[23:23] <lifeless> I may have some half done thing in this area
[23:24] <wallyworld> lifeless: ok. i'm working on another issue and came across this. if i were to do a quick fix to the linked_bugs property on the view, that works fine for now and there's no issue with collection sizes etc
[23:24] <wallyworld> that i can see
[23:25] <lifeless> wallyworld: uhm, I think thats stale
[23:25] <lifeless> check_permission will trigger late evaluation anyhow
[23:26] <wallyworld> late evaluation of what? the object having its permission checked?
[23:26] <lifeless> yes
[23:27] <lifeless> wallyworld: ok, I've paged this in
[23:27] <lifeless> I got half way through refactoring
[23:27] <lifeless> see Branch.getLinkedBugTasks
[23:28] <lifeless> most of the templates now use linked_bugtasks rather than linked_bugs (because in LP we work with bugtasks not bugs)
[23:28] <wallyworld> ok. but sometimes we want bugs
[23:28] <lifeless> wallyworld: no, we don't.
[23:28] <lifeless> wallyworld: we think we do, but its a mistake.
[23:28] <wallyworld> like showing the linked bugs for merge proposals? so we want to show linked bug tasks instead?
[23:29] <lifeless> we already show bug tasks.
[23:29] <lifeless> wallyworld: bugs have no importance or severity
[23:29] <wallyworld> ok
[23:29] <lifeless> wallyworld: see DecoratedBug.bugtask *or* the implementation of Branch.getLinkedBugTasks
[23:29] <lifeless> wallyworld: either of those show the selection for the bugtask to show
[23:30] <lifeless> wallyworld: so, yes, the data model links to *bug*, but the UI then *picks a task* to show.
[23:30] <wallyworld> BranchMergeProposalView has both linked_bugs AND linked_bugstasks properties, so it seems some work is needed there
[23:30] <lifeless> I added linked_bugtasks to land my performance for BugTask:+index
[23:30] <wallyworld> ah ok
[23:31] <lifeless> there was some stuff still referenving linked_bugs on the BMP template so I left it in the short term.
[23:31] <lifeless> wallyworld: we were doing 1000 queries dealing with linked_bugs
[23:31] <wallyworld> so getLinkedBugTasks does the right thing with respect to permissions/private etc?
[23:31] <lifeless> wallyworld: due entirely to late evaluation
[23:31] <maxb> Oh dear.... if I was to suggest that emailed in bundles to create merge proposals was completely broken, does that sound possible?
[23:31] <lifeless> wallyworld: yes, what it returns is usable, visible and [mostly] eager loaded already.
[23:31] <lifeless> maxb: yes
[23:31] <wallyworld> if so i'll remove linked_bugs from bmp view
[23:32] <wallyworld> and write some tests - there were none at all for this
[23:32] <lifeless> there are view tests
[23:32] <lifeless> but yeah, none dealing with permissions
[23:32] <lifeless> OTOH
[23:32] <lifeless> the *only* reason there are permission issues
[23:32] <wallyworld> not that check that private linked bug[tasks] are not visible
[23:32] <lifeless> is because code wasn't using the bugs interface for getting bugs.
[23:33] <wallyworld> because it was navigating the object model starting at a branch
[23:33] <lifeless> no
[23:33] <lifeless> because it used direct queries to grab bugs
[23:33] <lifeless> rather than getUtility(IBugTaskSet).search
[23:34] <lifeless> which is the -only- defined way to get bug tasks.
[23:34] <lifeless> everything else either:
[23:34] <lifeless> a) layers
[23:34] <lifeless> b) is buggy
[23:34] <wallyworld> it wasn't using direct queries that i saw - it was calling branch.linked_bugs
[23:34] <lifeless> linked_bugs = SQLRelatedJoin(..)
[23:34] <wallyworld> which may be using a direct query under the covers
[23:34] <lifeless> ^ - thats a direct query
[23:34] <wallyworld> but conceptually is an object model navigation thing
[23:34] <lifeless> but its broken
[23:35] <lifeless> we can't use object model navigation for most of our work
[23:35] <wallyworld> the direct query is an implementation detail :-)
[23:35] <lifeless> not IMO
[23:35] <wallyworld> agreed +100
[23:35] <lifeless> visibility depends on the current interaction
[23:35] <lifeless> we have some tensions
[23:35] <lifeless> we have half a container
[23:36] <wallyworld> yes, and using object model navigation sucks for that, i agree.
[23:36] <lifeless> we don't properly support (e.g. fast, efficient, clear) accessing current user from 'anywhere'
[23:36] <wallyworld> everything should be service based imho
[23:36] <lifeless> nor do we pass current user around pervasively
[23:36] <wallyworld> yep
[23:36] <lifeless> the big thing I want is thinness
[23:36] <lifeless> I want us to drop about 50% of our classes
[23:36] <lifeless> 60-70% of our interfaces
[23:37] <wallyworld> no argument from me for dropping code
[23:37] <lifeless> probably a 2012 thing
[23:37] <lifeless> we have many areas that are self-inflicted complexity.
[23:37] <wallyworld> so for now, i'll use your new getBugTasks stuff and refactor out the use of linked_bugs from bmp view
[23:37] <lifeless> (e.g. the conjoined master thing: I *still* haven't had an explanation of /why/ that exists)
[23:38] <lifeless> wallyworld: cool
[23:38] <wallyworld> that will solve the immediate permisisons issue
[23:38] <wallyworld> and then i can use that as the basis for the current issue i am working on, wheich is that the branch index page blows up if there's a private bug linked to a mp
[23:38] <lifeless> wallyworld: cool. Thats the last use of DecoratedBug btw
[23:39] <lifeless> you can delete it after you're done.
[23:39] <wallyworld> resulting from the previous stuff i did to show the mp and linked bugs for revisions
[23:39] <wallyworld> ok will do
[23:39] <wallyworld> lifeless: thanks for the input
[23:40] <lifeless> my pleasure
[23:44] <wgrant> StevenK: So, how much work is an extra Jenkins job with --layer=WindmillLayer?
[23:47] <StevenK> For just devel or both {,db-}devel?
[23:47] <wgrant> Just db-devel.
[23:48] <wgrant> Little point running it for both.
[23:48] <wgrant> And db-devel will catch everything.
[23:48] <StevenK> It's pretty easy
[23:48] <wgrant> It could even just run in the downtime when one of the other slaves is idle, if you don't have enough.
[23:49] <StevenK> I don't think that is required
[23:49] <StevenK> Wow. checkwatches.txt didn't break
[23:50] <wgrant> I introduced some new checkwatches tests yesterday, which may have caused it to reorder.
[23:50] <lifeless> wallyworld: this is one of the sample pages that was dying
[23:50] <lifeless> https://code.qastaging.launchpad.net/~software-store-developers/software-center/trunk/+index
[23:50] <wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/die-windmill-die/+merge/51836
[23:50] <StevenK> wgrant: Perhaps checkwatches.txt needs to die :-)
[23:50] <wallyworld> lifeless: yeah, i think from memory that one is listed in the bug report
[23:50] <lifeless> wallyworld: to give you an idea of data sizes with bug-branch links: 375 links
[23:51] <lifeless> wallyworld: which is also why I filed the bug about merge-proposal bug links - if that branch *ever* merges into another, the branch index will be showing 375 bugs against that revision.
[23:51] <lifeless> wallyworld: which is, frankly, insane.
[23:52] <lifeless> (a preexisting data model problem to your work: your using what we have in the model just made it much clearer)
[23:52] <wallyworld> lifeless: when i worked on the bug asking for this functionality, i never envisiaged a branch would have so many linked bugs
[23:52] <wallyworld> actually, a mp i mean
[23:53] <lifeless> right. But the mp 'related bugs' is defined as 'those of the source on in the linked bugs of the target'
[23:53] <wallyworld> because for each recent revision, we show the mp which resulted in that revision and the linked bugs are against the mp
[23:53] <lifeless> wallyworld: which is a definitional problem: really it should be 'the open bugs of the source from when the MP was started till it landed'
[23:54] <lifeless> wallyworld: a merge proposal represents a temporal range; but the bugs we show have no limits. There are other bugs about fallout from this
[23:54] <wgrant> StevenK: It probably needs to die, yes, but it's not breaking in buildbot so I care slightly less.
[23:55] <wallyworld> lifeless: the way we work in lp, the life of the source branch is closely tied to the life of the mp
[23:55] <wallyworld> so its a moot point for us
[23:55] <wallyworld> but you are saying other projects work differently?
[23:57] <lifeless> wallyworld: *we* don't work that way.
[23:57] <lifeless> wallyworld: you may. I don't. Stub doesn't. Others don't.
[23:57] <wallyworld> ah ok
[23:57] <lifeless> wallyworld: the lp project definitely doesn't: devel merges to stable multiple times a day.
[23:57] <wgrant> StevenK: So, I can has review?
[23:57] <wallyworld> i create a new feature branch for each new bug etc
[23:58] <lifeless> wallyworld: stable to db-devel. db-devel to db-stable.
[23:58] <wallyworld> yes, you are right
[23:58] <wallyworld> my thinking was far too narrow
[23:59] <lifeless> :)