[04:57] <lifeless> [04:57] <lifeless>     Hard / Soft  Page ID
[04:57] <lifeless>      270 /   12  Archive:EntryResource:getBuildSummariesForSourceIds
[04:57] <lifeless>      151 /   43  Person:+commentedbugs
[04:57] <lifeless>       75 /  269  CodeImportSchedulerApplication:CodeImportSchedulerAPI
[04:57] <lifeless>       58 /  230  BugTask:+index
[04:57] <lifeless>       10 /  265  Distribution:+bugs
[04:57] <lifeless>        7 /   63  Archive:+packages
[04:57] <lifeless>        7 /    7  ProjectGroup:+milestones
[04:57] <lifeless>        6 /   62  MaloneApplication:+bugs
[04:57] <lifeless>        6 /    9  Archive:+copy-packages
[04:57] <lifeless>        5 /    1  BugTask:+editstatus-page
[04:57] <thumper> lifeless: no mailing list ones :)
[04:58] <lifeless> because we raised the timeout
[04:59] <lifeless> hard_timeoutpageid:MailingListApplication:MailingListAPIView617000
[04:59] <lifeless> https://launchpad.net/+feature-rules
[05:01] <thumper> :(
[05:01] <thumper> I thought my fix would have fixed that
[05:01] <thumper> lifeless: it isn't in the top 10 statement counts any more
[05:02] <lifeless> thats good
[05:02] <thumper> lifeless: worth checking, but we should be able to get rid of that exception
[05:02] <lifeless> it may be that we can remove ... yeah
[05:02] <lifeless> see what the ppr shows
[05:02] <thumper> check tomorrow
[05:02]  * thumper agrees
[05:03] <thumper> lifeless: I am a little upset though, I was hoping it whould have made some of the code import scheduler ones go away
[05:03] <thumper> but not
[05:03] <thumper> s/not/no
[05:04] <lifeless> thumper: https://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-pageids.html
[05:04] <lifeless> thumper: it looks like its peaking at 10s now
[05:05] <lifeless> thumper: so improved - yes.
[05:05] <thumper> I'm guessing it is a different call that is peaking
[05:05] <lifeless> thumper: I think the big thing is getting it off of the one machine.
[05:05] <thumper> hence my desire for the method to be included in the pageid
[05:05] <lifeless> thumper: about 0.01% of calls I'd say
[05:05] <thumper> Failed to discover an OpenID server?
[05:05] <lifeless> thumper: I agree, definitely doit
[05:05] <lifeless> thumper: hmmm, I saw that the other day
[05:05] <lifeless> I think SSO is having trouble of some sort
[05:06]  * thumper feels a bid sad that he is on a work channel on Sunday afternoon :-|
[05:06]  * thumper leaves
[17:28] <lifeless> morning
[20:59] <thumper> morning
[21:05] <wallyworld> morning
[21:08] <thumper> wallyworld: hi, I'm just moving, we can have a chat after that if you like
[21:08] <thumper> wallyworld: or do you have kid stuff to do?
[21:09] <wallyworld> thumper: i have school drop off in 30 minutes; that will tale 20 minutes or so; i can ping you when i'm back?
[21:09] <thumper> yep
[21:09] <thumper> ttyl
[21:39] <lifeless> any soyuz folk around ?
[21:46] <wgrant> lifeless: Hi.
[21:50] <lifeless> hi
[21:50] <lifeless> so I'm fixing the timeout
[21:50] <lifeless> ArchivePublication injects itself into the code path for getBuilds
[21:50] <lifeless> and this makes things slow.
[21:50] <lifeless> there is a test that this works
[21:50] <lifeless> but its being done 'to make things fast' - oh, the irony.
[21:50] <wgrant> It makes things slow?
[21:50] <wgrant> Nice.
[21:50] <wgrant> How is it making things slow?
[21:51] <lifeless> the API doesn't use ArchivePublication
[21:51] <lifeless> https://bugs.launchpad.net/soyuz/+bug/662523
[21:51] <_mup_> Bug #662523: Archive:EntryResource:getBuildSummariesForSourceIds times out <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/662523>
[21:51] <wgrant> Ah, the API timeout, I see.
[21:51] <lifeless> so, I'm wondering whether, when the core is fast, if that delegate is still needed
[21:51] <lifeless> or if I need to support the injection (hacky) because other layers will be impacted
[21:51] <lifeless> I'm groupifying the code
[21:51] <wgrant> Where's the injection?
[21:52] <lifeless> gimme a minute to get the test passing and I'll push so you can see
[21:52] <lifeless> lib/lp/soyuz/adapters/archivesourcepublication.py
[21:52] <lifeless> getStatusSummaryForBuilds
[21:52] <wgrant> Ahh, I see.
[21:53] <wgrant> It should be fast enough if you rewrite it. It's only there to cache getBuilds and getUnpublishedBuilds.
[21:56] <lifeless> Using saved push location: lp:~lifeless/launchpad/getBuildSummariesForSourceIds
[21:57] <lifeless> rev 11878 should be there now
[22:13] <thumper> wallyworld: you back?
[22:17] <wallyworld> thumper: yep
[22:17] <thumper> wallyworld: shall we mumble?
[22:17] <wallyworld> already logged in
[22:54] <lifeless> aahhhhhhhhhh
[22:55] <wgrant> What have you fixed?
[22:55] <lifeless> getBuildStatusSummariesForSourceIdsAndArchive calls getBuildsForSourceIds and getUnpublishedBuilds
[22:55] <lifeless> getUnpublishedBuilds calls getBuildsForSourceIds
[22:56] <lifeless> oh
[22:56] <lifeless> and did I mention that we do a (potentially -huge-) set difference in python? :last line of getUnpublishedBuildsForSources
[22:58] <wgrant> lifeless: That .difference is a Storm operation.
[22:58] <wgrant> It's done in SQL.
[22:59] <lifeless> ok, thats a relief.
[22:59] <lifeless> But it totally sucks that you can't tel.
[22:59] <lifeless> *tell*
[22:59] <wgrant> I only know because I looked at the SQL and saw the EXCEPT.
[22:59] <lifeless> so
[23:00] <lifeless> we *still* end up doing the same thing twice
[23:00] <lifeless> except more of its on the db server
[23:00] <lifeless> I'll batch that up
[23:00] <wgrant> Great.
[23:00] <lifeless> and then call and end to this branch
[23:02] <wgrant> I might be able to look at it after Thursday.
[23:06] <lifeless> that would be sweet.
[23:06] <lifeless> I think it will be approximately constant queries now
[23:06] <lifeless> though I haven't reached the activation energy to write a test for that
[23:06] <wgrant> Excellent.
[23:06] <lifeless> I've just been refactoring as I read stuff
[23:06] <wgrant> It's very easy to get carried away refactoring in Soyuz...
[23:06] <lifeless> the core query is still going ot be slow (8.4 regression)
[23:07] <lifeless> I'd really like to see:
[23:07] <lifeless>  - the adapter nuked
[23:07] <wgrant> Now that we're more allowed to cache in model objects, sure.
[23:07] <lifeless>  - only one retrieval of the spph's (vs twice in my branch (or 3N in trunk))
[23:07] <lifeless> wgrant: caching isn't needed
[23:08] <lifeless> wgrant: you can cache, but its not what was needed
[23:08] <wgrant> ArchiveSourcePackageRelease may still be needed.
[23:08] <lifeless> sure
[23:08] <lifeless> theres a lot of refactoring to do
[23:08] <lifeless> many single object methods to nuke
[23:10] <wgrant> I wonder if we should strongly encourage query count tests for new APIs and views.
[23:12] <lifeless> certainly
[23:12] <lifeless> if you're writing something new, a query count test at the start is an awesome idea.
[23:13] <lifeless> https://code.launchpad.net/~lifeless/launchpad/getBuildSummariesForSourceIds/+merge/40298 if you're interested in this branch
[23:13] <wgrant> I was just looking for it.
[23:13] <wgrant> Any chance you could ec2 https://code.edge.launchpad.net/~wgrant/launchpad/destroy-distroseries-lucilleconfig/+merge/38648? PQM testfix-rejected it twice last week.
[23:14] <lifeless> what flags
[23:14] <lifeless> oh
[23:14] <lifeless> we're in testfix right now
[23:14] <lifeless> what are the odds
[23:14] <wgrant> Awesome.
[23:15] <lifeless> 38 mins to go
[23:15] <lifeless> that should have the fix from sinzui
[23:15] <wgrant> Ah, sinzui's fix still hasn't made it through?
[23:15] <wgrant> Right.
[23:15] <lifeless> so
[23:15] <lifeless> what flags do you want?
[23:15] <lifeless> --no-qa?
[23:15] <wgrant> --no-qa was what was used last time.
[23:15] <wgrant> So sounds good.
[23:16] <lifeless> ok, kicked it off
[23:16] <wgrant> Thanks.
[23:16] <lifeless> de nada
[23:20] <lifeless> jelmer: when you get up; https://bugs.launchpad.net/soyuz/+bug/669676 still needs qa, and its blocking further deployments.
[23:23] <wgrant> It's easy enough to QA that on qastaging like we did last week.
[23:24] <lifeless> a bit more involved
[23:24] <lifeless> given we're looking for race conditions
[23:25] <lifeless> anyhow, we'll be only 4 days back once we deploy up to it
[23:27] <wgrant> Hm.
[23:28] <wgrant> I think that has a bug.
[23:29] <wgrant> Yeah, it looks like it has the same bug that r11812 fixed.