[00:43] <StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/fetchpage-does-not-send-headers-always/+merge/132638
[00:46] <wallyworld_> StevenK: i don't know the code - why can't those 2 classes be change to use getPage()?
[00:46] <StevenK> I tried, and Trac broke
[00:48] <wallyworld_> it just seems very hacky is all - fetchPage seems to expect a Request and nothing else
[00:48] <wallyworld_> so it shouldn't be called with a string
[00:49] <wallyworld_> and if it is then that's a mistake to be corrected upstream so to speak
[00:49] <StevenK> It just tosses things into urlopen, and that will deal with either
[00:49] <StevenK> It's just that we want to send headers
[00:50] <StevenK> And for that, you need a Request
[00:50] <wallyworld_> ok, makes more sense now
[00:50] <wallyworld_> r=me
[01:02] <StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-distribution-upstreamreport/+merge/132639
[01:02] <StevenK> You two should enjoy this one, it's great.
[01:02] <wgrant> There must be more
[01:02] <wgrant> Look harder
[01:04] <StevenK> Really now?
[01:05] <wgrant> Also, we need to check with UE before this can go anywhere
[01:17] <StevenK> wgrant: Tempted to close bug 899388
[01:17] <_mup_> Bug #899388: Some translation statistics are not up to date <lp-translations> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/899388 >
[01:18] <wgrant> StevenK: Nope
[01:18] <StevenK> It's still broken?
[01:18] <wgrant> The stats jobs don't do everything
[01:26] <StevenK> wgrant: Ah, how do I read https://oops.canonical.com/oops/?oopsid=OOPS-13c0c8bb3601a44992977eaebb7c2d9e ? Since it's a high non-SQL time
[01:29] <wgrant> StevenK: You could try getting a profile from qastaging
[01:29] <wgrant> It's probably spending most of the time loading Storm objects
[01:29] <wgrant> Enum columns, in particular
[01:44] <StevenK> 3731696 function calls (3643838 primitive calls) in 28.437 CPU seconds
[01:48] <StevenK> wgrant: enumcol calls don't look high to me
[01:48] <StevenK>      2043    1.241    0.001    1.392    0.001 store.py:807(_add_to_alive)
[01:48] <StevenK>    119112    1.237    0.000    2.744    0.000 {method 'set' of 'storm.variables.Variable' objects}
[01:48] <StevenK>      2290    0.907    0.000    5.849    0.003 store.py:680(_load_object)
[01:49] <StevenK> Looks like it's loading stacks of objects
[02:10] <wgrant> StevenK: Found anything?
[02:40] <StevenK> wgrant: Indeed, I found my lunch, and it was good.
[02:40] <wgrant> You also found a way to break buildbot :P
[02:42] <StevenK> Oh, for the love of ...
[02:42]  * StevenK fixes
[02:47] <wgrant> Now to pretend to be a builder...
[02:49] <lifeless> wrong name
[03:22] <StevenK> Crumbs
[03:22] <StevenK> Last failure is test_rendered_query_counts_constant_with_team_memberships
[03:23] <wgrant> :(
[03:23] <StevenK> That didn't fail on buildbot, so I'm not sure why testr ran it
[03:29] <StevenK> wgrant: Reply from Bryce on the bug
[03:29] <wgrant> So I saw
[03:34] <StevenK> I'm not sure how to read that reply.
[03:35] <StevenK> It sounds like he's interested and used to graph the data, but doesn't answer my question on what we can do
[03:43] <StevenK> wallyworld_: You'll get a kick out of this: http://www.theregister.co.uk/2012/11/01/samsung_case_apple_told_to_apologise_again/
[03:43] <wallyworld_> lol
[03:44] <wallyworld_> too bad the us courts are so stupid
[03:44] <wallyworld_> i wonder how mych campaign money apple donates
[03:44] <wallyworld_> that would have nothing to do with it i'm sure
[03:46] <StevenK> wallyworld_: I'm just glad that 1. The UK court forced Apple to apologise. 2. That the UK judge is displeased with their first attempt and has told them they have 24 hours to fix it.
[03:46] <wallyworld_> yeah, me too. but i despair that crapple lost the battle here but is winning the war
[03:49] <sinzui> StevenK, Bryce and other upstream stewards want a chart of how many bugs in ubuntu packages are triaged, have an upstream task (a task that we presume is upstream), have a bug watch.
[03:49] <sinzui> StevenK, each number comes from a bug search, so api could produce just the numbers you need.
[03:50] <StevenK> sinzui: With no changes needed?
[03:51] <sinzui> StevenK, , but if you are patient to get the page to load, you will see that the top 100 packages are someone's acid-fueled fantasy...pidgin, firefox 3.5, f-spot, banshee, ekiga...
[03:52] <sinzui> StevenK, the page lists 6 x-related packages.
[03:52]  * sinzui gets the example urls for an x package
[04:00] <sinzui> StevenK, http://paste.ubuntu.com/1325544/
[04:01] <StevenK> sinzui: So you think we provide enough tools for Bryce to graph without having to screenscrape and we're free to destroy Distribution:+upstreamreport and the horse it rode in on?
[04:01] <sinzui> We know he cannot screenscrape. It took me 10 minutes to load the page
[04:02] <sinzui> We should delete the page because we should not be working about ekiga bugs. There is not mention of bugs in zeitgeist which is what unity needs
[04:03] <sinzui> Some one could write a better api script that does the same work for any package, then we let the community choose the packages instead of looking back at Hardy with fondness
[04:04] <wgrant> If we were to try to fix it, we would likely end up having to rewrite it based on a full new set of denormalised data structures
[04:04] <wgrant> Ubuntu has a few more bugs now than it did in 2008
[04:08] <StevenK> Well, I'm happy we have the data and if it's up to Ubuntu to write an API script to pull it per package, then that's fine and we can destroy the page.
[04:09] <sinzui> We can destroy the page. If some says they are using it for a package in Quantal, I will write the script on my Lunch
[04:10] <StevenK> sinzui: https://code.launchpad.net/~stevenk/launchpad/destroy-distribution-upstreamreport/+merge/132639 is an MP I prepared earlier
[04:10] <StevenK> wgrant said there must be more, but I'm not certain if he was trolling. :-)
[04:13] <sinzui> r=me
[04:14]  * sinzui find bed
[04:14] <StevenK> sinzui: Night!
[04:25] <StevenK> wgrant: Right, I think I'm undistracted enough.
[04:37] <wallyworld_> wgrant: if you are interested, this currently seems to work nicely. the garbo job needs work to handle existing records in the cache table and to do bulk updates (it is just designed right now to run against a totally empty cache table to get data in)
[04:37] <wallyworld_> https://code.launchpad.net/~wallyworld/launchpad/ppa-packages-timeout-1071581/+merge/132646
[04:37] <wgrant> wallyworld_: How does it handle the case where I maintain a package that has been uploaded by two different people?
[04:37] <wallyworld_> running it on dev produces the same results as previously as far as i can tell
[04:37] <wgrant> Also, ew
[04:38] <wallyworld_> there should be a row for each of those
[04:38] <wgrant> Sure
[04:38] <wgrant> That's the problem :)
[04:38] <wallyworld_> isn't that the case now?
[04:38] <wgrant> It's only meant to show on my +maintained-packages once, even if it was uploaded by multiple people
[04:38] <wgrant> Today we filter by maintainer and distinct on (archive, distroseries, sourcepackagename)
[04:38] <wallyworld_> oh ok, so just the latest shouyld show?
[04:39] <wgrant> but the cache is distincted on (archive, distroseries, sourcepackagename, creator, maintainer)
[04:39] <wallyworld_> sure, that's what i thought we needed
[04:39] <wallyworld_> but i can change it
[04:40] <wgrant> Also, the garbo job is very much non-wgrant compliant
[04:40] <wallyworld_> actually, maybe it does correctly only have the one row, will have to check
[04:40] <wallyworld_> why?
[04:40] <wgrant> One does not simply concatenate arbitrary strings into SQL.
[04:41] <wallyworld_> you mean for the job table update?
[04:42] <wgrant> Yes
[04:42] <wallyworld_> well, it's done elsewhere
[04:42] <wallyworld_> and it's just for a simple, safe update
[04:42] <wgrant> Where?
[04:42] <StevenK> That is not an argument.
[04:43] <wallyworld_> in many other garbo jobs
[04:43] <wgrant> I'm going to find that place and delete it :)
[04:43] <wallyworld_> have a look
[04:43] <wallyworld_> there's lots
[04:43] <StevenK> Cargo-culting existing bad practise is bad
[04:43] <wallyworld_> it's not worth introducing a whole new class
[04:43] <wgrant> Not seeing it...
[04:43] <wallyworld_> just to update one row in a job table
[04:43] <wgrant> Hmm?
[04:44] <wgrant> Using store.execute and using string contatenation to create SQL are orthogonal concepts :)
[04:44] <wgrant> store.execute is fine
[04:44] <wgrant> String concatenation is not
[04:44] <wallyworld_> see OpenIDConsumerAssociationPruner for example
[04:46] <wallyworld_> i agree though, this was just quick and dirty to get something going
[04:46] <wallyworld_> i could care less right now about the garbo job - it needs work as i said
[04:47] <wallyworld_> i'm more concerned with the correctness of the table population
[04:47] <wgrant> ("couldn't")
[04:48] <wgrant> But everyone needs to not even jump to string concatenation as a first step, ever
[04:48] <wallyworld_> i was using colloquial
[04:48] <wgrant> , params=('foo', 'bar') is not that hard :)
[04:48] <wallyworld_> sigh, it was a poc
[04:48] <wgrant> LP PoCs have a habit of making it to production :(
[04:49] <wallyworld_> not once i'm finnished with it
[04:49] <wallyworld_> i just want to get the data in palce to see if the business logic works
[04:49] <StevenK> wgrant: If you're done lambasting wallyworld_, could you help with ++profile++show?
[04:49] <wallyworld_> and the important logic is all strormified
[04:49] <wgrant> StevenK: What's the issue?
[04:50] <StevenK> wallyworld_: You're fun to argue with. When you get frustrated you start typing like sinzui.
[04:50] <wgrant> Heh
[04:50] <wallyworld_> yes i do.
[04:50] <StevenK> wgrant: Milestone:+index again, now I'm now undistracted.
[04:50] <wgrant> StevenK: Sure, but what about the profile?
[04:50] <wgrant> Like, what do you need help with?
[04:50] <wgrant> If it involves killing dogfood I'm all for it
[04:51] <StevenK> wgrant: I'm not sure how to use the profile to point out the terrible bits of code.
[04:51] <wgrant> heh
[04:51] <wgrant> type 'ubuntu-10' in awesomebar
[04:51] <wgrant> first result is https://qastaging.launchpad.net/ubuntu/+milestone/ubuntu-10.04.2/++profile++show
[04:51] <StevenK> Aside from 'all of it'
[04:51] <wgrant> That was months ago
[04:51] <StevenK> wgrant: Haha
[04:52] <StevenK> I have https://qastaging.launchpad.net/ubuntu/+milestone/ubuntu-10.10/++profile++show/+index loaded
[04:52] <wgrant> wallyworld_: So, the only obvious logic concern is the creator vs maintainer thingy
[04:52] <wallyworld_> i just added in maintainer to distinct on
[04:52] <wallyworld_> i think that wil fix it
[04:52] <wgrant> Well
[04:53] <wgrant> I'm not sure that a single table can satisfy both
[04:53] <wallyworld_> not sure, i'll have to look into it some more
[04:54] <wgrant> Given that the table is meant to satisfy the distinct itself, hmm
[04:54] <wgrant> But it probably can't, because it has to filter on either creator or maintainer
[04:55] <wallyworld_> maybe i need 2 tables then
[04:55] <wgrant> StevenK: You managed to get the table to not time out, then?
[04:55] <wallyworld_> or the record count will be ssmall enough that some live filtering can be done
[04:55] <wgrant> wallyworld_: Quite likely. They end up pretty much identical and can be maintained by a live job
[04:55] <wgrant> s/a live/the same/
[04:56] <StevenK> wgrant: Yeah, 10.10 loaded with qas, but ++profile++show's title shows Error: Timeout
[04:57] <wgrant> Hmm
[04:57] <wgrant> So your initial assessment was pretty accurate
[04:58] <StevenK> wgrant: That it's just loading 2,100 odd objects?
[04:59] <wgrant> No, the "all of it" bit
[04:59] <StevenK> Oh, right.
[05:02] <StevenK> Yeah, the code looks pretty terrible
[05:19] <wgrant> StevenK: http://people.canonical.com/~wgrant/output.png
[05:20] <StevenK> What's that from?
[05:20] <wgrant> gprof2dot
[05:20] <wgrant> ubuntu-10.04.2, not ubuntu-10.10, but it's slow too
[05:21] <StevenK> launchpad_dogfood=> SELECT productrelease, COUNT(libraryfile) FROM productreleasefile GROUP BY productrelease HAVING COUNT(libraryfile) > 0 ORDER BY COUNT(libraryfile) DESC LIMIT 1;
[05:21] <StevenK>  productrelease | count
[05:21] <StevenK> ----------------+-------
[05:21] <StevenK>           32517 |   899
[05:21] <wgrant> Yeah
[05:21] <StevenK> I dispair at our DB
[05:21] <wgrant> PRF is a bit crap
[05:22] <StevenK> wgrant: It's pretty, but it's a mess of colour :-)
[05:27] <StevenK> wgrant: So I do wonder if the mixin's _bugtasks method is too blame for most of the mess.
[05:27] <wgrant> Well
[05:27] <wgrant> That's probably just spending time loading lots of objects
[05:28] <StevenK> Which is most of the problem, no?
[05:28] <wgrant> Sure
[05:29] <StevenK> We only want the title, importance, assignee and status. Can't we just populate a list straight from BTF and skip most of the garbage?
[05:29] <wgrant> If you want to reimplement all of Launchpad, sure
[05:31] <StevenK> wgrant: I meant for Milestone:+index specifically, how does your comment follow on?
[05:31] <wgrant> StevenK: Milestone:+index is part of Launchpad, and relies on a lot of other parts of Launchpad, all of which rely on having these objects
[05:31] <wgrant> We may be able to optimise the loading of those objects
[05:31] <wgrant> The first step is probably to try to reproduce it locally.
[05:33] <StevenK> wgrant: Well, what I was actually suggesting lobomitizing Milestone:+index to not have the bug objects which is pulling in a lot of baggage, and work from a list populated by BTF.
[05:34] <wgrant> StevenK: The DB queries aren't the problem (and BTF doesn't have title)
[05:34] <wgrant> Eliminating the bug objects would work, but it's a lot of work
[05:35] <StevenK> Hmmm
[05:48] <StevenK> wgrant: Did you attempt wallyworld_'s QA?
[05:51] <wgrant> I've been at it on and off for the last 5 hours :/
[05:51] <wgrant> DF is a little bit slow
[05:51] <StevenK> A little? :-P
[05:51] <wgrant> I've about to give in and qa-ok it half-way through
[05:52] <StevenK> Might be a little late to deplay today, though
[05:52] <wgrant> The bug is fixed, and I don't see how it could have broken the existing functionality
[05:52] <wgrant> I can see my correctly defaulted binary in the queue, but acceptance times out...
[06:11]  * wgrant concedes defeat
[07:35] <czajkowski> morning
[08:50] <deryck> Morning, all.
[08:50] <czajkowski> deryck: ello
[11:10] <rick_h_> morning deryck
[11:12] <deryck> rick_h_, hey, man.
[11:12] <czajkowski> rick_h_: what time is it where you are?
[11:20] <rick_h_> czajkowski: 7am
[11:20] <rick_h_> trying to get back on schedule
[11:21] <czajkowski> wow
[11:37] <deryck> czajkowski, rick_h_ never stops. :)
[11:38] <czajkowski> finally choolate and nice fruit with ginger ale are on offer!
[11:39] <czajkowski> *chocolate
[11:53] <deryck> Sorry for all the email Orange Squad.  Trying to get cards on board categorized and prioritized.
[11:53] <deryck> Assuming you get board email :)
[11:56] <rick_h_> deryck: rgr yea noticed
[12:00] <deryck> lunch time!
[13:00] <abentley> deryck: I don't get board mail, so I'm happy.
[13:00] <deryck> abentley, excellent then :)
[13:03] <abentley> deryck: I plan to back-burner 1066905 in favour of 1074139, because the latter can lock users out of their projects, and it should be an easy fix.
[13:04] <deryck> abentley, I trust your judgement.  can't look right now, in meeting.
[16:12] <joey> Hi, what's the best way over the api to return a list of teams that match a wildcard?
[16:12] <joey> e.g. ubuntu-*
[16:51] <deryck> rick_h_: hey, there.
[16:51] <rick_h_> deryck: howdy
[16:52] <deryck> rick_h_: see my note on the timeout bug.  Did we change something with profile queries?  Or just now suddenly profile queries are slower for everyone?
[16:52] <deryck> or timing out for everyone, rather
[16:52] <sinzui> deryck, rick_h_, I am not seeing these profile page timeouts. I suspect I cannot because I am a CA...privacy security checks short circuit for me
[16:52] <rick_h_> deryck: not sure, I see the one query that's taking 4s around blueprints
[16:52] <deryck> sinzui: interesting
[16:52] <rick_h_> so looking at where it's at and what it's trying to do
[16:53] <sinzui> oh, but I still cannot see private blueprints/bugs/branches that I am not subscribed to, so that does not make sence
[16:53] <deryck> rick_h_: yeah, that query is slow, no doubt.  But did we just add the filter?  or is maybe something else also changed recently to make matters worse?
[16:53] <rick_h_> deryck: not sure, I've not tracked the bzr history of it atm. I can peek and try to see if there's some recent stuff that hit it
[16:53] <rick_h_> was more just trying to understand and figure out where the ineffecient part is
[16:54] <rick_h_> it's got somethjing like 5 or 6 subqueries in it atm ugh
[16:54] <deryck> rick_h_: yeah, the 4 second query is awful.  fixing that will most likely fix the issue.
[16:55] <deryck> rick_h_: it just seems strange that we *suddenly* started OOPSing every user page.  unless we just deployed this query, or something else changed around it.
[16:55] <rick_h_> everything else seems pretty light
[16:56] <rick_h_> deryck: ok, cool I didn't realize it was everyone hitting this and suddenly. I just assumed it was something for those with private blueprints started seeing recently
[16:57] <deryck> rick_h_: no, you don't need private blueprints to cause the problem.  we're querying to make sure you can see your blueprints, whether you have private ones or not.
[16:57] <rick_h_> right, my profile page loads fine
[16:57] <deryck> hmmm
[16:58] <deryck> rick_h_: not for me.
[16:58] <rick_h_> deryck: but this is coming from abentley's new assigned_specs_in_progress
[16:58] <rick_h_> so I bet that's why it's more recent.
[16:58] <rick_h_> that was just something last week that was worked on/added
[16:58] <rick_h_> while we were in CPH I believe
[16:59] <deryck> rick_h_: ah, right, I see the rev now.  yeah, that makes more sense.
[16:59] <deryck> rick_h_: so chat with abentley about it then.
[16:59] <rick_h_> will do, thanks for forcing me to look at that closer and connect the dots.
[16:59] <abentley> rick_h_: My profile page is zippy.
[16:59] <deryck> rick_h_: np.  hi, abentley
[16:59] <abentley> deryck: Hi.
[16:59] <rick_h_> abentley: yea, mine seems to be ok as well, but we're getting a bunch of these
[17:00] <deryck> so every profile page OOPes for me.  as does for other people in the bug.
[17:00] <rick_h_> but I don't have many BP and none are private
[17:00] <sinzui> We don't use blueprints, that is why we don't see them
[17:00] <rick_h_> right, that seems to fit for me as well. and this is one UGLY generated sql in the oops
[17:00] <abentley> deryck: Can I have an oops-id?
[17:00] <rick_h_> https://oops.canonical.com/oops/?oopsid=OOPS-8844d78960633507626e1325e8cad359
[17:00] <rick_h_> abentley: ^
[17:00] <deryck> abentley: https://oops.canonical.com/oops/?oopsid=OOPS-1390ee2cc5227ea249e6dbdc7c1f79f0
[17:00] <deryck> heh
[17:00] <deryck> OOPS, twice :)
[17:01] <rick_h_> you dare race me deryck :P
[17:01] <deryck> never :)
[17:01] <rick_h_> I'm back on a real keyboard in the office again bwuhahaha
[17:01] <abentley> deryck: Are you free to compare query times?
[17:01] <deryck> abentley: sure.
[17:02] <abentley> Aw man, even here we don't have the substitutions.
[17:08] <abentley> deryck: I can't see how to find hloeung's db id via the API.  Could you look that up, please?
[17:11] <deryck> abentley: https://pastebin.canonical.com/77605/ (private paste, since it's data for a query.)
[17:25] <deryck> abentley: I need to go down for dinner.  maybe sinzui could run queries for you?
[17:26] <sinzui> I can
[17:26] <abentley> sinzui: Thanks.
[17:26]  * sinzui is stuck so needs a diversion
[17:27] <deryck> awesome, thanks guy!
[17:27] <deryck> guys even
[17:29] <abentley> sinzui: Could you please time this against qastaging?  (I'm expecting it to run a long time.)
[17:29] <abentley> sinzui: https://pastebin.canonical.com/77606/
[17:31] <sinzui> oops, I ran on staging
[17:31] <abentley> sinzui: Should be fine.
[17:32] <sinzui> well both are subsecond to get 2 rowns
[17:32] <sinzui> rows
[17:32] <abentley> sinzui: That is not good, because it suggests I've done the substitutions wrong.
[17:32] <abentley> sinzui: Or else the query is not consistently slow.
[17:33] <sinzui> abentley, this is the query plan form qastaging: https://pastebin.canonical.com/77609/
[17:34] <sinzui> abentley, surely there is more private blueprints in production than qastaging.
[17:34] <sinzui> staging is 2 weeks old, so that would be better
[17:35] <abentley> sinzui: You ran it against staging, though.
[17:35] <sinzui> yes, but then I switched to qastaging...
[17:35] <sinzui> oh, the query plans are different!
[17:35] <abentley> sinzui: You said staging was subsecond.
[17:36] <sinzui> this is staging's explain, and yes, subsecond for both: https://pastebin.canonical.com/77610/
[17:38] <abentley> sinzui: I think we need to try it on prod.
[17:38] <sinzui> we need webops for that
[18:27] <abentley> rick_h_: Do we have a bug for these oopses?
[18:28] <rick_h_> abentley: yes, https://bugs.launchpad.net/launchpad/+bug/1074239
[18:28] <_mup_> Bug #1074239: Timeout error when trying to view any user profile page. <lp-blueprints> <private-projects> <timeout> <Launchpad itself:In Progress by rharding> < https://launchpad.net/bugs/1074239 >
[18:30] <abentley> rick_h_: ty
[22:41] <joey> well nobody liked my last question so how about a different one.. I need help optimizing a short piece of python code that's comparing LP object to LP object
[22:41] <joey> anyone up for that?
[22:47] <lifeless> joey: you know the rule... don't ask to ask ;)
[22:47] <joey> yeah but I'm just too polite
[22:48] <joey> google's not been very helpful... I think it's python thing...
[22:48] <joey> lifeless: how's the new job?
[22:49] <lifeless> joey: pretty good
[22:49] <lifeless> joey: --big-- company