wgrantSo pillarname is the most read table on the system by around 50%. And the main query that used it was 30x slower than it needed to be.00:12
wgrantOur performance situation is hilarious.00:12
wgrant== Most Bloated Tables ==00:13
wgrantlogintoken || 96% || 555 MB of 576 MB00:13
wgrantIs autovacuum broken?00:14
wgrantI think it's all slony00:17
wgrantThe performance regression in week 47 last year (the week of Nov 19) coincides with slony jumping from 20% to 85-110% master CPU usage.00:18
wgrantI wonder if that's when we upgraded.00:18
wgrantCompare https://devpad.canonical.com/~lpqateam/dbr/weekly/db-report-2011-11-11-2011-11-18.txt and https://devpad.canonical.com/~lpqateam/dbr/weekly/db-report-2011-11-18-2011-11-25.txt00:19
wgrantNo major write load changes.00:20
wgrantThe upgrade was the 18th.00:22
wgrantTo slony200:22
wgrantThat is slightly suspicious, if you ask me.00:22
* StevenK blinks at FormFields.select()00:27
StevenKIt's second argument is *names, and when I pass it a list, (Pdb) p names00:28
StevenK(['name', 'displayname', 'visibility', 'contactemail', 'teamdescription', 'subscriptionpolicy', 'defaultmembershipperiod', 'renewal_policy', 'defaultrenewalperiod', 'teamowner'],)00:28
wallyworld_StevenK: who is a lp.dev user with launchpad.Commercial?00:36
StevenKadmin ?00:37
wgrantwallyworld_: commercial-member@canonical.com00:37
wgrantor admin, yeah00:37
wgrantwallyworld_, sinzui: Why BugTaskVisibilityPolicy?00:40
wgrantIsn't it global to all artifacts?00:40
wallyworld_wgrant: it's not00:41
wallyworld_unless something went wrong with the landing00:41
wgrantOh, the commit message was wrong.00:41
wgrantThe code is right.00:41
wallyworld_bugger, didn't change it when i changed the code00:42
wallyworld_StevenK: i think the rule in checkAllowVisibility() is wrong? if ff is on, it no longer allows lp.Commercial if ff is on00:42
wgrantI had also intended that it be AccessPolicyType, but we can discuss that.00:42
wgrantSince it's not an actual policy.00:42
wallyworld_sure, easy to change00:42
wgrantJust a class of policy.00:43
=== Ursinha_ is now known as Guest63174
StevenKwallyworld_: Damn good point00:46
StevenKwallyworld_: I can subtly fix that in this branch if you want00:46
wallyworld_StevenK: yes please :-)00:46
wallyworld_and add a test00:46
wallyworld_StevenK: just noticed, the public/private field on Team+edit suffers the same issue as the private_bugs field00:52
wallyworld_StevenK: when you edit a private team, the public/private field says Public00:53
wallyworld_it's not being initialised from the context properly00:53
wallyworld_StevenK: lp:~wallyworld/launchpad/private_bugs-init-932721 has my solution00:55
wallyworld_to the problem which works00:55
lifelessStevenK: thats a list in a tuple00:57
StevenKlifeless: Well, yes.00:57
StevenKlifeless: I needed a *00:58
StevenKWhich blows the list apart into arguments, I think00:58
lifelessif thing = ([foo,bar],)00:59
lifelessquux(*thing) == quux([foo,bar])01:00
StevenKthing = [foo, bar]01:01
lifelessquux(*thing) == quux(foo, bar)01:01
StevenKRight, so I was right01:01
lifeless(but your names above was ([foo, bar],)01:01
StevenKlifeless: I was calling select(list), where it was 'def select(self, *names)'01:02
StevenKCalling select(list) got me ([foo, bar],)01:02
StevenKCalling select(*list) got me (foo, bar)01:02
* StevenK grumbles at IPerson.checkAllowVisibility()01:13
mwhudsonwhat is the intent of the register merge button doing ajax rather than a direct post?02:16
mwhudsonis it supposed to generate the diff immediately?02:17
StevenKIt is supposed to tell you when stuff is wrong so you can fix it02:18
wgrantIt'll be unbroken in a few hours.02:19
mwhudsonStevenK: ah ok02:33
mwhudsonStevenK: 'wrong' ?02:33
StevenKwallyworld_: ^02:35
wgrantRequesting a review from someone who can't see the branches.02:36
wallyworld_StevenK: ?02:39
wallyworld_it will warn you if you are about to expose private branches02:39
wallyworld_and ask you to confirm02:39
mwhudsonah ok02:43
wallyworld_mwhudson: since now nominated reviewers are subscribed to the source/target branch if the branch(es) would otherwise have been invisible to the reviewer02:55
StevenKI see what broke the test03:02
StevenKWe only want to render the context for edit03:02
StevenKOtherwise it looks for IPersonSet.visibility, and then the toys get de-pramed03:03
StevenKwallyworld_: I think I'm done with your changes.03:04
wallyworld_StevenK: excellent03:04
StevenKwallyworld_: I'll get you to review the changes I added, if you don't mind.03:05
StevenKBut that will about ten, since the kettle is calling03:05
wallyworld_can relate to that03:05
StevenKwallyworld_: Sorry, I also cleaned up after lunch, but r14801 only on https://code.launchpad.net/~stevenk/launchpad/better-name-field/+merge/9331703:29
* wallyworld_ looks03:29
StevenKwallyworld_: Feel free to add your +1 on the MP if you want.03:29
wgrantStevenK: Do you plan a fastdowntime tonight?03:32
wallyworld_StevenK: why not use check_permission('launchpad.Commercial', self)? it was just the placement that was wrong, not the method call itself03:33
StevenKwallyworld_: Because it wasn't working, and check_permission() in model code is always a bit suspect03:33
StevenKwgrant: That sounds like a good plan03:33
StevenKI think the mass index death patch is first03:34
wgrantIt is indeed.03:34
* StevenK reaches for staging logs03:34
wgrantNow, unity, give me back my coding terminal.03:34
StevenKwgrant: gvim? :-P03:34
wgrantIt's still there in Alt-Tab, but has otherwise disappeared.03:34
wallyworld_StevenK: ah, didn't notice it was in model. in such circumstances, i would instantiate and call the security adaptor.'03:35
StevenKwallyworld_: Example?03:35
wallyworld_so that we know the same security checks are used03:35
StevenKwallyworld_: I quite like the use of IPersonRoles03:35
wgrantIPersonRoles is the right thing to do here.03:35
wallyworld_StevenK: except that it used to be launchpad.Commercial03:36
wallyworld_and now it's different03:36
StevenKin_commercial_admin() is identical03:36
StevenKFor this purpose03:36
wallyworld_so we really want to allow in_admin as well?03:37
StevenKBoth should be allowed03:37
StevenK.in_admin == ~admins03:37
wallyworld_sure, but it's new behaviour03:37
wallyworld_just checking that it's really required03:38
StevenKNot really, now it's explicit03:38
StevenKadmins have lp.Commercial03:38
wallyworld_right, didn't know that03:38
StevenKadmins also have lp.Moderate and I think they have lp.Special03:39
wgrantlp.Special exists solely to exclude admins.03:39
wgrantThat's its entire purpose.03:39
lifelesswhich is a bit backwards way of doing it03:40
lifelessits on the wrong dimension03:40
wallyworld_StevenK: r=me03:41
StevenKwallyworld_: Thanks!03:42
StevenKwallyworld_: Did you want to scribble the MP, or you don't care?03:42
wallyworld_np. be good to get all this stuff landed03:42
wallyworld_StevenK: already have :-)03:42
StevenKwgrant: I'm not sure I trust dbupgrade.log03:45
wgrantStevenK: Oh?03:45
wgrantThe times in there should be accurate.03:45
StevenK2012-02-15 11:15:36 INFO    2209-10-0 applied just now in 0.1 seconds03:46
lifelessyes, any ?03:46
StevenKI don't believe it. That patch drops 63 indices03:47
lifelessdropping an index is cheap03:50
StevenKSo stub said, delete a file, sync, move on03:50
lifelesslittle more than that but yeah03:51
lifeless(metadata updates, locks, etc)03:51
StevenKI'd just expect dropping 63 to take more than 100ms03:51
lifelessdid you confirm my question about FK indices ?03:51
lifelessStevenK: its one transaction, so only a few syncs, not one per index.03:52
StevenKI saw it, I didn't think we were dropping any03:52
lifelessbranch.owner ?03:52
lifelessI haven't gone looking for others but I'd expect that to be the index we need for efficient referential-integrity queries03:53
StevenKBut we don't select via owner. The index has been untouched03:54
wgrantIt was branch__owner_name__idx that was dropped03:54
wgrantNot owner03:54
wgrantowner_name probably hasn't been used since unique_name was introduced.03:55
mwhudsonthat index was probably a hang over from before source package branches03:55
lifelessah, cool.03:55
lifelessStevenK: the indexes I'm concerned about *wouldn't be used*03:55
lifelessor at least, very rarely03:55
wgrantPerson merges would use them.03:55
StevenKlifeless: So, I trust stub's judgement on this03:55
lifelessI will ask if he checked03:56
lifelesswhich is all I was asking you03:56
lifelessthat someone took the time to remember this step03:56
StevenKwgrant, I, and stub spent a few hours talking over all of them03:56
wgrantI disagree with the removal of some.03:56
wgranteg the ones on incrementaldiff03:56
wgrantThey're only unused because the feature is disabled.03:57
wgrantBut I don't care enough to revert the patch.03:57
StevenKIndex creation can be done without downtime03:57
wgrantThey're easy to add back later if we don't want to delete the feature.03:57
StevenKwgrant: FDT request up03:57
* StevenK glares at his laptop03:58
StevenKIf I double a text file on my desktop to bring up gedit, Unity crashes03:58
StevenKIf I use gnome-do to bring up the same file, it doesn't03:58
* wgrant blames nautilus03:59
wgrantAnd thumper.03:59
StevenKIt's oneiric, so his care factor is probably a large negative number.03:59
* StevenK stabs _lock_actions more.04:49
cody-somervilleWhy does buildmailman.py take so long to run?04:55
StevenKBecause mailman is large04:56
cody-somervilleand wtf04:56
cody-somervilleImportError: Bad magic number in /home/cody-somerville/Projects/launchpad/lp-branches/devel/lib/lp/services/timeline/timeline.pyc04:56
cody-somervillefixie fixie04:58
cody-somervilleanyhow, either launchpad has gotten slower to start or my machine has gotten slower :(04:58
cody-somervillehmm... ImportError: No module named convoy.meta. Am I missing a dep?04:59
StevenKlaunchpad-developer-dependencies depends on it05:00
cody-somervilleWhat is convoy anyhow?05:02
cody-somervilleand can I copy the lucid build with binaries of it into natty series of ppa?05:04
StevenKWhy are you developing LP on Natty?05:05
StevenKI was going to remove the Natty packages from the PPA05:05
cody-somerville:( cause I like Natty05:06
StevenKcody-somerville: We offically support the latest LTS and the latest release only.05:07
StevenKHowever, I'll fix the convoy recipe to build for Natty.05:08
cody-somervilleStevenK, You're the best.05:08
StevenKcody-somerville: And to answer your question: Convoy is a WSGI app for loading multiple files in the same request.05:09
StevenKOur plan is to use it for JS rather than the horrible launchpad.js05:09
cody-somervilleIs it faster?05:09
StevenKIt will result in domReady happening quicker, because your browser no longer has to digest 3MiB of JS05:10
StevenKcody-somerville: launchpad.js is a concatenation of every JS file in our tree + YUI. convoy means that we only send down the files that are needed for each page in one request05:12
cody-somervilleCool. Are you taking advantage of xsendfile header?05:13
StevenKNo, we're not using XSendFile05:14
wgrantstub: I found some more queries today which have massive planner overhead due to the stats things.05:21
wgrantstub: Should we drop the target down to something more sensible except for problematic cols?05:22
stubwgrant: I'm considering that, yes. Unfortunately, for the problematic issues we would need to set it to a value we know is too low (100).05:22
wgrantstub: What if we drop to like 500 and see how it goes?05:23
stubI'm trying to assemble a test case for upstream, so the more the merrier.05:23
wgrantstub: Also, slony 2 is a performance regression.05:23
stubDropping it to 500 push the issue below the radar, but it will still be lurking. Not sure if that is the best idea.05:24
wgrantThe unidentified global performance regression from November coincides exactly with the slony 2 upgrade, which coincides with 15x more sl_log_* tuple reads and slony going from 15% DB CPU usage to 110%05:24
stubSo I have a collection of ideas I'm not happy with :)05:24
stubwgrant: There is a known regression in Slony that is fixed with the next patch level that matches what you are seeing.05:25
wgranthttp://www.slony.info/bugzilla/show_bug.cgi?id=167 is possibly relevant05:25
wgrantWe should upgrade to 2.1, or possibly increase the intervals back to their defaults.05:25
wgrantSee if that alleviates it.05:25
stubIs it affecting perceived performance? atm we have CPU and disk to spare I think, and the sl_log_ entries are going to end up in RAM anyway.05:26
wgrantCompare https://devpad.canonical.com/~lpqateam/dbr/daily/db-report-2011-11-16.txt and https://devpad.canonical.com/~lpqateam/dbr/daily/db-report-2011-11-18.txt05:27
wgrantI agree that it shouldn't be a problem.05:27
wgrantBut the PPR graphs say otherwise.05:27
wgrantWeek 47 is the slony upgrade05:28
wgrantweb increases by ~0.8s during that week and stays there05:28
wgrantIt's possibly not related.05:29
wgrantBut slony eating more than a core sounds undesirable anyway.05:29
=== almaisan-away is now known as al-maisan
stubwgrant: I have a suspicion we did a point release upgrade of PostgreSQL to 8.4.9 at the same time05:48
wgrant21:18 < mthaddon> stub: any objections if we upgrade postgresql while we're at it? (security update)05:50
stubHmm... or would have have been the previous month? That point release was out 26 sept, but not sure how long it took for the packages to be made, dribble through to lucid and actually get deployed.05:50
wgrantBuildbot broke in late October05:50
stubRight. So the PPR graph is quite probably the planner issue.05:50
wgrantFix join selectivity estimation for unique columns (Tom Lane)05:51
wgrantThis fixes an erroneous planner heuristic that could lead to poor estimates of the result size of a join.05:51
wgrantThat doesn't look irrelevant.05:51
stubIt was suspected when I've discussed this on IRC, but I'll need to prove it to Tom.05:52
wgrantFortunately we have a nice sacrificial sourcherry where we can reproduce the issue...05:52
stubNot really sure how PostgreSQL downgrades work :)05:53
wgrantNo, but patching out the planner changes does :)05:53
* stub gets dragged out to lunch05:55
wgrantstub: Setting branch.owner and teamparticipation.{person,team} to 2500 is enough to get it to 40ms on DF06:34
wgrantFor one query I tested.06:34
wgranthttps://pastebin.canonical.com/60335/ is the one.06:36
wgrant*Up* to 50ms if I remove the second part of the UNION06:38
wgrantstub: http://paste.ubuntu.com/844047/ is a reasonably minimal (but LP prod-specific) example with timings.07:01
wgrantSo it gets pretty serious pretty quickly.07:02
wgrantAnd reproduced all those results to within 1ms on clones of those tables with only the relevant columns and no indices.07:09
stubwgrant: You are *upping* the stats to improve the planner time?07:12
stub2500 is the current production and staging default07:13
=== al-maisan is now known as almaisan-away
wgrantstub: No, this is on DF, which defaults to 10007:36
wgrantThe first column is the list of columns that I changed to 2500 and reanalyzed.07:37
stubwgrant: "Setting branch.owner and teamparticipation.{person,team} to 2500 is enough to get it to 40ms on DF". Did upping the stats to 2500 improve planner time to 40ms, or are you talking about query time?07:38
wgrantstub: enough to get it to 40ms from 1ms07:38
wgrantRemember that on DF planning is quick07:38
wgrantbecause of the tiny stats07:38
wgrantstub: Hm, did you tweak stuff on prod? I have yesterday's 200ms query taking ~15ms now.07:53
stubI applied a patch changing the statistics target on the TeamMembership.person column to 10007:55
wgrantAh, assumed that wasn't live, since it was on db-devel.07:56
stub(all columns on that table could happily go to 100, but I didn't want to be invasive yet as there is a reasonable chance I'll be changing the default)07:56
stubGuess I should have landed it on devel now we have twiddled the pqm rule.07:56
wgrantYeah, otherwise scheduling fastdowntime gets confusing.07:57
wgrantFor us mortals without access to prod LDR :)07:57
stubShould wire that up on a web page in Launchpad. We never made a /+admin or similar jump off page did we?07:58
wgrantI've been considering that for a while.07:58
wgrantJust needs to be something trivial and text line +opstats, I guess.07:59
wgrantThere's still significant overhead on a lot of queries, but it's vastly reduced on yesterday's.08:00
wgrantMy pillarname fix worked well08:02
wgrantReads are down by 99%08:02
wgrantpillarname has in fact dropped off the DBR entirely, from being the top read table by like 50%08:12
stubwgrant: When do Bug.access_policy and branch.access_policy get dropped or repaired?08:19
stubwgrant: What was the pillarname fix?08:20
wgrantstub: PillarNameSet.getByName was doing a full seq scan on pillarname08:22
wgrantThat's the API that's, you know, used to traverse to products and distros.08:22
stubYer. Always wondering why that table was so popular for reads :)08:22
wgrantIt's probably read by more requests than any other table.08:23
wgrantBut the reads should each be roughly one tuple.08:23
wgrantstub: Hmm, I should probably drop them with the rest of the schema changes.08:23
wgrantForgot that detail.08:23
wgrantbranch.access_policy will probably come back at some point, but bug.access_policy is dead to us.08:24
StevenKI love it when a branch goes through ec2 first time08:29
StevenKYay for 4 changes as well as knowing what tests are impacted08:29
wgrantstub: Thanks08:39
wgrantThose columns are gone in -208:40
stubwgrant: So people get access to an artifact either via being granted access to a policy or being granted directly to the artifact?08:49
wgrantstub: Right. Same as the old schema, just with multiple policies per artifact, and modelled slightly differently with denormalisation.08:50
wgrantSo to see an Ubuntu security bug you need to either be in ubuntu-security (which has a grant for the ubuntu security policy), or have a specific grant for the bug.08:52
stubwgrant: I don't understand why any of the 'FK to be added later' columns need to have their foreign keys added later rather than now.08:52
wgrantstub: Person merging and branch deletion need special code to handle foreign keys to their tables.08:53
wgrant-3 adds that code, -4 will add the foreign keys08:53
wgrantThe ugliest part of fastdowntime :(08:53
wgrantWe really should rework that code to make it easier, but I'm not quite sure how.08:54
stubHey, you guys wanted it like this :-)08:54
stubYou could add the columns to the 'ignore these columns' sets in -1 and -3, but probably no real gain08:55
stubwell... one less fast downtime08:56
wgrantAh, yes, could this time, because there's an initial patch.08:56
wgrantMight as well do that.08:57
stubI'm pretty certain person merge has the ignores stuff you need. I don't know about the branch removal code.08:57
stubProbably not - ignores doesn't make sense there except for this sort of db patch juggling.08:58
wgrantBranch removal doesn't actually check. It just has a doctest that lists all the FKs.08:58
StevenKOh, twitch08:58
StevenKThat's a pointless test08:58
StevenKHow does listing all FKs on branch help in any way?08:59
wgrantBecause it will fail if you've added one, hinting that you need to add new rules to handle it.08:59
stubAccessPolicyArtifact needs a PRIMARY KEY09:00
wgrantCurses, you're right.09:05
wgrantMorning czajkowski.09:05
czajkowskihas anything happens since last friday with launhcadlib , a user could login on friday and today is seeing errors and not not able to use it09:06
wgrantczajkowski: The user is using staging. staging's database gets reset over the weekend, so the credentials from last week are no longer valid.09:06
czajkowskiwgrant: thank you09:07
wgrantThey may need to use a keyring manager to remove the cached credentials.09:07
czajkowskithanks for explaining09:07
StevenKczajkowski: That's one thing to keep in mind -- "I'm having a problem with LP" covers a multitude of sins, you should get into the habit of asking if they're using production, edge, staging or qastaging if they don't say so themselves.09:08
wgrant(in this case the error showed it was staging)09:08
czajkowskiStevenK: wgrant nods ok09:08
StevenKczajkowski: mrevell can explain the gory history about that four instances (and there is a fifth, but non developers tend not use it) if you care. :-)09:09
lifelessStevenK: czajkowski: s/edge//09:10
czajkowskithanks folks09:11
StevenKOh, right, edge redirects now09:11
wgrantlifeless, StevenK: Not for the API09:11
cody-somervilleHow long does the Soyuz test suite take to run these days?09:16
StevenKLike an hour?09:16
StevenKbuildmaster doesn't add much09:17
StevenKcody-somerville: The entire testsuite on buildbot is now taking about 5.5 hours09:17
cody-somervillelol. I just ran it in 123.1 seconds - Tests with failures: runTest09:18
cody-somervilleOkay. So I'm looking to export the authorized_size attribute on PPAs. StevenK: I'll buy you a beer at UDS if you do it for me :)09:19
wgrantstub: Declared the PK, dropped an index that was redundant with it, and renamed one which had an obsolete name.09:19
StevenKcody-somerville: And if I'm not at UDS? :-)09:19
wgrantIt's a trap, because we don't go to UDS :)09:20
StevenKcody-somerville: To export it read-only, it's fairly easy09:20
StevenKTo *change* it over the API, that's a little harder09:20
cody-somervilleI want to be able to change it (provided one has the necessary permissions)09:20
stubta. just looking at the triggers09:21
StevenKYes, I thought you might. :-)09:21
cody-somervilleIsn't it just an int field?09:21
StevenKcody-somerville: So, you can wrap it in exported(), and write a bunch of tests to see :-)09:24
cody-somervilleIf only that was easier :(09:25
stubwgrant: SECURITY DEFINER functions also need to add a 'SET search_path TO public' clause for security reasons (patch-2209-00-5.sql has an example).09:25
StevenKcody-somerville: webservice tests aren't too bad09:25
cody-somervillewhen I run make schema, I get 'ProgrammingError: text search configuration "default" does not exist'09:25
stubNot that it should be possible to exploit that in our setup, but defence in depth and all that.09:26
wgrantcody-somerville: Have you run launchpad-database-setup?09:27
wgrantIt's in utilities/09:27
cody-somervilleI have in the past09:27
StevenKYou probably need to again09:28
cody-somervilleugh. it destroyed my database, lol.09:29
stubcody-somerville: In a good way?09:32
stubThat script scares me09:32
cody-somervilleNo. I thought I could provide a different account name as argument and it would just destroy stuff belonging to that09:33
cody-somervillebut nope. it just went ahead and dropped the entire cluster09:33
wgrantThat's why it gives you a warning :)09:34
wgrantOh, but it lies.09:35
wgrantBecause it indeed drops the whole cluster.09:35
cody-somerville'THIS SCRIPT WILL DESTROY ALL POSTGRESQL DATA for the given user'09:35
stubThat isn't a lie, it just doesn't tell you everything09:36
cody-somervilleI gave it a non-existent user as an argument, lol09:36
cody-somervillewelp, at leasts the tests are running correctly now09:37
wgrantstub: Is there any way we can tell what's reading 1.5 million branch tuples/sec?09:37
wgrantThat seems pretty unlikely to be legit.09:37
stubwgrant: Those stats are not linked to particular queries or connections unfortunately.09:39
wgrantWill see if the go away in 15 minutes when everything's disabled.09:39
stubwgrant: The only way I can think of would be to turn on statement + plan logging and analyze that.09:39
wgrantThat will narrow it down to the webapp.09:39
=== almaisan-away is now known as al-maisan
wgrantstub: Ah, thanks for the approval.09:46
wgrantUnless it's translations_import_queue_gardener it must be the webapp...09:55
gmbFolks, I know I'm being dumb, but what do I need to do to make this go away:10:07
gmbTraceback (most recent call last):10:07
gmb  File "utilities/js-deps", line 4, in <module>10:07
gmb    from convoy.meta import main10:07
gmbImportError: No module named convoy.meta10:07
gmb(when running `make`)10:07
StevenKUpgrade your dependencies10:08
gmbStevenK, That's what I thought... I'll try again.10:08
wgrantgmb: apt dependencies10:08
wgrantMake sure launchpad-developer-dependencies is installed10:08
wgrantAnd that you're not running natty, which IIRC you are.10:08
gmbwgrant, I'm on Precise.10:08
wgrantOh, OK10:09
gmbFor juju goodness.10:09
gmbAnd lxc10:09
wgrantOne of the other MacBookers was on Natty at the 'dome10:09
gmband other crazy-ass stuff.10:09
gmbwgrant, Yeah, but I'm running in a VM; I'm too lazy to do the work to get Ubuntu working properly on the metal.10:09
gmbAnd whaddaya know, lp-dev-deps isn't installed any more.10:11
gmbGoodness knows why.10:11
gmbThanks fellas.10:11
bigjoolsgmb: release upgrade removes it10:15
gmbbigjools, Yeah, I must've forgotten to re-add it afterwards.10:16
jmlbigjools: testtools 0.9.14 released, fixing the subunit incompatibility.10:52
jmlI really want that in precise.10:53
StevenKjml: Feature freeze is today, get cracking11:16
jmlStevenK: I don't know what to do. Honest.11:22
jmlStevenK: otherwise I would.11:23
StevenKLooks like it has been synced from Debian in every release except precise11:23
bigjoolsjml: awesome11:29
jmlStevenK: normally lifeless takes care of it, but he's been busy this cycle.11:30
bigjoolsjml: I'll try and get it poked in11:33
jmlbigjools: thanks.11:33
wgrantstub: I think I've found the thing that's reading branch a lot. The planner makes very bad life choices: http://paste.ubuntu.com/844328/ is the >1s query issued by BranchLookup.getUniqueName, <1ms if I extract the distroseries bits into a CTE.13:01
wgrantBranch.owner is extraordinarily skewed, but the planner should be able to tell that those joins map perfectly through unique indices :/13:03
wgrantUnless I'm missing something.13:03
stubYes, it is expecting a user to not have many branches13:05
wgrantYeah, which is a reasonable assumption.13:05
wgrantI can't fault it for that.13:05
wgrantBut I can fault it for not doing the dead-simple plan which evaluates the four unique joins individually :)13:06
stubIt might know that id XXXX has lots of branches (if the sample size is high enough). But it won't know that that Person.name matches that id.13:06
wgrantRight, which is why it normally works, because we usually query by ID.13:06
wgrantBut in this case the indices should tell it that the WHERE clause constrains each join to a single row, surely.13:07
stubThe planner estimates agree with that13:11
stubIt was expecting a single row at every stage of the plan, but things went pear shaped when it found that user actually owned 21k branches13:12
StevenKThat's what happens for ubuntu-branches, though13:12
wgrantI'm still not sure how that can work out cheaper than the plan I suggest, but I guess index reads might do it.13:13
StevenKI say go for the CTE13:13
stubYes, planner being too thick. For every table in there we can guarantee a single row due to uniqueness on the id column, except for Branch which *probably* will return 1 row but that is a guess, not a guarantee.13:15
stubWell, 0 or 1 rows13:15
wgrantIt can reason through the unique indices and say there's only one branch row.13:16
wgrantBecause it knows there's only one row for every element of one of its unique keys.13:16
stubI think the problem is that it doesn't know that. It could, but not implemented.13:18
wgrantSeems like it would be a useful sort of thing to know :/13:19
stubwgrant: Paste your rewritten query?13:20
wgrantJoining the CTE is slow. It may be the one-row hint provided by the = that does it.13:21
wgrantIt's still not doing the right thing, but it works.13:21
wgrant(it does a nested loop from branch onto person)13:22
stubSo you want to filter the Branch table on those criteria? Where do you start? Join with distroseries, you end up with a working set of maybe 20k. Join with SourcePackageName you end up with a working set of maybe 30. But join with Person, you are probably going to end up with 1 row in your working set.13:26
stubRather than rewrite for a CTE, we might be able to craft an index that will be used for this and similar queries13:27
stubSay (person, distroseries, sourcepackagename)13:28
stubSuspect we always limit on both distroseries and sourcepackagename, never just sourcepackagename?13:28
wgrantThis query is the only one that appears to be particularly problematic.13:29
wgrantBecause it's called as the sole query behind an XML-RPC API13:29
wgrantSo has none of the objects.13:29
wgrantSo the stats are useless.13:29
wgrantBecause it's all string-based.13:29
wgrantSo, yeah, never just sourcepackagename.13:30
stubIndex fails for me13:31
* wgrant sleeps.13:43
stubNight. I can't improve on what you have.13:44
=== danhg_ is now known as danhg
wgrantstub: Thanks. I might land that tomorrow, then.13:46
stubhttp://paste.ubuntu.com/844395/ without the CTE13:47
=== al-maisan is now known as almaisan-away
=== almaisan-away is now known as al-maisan
=== jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugtasks: 4*10^2
=== al-maisan is now known as almaisan-away
=== rick_h changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett, rick_h* | Firefighting: - | Critical bugtasks: 4*10^2
rick_hjcsackett: can you look over this when you get a sec pls? https://code.launchpad.net/~rharding/launchpad/gallery-accordian_fix/+merge/9340614:58
=== matsubara is now known as matsubara-lunch
czajkowskibigjools: did the person on this ticket contact you on irc to resolve the issue https://support.one.ubuntu.com/Ticket/Display.html?id=499715:02
jcsackettrick_h: sure, i'll take a look in a sec.15:05
czajkowskisinzui: you about ?15:10
sinzuiI am15:11
czajkowskiready for G+ ?15:11
sinzuiI am15:12
czajkowskisinzui: https://plus.google.com/hangouts/133f83a9936535af8066fcbc7938f0325c3f4d8b?authuser=0&hl=en-GB#15:13
czajkowskisorry for the delay15:13
czajkowskiwe've been cleaning the RT system15:13
jcsackettdid our diff builder get much smarter? or has it always just shown one line when a file has just been moved?15:18
sinzuiczajkowski, bzr+ssh://bazaar.launchpad.net/~launchpad/lp-dev-utils/trunk/15:38
mabacjcsackett, thanks for reviewing https://code.launchpad.net/~linaro-infrastructure/launchpad/workitems-model-classes/+merge/92174. would you also land it for us?16:01
abner`folks, I'm planning to have a new bts installed in my server, but there's a complicated requirement here: I need two instances in two servers being synchronized. Any idea if it's possible to be done with launchpad?16:01
sinzuiczajkowski, this is the command to disable a translation project because the project is already registered ./disable_projects.py --msg=2 gmusicbrowserhutranslate16:02
jcsackettlifeless: you around?16:05
bigjoolsjml: https://launchpad.net/ubuntu/+source/python-testtools16:05
jmlbigjools: \o/16:06
jmlbigjools: thanks :)16:06
bigjoolsjml: my extreme pleasure :)16:06
jcsackettmabac: yes, that's on my todo list. :-)16:07
mabacjcsackett, cool, thank you very much! :)16:07
=== matsubara-lunch is now known as matsubara
sinzuiczajkowski, this is the command to disable one or more projects that sends an explanation to the project maintainer: ./disable_projects.py totten2516:15
=== salgado is now known as salgado-lunch
sinzuiczajkowski, http://opensource.org/docs/osd16:18
=== salgado-lunch is now known as salgado
syst3mw0rmI would like to know the status of grackle project...as mailman is investigating the possibility of using grackle as new archiver framework16:57
=== deryck is now known as deryck[lunch]
sinzuisyst3mw0rm, It is not yet operational17:13
sinzuisyst3mw0rm, There are two needed parts to start testing. I am working on the client library right now. It might be done tomorrow17:14
syst3mw0rmsinzui, what is the second part ?17:14
sinzuisyst3mw0rm, The Cassandra-based store is yet to be built and might be a month a way give that my squad does not have time to work on it17:15
syst3mw0rmsinzui: Well, what do you think about the idea of using grackle as new archiver framework ?17:16
syst3mw0rmfor mailman17:16
sinzuisyst3mw0rm, I have been using a memory implementation store to test the client lib. I suspect the memory implementation will evolve into a a reference implementation that can be used to create other backends.17:16
sinzuisyst3mw0rm, That is my dream, but I think Cassandra's java is a nuance. I think something else with a python heritage would be more suitable.17:17
syst3mw0rmso you mean that grackle can be used with other backends as well, and you will be using cassandra-based store for launchpad ?17:18
sinzuisyst3mw0rm, I already have a branch queued to land that makes out mailman use grackle17:18
syst3mw0rmcan you provide me with the link ?17:18
syst3mw0rmsinzui, why did you choose cassandra when you think that cassandra's java is a nuance ?17:19
sinzuisyst3mw0rm, correct. grackle is providing a fast way to inject a message and a mechanism to get JSON encoded messages back. We have already completed the JS/AJAX lib that will allow the browser to read the archive doing a minimal number of server queries17:20
sinzuiI do not like java. I do not want to maintain a java-stack and python stack to run an archive.17:21
sinzuiAs a developer, or someone deploying the code, I want few dependencies, not many17:22
sinzuisyst3mw0rm, Canonical like Cassandra it is not considered an extra dependency since it is the preferred blob store17:22
sinzuiAnd to be fair to Cassandra, It was created to store messages by Facebook. I was surprised that an archive did not already exist17:23
syst3mw0rmSo, what they used to store the messages when Cassandra was not open source ?17:25
sinzuisyst3mw0rm, yes. I think Facebook has a one or more message stores in Cassandra. Their messages though are short, and without attachments.17:27
syst3mw0rmsinzui: can you give me link of the branch, where makes mailman use grackle ?17:30
sinzuisyst3mw0rm, If I were to hack on the store on my own time. I would keep the mbox and use a db (sqlite default, but allow mysql, drizzle, postgresql) sot the indexes and json data. I think that will be fast enough for giant lists an easy to deploy17:30
sinzuiThe mbox would only been needed to get the non-text parts17:31
sinzuisyst3mw0rm, https://code.launchpad.net/grackle17:31
sinzuiI pushed some changes a few days ago17:31
syst3mw0rmsinzui:  non-text part parts, like attachments ?17:32
sinzui^ syst3mw0rm17:37
syst3mw0rmsinzui: ok.17:37
syst3mw0rmsinzui: The link you have provided just have some unit tests..17:38
sinzuisyst3mw0rm, there is code in there for the client lib. It is very small. The memory implementation is in the test. I will extract it after I have complete the client API17:48
sinzuisyst3mw0rm, the javascript lib will be moved into grackle soon. It is in Lp at the moment because it had the YUI infrastructure we depend on.17:49
syst3mw0rmsinzui: nice..Can i also get involved with development of grackle client lib... ? As i am planning to make mailman use grackle..it would be better if i am quite familiar with grackle internals..17:51
sinzuisyst3mw0rm, that would rock. I expect to have the client completed this week. That allows ud to refine it next week, integrate the js, and work on the server. wgrant was working on the server, but he has more important things to work on. We can work on the server.17:55
syst3mw0rmsinzui: seems like a great opportunity..17:56
syst3mw0rmso i think i will get myself familiarized with cassandra till you finish the client part..and then we can take it forward form there itself..what do you say ?17:56
syst3mw0rmwe are going to work on cassandra itself, right ?17:57
syst3mw0rmsinzui: ^17:57
lifelesswith, not on17:58
lifelessbeing familiar with how cassandra modelling is done is a good idea though17:58
sinzuisyst3mw0rm, maybe. I certainly will use Cassandra when working on Canonical's time. On my own time I would write something that easier for someone to deploy with mailman17:58
lifelesssinzui: cassandra is no harder to deploy than postgresql once packaged :)17:59
syst3mw0rmlifeless: "with, not on" ?17:59
lifelesssyst3mw0rm: we don't expect to be making changes to cassandrra17:59
sinzuilifeless, There are worlds ouside of Ubuntu's JuJu paradise.17:59
lifelesssinzui: oh, I know - I wasn't meaning juju18:00
sinzuilifeless, Most mailman installs would want a batteries-included option that meets the needs of 80% of mailman installs.18:00
lifelesssinzui: cassandra will run single-node quite happily18:00
lifelesssinzui: your time is of course your time; I just don't see much reason to do a different backend for other users: I expect most mailman installs are from Ubuntu packages already18:01
lifelesssinzui: (or Debian with packages)18:01
sinzuilifeless, I will not ever install a java stack to get mailman running for my biz/org The mediocre choices of archivers we have is partially because they are awesomely easy to setup at the moment of mailman's install18:03
syst3mw0rmsinzui, lifeless i think that i should discuss it with barry warsaw who maintains mailman, so he might be having better idea about which backend service might be good for mailman installations...What do you guys say ?18:03
sinzuisyst3mw0rm, I certainly will talk to barry about the replacement of pipermail. That is why I think the server/store implementation needs to have options18:06
sinzuipython can needs of most cases, galaxy-size hosts need enterprise level backends18:07
syst3mw0rmsinzui: your last statement isn't clear to me...18:11
syst3mw0rmare you saying that we should have two options, cassandra for galaxy-size hosts and python based service for most of the other cases ?18:12
lifelessthat is what sinzui is saying18:12
lifelesswhere Launchpad counts as galaxy size18:12
lifelesssalgado: nice finding of deletable code !18:13
sinzuilifeless, I would not have said that two years ago. The terrible performance we see now is cause by some monster lists we now host18:14
salgadolifeless, heh, it was danilo who told me about it :)18:14
sinzuisalgado, there are still map views and locate tables18:15
sinzuipersonlocation tabled18:15
* sinzui stops typing18:15
salgadosinzui, map views?!  those I'd like to delete18:15
sinzuiI think we have two view, possibly unregistered sitting in lp.registry.browser.team18:16
* salgado goes find them!18:16
sinzuiwtf: salgado person.index still calls this: <div tal:content="structure context/@@+portlet-map" />18:17
sinzuiI suck I should have removed that 14 months ago18:17
salgadoI'm glad you did not18:18
salgadothe +map view on teams is still registered though18:18
salgadoyay, I'll kill 'em all18:18
lifelessderyck[lunch]: ohhai18:19
sinzuisalgado, I removed a vocab a few months ago that was not being used.  There is not obvious way to locate an unused vocab, But i think there might be 2000 deletable lines18:21
salgadosinzui, other vocabs, you mean?18:23
salgadoniice, I'll have a look and see if I find any others18:23
abner`folks, I'm planning to have a new bug tracking system installed in my server, but there's a complicated requirement here: I need two instances in two servers being synchronized. Any idea if it's possible to be done with launchpad?18:25
sinzuisalgado, tar up all the windmill directories to in the tree and put it in the dev wiki. you might delete 20,000 lines of code that is never executed18:25
lifelessabner`: in principle yes, in reality no for two reasons; the federation system isn't all that mature (e.g. we have bits turned off because of issues) and we don't have an LP<-> federation module written18:27
=== deryck[lunch] is now known as deryck
lifelessabner`: but you could just use postgresql replication ..18:27
salgadosinzui, why put that in the wiki?18:29
abner`lifeless, I have been trying to do this replication on the db layer, but it's not attending to all requirements.18:29
sinzuiI think deryck wanted to keep them as reference to what we did test when writing yui replacement tests18:29
abner`lifeless, how launchpad stores the data? is it one big db with all projects inside?18:29
lifelesswe have 5 or 6 data stores (db, mailman, bazaar, debbugs replica, debian replica, librarian) with data stored according to its nature18:30
lifelessthere is a big DB, but its not the largest store if you measure by disk size; and possibly not largest by addressable-items either18:31
derycklifeless, I'll ping in about 30-1hr for call.  cool?18:34
czajkowskilifeless: sorry for the multiple pings in -meeting18:34
czajkowskii added the PPA CoC topic to the CC agenda to get some discussion/answers18:35
abner`lifeless, hmm.. I'm asking because during the synchronization/replication I would need to transfer only the data of a specific project, not all project. So having an unique db with all project can also be a problem.18:36
lifelessderyck: ok18:38
lifelessczajkowski: my irc client doesn't see any pings18:39
sinzuisalgado, I am disappointed to see that windmill is only 2593 linez18:39
czajkowskilifeless: what irc client do you use ?18:40
lifelessczajkowski: irssi; it needs a : to invoke highlight18:40
salgadosinzui, come on, every windmill line, when removed, should probably be worth like a dozen lines of proper code ;)18:41
lifelessczajkowski: I could change that, but its good ATM - if someone doesn't want to invoke me, they don't accidentally do so.18:41
czajkowskiah yes same here18:41
czajkowskigood to know18:41
czajkowskianyway we got an answer yes they do need to sign the CoC for ppas and it should be enforced18:41
lifelessczajkowski: thanks fo rletting me know about the discussion18:41
czajkowskishall get the mins later and post to the list tomorrow18:41
salgadosinzui, btw, I got excited and started removing the location-related (except timezone) from IPerson but then realized .latitude and .longitued are exported on the API.  do we need to follow a process to remove those or can we just assume nobody will care as those things have been removed from the UI anyway?18:42
czajkowskilifeless: in fact before I head out, here you go http://ubottu.com/meetingology/logs/ubuntu-meeting/2012/ubuntu-meeting.2012-02-16-17.02.moin.txt and you can see the discussion http://ubottu.com/meetingology/logs/ubuntu-meeting/2012/ubuntu-meeting.2012-02-16-17.02.moin.txt18:42
lifelesssalgado: keep the attribute, return None for them, for old API versions18:43
lifelesssalgado: for devel, unexpect it entirely18:43
lifelessczajkowski: I used my backlog :P18:43
salgadolifeless, right, makes sense18:43
czajkowskiwell in case others were curious :)18:43
sinzuisalgado, yes, what lifeless said.18:43
salgadolifeless, what about stuff that was only exported on the beta API? I guess those can be removed altogether?18:45
lifelessIMO yes18:45
lifelesssame as devel. YMMV though, I think some folk used beta heavily?18:45
barrylifeless: hiya18:45
lifelessbarry: oh hey18:46
lifelesssinzui: and syst3mw0rm: were talking about mailman archivers18:46
lifelessand we got to a point where consult-barry was raised. I thought, why wait?18:46
barrysyst3mw0rm: hi, thanks for joining us here18:47
barrylifeless: good thought :)18:47
=== abner` is now known as abner
syst3mw0rmbarry: no problem18:47
sinzuibarry, we were discussing ideals of archivers. We are using java-based Cassandra as the message store for Lp. I believe that is over kill for most mailman installs. A simpler store possibly taking the batteries-included approach would work18:50
lifelessI was putting forward the opinion that apt-get install cassandra is pretty simple :>18:51
barryright, anything java-based is overkill for standard mm installs :)18:51
barrylifeless: certainly for lp, that would be fine.  mm3 though runs on lots of *nixen18:52
barrysinzui: do you have apis designed for the backend store?  what are the basic requirements for that store?18:52
sinzuibarry yes18:53
sinzuiin, fact we have accidentally written a memory-based reference implementation for testing18:53
barrysinzui: would it be possible to use sqlite + filesystem ?18:53
sinzuiI think it will be easy to play tests against another implementation or mock/fake to verify conformance18:54
barryor should i say storm + possibly filesystem18:54
sinzuibarry, friend, man-of-my-mind, yes. I think mbox + sqlite will work for 80% of the cases18:54
barrysinzui: that would be perfect, if by 'mbox' you mean http://docs.python.org/library/mailbox.html18:55
sinzuibarry, sqlite could also be a step to switch to drizzle, mysql, etc ...18:55
sinzuibarry, yes, I do mean that18:55
barrysinzui: exactly.  and if done through storm, it's an easy-ish config var to change.  e.g. we have postgres support in mm3 bzr right now18:55
barrysinzui: beauty.  that would let us use maildir for example18:55
sinzuioh, yes. I forgot that18:56
barry(maildir on disk format)18:56
barrysinzui: btw, i *love* the idea of a backend store that does all the threading and whatnot, and vends the necessary info through rest for whatever formatter you want to use18:56
syst3mw0rmbarry: i guess you are talking about the grackle's idea ?18:58
sinzuibarry, I don't have any code yet, but I image the mbox is only used to store the attachments and is the portable-part of the archive. The db manages, indexes, threads, and stores JSON for fast returns os massage data18:58
barrysyst3mw0rm: yep18:58
salgadolifeless, I can't seem to find how to tell exported() to do so only for certain versions of the API... any pointers? :)18:58
barrysinzui: yep.  mbox would store the raw email messages too, which you might eventually want to have access to18:59
barrysinzui: the nice thing is that the db would also store things like takedown flags.  and the really nice thing is that the consumer of the json can do things like email obfuscation algorithms, or body massaging (e.g. hyperlink bug tags, etc.)19:00
lifelesssalgado: something_for_version IIRC19:00
salgadooh, a different thing, like operation_for_version I guess19:01
* salgado digs19:01
sinzuibarry, we are in agreement. we even intend to have the Cassandra version do the same.19:01
lifelesssalgado: oh oh, I know what you mean19:01
lifelesssalgado: its a dict at the end of the export19:01
salgadooh, right19:02
lifelesstakes a mapping of versions and shit. Uhm, grep for '\<beta\>' and you'll likely find it quickly19:02
barrysinzui, syst3mw0rm: this would be so cool.  the state-of-the-art in floss archivers has been abysmal for at least a decade.  this would leapfrog everything that i'm aware of19:02
syst3mw0rmbarry: seems cool..but frankly, it will take some time to digest totally whatever you were discussing...:)19:03
sinzuibarry, We will use Lp as a pass-through to do linking in real-time because of permission/data changes. The page code/js does not know it is talking to Lp...it can talk directly to the archive (a wsgi app) if you want19:03
barrysyst3mw0rm: no worries :)19:03
barrysinzui: rock19:04
lifelesssinzui: hmm, the js will need to be querying LP; I think yo umean 'LP will be offering the same API the archiver does'19:04
lifelesssinzui: so they can be substituted transparently19:05
lifeless(Do I understand correctly?)19:05
sinzuilifeless, correct19:05
=== matsubara is now known as matsubara-afk
barrylifeless: is there *anything* y'all can do to bump up the timeout on https://launchpad.net/ubuntu/+search?  it's so painful, but such a basic interface20:59
lifelessbarry: patches appreciated21:00
lifelesslet me link you the bug21:00
lifelessbug 81687021:00
_mup_Bug #816870: Distribution:+search (package search) timeouts <critical-analysis> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816870 >21:00
lifelessthe count alone is 11 seconds to complete21:01
barryyay!  4 hotter21:01
lifelessthe actual search will then be (probably but not always) less than that21:01
lifelesswe'd likely need 20seconds to have -any- chance of reliability21:01
barrylifeless: jfdi!21:02
lifelessand thats as high as we /can/ go before other components in the chain will time it out21:02
barryi'm willing to wait :)21:02
lifelessbarry: when we raise such timeouts, we consume more resources, increases contention for service points -> more issues elsewhere21:02
lifelessbarry: this needs to be fixed, not tolerated. Thus my 'patches appreciated' comment.21:02
barrylifeless: i know.  i'm just being curmudgeonly.21:03
lifelessif we could just toss hardware at it, and tolerate the timeouts, it would be a different matter; but anyone who uses LP will be on master most of the time, and thats much harder to scale21:04
lifelessand this is all DB server CPU time, so we only have 16 concurrent queries we can run21:04
lifeless-> not many.21:04
lifelessbarry: I'd love it if you fixed it.21:04
barrylifeless: no, i do appreciate the constraints21:05
wallyworld_sinzui: jcsackett: wherefor art thee?22:05
jcsackettwallyworld: be there in just a moment.22:05
* thumper saw the word22:05
sinzuiI lost X for 30 minutes22:05
wallyworld_thumper: unity is making me sad today22:05
StevenKthumper: Do you hilight on unity or something?22:05
thumperwallyworld_: why?22:05
thumperStevenK: no, but quassel has a summary buffer22:06
wallyworld_thumper: dash broken, random stuff left on screen22:06
thumperwhich shows all chatter on all channels22:06
thumperand I happened to see it22:06
sinzuiI am not sure unity is my problem. X went belly up and I had no harware detection for display, mouse, or keyboard22:06
wallyworld_thumper: i just upgraded this morning. often when  stuff crashes and i go to report the bug, it says i am not running an official ubuntu package and so won't do it22:08
thumperwallyworld_: which are you using?22:09
wallyworld_thumper: let me check22:09
thumperthere have been X graphics driver issues22:09
thumpervisual corruption et al22:10
wallyworld_thumper: 5.2.0+bzr1975ubuntu0+644.really197722:10
thumperwallyworld_: apt-cache policy unity22:13
wallyworld_thumper: https://pastebin.canonical.com/60465/22:14
wallyworld_thumper: the dash issue is that when i click on an icon to launch an app, nothing happens22:15
thumperwallyworld_: hmm... that is the latest and greatest22:19
wgrantwallyworld_: https://devpad.canonical.com/~lpqateam/dbr22:19
wallyworld_thumper: yes, i like the bleeding edge :-)22:20
thumperwallyworld_: so bleed22:21
wallyworld_thumper: indeed. i was just mentioning it :-)22:22
=== jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
thumperwallyworld_: I think I've found your bug22:36
wallyworld_thumper: you're a legend22:36
thumperwallyworld_: I didn't say I've fixed it :)22:37
wallyworld_thumper: that's ok. finding it is the firs tstep :-)22:37
wallyworld_thumper: and fewer others will be affected hopefully22:38
jcsackettwallyworld_: ok, i've made my vote on the MP and added you directly as a requested look. enjoy. :-P22:39
=== salgado is now known as salgado-afk
wallyworld_jcsackett: thanks, i think :-)22:39
* jcsackett grins22:39
jcsackettwallyworld_: are you the reviewer today, or just planning on looking at that MP out of the goodness of your heart?22:52
jcsackettwallyworld_: when you have a moment (say when your day fully begins), can you look at https://code.launchpad.net/~jcsackett/launchpad/trivially-add-sharing-ui-stuff/+merge/93501? it's much shorter than the other MP. :-)22:53
wallyworld_jcsackett: i had already clicked on it but haven't read it yet :-)22:54
wgrantwallyworld_: https://code.launchpad.net/~wgrant/launchpad/bug-933853/+merge/93508 is the branch performance fix I mentioned earlier.23:20
StevenKsinzui: http://pastebin.ubuntu.com/845139/23:26
wgrantlifeless: I was going to do a FF cleanup today anyway...23:35
=== wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugtasks: 4*10^2
wgrantwork items landed23:46

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!