[00:26] <wallyworld_> wgrant: i hadn't forgotten the qa - i needed to finish some stuff first. did you check that the branch badges displayed ok?
[00:27] <wgrant> wallyworld_: Yep, seems to work fine.
[00:27] <wallyworld_> ok. thanks.
[00:27] <wgrant> I had 12 QA items of my own, so it wasn't much extra work :)
[00:28] <wallyworld_> np. was just uploading a new oops-amqp to pypi for james_w
[00:33] <wallyworld_> wgrant: so now that oops-amqp is at 0.0.6 on pypi, what causes the ubuntu package to get updated?
[00:45] <lifeless> wallyworld_: nothing; they probably have a watch file tht will alert the package maintainer
[00:46] <lifeless> wallyworld_: we don' worry about that at the moment
[00:46] <wallyworld_> ok. thanks. just wanted to be sure i didn't stuff anything up
[00:46] <lifeless> wallyworld_: you did an sdist upload -s ?
[00:46] <wallyworld_> yes
[00:46] <lifeless> wallyworld_: close off any fix committed bugs in lp; and you're done AFAIK
[00:46] <wallyworld_> had to register my ssh key etc
[01:01] <wallyworld_> wgrant: StevenK: if i ask nicely, can one of you +1 rick's review so i can get this landed? https://code.launchpad.net/~wallyworld/launchpad/private-by-default-ui-885503/+merge/92273
[01:48] <lifeless> interesting - https://www.ebayopensource.org/index.php/Turmeric/HomePage
[01:49] <StevenK> Sorry, you lost me after Java.
[02:22] <wgrant> wallyworld_: Looking
[02:28] <StevenK> lifeless: But teams don't have gpg keys. You'd ask the owner the sign the CoC and be responsible for the PPA?
[02:30] <lifeless> StevenK: there are a number of ways it could be implemented
[02:30] <lifeless> StevenK: one; force every member to be a CoC signer
[02:31] <lifeless> two; reject uploads from non CoC signers
[02:32] <lifeless> three; I'm not sure right now, but give me a few minutes :P
[02:33] <StevenK> 1) Could result in "Damn it, I can't create a PPA because x, y and z haven't signed the CoC, and they're on holidays."
[02:34] <lifeless> indeed
[02:35] <wgrant>  * 471 Exceptions
[02:35] <wgrant> Yay
[02:37] <lifeless> nice
[02:38]  * wgrant fixes 110 of them
[02:41] <wgrant> wallyworld_: Done
[02:41] <wallyworld_> wgrant: thank you
[02:41] <wgrant> wallyworld_: Also, you can't land this until the DB patch is deployed.
[02:41] <wgrant> Oh, you won't be thanking me :)
[02:41] <wallyworld_> yeah, hopefully tomorrow
[02:57] <wgrant> I hate our hand-generated SQL.
[03:18] <wallyworld_> wgrant: thanks for review - i've responded
[03:20] <wgrant> wallyworld_: My premise was that the product argument to person.hasCurrentCommercialSubscription should not exist.
[03:21] <wallyworld_> why? i think it's fine to ask the question "does the user have a commercial subscription to this product". we also discussed this on the standup last week and curtis was ok with it
[03:21] <StevenK> Didn't we say that on the call on Friday too? :-)
[03:21] <wgrant> That's not a sensible question.
[03:21] <StevenK> wallyworld_: Right, but both wgrant and I disagreed.
[03:21] <wallyworld_> 2 vs 2 :-)
[03:21] <wgrant> A product has a commercial subscription.
[03:21] <wgrant> A person is not involved.
[03:22] <wallyworld_> the current api says that a person is involved - the one before i modified it i mean
[03:22]  * StevenK stabs this branch, pulls the knife off and it stabs it again.
[03:23] <wallyworld_> since i can't land till db patch lands, let's discuss again tomorrow on the call?
[03:25] <wgrant> That was written two weeks ago to see if a person has any projects which have subscriptions.
[03:25] <wgrant> It's a suboptimal hack to make private teams more accessible.
[03:25] <wgrant> It must not be perpetuated.
[03:25] <wgrant> Replied.
[03:25] <wallyworld_> why suboptimal hack? i think it's ok to have such a method
[03:27] <james_w> thanks wallyworld_!
[03:27] <wallyworld_> james_w: np. the 0.0.6 version is on pypi too fwiw
[03:28] <wgrant> It's conflating two checks that your method should be performing
[03:28] <wgrant> Aha
[03:28] <wgrant> Now freenode is alive.
[03:28] <james_w> wallyworld_, great, thanks
[03:30] <StevenK> wgrant: Can you please look at http://pastebin.ubuntu.com/839965/ which is the test failure I'm debugging, along with a current diff before I slam my head through my monitor and the wall behind it in frustration?
[03:30] <wallyworld_> wgrant: re: Unauthorised - that is used in many other places for similar semantics
[03:31] <wgrant> NAFAIK
[03:32] <wgrant> All the 'raise Unauthorized's that I can see are cases of inadequate role membership.
[03:34] <wgrant> statik: Was visibility formerly public?
[03:34] <wgrant> Er
[03:34] <wgrant> StevenK: ^^
[03:35] <StevenK> wgrant: The Unauthorized I'm not concerned with yet, it's the rest of the doctest failures
[03:36] <wgrant> StevenK: They are likely cascading failures.
[03:38] <StevenK> wgrant: What makes you think that?
[03:38] <StevenK> Oh, that the team hasn't actually be turned private?
[03:38] <wgrant> Right.
[03:39] <StevenK> wgrant: visibility doesn't appear in configure.zcml, just IPersonCommAdminWriteRestricted
[03:40] <StevenK> It has a require permission .Commercial, and it's allowed in IPerson's interface
[03:41] <wgrant> <allow /> == <require permission="zope.Public" />
[03:41] <StevenK> Right.
[03:41] <StevenK> It was publically readable, and only settable for .Commercial
[03:42] <StevenK> Now, how do I fix that ...
[03:43] <wallyworld_> wgrant: inadequate role membership and not having the required previliges to turn on private bugs is pretty much the same thing, no? i don't thing bad request is a correct alternative regardless
[03:46] <StevenK> wgrant: So I guess IPersonEditRestricted is the wrong place for visibility
[03:52] <wgrant> wallyworld_: It's not really a lack of privilege of the person. It's a lack of privilege of the project.
[03:52] <wgrant> wallyworld_: So it's slightly different, and not really something we use Unauthorized for elsewhere right now.
[03:53] <wallyworld_> which still results in the person being authorised
[03:53] <wallyworld_> hmmm
[03:53] <StevenK> wgrant: Getting closer. First failure is now "- A private team cannot change visibility."
[03:53] <wallyworld_> if the person were in a different role eg ~registry then they would be authorised
[03:54] <wgrant> wallyworld_: Right, but that's not usually how people would solve it.
[03:54] <wgrant> In order for Joe Random to solve the error, he has to change the project.
[03:54] <wgrant> Not become a member of ~registry.
[03:54] <wallyworld_> sure, but i used it to illustrate that it can be considered a limitation of what the person is allowed to do
[03:54] <wgrant> It could go either way, but I think BadRequest is slightly better.
[03:55] <wallyworld_> but badrequest is for stuff like protocol issues etc
[03:55] <wallyworld_> afaiui
[03:55] <wgrant> We use a derivative of BadRequest for most user errors.
[03:55] <wallyworld_> why?
[03:56] <wgrant> Because in most cases they're bad requests.
[03:57] <wallyworld_> really? the syntax is bad?
[03:57] <wgrant> Hm?
[03:57] <wgrant> Bad Request doesn't meant the HTTP syntax is bad.
[03:58] <wallyworld_> i thought it was a request that could not be fulfilled because not all info was provided or whatever
[03:58] <wallyworld_> different to not being allowed to do something
[03:59] <wallyworld_> so like leaving out a mandatory parameter = bad syntax
[03:59] <wallyworld_> but here there is no such issue but there is an authorisation issue
[04:00] <wgrant> A 403 is possibly more appropriate, but the rest of the API uses 400 here.
[04:00] <wgrant> 401 is not appropriate for normal users, as becoming a ~registry is not a way out for them.
[04:01] <wallyworld_> i wasn't suggesting becoming ~reg was a way out
[04:02] <wallyworld_> i think 403 is ok
[04:02] <wallyworld_> i'd like to know why the rest of the api uses 400 in these cases
[04:03] <lifeless> so 400 is definitely wrong per rfc2616 but
[04:04] <lifeless> httpbis changes this
[04:04] <lifeless> http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-18#section-7.4.1
[04:04] <wgrant> The RFC isn't relevant here.
[04:04] <wgrant> It's 15 years behind the web.
[04:04] <lifeless> specifically, it says "The server cannot or will not process the request, due to a client
[04:04] <lifeless>    error (e.g., malformed syntax)."
[04:04] <lifeless> wgrant: a little patience kemosabi
[04:04] <lifeless> wgrant: and you might find your local friendly http nazi supporting your case
[04:05] <wgrant> Heh
[04:05] <lifeless> 400 has become the defacto 'go away and it is your fault'
[04:06] <wgrant> Exactly.
[04:06] <lifeless> you could return 401 if you want to trigger *http* authentication
[04:06] <lifeless> but
[04:06] <lifeless> we don't use http authentication anywhere
[04:07] <lifeless> so we should never use 401 or 403, more or less
[04:07] <wgrant> 403 is fine
[04:08] <wgrant> 401 is illegal, but we do it in the API anyway
[04:08] <wallyworld_> lifeless: isn't 'go away, it's your fault' different to 'you sent all the right info to do this but you are not allowed ie forbidden"
[04:08] <wallyworld_> so why should we never use 403?
[04:08] <lifeless> same reason 401
[04:08] <wgrant> IIRC 401 says the response MUST include WWW-Authenticate
[04:08] <lifeless> 401 and 403 are talking about the ability and limits of dealing with http authentication credentials
[04:08] <wgrant> 403 doesn't say anything about HTTP auth
[04:09] <wallyworld_> what he said
[04:09] <wallyworld_> hence 403 is ok here i think?
[04:09] <wgrant> I still think 400 is more in line with the rest of the application, and indeed the rest of the modern world.
[04:09] <wallyworld_> well the world is wrong :-)
[04:10] <lifeless> http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-18#section-7.4.4 if you want to read it
[04:10] <wallyworld_> i still don't get why the world would be using an incorrect code
[04:10] <wallyworld_> ok, i'll read it
[04:10] <lifeless> I see no value in spec lawyering without reference to the current spec
[04:10] <wgrant> Current *draft* spec that's already obsolete? :)
[04:10] <lifeless> final stage about to be ratified
[04:11] <wallyworld_> reading that link reinforces my view that 403 is ok here
[04:11] <wallyworld_> but i'll make it 400 if that's what i need to do
[04:12] <StevenK> Hmmmmm.
[04:12] <wallyworld_> even though it's not a 400 ie all necessary info for the server to process the request was provided correctly
[04:12] <StevenK> I think hasCCS has a bug
[04:12] <wgrant> wallyworld_: "due to a client error" in the new spec.
[04:12] <wallyworld_> but it's not a client error
[04:13] <wgrant> It is.
[04:13] <wgrant> It's asking for something which can't be done.
[04:13] <lifeless> wallyworld_: so the sequence is - you start with nothing, you make a request, server says 401 (gotta be authenticated), so you auth and try again and it says 403( you picked the wrong usercode)
[04:13] <wallyworld_> agree so far
[04:13] <wallyworld_> or a user code that's not allowed
[04:14] <lifeless> wallyworld_: 401 (which references http://tools.ietf.org/html/draft-ietf-httpbis-p7-auth-18) makes no sense if you are not talking about HTTP auth
[04:14] <wallyworld_> yes, i should not have used a 401 in the mp. i should have used 403
[04:14] <lifeless> 403, which layers on 401 - telling you to go back and reauthenticate - makes no sense unless 401 makes sense.
[04:14] <lifeless> 403 is flat out wrong for us.
[04:14] <wgrant> 403 doesn't layer on 401
[04:14] <wgrant> In practice it did 15 years ago.
[04:14] <wgrant> But even 1.1's 403 doesn't mention 401, or HTTP auth at all.
[04:14] <lifeless> 403 is what you use to trigger a browser login window regardless of their current login status.
[04:15] <wgrant> No, that's 401
[04:15] <wgrant> 403 doesn't; the webapp uses it.
[04:15] <StevenK> Meh, let's just raise 402
[04:16] <wgrant> 402 is actually almost appropriate here, for like the first time ever :)
[04:16] <wallyworld_> hah
[04:16] <StevenK> I know :-)
[04:16] <StevenK> (402 is Payment Required, for those playing at home.)
[04:17] <StevenK> AH! I get it!
[04:17] <StevenK> The logged in user has a subscription, but the new team doesn't.
[04:28] <wgrant> Ah, finally.
[04:28] <wgrant> https://launchpad.net/launchpad renders in <1s now.
[04:28] <wgrant> Next stop <500ms.
[04:43] <StevenK> wgrant: If I change TeamEditView to have class schema(ITeam):\n visibility = copy_field(...  I get TypeError: ('Could not adapt', <Person at 0xfb05050 team-name-100028 (Team Name 100028)>, <InterfaceClass lp.registry.browser.team.schema>) in the tests.
[04:44] <wgrant> StevenK: That's inconvenient.
[04:45] <StevenK> But it works for TeamAddView and ITeamCreation
[04:47] <StevenK> wgrant: Can I force Zope to give me a better error?
[04:47] <wgrant> StevenK: You may have to do it in setUpFields with copy_field. See BugTaskEditView
[05:12] <StevenK> wgrant: Down to one failure.
[05:13] <wgrant> :(
[05:14] <StevenK> Hmm?
[05:17] <wgrant> failplanner is fail
[05:48]  * StevenK stabs longpoll
[05:58] <StevenK> wgrant, wallyworld_: Do either of you want to review https://code.launchpad.net/~stevenk/launchpad/make-person-visibility-mutate/+merge/92715 ?
[05:58] <wallyworld_> StevenK: i'll do it
[06:00] <StevenK> Grrr. Now I have to wait 7 hours to see if combo-url breaks qas.
[06:01]  * StevenK glares at wallyworld_.
[06:01] <wallyworld_> what have i done this time?
[06:02] <StevenK> wallyworld_: I was expecting combo-url to be the first branch to land this morning, and it wasn't.
[06:02] <wallyworld_> the early nerd gets the worm
[06:02] <StevenK> I tossed it at ec2 at 7:40!
[06:02] <wallyworld_> mine was a small test fix which i lp-landed
[06:03] <wallyworld_> how long till gary makes bb run in 10 seconds?
[06:04] <StevenK> Years?
[06:09] <wallyworld_> StevenK: 41	+ visibility = data.get('visibility')..... i think you will find you need to del data['visibility']
[06:09] <wallyworld_> if you have made visibility field readonly, unless you remove it from data, it should error
[06:10] <wallyworld_> if your tests pass as is, i'm not sure why
[06:10] <StevenK> wallyworld_: It was made readonly in the interface due to the mutator method. Like I said, I had to do magic to get it to NOT be readonly for the two views.
[06:13] <wallyworld_> StevenK: IPerson.visibility is readonly and doesn't updatecontextfromdata() attempt to do "context.visibility = data('visibility')"
[06:13] <StevenK> It doesn't seem to.
[06:14] <wgrant> Hm, I would expect it to.
[06:14] <wallyworld_> it does field.set(...)
[06:14] <StevenK> wallyworld_: I'm happy to delete it from data after I grab it
[06:14] <wallyworld_> i think you need to but i'm concerned no test failed
[06:15] <wallyworld_> there's several other places where a readonly field is set using a mutator and it has to be deleted from the data dict
[06:16] <StevenK> wallyworld_: I think what is perhaps going on is the mutator is setting it, and then it gets changed to the same value
[06:16] <wallyworld_> ah, that may explain it perhaps
[06:18] <StevenK> wallyworld_: http://pastebin.ubuntu.com/840063/
[06:18] <wallyworld_> StevenK: i think that is ok. it's more explicitly correct if that makes sense
[06:19] <wallyworld_> StevenK: i'm wondering if the new code in setupFields should be moved to conditionallyOmitVisibility() and the method renamed to setupVisibility(0 so that it all lives together?
[06:20] <wgrant> lifeless: Is it evil to have partial indices where distribution IS DISTINCT FROM 1? :)
[06:20] <StevenK> wallyworld_: wgrant didn't think it belongs in conditionallyOmitVisibility(), and I agree.
[06:20] <wgrant> StevenK: It might be OK if you rename it.
[06:20] <wallyworld_> not if it's not renamed
[06:20] <wgrant> But I forget what it was.
[06:21] <wallyworld_> but it's code that's all to do with initing the view wrt the visibility field
[06:21] <StevenK> wallyworld_: The other thing is I only need to do setUpFields for the Edit view
[06:21] <StevenK> wallyworld_: The class schema is fine for the Add view, so it doesn't need to be done
[06:22] <wallyworld_> let me have a closer look
[06:22] <wgrant> StevenK: I'd use the same for both.
[06:22] <wgrant> No point duplicating it.
[06:22] <StevenK> I was happy with class schema, but it gave the can not adapt error for Edit
[06:23] <wgrant> Sure, so it's not useful.
[06:23] <wgrant> If you have one solution that isn't vile and works for both, use it for both.
[06:28] <StevenK> wallyworld_: What do you think of http://pastebin.ubuntu.com/840071/ ?
[06:30] <wallyworld_> StevenK: i think that looks nicer thanks. maybe s/delete_visibility/omit_visibility to user consistent terminology?
[06:31]  * wallyworld_ stabs unity. another crash, another set of duplicate icons on the launcher
[06:31]  * StevenK wields s/delete\(_visibility\)/omit\1/g
[06:31]  * wgrant curses the bug sort tiebreaker.
[06:36] <wgrant> Hmm
[06:36] <wgrant> By doing partial indices by openness I can get the main query on https://bugs.dogfood.launchpad.net/ubuntu down to 6ms
[06:36] <wgrant> Which is possibly a good thing.
[06:36] <wallyworld_> StevenK: should the transitionVisibility method check that new visbility != old visibility before it complains?
[06:40] <StevenK> wallyworld_: Hmmmm.
[06:40] <StevenK> Probably
[06:40]  * StevenK wishes for a Perl-ism
[06:41] <wallyworld_> StevenK: and there's no difference between turning it on vs off?
[06:42] <StevenK> #define it ?
[06:42] <wallyworld_> visibility
[06:42] <wallyworld_> in terms of permissions i mean
[06:43] <StevenK> Sorry, I don't get it. Please be clearer.
[06:43] <wallyworld_> so for the private bugs setting, you need more privileges to turn private bugs on than to turn it off
[06:43] <wallyworld_> but i guess for visibility it's symetrical
[06:43] <StevenK> Right
[06:43] <wallyworld_> ok
[06:43] <StevenK> You can either it change it, or you can't.
[06:44] <wallyworld_> np. sorry
[06:44] <wgrant> It's probably not symmetrical.
[06:44] <wgrant> eg. if my commercial subscription expires, surely I can unprivate my team?
[06:45] <wallyworld_> StevenK: i'm also wondering why we're not using validate() to setField error
[06:46] <wallyworld_> oh, atm, field is not shown if not permitted
[06:46] <wallyworld_> so no need
[06:46] <wallyworld_> this is turning out very much like the private bugs branch
[06:48] <StevenK> wallyworld_: Yes.
[06:49] <StevenK> wgrant: Mmmmm. I think I'd like to do that as a seperate branch, this one is already hairy enough.
[06:51] <wallyworld_> StevenK: what is nagging at me for this branch is that the rules for visibility allowed or not are in 2 places - the view to omit the field or not plus the model to do the save
[06:51] <wgrant> Well I think I'd like postgres to be smart and do a union over sorted index components, but it won't :)
[06:52] <StevenK> wallyworld_: Right. Perhaps I need to refactor it out.
[06:52] <wallyworld_> in my branch, i now have a checkPrivateBugsTransition() method which raises if not allowed. the validate() calls this and uses ex.message to set the field error
[06:53] <wallyworld_> and the model nutator also calls checkxxx() which raises if the api is used
[06:53] <wallyworld_> for example
[06:53] <wallyworld_> so it's trying to put all the permission checks in one place
[06:54] <StevenK> wallyworld_: So I can refactor out into IPerson.checkAllowVisibility if you think that's appropiate
[06:54] <wallyworld_> StevenK: i think the same technique is used in the team edit view for the team moderated/restricted etc radio buttons
[06:54] <wallyworld_> StevenK: i think that works
[06:55] <wallyworld_> StevenK: with those moderated/restricted radio buttons, i catch the exception and use the message to put in that little info line at the top of the widget
[07:00] <StevenK> wallyworld_: Anything else?
[07:02] <wallyworld_> StevenK: don't think so.you'll need to add different tests depending on what changes you make. there should probs be a -ve test for the test_person_with_cs_can_create_private_team test
[07:02] <StevenK> I'm so over this branch
[07:03] <wallyworld_> StevenK: that's how i feel about the private bugs one :-(
[07:03] <StevenK> wallyworld_: Er, but if the user has no CS, they don't see the field?
[07:04] <StevenK> There are existing tests for that case, the branch adds an end-to-end test to make sure the team gets created.
[07:06] <wallyworld_> StevenK: yes, you are right. i was thinking about any future asymetric permission workflow
[07:06] <StevenK> Which we haven't done yet. :-P
[07:07] <wallyworld_> StevenK: so i've got to run to the shops before it hails (sky is green and hail is expected). i'll +1 after dinner
[07:07] <StevenK> Bleh
[07:07] <wallyworld_> or sooner if you want
[07:07] <wallyworld_> give me 20 minutes?
[07:07] <StevenK> wallyworld_: Yeah, sure.
[07:08] <wallyworld_> thanks. will do it as soon as i can
[07:47] <wallyworld_> StevenK: r=me with a suggestion for a test for the ImmutableVisibilityError
[08:35] <wgrant> stub: Do you have much idea how much DB IO we'd save by not reading large fields like Bug.description during searches?
[08:36] <stub> Are we scanning them, or looking at the fti indexes?
[08:37] <wgrant> FTI, so the description column isn't used at all.
[08:37] <wgrant> Except Storm asks for it, even though it won't use it.
[08:37] <stub> I have no real metrics. It would certainly be faster to display a list of 70 odd bugs (a batch from a search results). I don't know how much.
[08:37] <wgrant> It's hopefully the only large toasty field that Storm asks for
[08:37] <wgrant> So it seems like we could save a lot of toast table reads.
[08:38] <stub> For the actual search, the fti index is used - removing description would make it smaller but I don't think it would save that much.
[08:38] <wgrant> Oh, blah, fti gets rechecked even after the GIN.
[08:39] <wgrant> So I guess we can't skip toast reads for that, only description.
[08:40] <wgrant> stub: I'm going to experiment with using GIN for tag searches tomorrow.
[08:41] <wgrant> Since they're currently the slowest common search in my new schema.
[08:42] <stub> wgrant: So the trick with PG < 9.1 is that you need to determine if the query is going to do a full index scan and fail it, rather than generate an OOPS because the index gets asked to do something it doesn't support.
[08:43] <stub> With 9.1, GIN is drop in - we just get slower writes and faster reads
[08:43] <wgrant> Yeah, but I'm hoping we'll be on 9.1 soon enough :)
[08:43] <wgrant> Since it makes Ubuntu searches 5x faster.
[08:43] <wgrant> s/it/GIN/
[08:43] <stub> Yes. I'll be converting all our GIST indexes to GIN.
[08:50] <wgrant> stub: Could you please pastebin index usage numbers from bug and bugtask?
[08:55] <stub> k
[08:56] <stub> I should snapshot these like I do the disk space ones for generating deltas
[08:56] <wgrant> Yeah, that sounds pretty easy and potentially very useful.
[08:57] <wgrant> But not critical for my uses right now :)
[09:00] <stub> https://pastebin.canonical.com/60003/
[09:00] <stub> couple of obvious dead ones there...
[09:02] <wgrant> I'll be deleting most of them in a month or so anyway :)
[09:04] <StevenK> bugtask__binarypackagename__idx seems utterly useless
[09:04] <mrevell> Hello!
[09:04] <wgrant> StevenK: Considering the column should never have existed, and hasn't been used since at least 2005, yeah.
[09:04] <wgrant> It's all part of The Plan.
[09:05] <StevenK> wgrant: But like you say, it's probably not worth just deleting the unused indices yet
[09:05] <wgrant> Right, considering I will be taking a chainsaw to those tables in the coming little while.
[09:06] <StevenK> I'm looking forward to it.
[09:06] <StevenK> stub: Is it easy to see if any indices aren't used at all, ignoring those on bug and bugtask?
[09:07] <stub> Yes. There are false positives though, such as indexes rarely used but need to be there anyway to support that once-every-three-months operation
[09:08] <StevenK> If they're used once-every-while, they should have some non-negative read and scan results, right?
[09:08] <stub> I also can't generate deltas, so the numbers are from whenever the stats were last reset, and I don't know when that was :)
[09:09] <adeuring> good morning
[09:09] <StevenK> I can't imagine we reset the stats at all. Some of the those numbers look very large.
[09:10] <stub> https://pastebin.canonical.com/60004/
[09:10] <stub> That needs to go in the database report really
[09:10] <StevenK> Holy crap, 208
[09:10] <StevenK> Er, 280
[09:11] <StevenK> bugnotificationarchive ? Really? Is that even used?
[09:11] <stub> That really should be filtered by num tuples in the table or something too. There are a lot of indexes there to support deletions and such if the tables grow.
[09:12] <StevenK> stub: If you don't think the gardening is much of a help, that's fine, I'm effectively offering to grab a pair of shears.
[09:12] <stub> The gardening will help. Just needs a little thought on each one though, which I'll double check on review.
[09:13] <stub> Leave all the indexes on person
[09:13] <stub> And libraryfilealias
[09:13] <stub> And the primary keys, and uniques...
[09:14] <stub> Hmm... can I filter them out?
[09:14] <StevenK> Hmm, like on the indexrelname?
[09:15] <stub> _key, _unq, _pk, _pkey are good hints, but we don't always follow conventions unfortunately.
[09:21] <mrevell> Everyone say hello to czajkowski, who joins the LP team at Canonical today :)
[09:21] <stub> StevenK: https://pastebin.canonical.com/60006/ is better to work from. It skips all the unique indexes, and shows the number of tuples in the table (for indexes that would be used if we had more data)
[09:22] <stub> Hello czajkowski, who joins the LP team at Canonical today
[09:22]  * czajkowski waves hi 
[09:23] <wgrant> Morning czajkowski.
[09:23] <stub> What timezone are you in czajkowski?
[09:23] <czajkowski> UTC, I live in London
[09:24] <stub> In the office or at home?
[09:24] <czajkowski> at home, will pop in from time to time in the office though as I live nearby
[09:24] <adeuring> morning czajkowski
[09:24] <czajkowski> wgrant: adeuring ello :)
[09:25] <stub> I hated commuting on the tube for the few months I was there.
[09:25] <czajkowski> it's about a 15-20 min walk from here
[09:29] <StevenK> stub: Haha, did you see tm__potmsgset__language__not_used__idx in that?
[09:47] <nigelb> 34
[09:49] <nigelb> ugh
[12:24] <StevenK> rick_h: O hai!
[12:24] <rick_h> StevenK: howdy
[12:24] <rick_h> how went the whole thing? Saw your email, will probably get a reply going with some notes.
[12:25] <StevenK> rick_h: Sweet. Buildbot finished ~15 minutes ago, I was hoping that either qas would be a burning ball of fail or working fine before I went to bed.
[12:25] <rick_h> StevenK: heh ok. So it's not hit qas yet then?
[12:25] <StevenK> rick_h: qas takes ~40 minutes to update.
[12:25] <rick_h> ok
[12:25] <StevenK> Next cron for qas kicks off at :48
[12:26] <StevenK> I doubt it hit the :18 run
[12:26]  * rick_h crosses fingers
[12:26] <jelmer> mogguh StevenK, rick_h
[12:26] <StevenK> jelmer: G'tag (if you remember that one :-)
[12:26]  * rick_h looks up mogguh
[12:26] <jelmer> StevenK: yeah :)
[12:30] <StevenK> rick_h: Did you get a chance to look at the AJAX log thing?
[12:30] <rick_h> StevenK: not yet, created a ticket for it and it's on the board
[12:30] <StevenK> s/ticket/card/
[12:30] <StevenK> Ticket makes think of RT.
[12:30] <rick_h> sorry, created a bug and a card and the card is on the board :)
[12:30] <rick_h> ticket == bug sorry
[14:01] <deryck> Morning, all.
[14:01] <deryck> Oh happy uninterrupted-duty day!
[14:03] <rick_h> lol
[14:13] <flacoste> morning deryck!
[14:31] <deryck> abentley, adeuring, rick_h - https://plus.google.com/hangouts/extras/talk.google.com/orange-standup
[14:53] <deryck> mrevell, my wife's van died at her shop where I'm working. And a tow truck is coming to get it. I'll have to go out when he arrives (in 10 mins likely).
[14:53] <deryck> mrevell, should we push back our call with czajkowski or schedule for tomorrow?
[14:54] <czajkowski> deryck: ello
[14:54] <deryck> hi czajkowski!  You have no idea how happy the Orange Squad is that you have joined Launchpad! :)
[14:54] <czajkowski> I'm going to hope this is a good thing :)
[14:55] <deryck> czajkowski, it is! :)
[14:55] <deryck> flacoste, look at my luck with this stupid van ^^ :)
[14:55] <mrevell> czajkowski, deryck: Tomorrow is fine if you're not sure when you'll be done with the tow truck. If not, how about 16utc? I won't be available then but you and czajkowski can talk.
[14:56] <deryck> mrevell, either way is fine with me.  Shouldn't take but about 10-20 minutes to get him a key and on his way.  but unfortunately, it just hits during our call.
[14:56] <deryck> mrevell, I called when I got in this morning, but he's only now able to come here.
[14:56] <mrevell> No prob :) Let's say 16utc, if that suits you czajkowski.
[14:57] <flacoste> deryck: is it again your wife's doing this time? her general delicacy in handling mechanical/technology device?
[14:57] <deryck> ha!
[14:57] <czajkowski> mrevell: deryck sounds good to me
[14:57] <deryck> flacoste, one can never know for sure. :)
[14:57] <deryck> mrevell, czajkowski -- sounds good to me too.
[14:57] <mrevell> great :)
[14:58] <deryck> flacoste, I think it's just an alternator though, which is just unfortunate timing.
[14:58] <flacoste> ah right
[14:58] <flacoste> that sucks
[15:38] <sinzui> jcsackett, are you back yet. I shot myself in the foot. I want to talk about invalidating a cache to apply more than one commercial subscription to a projects
[16:09] <jcsackett> sinzui: i'm back now. my apologies, my timing estimates didn't account for a traffic accident on the way home.
[16:11] <rick_h> jcsackett: quit running into people. It's not good for you
[16:11] <jcsackett> rick_h: thankfully, i wasn't *in* the accident. i was just blocked by it. :-P
[16:11] <sinzui> That accident was a MiB conspiracy. Your domicile is bugged
[16:12]  * jcsackett laughs
[16:12] <sinzui> mumble?
[16:13] <sinzui> jcsackett, or hangout
[16:14] <jcsackett> let's go mumble. google+ was crashing my computer yesterday and i don't yet trust it again.
[16:37] <rick_h> deryck: so finally changed my last .html file for a bit hopefully! https://code.launchpad.net/~rharding/launchpad/combo_yui_tests5/+merge/92092
[16:37] <deryck> nice!
[16:37] <rick_h> deryck: if you get a few ^ first one is on ec2 and once that goes can land these other 4 side by side
[16:38] <deryck> rick_h, is that a review request? :)
[16:38] <rick_h> deryck: yes please :)
[16:38] <deryck> rick_h, will do :)
[16:38] <rick_h> I htink benji would hunt me down if I hit up ocr
[16:38] <deryck> meh. what's 4200 lines among friends.
[16:39] <rick_h> heh, across 34 files
[16:42] <sinzui> jcsackett, bzr+ssh://bazaar.launchpad.net/~launchpad-purple-squad/%2Bjunk/disclosure_mockups/
[16:46]  * benji prepares the specially-trained rick_h hunting dogs.
[16:58] <czajkowski> deryck: nice to put the face to the nick btw
[16:59] <deryck> czajkowski, thanks!  Likewise.
[17:20] <abentley> deryck: With further optimization, I completed the branch scan in 3 minutes!
[17:21] <deryck> abentley, woah!  nice!
[17:23] <abentley> deryck: I had to use store._connection.build_raw_cursor().executemany to do it, so there's some work required to de-hackify it, but at least we know it's possible.
[17:24] <deryck> abentley, nice that it's possible, and yuck that it's so non-obvious to do multiple inserts.
[17:38] <lifeless> mornin
[17:38] <sinzui> czajkowski, https://dev.launchpad.net/MaintenanceRotationSchedule
[18:21] <abentley> lifeless: mornin.
[18:22] <abentley> lifeless: I'd like to chat with you about task queues.  Are you up for that sometime today?
[18:23] <lifeless> sure
[18:23] <lifeless> how many hours till your EOD ?
[18:36] <danilo> benji, heya, I've put a few merge proposals up for pygettextpo (https://code.launchpad.net/pygettextpo/+activereviews): it's used only by LP but I want to make it more complete and usable (and perhaps able to replace gettext_po_parser.py in Launchpad as well)
[18:37] <benji> danilo: ok, I'll take a look
[18:38] <danilo> lifeless, flacoste: I wonder if we should create a separate maintainer team for pygettextpo: I do want to contribute quite a bit of code to it, and I am guessing I might be the one with the biggest interest in driving that :)
[18:40] <danilo> benji, thanks!
[18:41] <lifeless> danilo: we can, but you should be in lp-canonical-emeritus anyway, which should give landing rights
[18:42] <danilo> lifeless, right, cool, I am (in canonical-launchpad-emeritus)
[18:44] <danilo> lifeless, (and actually, I was still in ~launchpad, so I left that team)
[18:44] <lifeless> :P
[18:44] <lifeless> So, while that remains true I wouldn't expect any unusual friction; you still count as a reviewer, the policies don't seem onerous or irrelevant, ...
[18:45] <lifeless> but like I say, we certainly can give it separate governance if thats useful
[18:47] <abentley> lifeless: EOD in 3+ hours.
[18:48] <lifeless> abentley: ok, cool. I have 2 other calls (one with the UK) but we should be able to find a timeslot.
[18:48] <danilo> lifeless, sure, I don't have a particular need for it, as long as this works
[18:48] <lifeless> danilo: if it doesn't, lets try to fix it first (its a new thing after all) and then iterate from there
[18:48] <abentley> lifeless: cool.
[18:48] <danilo> lifeless, agreed, thanks
[18:54] <abentley> lifeless: I can't find a non-hacky way of using cursor.executemany via Storm, but my tests show a significant performance win for branch scanning.  Do you know why it's not provided?
[19:10] <rick_h> benji: if you have a sec to review: https://code.launchpad.net/~rharding/launchpad/ajax_log_930141/+merge/92827
[19:10] <benji> rick_h: sure
[19:11] <rick_h> StevenK: ^^ for you since I know you were asking about it
[19:11] <rick_h> benji: ty
[19:53] <flacoste> lifeless: i wouldn't speaking a little later today, that could make it convenient to talk to abentley
[19:55] <lifeless> cool, thanks
[20:24] <sinzui> Sweet, bringing thunderbird to focus also kills X.
[20:25] <lifeless> abentley: lets try for a chat now?
[20:25] <abentley> lifeless: sure.  Preferred chat tech?
[20:25] <lifeless> abentley: I have a call with elmo but not sure just when; uhm skype is le trivial for me
[21:40] <flacoste> lifeless: let me know when you are available
[22:04] <sinzui> wgrant, I cannot hear you
[22:08] <lifeless> flacoste: in 5 ?
[22:08] <flacoste> lifeless: might be best to reschedule
[22:08] <StevenK> rick_h: Yay!
[22:08] <StevenK> rick_h: That looks awesome.
[22:09] <flacoste> lifeless: unless your agenda is short
[22:13] <lifeless> flacoste: we could time box to 15m and see? If you're past EOD lets just adhoc it tomorrow
[22:13] <flacoste> lifeless: timeboxing to 15 minutes suits me
[22:37] <wgrant> sinzui, wallyworld_: https://launchpad.net/ubuntu/precise/+localpackagediffs
[22:37] <wgrant> See the column headers.
[23:03] <lifeless> wgrant: ok, you have mail
[23:03] <lifeless> I am popping out for a little bit, back soon
[23:03] <sinzui> wgrant, StevenK, jcsackett, wallyworld_: I published part one of the terminology http://blog.launchpad.net/
[23:12] <wgrant> lifeless: Thanks.
[23:47] <wgrant> jelmer: Do you know what caused the performance regressions in your revision_history() branch last time?
[23:53] <StevenK> wgrant: Do I need to drop all indices first, or will dropping the table take care of everything?
[23:55] <wgrant> StevenK: You can't drop the table directly. Change its schema to todrop instead.
[23:55] <wgrant> upgrade.py does the special slony dance to drop them from there.
[23:55] <StevenK> wgrant: Ah yes. But will the indices disappear as well?
[23:56] <wgrant> StevenK: This is Entitlement?
[23:57] <StevenK> wgrant: Yes.
[23:59] <wgrant> StevenK: So, yes, indices go with the table automatically.