/srv/irclogs.ubuntu.com/2012/01/24/#launchpad-dev.txt

wgrantlifeless: So, access policy model.00:05
rick_hStevenK: wallyworld ok, boy is in bed. You guys need anything from me then or are we all set?00:05
lifelesswgrant: shure00:05
wgrantlifeless: The queries end up sufficiently complex that the array indices don't seem to help.00:05
wgrantlifeless: Adding an extra intermediate table to link to multiple policies might perform OK, but it's really hard to say on DF.00:06
lifelesshmm00:07
wgrantI have schemas + populations + queries at http://people.canonical.com/~wgrant/launchpad/access-policies00:07
lifelesshow do they perform00:07
wgrantI can't tell.00:07
wgrantDF's performance is too variable.00:07
lifelesswould you like me to run some shit somewhere00:07
wallyworldrick_h: just waiting for the buildbot upgrade. thanks for asking. next thing is to get the convoy patch done etc but you have that in hand already00:07
rick_hwallyworld: ok cool.00:08
wallyworldrick_h: i merged your branch and will land mine as soon as we are ready to go. i'll let you know what transpires today00:08
rick_hwallyworld: ok, sounds like a plan00:09
wgrantlifeless: I'd like if you could experiment a little. There are two there for which performance on qastaging would be interesting. accesspolicy*.sql set up the schema and populate it with real data. The other two are queries for the corresponding schemas.00:13
lifelessare they mutually exclusive ?00:14
wgrantYes.00:14
lifelesswgrant: your tests are missing analyzes of some temp tables00:18
lifelessand with-arrays has a syntax error00:18
lifelesswgrant: some initial results: http://paste.ubuntu.com/814915/00:19
wgrantlifeless: The two sets of queries are basically just a dumping ground for things that I tried.00:20
wgrantUnlike the schema bits, they're not meant to be executed in one go verbatim.00:20
wgrantHah, one of those is for launchpad, the other for ubuntu.00:21
lifelesshmm, I'm surprised at the scope fo rthe CTE working in the second half of the union00:24
wgrantWhere?00:26
wgrantA lot of these queries are probably horrible, as I maimed them in horrible ways to try to get different plans.00:26
lifeless  OR EXISTS ( WITH teams AS (SELECT team FROM TeamParticipation WHERE person = 21997)00:26
lifeless..00:26
lifelessUNION00:26
lifeless...00:26
lifelessteams00:26
lifelessthe LP with-arrays example00:27
wgrantAh, right.00:27
lifelessanyhow, LP is 300ms which is reasonably snappy (for the batch)00:27
wgrantYeah, that's reasonable for LP.00:27
wgrantWhich is the normal case.00:27
wgrantUbuntu is another story.00:27
wgrantI was able to do any LP thing in 500-700ms on DF.00:27
lifelessits bringing back 5800 bugs00:28
wgrantHm, I apparently lost the original query at some point.00:28
lifelessso the ubuntu case would be 100 times slower00:28
wgrantThat is the one with the current subscription-based queries.00:29
wgrantRight, except it uses a different plan.00:29
lifelessalso00:29
lifelessI tihnk the sort is wrong - see the sort bug I filed today00:29
lifelesswe do -importance, bugtask.id00:30
lifelessthese days, at least for some scopes.00:30
lifeless5.1 seconds for ubuntu00:30
lifeless         ->  Nested Loop  (cost=0.00..4068090.66 rows=152081 width=69) (actual time=0.206..5084.616 rows=98557 loops=1)00:30
lifelessthis00:33
lifelessORDER BY BugTask.importance DESC, BugTask.id LIMIT 7600:33
lifelessmakes it00:33
lifeless Total runtime: 160.709 ms00:33
wgrantorly00:33
lifelessdunno how well it will scale when you're at offset 5000 (or constraints based that deep in)00:34
lifelesshttp://paste.ubuntu.com/814929/00:34
wgranthttps://pastebin.canonical.com/58655/ is the current query, for comparison00:35
lifelessso, thats fast because 1/2 the bugs are ubuntu bugs, and the index matches the sort, so it goes 'wha-hay, lets just walk the index'00:35
wgrantYep00:36
wgrantLike we saw a couple of months abck00:36
lifelesscount(*) is  Total runtime: 3892.111 ms00:36
lifelessbut that is largely separate00:37
wgrantHow I envy every other bugtracker :(00:40
lifelessbecause they don't have to handle our scale?00:40
wgrantRight.00:41
wgrantI guess they also lack bugtasks.00:41
=== ajmitch__ is now known as ajmitch
lifelesswgrant: I have a possible search schema00:43
lifelessnow where did I put it00:43
wgrantThat's what I was hoping :)00:43
lifelesshttp://paste.ubuntu.com/814938/00:44
lifelesswgrant: does that help ?02:02
wgrantlifeless: Maybe :)02:04
StevenKHmmmmm. Looks like there is a bug with how nominations check04:58
* StevenK checks if he can break it, to see if he should notify and redirect04:58
* StevenK peers at the dev sign-in form.05:04
StevenKOh yes, wgrant ripped out passwords.05:04
wgrantYeah05:05
wgrantNo passwords for you.05:05
StevenKHmmmm, if wgrant isn't being evil to DF ...05:08
wgrantI'm not, no.05:11
StevenKwgrant: O hai, Mr OCR -- https://code.launchpad.net/~stevenk/launchpad/notification-already-targetted/+merge/8982605:15
wgrantStevenK: Shouldn't those permissions work?05:18
wgrantIf they're defined at all for those objects, they should surely be transitive.05:18
StevenKwgrant: They don't seem to, and we spent a while hammering out userHas*Privileges()05:18
StevenKSo we should use them ... :-)05:19
wgrantSure, but if there are incorrect launchpad.Driver and launchpad.BugSupervisor for ISourcePackage, we should probably fix that.05:19
wgrantIt's fine if they don't exist.05:19
wgrantBut not if they exist but are incorrect.05:19
StevenKI think it's being called on a SSP05:20
wgrantEr05:20
wgrantDoesn't only code use SSP?05:20
wgrants/code/Code/05:20
StevenKhttps://bugs.launchpad.net/ubuntu/maverick/+source/libvirt/+bug/632696/+nominate should work for me, I have bug supervisor superpowahs, right?05:21
_mup_Bug #632696: libvirt won't start a VM with serial or console <amd64> <apport-bug> <maverick> <server-mrs> <libvirt (Ubuntu):Invalid> <libvirt (Ubuntu Maverick):Confirmed> < https://launchpad.net/bugs/632696 >05:21
wgrantSure.05:21
wgrantBut is that because the permissions are incorrect, or because they don't exist?05:22
wgrantIf they don05:22
wgrantdon't exist, this fix is correct.05:22
StevenKI'm not sure how to check05:22
wgrantLook for corresponding adapters.05:22
wgrantOr, better still, use bin/harness and try to adapt them :)05:23
StevenKYou just lost me05:23
wgrantCheck which security adapter happens when you try to adapt ISP to launchpad.Driver or launchpad.BugSupervisor.05:23
* StevenK drowns in zope calls05:36
StevenKAnd checkPermission() is utterly incomprehensible. :-(05:42
wgrantIAuthorization(someobj, name='launchpad.Something')05:44
StevenKI was just coming to that conclusion, due to iter_authorization.05:45
StevenKIn [5]: IAuthorization(spn, 'launchpad.BugSupervisor')05:47
StevenKOut[5]: 'launchpad.BugSupervisor'05:47
wgrantname=05:48
wgrantThe second argh is the default.05:48
StevenKBah05:48
wgrant-h05:48
StevenKTypeError: 'name' is an invalid keyword argument for this function05:48
wgrantMay have to use queryAdapter or getAdapter directly.05:49
StevenKIn [16]: auth = queryAdapter(spn, IAuthorization, 'launchpad.BugSupervisor')05:50
StevenKIn [17]: print auth05:50
StevenK-------> print(auth)05:50
StevenKNone05:50
wgrantspn?05:50
StevenKBleh05:51
StevenKfactory.makeSourcePackage(), and then the above on that results in the same05:51
wgrantThat suggests there's no adapter at all.05:52
wgrantSo your solution may be correct.05:52
wgrantBut check similar for launchpad.Driver05:52
StevenKIn [23]: auth = queryAdapter(sp, IAuthorization, 'launchpad.Driver')05:52
StevenKIn [24]: print(auth)05:52
StevenK<lp.security.SeriesDrivers object at 0x80e1dd0>05:52
StevenKwgrant: I had secessfully managed to block lib/lp/security.py from my mind, damn you.05:55
StevenKThere are no adapters for ISP at all05:56
wgrantStevenK: Right.06:06
wgrantSo your description is incorrect, but your code is right.06:06
StevenKwgrant: "Work around lack of ISP security adapters by using IBugTask.userHas*Privleges()" ?06:14
wgrantStevenK: Something like that.06:15
adeuringgood morning08:57
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
=== matsubara-afk is now known as matsubara
rick_hmorning11:10
rick_hruh roh, did the use-convoy not make it last night?11:16
StevenKrick_h: I don't think buildbot hasn't been upgraded11:26
StevenKSigh, grammer fail11:27
rick_hheh, ok11:27
StevenKWhy is the BranchRevision SQL on the branch page taking 8.8 seconds? :-(11:47
wgrantStevenK: Probably the scanner being a bit greedy with its locks.11:47
StevenKAnd this is terrible. Thanks the base64 disease in OOPSes, you can't trigger an OOPS on one machine and then eyeball-copy the OOPS to the other.11:48
wgrantIt solves more problems than it causes :)11:49
wallyworldrick_h: hi, still waiting on the rt to be done before we can land11:59
rick_hwallyworld: ok cool12:00
wallyworldi was hoping to be able to tell you it was done12:00
rick_hheh, np12:00
rick_hI need to work off that branch to start work on the other issues so I'm ancy. I'll be patient :)12:00
wallyworldi want it done too12:01
wallyworldi initially needed to update the rt with the required version info so there was a hold up there, but hopefully there's no more roadblocks12:02
=== Topic unset by gmb on #launchpad-dev
gmbOh, sod it.12:25
=== gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugtasks: 3*10^2
gmbAlthough, that said...12:29
* gmb lunches12:29
=== almaisan-away is now known as al-maisan
tumbleweedlifeless: you appear not to be subscribed to bug 583426. Are you ok with the dh_python2 namespace approach for lazr?13:16
_mup_Bug #583426: Missing __init__.py <apport-bug> <i386> <lazr.restfulclient (Ubuntu):Confirmed> <lazr.uri (Ubuntu):Confirmed> <protobuf (Ubuntu):New> < https://launchpad.net/bugs/583426 >13:16
=== al-maisan is now known as almaisan-away
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
=== almaisan-away is now known as al-maisan
=== matsubara is now known as matsubara-lunch
deryckabentley, adeuring, rick_h -- hey there, guys.  did you guys hold the standup?14:48
abentleyderyck: yes./14:48
deryckabentley, adeuring, rick_h -- shall we do it now, or in 10 at the top of the hour?14:49
abentleyderyck: I'm ready now14:49
adeuringeither time is fine for me14:49
rick_hyea, fine either way14:49
deryckok, let's do it now then.14:49
Laneybigjools: am I right in thinking that spph.creator wasn't there from the start of copyPackage? i.e. early syncs using that method won't have that set15:32
bigjoolsLaney: ummmm I can't remember.  It's NULL for non-syncs for sure.  rvba, can you remember?15:33
* tumbleweed recalls that being the case15:33
rvbaLaney: bigjools yes, that's true, creator was introduced afterwards.15:34
Laneyhmm OK15:34
Laneywas wondering why we misidentified some early syncs. guess there's no way for us to tell15:34
tumbleweedLaney: date range...15:35
Laneytell /who/ synced15:35
Laneyrvba: thanks for the info15:36
rvbawelcome.15:36
deryckabentley, I'm free now.  Sorry it took so long.  mumble or g+ hangout?15:38
abentleyderyck: mumble15:38
=== matsubara-lunch is now known as matsubara
jcsackettsinzui: regarding the grackle-lp-views branch. create bug to land, or just land no-qa?16:15
sinzuicorrect16:15
sinzuijcsackett, I unassigned myself from grackle-client. I need to announce the terminology/use cases decisions from Budapest, and revise  interactive mockup to hand off to Huw and Ian16:17
sinzuijcsackett, I will take the card back tomorrow if all goes well16:18
jcsackettsinzui: ok. when you say correct, was that to no-qa? could have applied to either clause of my question. :-P16:18
sinzuijcsackett, does it fix a specific bug?16:19
sinzuiI think not so no-qa16:19
jcsackettsinzui: that's where i was leaning, but i like second opinions. :-P16:19
jmlis there any auth on the LP oops system?16:19
jmli.e. what mechanism do you use to prevent random bozos on the Internet submitting oopses?16:20
sinzuijcsackett, I might be the wrong person to ever ask the question. I QA most of my no-qa bugs because I am paranoid. I would still visit the new view you created and verify the page title and headings16:20
jcsackettsinzui: oh yeah, i'm not thinking fire and forget. just whether it's worth filing a filler-bug.16:21
jcsacketti think we agree that it's not, but i totally plan on looking at the branch on qastaging.16:21
sinzuijcsackett, I do not think a bug is needed. The view does not provide value to anyone yet16:22
abentleyjcsackett: Are you handling the case where message body is too long?16:24
abentleyjcsackett: I'm a bit obsessed with that right now, because that's the root cause of bug 91109016:24
_mup_Bug #911090: merge proposal is broken with consistent timeouts <timeout> <Launchpad itself:Triaged by abentley> <Tarmac:New> < https://launchpad.net/bugs/911090 >16:24
jcsackettabentley: our plan is to auto-truncate the body when displaying messages, iirc.16:25
jcsackett...auto-truncate is redundant.16:25
jcsackettwe'll truncate long messages, and allow another means of seeing the whole thing.16:25
jcsackettbut we're not a stage where any solution to that is set yet.16:25
abentleyjcsackett: Cool.  Yeah, I'm implementing that for comments now.16:25
jcsackettabentley: yeah, i can see that being a pain in comments.16:25
abentleyjcsackett: We have three classes of message bodies: Short, long and excessive.16:27
abentleyjcsackett: long and excessive messages get truncated, with a download link.16:28
abentleyjcsackett: long messages also get a "Read more" link that lets you view the message in the Launchpad UI.16:28
jcsackettabentley: that sounds like a good solution. i'll ape it so we don't have two means of dealing with the same problem. :-)16:29
abentleyjcsackett: Well, don't go overboard.  Since most comments can be adapted to IMessages, maybe we should just move the code to IMessage.16:31
* jcsackett nods16:31
jcsackettabentley: when do you think you're code is likely to start landing?16:31
abentleyjcsackett: this week.  Tomorrow if I'm lucky.16:32
jcsackettwe're doing a lot of incremental landings for grackle integration, and nothing is going to get linked to and really wired in until we can integrate grackle properly, so i can probably move relevant bits into IMessage as i go.16:32
jcsackettawesome. i'll take a look as i'm going then and if it works take advantage of that code.16:32
jcsackettabentley: moving it into IMessage though may best wait to the day we start moving all of our message stuff into this same archive idea, which i think is longer-range than getting grackle in for mailing lists.16:33
abentleyjcsackett: Part 1 is here, but there's a lot of other stuff: https://code.launchpad.net/~abentley/launchpad/bugcomment-as-icomment/+merge/8949716:34
jcsackettabentley: dig. i think i recall you mentioning some of this at the epic.16:35
=== al-maisan is now known as almaisan-away
abentleyrockstar: I want to turn ICodeReviewComment.message_body into a method.  It used to be one, but you changed it into a property.  Can you comment?16:44
abentleyrockstar: (I want to do this because properties are always rendered, and Tarmac is making 16MB message bodies.)16:46
rockstarabentley, *groan* I can't comment.  It's been too long.16:46
abentleyrockstar: Okay.16:47
rockstarI think that might have been the result of a code review, actually.16:47
abentleyrockstar: It didn't look like it.  I was the reviewer, and I don't remember either.16:48
rockstarOh, uh, probably ignorance on my part then.16:48
abentleyrockstar: r7719.1.4 if you're curious.16:50
rockstarOh, it was part of the API!16:51
rockstarleonardr told me to do it that way.16:51
jmlflacoste: fwiw, the thing I talk about http://code.mumak.net/2012/01/undistract-me.html addresses one of your past concerns about LP's test suite16:51
flacostejml: i saw!16:52
abentleyrockstar: Doh!16:52
jmlflacoste: cool :) it's working out really well for me16:53
rockstarabentley, I think there was something about the API decorators that was causing problems.16:54
=== beuno is now known as beuno-lunch
=== beuno-lunch is now known as beuno
=== dpm is now known as dpm-afk
lifelessrick_h: hey20:01
lifelessrick_h: have you looked closely at handlebars?20:01
lifelessjml: our oops system is firewalled off20:02
lifelessjml: we have a plan for a javascript oops system, which will let random bozos submit oopses20:02
lifeless-> automated signal detection will matter, but the fact is random bozos will need access (can't prove a browser was on our website...)20:03
lifelessabentley: perhaps we should reject the 16MB messages (and offer attachments instead) ?20:05
abentleylifeless: We generally render textual attachments, so I don't think that helps, and I prefer systems that don't have discontinuities.20:08
lifelessabentley: we don't render textual attachments in bugs - at all20:38
lifelessabentley: diffs are a dicontinuity already :)20:38
abentleylifeless: I'm talking about code review comments.20:39
=== salgado is now known as salgado-afk
rick_hlifeless: I'm using it for my personal app. I'm liking it so far.21:45
rick_hlifeless: I've not looked at the source much though, so could be evil, but doubt it if YUI is building it into 3.521:46
sinzuijcsackett, mumble?22:04
deryckG'night, all.22:34
wallyworldrick_h: use-convoy branch now in ec2 for real :-D22:37
StevenKIt was for real last time, too. Now it will land if the tests passs.22:37
StevenKs/\(ss\)s/\1/22:38
wallyworldStevenK: that's what i meant by "for real"22:38
StevenKHah, another vuln in wordpress <= 3.3.122:39
rick_h_droidawesome22:43
rick_h_droidstevek I might need a hm. f22:43
rick_h_droidbah22:43
rick_h_droida hand with some make/buildout magic to help tests setup22:44
rick_h_droidI think we're still going to want a copy of yui and such in build/js22:44
wallyworldrick_h: so, you have the convoy patch sorted, and are now looking at the tests? the first thing i'd do is change the yui seed script location in all the html files  so that it doesn't use the symlink in icing22:53
rick_h_droidwallyword yea, I had a discussion with deryck on it today. I think the key will be pointing all seed files at the build js directory and create a symlink there to change yui versions as required23:13
lifelessrick_h: hey hey23:17
rick_h_droidhowdy23:17
lifelessrick_h: so yeah, I've been looking at it to asses a pybars port23:18
rick_hlifeless: ah, nice23:18
wallyworldrick_h: what you say above about yui tests matches how i envisaged it would work also23:30
rick_hwallyworld: yea, so I'm working on my make/buildbot skills because right now things are built straight into the convoy directory23:31
rick_hand we'll need a new command to move the symlink in the build/js directory from one version of YUI to another and such23:31
rick_hI did it with a bunch of magic JS code today and deryck suggested I might have overcomplicated the issue a little bit :)23:31
wallyworldyeah, kiss is good23:33
rick_hit's ok, I got the oreilly gnu make book and I'm starting to try to make it less scary for me23:34
wallyworlda goal here for me at least is to totally remove the yui symlink and anu other build cruft from the icing dir23:34
rick_hright, I'm making sure nothing but the link to combo.css comes from icing any more23:35
wallyworldsounds good23:35
rick_hanyway, work for tomorrow. Good luck with the landing23:35
rick_htime to work on getting the boy to bed23:35
wallyworldthanks :-) will let you know23:35
lifelessrick_h: blah, elocal23:43
lifelessrick_h: so, it seems to me that handlebars has kindof missed the 'separation of code and presentation' angle of moustache23:44
lifelessrick_h: I was wondering if you had any insight into that23:44
StevenKwgrant: QA for r1471723:46
rick_hlifeless: I find it provides a decent middle ground. You can define new blocks and helpers23:48
rick_hbut it's still fairly limiting as a template language goes23:48
rick_hI mean, I know there's a very anti-code in the templates kind of thing, but I also feel that view belongs in the view (template) and things like concat'ing, prettying dates, etc aren't model duties23:49
rick_hlifeless: which part do you find crosses the border?23:50
StevenKwallyworld: Can you look at kanban and see if any cards can move? I know the convoy one can't yet, but some others may be able to23:53
wallyworldok23:53
wallyworldStevenK: the two cards in landing are part of the use-convoy branch, so there's nothing obvious to move23:54
StevenKwallyworld: Right23:55
lifelessrick_h: e.g. on here http://handlebarsjs.com/ - the 'list' helper.23:56
lifelessrick_h: seems like a terrible reason to have that facility.23:56
StevenKwgrant: I have a deployment request prepared for when we're good.23:57

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!