lifelessflacoste: http://blog.mountaingoatsoftware.com/agile-succeeds-three-times-more-often-than-waterfall00:00
StevenKwgrant: I should fix security.cfg in the code branch, or the db branch?00:03
wgrantStevenK: You can do it in the DB branch, so might as well do it there.00:06
wgrantIn fact, you have to do it there.00:06
wgrantBecause of person foreign keys.00:06
StevenKHmm, so I have to drop the column from person too00:07
StevenKThat is going to be fun.00:07
StevenKOh, it's a join, not a column00:07
rick_hStevenK: ping00:08
StevenKrick_h: O hai00:08
rick_hStevenK: hey, I started landing the test fixes00:08
StevenKI saw.00:08
StevenKI think.00:08
rick_hStevenK: in my dev work I noticed that make jsbuild doesn't update the build dir files00:08
rick_hSo I saw combobuild00:09
rick_hbut that didn't update an existing file either00:09
rick_hI kept having to make clean && make to get the build dir updated to work on a js file as I worked00:09
rick_hwhich was a tad slow as you can imagine00:09
rick_hI tried to see if the combobuild didn't work right because it wasn't listed in PHONY, but that wasn't it.00:09
rick_hI didn't get to look at all of it before EOD00:09
StevenKbin/combo-rootdir will exit with a message if build/js/lp exists00:10
rick_hStevenK: right, it basically ran without cp'ing the files00:10
rick_hStevenK: we need some way to work on a .js file and getting it rebuilt like make jsbuild00:10
StevenKrick_h: Right, so now that icing depends on the YUI directory inside build/js, I can't just blow away the entire directory with impunity like I used to.00:11
rick_hStevenK: right, I have on my todo to work on a script you can run using pyinotify to auto rebuild changed files00:12
rick_hbut have things keeping me from getting to it yet00:12
rick_hplus, that's going to be a pain since we've got build exceptions :/00:12
StevenKrick_h: Right, so I've not come up with a good plan yet.00:13
rick_hStevenK: ok, well that's part of it. Wanted to make sure I wasn't expecting something to work that wasn't00:13
rick_hsounds like it's on the todo list then00:13
rick_hwhich is ok, now that I know00:13
StevenKI think I can delete everything but yui and then cope if yui is a symlink00:14
rick_hyea, I was thinking of just making a temp make command that blew away /build/lp and rebuilt it00:14
StevenKMeh, don't do that00:15
StevenKFix combo-rootdir to not be so naive :-)00:15
rick_hit was a sledgehammer solution :)00:15
StevenKrick_h: I'll look at fixing combo-rootdir today00:15
rick_hyea, I'll look at it then. Thanks for filling me in00:15
rick_hok, well no big deal. No one's really using it yet00:16
* StevenK is destroying code and tables00:16
rick_hStevenK: so qas ok? Is it setup I can run combo loader on qas if I set the FF?00:16
StevenKrick_h: You can't set the flag, and no, it's still waiting for that RT.00:16
rick_hok, cool00:17
lifelesswgrant: I have some bad news w/respect heat00:23
lifelesswgrant: we're missing some update cases.00:23
lifelesswgrant: I suspect dupes, subscribers and affects-me-too are not updating the bug record heat00:24
wgrantlifeless: Are too.00:28
wgrantat least affects-me-too is.00:28
wgrantSubscribers work too00:28
wgrantAnd I can't test duping non-destructively.00:28
StevenKwgrant: Easy review https://code.launchpad.net/~stevenk/launchpad/kill-entitlements/+register-merge00:29
StevenKWhen did that stop working?00:29
wgrantOh, you're afflicted too?00:29
wgrantwallyworld_: ^^00:29
wgrantWe have a sane reproducer.00:29
StevenKwgrant: https://code.launchpad.net/~stevenk/launchpad/kill-entitlements/+merge/9288800:29
lifelesswgrant: really? oh good. I was watching heat fail to change on prod when doin stuff the other day00:30
wallyworld_wgrant: what's up?00:30
wgrantwallyworld_: The MP bug00:30
wgrantWe have a reproducer who is not stupid :)00:30
wallyworld_bug nr?00:30
wgrantBug #92942200:31
_mup_Bug #929422: Fails to refresh the URL when making a merge request <Launchpad itself:Triaged> < https://launchpad.net/bugs/929422 >00:31
StevenKThat's it00:31
wgrantApparently it replaces the page content inline?00:31
wgrantThat sounds extremely evil.00:31
StevenKIt should stop doing so.00:32
wallyworld_if you can come up with a solution that can avoid replacing the page content, i'm all ears00:33
wgrantwallyworld_: Redirect!00:33
wgrantwindow.location = whatever00:34
wallyworld_isn't that another server request though00:34
StevenKIt used to redirect and it was fine!00:34
wgrantwallyworld_: Don't you have to make the request to get the content anyway?00:34
wgrantIt will just be a normal navigation request rather than an AJAX one.00:34
wallyworld_hard to explain - an xhr request is made and it may error so we need to stay on the current page00:35
wallyworld_so that if there is an error00:35
wallyworld_if can be displayed00:35
wgrantAnd when it succeeds it will give a 303, right?00:35
wallyworld_atm it gives 20000:35
wgrantOr 201 or something like that.00:35
StevenKwallyworld_: The URL can updated via JS00:35
* wgrant reads the code.00:36
wallyworld_right, so that's all i need to do00:36
wgrantThat's evil.00:36
wallyworld_i just forgot to update the url via js00:36
wgrantWe should avoid it if we can; it's not widely supported.00:36
StevenKwgrant: Thank you for the +100:36
wallyworld_is it part of the standards?00:36
wgrantIt's part of HTML500:36
wallyworld_so only ie < 8 or something will have an issue?00:37
wgrantwallyworld_: No, not even IE9 supports it.00:41
wallyworld_sigh. i hate ie00:41
wgrantIt's a new API and what you're doing is reasonably evil.00:41
wgrantReplacing the page content entirely is unprecedentedly evil.00:42
wgrantThat is not arguable :)00:42
wallyworld_i'll take your word for it i suppose00:42
wallyworld_i mean, it's merely doing what the browser does when it gets stuff to render00:43
wgrantdocument.write is evil :)00:43
wallyworld_why? it's a public api, or?00:43
wgrantThere are lots of evil public APIs.00:44
wallyworld_so the issue is that the lp form infrastructure returns the html resulting from doing a form submit, and the xhr call needs to do something with this data or else just throw it away and waste the rendering effort00:44
wgrantI had assumed that you used an API call.00:44
wgrantI didn't realise until a few minutes ago that you actually POSTed to +register-merge.00:45
lifelessso, FWIW00:45
lifelessI think avoiding roundtrips is good; but is this actually doing that ?00:45
lifelessin which case, the UI impact is a regression, but it is otherwise tolerable, at least in the absence of e.g. doing the form via *stache00:46
wallyworld_i posted to +register-merge to reuse all the existing code and avoid writing a bunch of new stuff and keep easy backward compat with non js browsers00:47
lifelessI heartily approve00:48
wgrantI approve of eliminating roundtrips, but I disapprove of introducing yet another non-standard way of doing forms.00:48
wgrantParticularly one that is fragile and uses widely discouraged APIs like document.write.00:48
wallyworld_except i need to figure out a non evil way to handle the form submission via the xhr call without adding a rt and wasting rendering effort00:49
wallyworld_maybe a server side redirect?00:49
lifelessthats what LP normally does00:49
lifelessAre you sure the xhr call isn't following it itself?00:49
wgrantThere's still likely to be a round-trip, but it's not too terrible for this sort of page.00:49
wallyworld_it would cause the server side rendering to be duplicated00:49
wgrantThe way we normally avoid duplication is by not POSTing to the view.00:49
wgrantBut using an API instead.00:50
lifelessdid you set out to eliminate roundtrips, or do something else?00:50
wgrantThat's how all the other AJAX forms work, TTBOMK.00:50
wallyworld_lifeless: i set out to convert what was a bog standard html form into a page that allowed the server to check for issues and send back an error message for display without leaving the page00:51
wallyworld_and still act like a form submission on success00:51
wallyworld_and live within the lp form infrastructure00:51
lifelessso, neither fish nor fowl :)00:52
wallyworld_yeah, guess so00:52
wallyworld_wanted to avoid too much new code / change etc00:52
lifelessso, deconstructing this situation00:52
wallyworld_perhaps should look at using an api00:52
StevenKThere is one?00:53
lifelessthe new thing here was doin it in the picker00:53
wallyworld_the work was already 5 or 6 branches long00:53
wgranteg. AJAX milestone creation is the normal form mapped to the API00:53
lifelessyou could have, for instance, let the picker choose anyone, and show an error on submission00:53
lifelesswould not have been as nice00:53
wallyworld_lifeless: the picker does a check, but the submission also needs to do a check00:53
lifelesswallyworld_: I know00:53
lifelesswallyworld_: the picker check is redundant00:54
wallyworld_so i did both00:54
wallyworld_lifeless: yes, but it gives a nicer user experience00:54
lifelesswhich I applaud00:54
wallyworld_catch the error early00:54
wallyworld_in the workflow00:54
wallyworld_so, i needed a way to hit submit button but not leave the page and refresh if there is an issue, hence i made the xhr call, but the lp form stuff gives back a 200 with the new html for the resulting page on success00:56
wallyworld_hence my use of document.write to render - it's the same data over the wire after all00:56
lifelesswallyworld_: I don't quite follow that last bit; why couldn't it just do what our other forms do - render with an error?00:57
lifeless(I'm not sayin that is good or optimal; just trying to understand)00:57
wallyworld_lifeless: sure. there is no single field  in error - it's a confirmation popup00:58
wallyworld_not an error00:58
wallyworld_and the form error stuff is a page refresh00:58
wallyworld_which i didn't want00:58
wallyworld_the user sees the confirmation popup and hits yes and a normal submission then occures00:58
wallyworld_or if no, the popup goes away and they keep editing the mp00:59
lifelessI understand, but how is that different to a fresh page with a 'please click here to ok adding the user as a subscriber to  ...' ?00:59
wallyworld_better , more web 2.0 experience00:59
wallyworld_i didn't think doing a separate confirmation html page was a good thing to implement01:00
wallyworld_so if no objections, let me see if a server side redirect works. might double render, not sure, but better than the current bug01:02
lifelessStevenK: can you check with webdev toolkit and see if a 30x is returned from the appserver?01:05
StevenK... better?01:05
StevenKI'll be putting up an MP shortly01:05
lifelesswallyworld_: so, as a datapoint, *all* our forms today double the same form as the confirmation page for all data entry issues01:05
lifelesswallyworld_: e.g. with red boxes around bad data etc01:05
lifelesswallyworld_: fitting in with that may well have been much less effort, and certainly would have been more consistent01:06
wallyworld_lifeless: but here, there is no bad data01:06
wallyworld_there's no error01:06
lifelesswallyworld_: model it as a default-hidden field and there is01:06
lifelessthat said, what we do today isn't all that nice01:07
lifelessit is however js-free01:07
wallyworld_i was trying to avoid propagating the non-nice stuff01:07
wallyworld_and we are allowed to do js only stuff now, right?01:08
lifelessthat is still vague, sadly.01:08
lifelesswe need to pin down the exact non-js use cases.01:08
lifelessthere are, AFAIK, three:01:08
wallyworld_well, too late, horse has well and truly bolted01:08
lifeless - login01:08
lifeless - apport bug filing01:08
lifeless - google indexing (all stuff we want indexed must have a server side render)01:09
lifelessbut, TTBOMK we haven't got a solid ratification around that bein the complete set, yet.01:09
wallyworld_lifeless: yes, it was extra work, but you need to do that to establish a new way forward to get away from the old, "bad", stuff, even if we decide we want to do it another way, we need to blaze a trail01:09
wallyworld_see what works, what doesn't01:10
lifelessI haven't, I hope, been critical so far01:10
wallyworld_oh, no not at all01:10
lifelessjust trying to ensure I have a clear understanding01:10
wallyworld_i appreciate a robust discussion, especially since i broke stuff :-(01:10
wallyworld_i have a thick skin :-)01:11
wallyworld_i do like the new behaviour, so would like to make that work rather than reverting to a confirmation page01:11
StevenKYou have to, to be a Queenslander01:11
wgrantDamn, beat me to it.01:11
lifelessso, here is my take on it01:14
lifelesswe probably need two forms of forms01:14
lifelessthe bug filing / login case01:14
lifelesseverything else01:14
lifelesswe -don't- need server side rendering of forms, google doesn't need to index forms01:15
lifelessthe current approach for populating and validating forms / form widgets / etc is API calls, not view calls.01:15
lifelessI'd rather we push on that unless we think its flawed, than overload the behaviour of server side forms01:16
lifelesswhich this seems to do to me01:16
wallyworld_lifeless: validation is done via form submission01:16
wallyworld_the lp form view converts the post params into objects etc and then calls validate01:17
lifelesswallyworld_: why? I mean, I know that is what you did, but API calls that mutate/create do their own validation01:17
wallyworld_i'm relying on the standard lp form submission code to get the form data and convert to model objects01:18
wallyworld_using the schema01:18
wgrantThe API does that too.01:18
wallyworld_yes. but i also then resuse the exisiting form validation callbacks01:19
wallyworld_but using the standard submit01:19
wgrantAren't they on the API too?01:19
=== nhandler_ is now known as nhandler
wallyworld_they are? how? one defines validate() on LaunchpadFormView01:20
lifelessthe parameters are typed01:20
lifelesswith schema validators attached to that (e.g. Choice())01:20
wallyworld_sure, i'm not talking about param conversion, but validation01:20
wgrantThat should all be in the model.01:20
wgrantThe view is just meant to present it as HTML.01:21
wallyworld_the LaunchpadFormView subclass offers a validate() method01:21
wallyworld_which people use01:21
wallyworld_and then call field.setError01:21
wgrantSure, but that should only be for formatting model errors as HTML.01:21
wgrantSince the model should be doing the validation, not the form.01:21
wallyworld_should be but doesn't always01:22
wallyworld_i'll have to recheck this particular case01:22
wallyworld_to see what it does01:22
lifelesswallyworld_: if the model does not do the validation, that is a bug01:22
wallyworld_then we have lots of bugs i think01:22
wgrantBecause there is an API for this.01:22
lifelesswallyworld_: We have had many OOPSes - criticals - due to exactly that.01:22
lifelessthis discussion has been good, I think I can suggest a path now01:23
lifeless(the one above), will send to list01:23
wallyworld_lifeless: wgrant: the RegisterMergeProposalView does do it's validation in the view - so bug :-(01:23
wallyworld_there is no well defined pattern that is in common use across lp for this01:25
lifelesswell, I beg to differ01:25
lifelesswe have two patterns and a bunch of sometimes poorly migrated forms01:25
lifelessthe migration *should* have been to move all validation to API's01:26
wallyworld_the code i look at often suggests otherwise :-)01:26
lifelessso there is an expected layout01:26
lifelessIf there is an API, it *must* do its own validation.01:26
lifelessIf there is a view and an API, the view should catch appropriate errors from the model, rather than duplicating validation code01:27
wallyworld_yep, but mostly don't01:27
lifelessIf there is a view and no API, its old unmigrated code.01:27
lifelesswallyworld_: -> lots of bugs01:27
wallyworld_lifeless: would you consider it a bug if a factory method makeFooBar() caused a side effect of leaving behind an admin login such that a check_permission in a test erroneously passes?01:35
StevenKI would.01:35
wgrantI ran into a similar thing last week.01:35
wgrantExcept it was logging out.01:35
* wallyworld_ stabs factory 01:35
wgrantI was tempted to wrap the factory in an interaction preserver.01:36
StevenKFix the factory to do with celebrity_logged_in():01:36
wallyworld_in this case, makeRegistryExpert() is evil01:36
wallyworld_yes, will fix, just wanted to check01:36
* wallyworld_ wonders how many tests will fail now01:36
StevenKwallyworld_: You will probably get fallout01:36
StevenKAh, you guessed :-)01:36
* wallyworld_ sighs01:36
StevenKWelcome to Launchpad.01:36
wallyworld_is it too late to run away?01:37
wallyworld_StevenK: what's bad is that the method just above uses the correct pattern, so somehow that was not noticed01:38
lifelesswallyworld_: yes; an undocumented sideeffect is almost certainly a bug01:39
StevenKlifeless: Yes, it returns a 303.01:39
lifelessStevenK: thanks01:40
lifelesswallyworld_: ^ :)01:40
lifelesswallyworld_: server side is doing the right thing01:40
wallyworld_for the register mp form?01:40
StevenKPOST => 303 ; GET <MP ID> => 20001:41
StevenKAnd then Firebug refreshes the Net panel, because it's evil.01:41
wallyworld_ok, so if the xhr call gets back stuff with code=303 and content type = text/html and responseText = the page html, what do i do?01:42
wallyworld_i could have sworn i got code = 20001:42
lifelessyou do01:43
lifelessthe browser follows the 30301:43
wallyworld_ah right, i remember now01:43
lifelessquestion your assumptions, always :)01:43
wallyworld_yes, i did figure that out01:43
StevenKwgrant: https://code.launchpad.net/~stevenk/launchpad/smarter-combo-rootdir/+merge/9289701:43
wallyworld_i had ust forgotten01:43
wallyworld_but i now recall my dilema01:43
wallyworld_the server side 303 is mapped to a 200 by the browser01:44
wallyworld_and the repsonseText has the html to render01:44
StevenKIt looked like two seperate requests to me01:44
wallyworld_not if i recall correctly01:44
wallyworld_but i could be wrong01:45
StevenKIn Firebug there was a POST, which returned a 303 See Other, and then a GET which returned 200 OK01:45
wallyworld_but that's all at the browser level01:45
wallyworld_i don't think the xhr call sees that01:45
wallyworld_the xhr call does a post and the browser does it's stuff and ends up giving a repsonse with code = 20001:46
lifelessStevenK: yes, thats right01:46
wallyworld_so the xhr call is none the wiser that the redirect has occurred01:47
wallyworld_so, even if i use an api, that's still an extra round trip01:48
wallyworld_1. the api call to do the submission, 2. the call to get the correct url on success01:48
wallyworld_whereas now it's just the one form submission01:49
lifelesswallyworld_: actually no, an API can be less roundtrips01:49
wallyworld_it's only one now atm01:49
lifelesswallyworld_: load the form template (moustache) on the initial page load; get the data back from the API (a single POST : reply) and render in client01:49
lifelesswallyworld_: its 2 POST:303 + final url:20001:50
wallyworld_it will be a huge job to convert the mp page(s) to mustache01:50
wallyworld_i was trying to keep it all simple01:51
wallyworld_ah yes, it is 2.01:52
wallyworld_i'll have a look after i get this current branch done01:52
wallyworld_now that i have figured out how factory is screwing me over01:52
lifelesssure, like I said though, we need a consistent story01:53
lifelesswhat you're doing isn't consistent, nor is it any faster than the API can/should be in principle [modulo headdesk bugs, of course]01:54
lifelessit might be more efficient for porting existing forms01:54
wallyworld_but it is a much nicer user experience01:54
wallyworld_if we stay consistent, we keep the crappy ui forever01:54
lifelessno, you misunderstand me01:54
lifelesswe have other ajax forms with lovely UI01:54
lifelessthat don't do what you have done01:55
lifelessof course we have to break consistency to migrate to something else01:55
wallyworld_do they need to be backwards compatible though?01:55
wallyworld_mine does01:55
lifelesswell, its not01:55
lifeless(because it breaks non-js browsers)01:55
wallyworld_no, it works01:55
lifelessso you have all the overhead of server side widget stuff etc01:56
wallyworld_for non js-browsers, it just submits as normal01:56
lifelessyou've got me tied up in knots01:56
wallyworld_but you don't get the popup confirmation but you do get an informational01:56
lifelessto be functionally equivalent you have to be able to ack the confirmation box01:56
lifelessso you must have added a widget, which you said 'yuck' to before01:57
wallyworld_here, it's not quite functionally equivalent01:57
wallyworld_non js just get the informational message01:57
wallyworld_after form submission01:57
wallyworld_js get the popup01:57
wgrantwallyworld_: Doing it the old way (setting document.location) is no more roundtrips.02:04
wgrantwallyworld_: You can tell XHR to not follow the redirect.02:04
wgrantwallyworld_: So you'll get back a 303 with a Location02:04
wgrantThen you set window.location = location02:04
wallyworld_ok, cool. didn't know that02:04
lifelessso thats a decent answer here02:05
wgrantOh, blah, you can't actually block redirects yet.02:05
wgrantSo the cheapest may be to say if is_ajax then return a 201 instead.02:05
wgrantIt's still evil, but a far, far milder variety.02:05
wallyworld_can't block redirects? not part of the standard yet?02:06
wgrantRight, it may be in future.02:06
wallyworld_i did look for how to do that but didn't see anything i could use02:07
wgrantRight. So the view needs to return a 201 like the API, or you need to use the API.02:07
wgrantBugSecrecyEditView already does something similar.02:08
wallyworld_yes, i wrote the ajax bit02:11
wallyworld_but it's used as a portlet inside a page02:12
StevenKwgrant: Can haz review?02:53
wgrantStevenK: Why don't we want to clobber yui(2)?02:57
wgrantlifeless: For optimising counts, have we tried doing a "count = SELECT COUNT(*) FROM (real query LIMIT 10000); if count == 10000 then count = 'shitloads';" sort of thing?03:18
lifelesswgrant: we haven't03:19
lifelesswgrant: I can see a few ways that that won't help much03:19
lifelesswgrant: but yeah, sure, that could be a good approach.03:20
lifelessalso grr @ bad triage from users that that pump and run03:20
wgrantIn some cases it won't help, sure.03:20
wgrantBut a lot of common queries just end up being an index walk.03:20
StevenKwgrant: Because it isn't going to change.03:25
wgrantStevenK: Not even with a YUI upgrade?03:25
StevenKwgrant: YUI2 won't change, and we can't upgrade YUI past 3.3 currently03:26
StevenKWe can once we stop using yui-deps and related garbage that needs deleting.03:26
wgrantSure, we can't do it now, but when we want to...03:26
StevenKwgrant: combo-rootdir will likely change again to no longer create a symlink for yui once the icing dependancy dies03:27
StevenKSo this isn't the final solution03:27
StevenKAnd then it can stop being buildout generated.03:28
StevenKwgrant: Ah, thanks. I wasn't sure that I'd convinced you.03:40
wgrantwallyworld_: Sorry, missed the review email before. I've just replied.03:42
wgrantStevenK: Indeed.03:42
wallyworld_wgrant: np. i'm just about to put up the fix for the mp registration03:43
wgrantExcellent, thanks.03:43
StevenKhttp://www.smh.com.au/environment/weather/weather-settles-after-a-day-of-storms-hail---and-a-tornado-20120214-1t2su.html << Sydney weather is whacked03:43
wallyworld_it's quite simple in the end03:43
wgrantwallyworld_: The cheap fix I suggested should be about a 10 line diff + tests.03:44
wallyworld_wgrant: sadly, other model code calls check_permission so i figured it was 'tolerated'03:44
wgrantwallyworld_: There's only a couple of other places.03:44
wgrantPersonRoles makes it pretty cheap to avoid now.03:45
wallyworld_well, only for those simple checks03:45
wallyworld_the logic needs to be broken out and the security adaptor call that03:45
wgrantIn the complex cases (mostly bugs, branches) it's already delegated to the model.03:45
StevenKAnd that is going to change anyway?03:46
StevenKWe're going to change the complex case of bugs and branches for disclosure?03:47
wgrantYes, but the precise rules are irrelevant here.03:47
wgrantIt's the code structure around them that is under scrutiny.03:48
wgrantTomboy fails at crash survival :/03:56
wgrantIt mustn't have flushed my notes for about 6 hours yesterday :/03:57
lifelessless crashing thanks03:57
StevenKwgrant: I have a test here that is doing setupBrowser(auth='Basic mark@ex...:password')  I thought you removed that terribleness?04:09
wgrantStevenK: Hm, I suspect that's no longer doing what it thought.04:11
wgrantStevenK: Especially since the domain seems to be ellipsised.04:11
wgrantAll the passwords are now 'test'04:11
wgrantTests still use basic auth.04:11
StevenKwgrant: I'm happy to fix it with a hammer (in a Jeremy Clarkson style) while I'm here.04:12
=== nhandler_ is now known as nhandler
StevenKwgrant: What should it be doing instead?04:12
wgrantStevenK: Which test?04:13
StevenKtest_adddownloadfile_nonascii_filename in TestProductFiles04:13
wgrantSo, you can usually just say self.getUserBrowser(user=someone)04:14
wgrantIt also takes a url arg.04:14
StevenKDoes that work under TestCase, or does it have to be BrowserTestCase?04:14
wgrantTestCaseWithFactory is sufficient.04:15
wgrantBrowserTestCase should probably be eliminated.04:15
StevenKwgrant: user= is an IPerson, so I have to reach into IPersonSet to grab mark out?04:16
wgrantOr you can continue using setupBrowser04:17
wgrantBut I suspect that test is bad04:17
wgrantBecause 'password' is no longer the right password.04:17
StevenKIt isn't password, it is test04:17
wgrantI thought I sedded all of them out :)04:18
StevenKI just thought auth='Basic ...' was sort of evil and needed to be killed04:18
wgrantIt is vaguely.04:18
wgrantBut it's still used widely.04:18
wgrantI didn't want another 20000 line diff to my name.04:19
StevenKException-Type: Unauthorized04:23
StevenKException-Value: (<zope.browserpage.metaconfigure.SimpleViewClass from /home/steven/launchpad/lp-branches/moar-factory-cs/lib/lp/registry/browser/../templates/productrelease-file-add.pt object at 0x11e60950>, 'browserDefault', 'launchpad.Edit')04:23
* StevenK blinks04:23
* StevenK stabs wallyworld_.04:55
wallyworld_ok, i give in04:56
StevenKwallyworld_: Where is the fix for +register-merge? It's getting quite annoying.04:56
wallyworld_it's up from review04:56
StevenKI have created a large number of MPs today, though.04:56
StevenKwallyworld_: Do you mind reviewing https://code.launchpad.net/~stevenk/launchpad/moar-factory-cs/+merge/92910 , so I can give wgrant a break? It's nice and small.04:57
wallyworld_StevenK: i think you need to pass in some attributes to the favtory method05:00
StevenKwallyworld_: Are you sure, the test passes? :-)05:00
wallyworld_eg there's one test where sales_system_id is 'foo' and then later 'new' and we need to be sure it's being reset05:00
wallyworld_an empty test also passes but doesn't test what's required :-)05:01
StevenKwallyworld_: Ah ha.05:03
StevenKwallyworld_: The voucher is redemmed, which ignores whatever is on the CS. Then we remove it, and recreate it. No voucher, so it's new.05:03
StevenKThe factory method uses sales_system_id='new' all the time05:04
wallyworld_and the test as originally written first used an id of 'foo'05:04
StevenKYes, and it tested for 'hello'05:04
StevenKSo the 'foo' was clearly ignored05:05
wallyworld_yes, seems so in this case05:05
StevenKwallyworld_: So, fix, or "meh" ?05:06
wallyworld_perhaps it's ok then05:06
wallyworld_let me double check05:06
StevenKwallyworld_: Pushing a change, reformating the asserts05:07
wallyworld_StevenK: i think we can ignore it05:07
wallyworld_yeah, there's a lot of asserts that are the wrong way around05:07
StevenKYou can see one of them in the current diff, line 6305:07
wallyworld_and 45 and ....05:08
StevenKYeah, fixed that too05:08
StevenKassertIs is better05:08
* wallyworld_ taps fingers waiting for diff05:10
* StevenK too05:10
wgrantDynamic bug listings just magically work with a StormRangeFactory.05:11
wgrantI'm shocked.05:11
wallyworld_StevenK: r=me05:11
StevenKwallyworld_: Thanks!05:12
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
wgrantAnyone feel like reviewing https://code.launchpad.net/~wgrant/launchpad/fix-debug-timeline-js/+merge/92916?06:22
StevenKwgrant: Oooh?06:44
wgrantFlattening access information into bugtaskflat brings COUNT(*) on the 92000 open Ubuntu bugs down from 1100ms to 250ms when hot, because it doesn't have to join against BugSubscription.06:45
StevenKwgrant: I find myself curious why the death of Mochi killed debug_timeline, and why you have href="javascript:void(0)"06:46
wgrantStevenK: removeElementClass doesn't exist, so I assume it was mochi.06:46
wgrantI do that so the link does nothing, because I am too lazy to change it to only create the link when it adds the handler.06:47
wgrantOtherwise it will try to take you to the href :)06:47
StevenKwgrant: r=me, land it06:48
wgrantI've been using it on DF for a while, so I know it works :)06:48
=== almaisan-away is now known as al-maisan
=== wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
wgrantlifeless: How significant is the overhead of rabbiting an oops?07:02
wgrantlifeless: I'd basically like to set the soft timeout of some bug searches to 0 when making large changes.07:02
StevenKstub: Thank you for the +1.07:06
StevenKstub: I'll set it to approved, but I need to wait for the previous branch to hit db-devel before landing this.07:06
stubyup. np.07:07
=== danilo is now known as Guest53490
stubStevenK: mrevell might cry to see entitlements go, but nothing on that is scheduled as far as I know.07:24
StevenKstub: They've never been used, ever. CommercialSubscription handles everything07:26
=== al-maisan is now known as almaisan-away
stubYer, I was wondering if it was one of those eternal 'we want to use it soon' bits. I'm all for it going.07:27
StevenKIf wgrant and I had figured it was this easy to rip out, we may have done it as part of the Doomed 3707:28
wgrantCurtis confirmed that it can die.07:29
StevenKstub: Working from https://pastebin.canonical.com/60006/ , what do you think of https://pastebin.canonical.com/60117/ as a start?07:38
stubbugnotificationattachment__message__idx is a worry as we should be using that when we delete a message (I suspect we don't delete, just hide at the moment, but that should probably change)07:40
stubbugsummaryjournal.viewed_by - looks like it is missing its foreign key definition?07:42
wgrantStevenK: That may be deliberate.07:42
wgrantEr, stub.07:43
StevenKstub: I'm happy to drop anything we're not 100% sure about and play it safe.07:43
wgrantIt may rely on the triggers on BugSubscription to update it.07:43
stubwgrant: yer, just refreshing my memory07:44
StevenKI did ponder adding the unused accesspolicygrant index to that list just to see wgrant be very sad.07:45
wgrantI'll be modifying that schema soonish anyway :)07:45
stubSo yes, it shouldn't have a FK constraint. And it can go - looks like we never select on it directly, but only when we already have a subset of rows using product/distro etc.07:46
StevenKstub: Shall I drop bugnotificationattachment__message__idx from the patch?07:47
* stub ponders07:49
stubThe table has 0 rows...07:50
StevenKbugnotificationattachment is empty?07:50
* StevenK greps07:50
StevenKCOMMENT ON TABLE BugNotificationAttachment IS 'Attachments to be attached to a bug notification.';07:51
StevenK... That sounds like bong07:51
stubMaybe some odd 'send patches' use case that will never land.07:52
* StevenK blames07:52
stubSo remove the 'DROP INDEX' from your branch. If it goes, it goes because we drop the whole table.07:53
stubI hate the hwdevicedb schema07:54
StevenKstub: Yes, I'm not amused that 13 are HW indices07:55
stubSame for hwdevicenamevariant - 0 rows. Leave the indexes. If they go, go because we drop the whole table.07:56
stubAnd hwdmihandle07:56
stubAnd hwtestanswer07:57
stuball of those actually....07:57
StevenKhwtest* ?07:57
stubNo point removing an index on a table containing 0 rows. No gain, and as we can see helps point to potentially bogus tables07:58
StevenKLeaving bugjob, bugsummaryjournal, hwdmivalue and packagesetinclusion07:58
stubYes, hwtest*07:58
wgrantI excluded hw* from my purge because I hope to delete them all in one hit within a year :)07:58
stubpackagesetinclusion is 0 ows too07:58
StevenKWhy is that empty?07:59
StevenKpackagesets are widely used07:59
stubMaybe nobody is using nested yet?07:59
stubMaybe we haven't implemented the UI for that?07:59
StevenKstub: Well. There is packagesetinclusion and flatpackagesetinclusion08:00
StevenKCompare and constrant the schema :-)08:00
stubLooks like we neglected to populate packagesetinclusion08:01
StevenKI know flatpackagesetinclusion is used, since I hit it on the derivation work08:01
stubRight. So that will be the exploded version, or should be.08:02
stubSo flatpackagesetinclusion contains 118 rows, each of which parent==child.08:03
StevenKHeh, handy08:03
stubSo back to my earlier theory that nobody is using nested packagesets08:04
StevenKI'll dump it then08:04
StevenKSo my 17 is down to 3.08:04
stubThe revision ones look juicy if that makes you feel bad.08:04
StevenKOh, which?08:04
StevenKI was ignoring any index that had n_live_tup > 008:05
stubrevision, revisionauthor, revisioncache08:05
stubIgnoring? The higher that number, the bigger the win08:05
StevenKOh, crap08:06
wgrantStevenK: Don't dump PSI08:06
wgrantIt's just a feature that Ubuntu isn't using yet.08:06
StevenKstub: What does the number mean, then?08:06
stubThe number of rows in the table08:06
stubnumber of live tuples08:06
StevenKAhh, and the indices listed are utterly unused.08:07
stubThese are all indexes that are unused, yes.08:07
stubSo unused indexes on hwsubmissiondevice are a win as it has 100 million rows08:07
StevenKstub: I'm concerned sl_log_[12] is in your list. :-)08:08
stubIt is?08:08
lifelesswgrant: its proportional to the # of queries08:09
StevenKLine 145 and 14608:09
stuboh, thought I'd filtered those08:09
stubForgot 'public schema only'08:09
lifelesswgrant: you can test locally to get a handle on it08:09
StevenKstub: Right, let ignore those 3. Going by largest number first, do you want to get the DB to do the heavy lifting, and ORDER BY n_live_tup LIMIT 15 and I'll hack my patch to drop those?08:10
StevenKstub: If you think we can do more than 15 in a FDT, feel free to bump up the LIMIT08:10
stubStevenK: Why the cleanup push? You trying to get loc credit before the market boom? Have you decided on a beer:loc exchange rate you are going to start selling at?08:10
StevenKstub: No particular reason08:11
StevenKOur DB is large, unwieldy and can be used to scare small children.08:11
stubDropping indexes should be fast. Should be able to cram a lot of them in.08:12
stubI could time it, but...08:12
StevenKI could time it on DF, but that would be pointless08:12
StevenKEspecially since wgrant keeps being evil to DF.08:13
StevenKstub: I thought index deletion scaled with the size of the index?08:14
stubWe still need to investigate the targetted ones. eg. LibraryFileContent.sha256. The column should probably be dropped instead as it is WHUI (not sure why it was urgent before)08:14
stubStevenK: rm file_on_disk;sync;08:14
StevenKBecause there is a little panic surrounding SHA1 and the librarian uses it quite heavily08:15
stubSo we shouldn't drop either the column nor the index, and add the code to the Librarian to populate it on upload.08:15
StevenKI'll let lifeless rule on that.08:16
stubWe don't want to drop indexes we will need because that will cause timeouts in the future (because we are unlikely to notice the indexes we need are not there until production).08:17
stubmessage__parent__idx unused. The DB model supports threading, but we never display it. UI decision08:20
StevenKstub: I'm paring down your list. Ignoring all indices whose n_live_tup is 3 digits or less and splitting out sl_log*, bug*, lp_*, hw* and specification__date_last_changed__idx, I'm down to 76.08:20
* StevenK grep's out the sha256 index, based on previous discussion08:21
StevenKstub: I'm concerned there are 6 FTI indices implicated08:22
* StevenK ignores branch__access_policy__idx too, glancing at wgrant.08:22
stubStevenK: Right. Probably shouldn't drop the fti indexes, and consider dropping the column (and fti trigger) instead08:23
StevenKstub: Right.08:23
StevenKHah. sourcepackagepublishinghistory__ancestor__idx08:24
StevenKThat one is my fault.08:24
stubwgrant: Have all the no-more-bug-heat-rewriting changes been rolled out?08:25
StevenKstub: I'm down to 68 -- nothing obvious is jumping out, but I can pastebin what I'm down to.08:25
wgrantstub: No, we have to wait 6 days for all the bugs to be updated.08:25
wgrantstub: The new function was deployed last night, and the garbo job updates everything weekly.08:25
stubok. I want to rebuild the fti indexes, hopefully for the last time.08:26
wgrantbug.fti will hopefully die soon :)08:26
wgrantAlthough just to be replaced with an equivalent.08:26
wgrantEr, not bug.fti, but its index.08:26
stubHave you been investigating tsearch2, or are we finally looking at an external text search engine?08:27
wgrantNah, still plain old tsearch.08:28
wgrantJust on a flattened table.08:28
wgrantPerforms pretty well with GIN and bitmap index combination.08:28
* StevenK ignores DSD too, since he isn't sure about its indices at all.08:28
StevenKstub: https://pastebin.canonical.com/60118/08:31
wgrantI'm surprised that branch__date_created__idx isn't used08:32
wgrantLikewise the codeimport* ones.08:33
StevenKI'm concerned about karmacache and oauth*08:33
wgrantstub: What about master vs slave?08:33
stubwgrant: We never list 'all branches in order of date created'. And I don't think it will ever be used for ordering when tuples have been selected using some other selection criteria.08:34
wgrantstub: Except that we list the first 5 branches in order of date created.08:35
wgranthttps://code.launchpad.net/ bottom middle08:35
stubwgrant: Good call. Should pull the index usage details on the slaves too.08:35
stubwgrant: probably using 'id' instead, since it guarantees order in tests.08:35
stubHmm... that is interesting. A lot more unused indexes on _prod_1 than _prod_2. We point all our scripts to _prod_2 IIRC, and prod_1 actually benefits from this lack of load balancing because it will have fewer hot indexes.08:45
stubStevenK: Ignore indexes that are not also listed in https://pastebin.canonical.com/60120/ https://pastebin.canonical.com/60121/08:45
stubStevenK: Or I can check it during review08:45
StevenKstub: My current list is https://pastebin.canonical.com/60118/08:47
stubhttps://pastebin.canonical.com/60119/ is the reordered master list08:48
stubWill we ever delete a GPG key? I guess not, as even if a user removes it we need to keep it around since there are old published archives signed with it.08:50
wgrantstub: We've done it once AFAIK.08:51
wgrantThat may be why those indices exist.08:51
stubStevenK: branch__merge_queue__idx will be needed if merge_queue is ever used. Is that exposed by the API, or is automatic merging a proposed feature in lp?08:54
wgrantMorning czajkowski.08:54
czajkowskiwgrant: morning :)08:54
adeuringgood morning09:00
czajkowskiadeuring: hiya09:00
adeuringhi czajkowski!09:00
czajkowskimrevell: ello09:01
=== almaisan-away is now known as al-maisan
StevenKstub: merge queues are a half-done feature09:20
StevenKstub: I'd like to see them finished, since PQM needs to die a horrible, painful and above all drawn-out death09:20
stubWe still can't disable or remove code imports, huh?09:28
wgrantstub: Suspend09:29
wgrantAnd you delete them by deleting the branch.09:29
stuboauthaccesstoken allows you to specify limitations on an access token, but it isn't used and thus the __product, __project etc. indexes show up09:32
stubI guess that isn't on the radar at the moment09:32
stubShould be be dropping those columns instead?09:34
StevenKstub: Are you happy enough with that patch that I should claim -10 as my very own, put an MP up for the branch and we can nut out the rest in review?09:34
stuboauthaccesstoken + oauthrequesttoken09:34
stubStevenK: Sure09:34
wgrantstub: I don't think the approach they were designed for is adequate.09:34
StevenKstub: I wonder if any of those columns even contain data09:34
wgrantSo they should probably go.09:34
stubStevenK: They don't.09:34
wgrantthey've certainly never been used.09:35
adeuringwgrant: thanks for the notice on bug 72632009:35
_mup_Bug #726320: <type 'exceptions.TypeError'>: add_repository() got multiple values for keyword argument 'src_type' <apport-crash> <i386> <natty> <aptdaemon (Ubuntu):New> < https://launchpad.net/bugs/726320 >09:35
StevenKSo perhaps we should not kill the indicies and just drop the columns09:35
adeuringwgrant: do you have already an idea how to dealwith sorting by rank(Bug.fti, ftq('whatever'))09:36
wgrantadeuring: StormRangeFactory is a bit of a challenge due to all the new sort orders.09:36
wgrantadeuring: And that, yeah, I don't know :/09:36
stubStevenK: Do you want to open a bug or me?09:36
StevenKstub: I'm dealing with allocated.txt and pushing up a branch, so can you?09:36
stubWe can drop the indexes now, but we need a bug open for the column removal so that doesn't get lost (well... )09:36
wgrantadeuring: Sorting by milestone, person, reporter names has the same sort of issue.09:37
wgrantadeuring: We need to return more data than usual.09:37
stubFodder for salgado to reduce his loc impact :-)09:37
stubWe should have a tag for that actually...09:37
adeuringwgrant: ah, right, each sort vaule must appear in the result set09:37
StevenKstub: 'here-salgado-salgado' ?09:37
adeuringwgrant: but adding that shoulkd not be difficult09:37
wgrantadeuring: Yeah, it's just a bit messy due to three types of queries that are produced.09:38
stubStevenK: Anyone working on features would find it useful ;)09:38
wgrantadeuring: some of them UNIONs, others wrapped in extra layers to eliminate dupes.09:38
wgrantI'm reworking that stuff atm, which is why I advise not to touch it.09:38
wgrantHopefully get down to just one or two types of queries.09:38
wgrantAnd then work out how to get SRF working on top of them.09:38
adeuringwgrant: right -- my basic idea was anyway to use stormrangefactory ;)09:39
wgrantadeuring: My changes make this a bit less important, because scans are about 5x faster in the new schema.09:39
wgrantBut I expect to do it afterwards anyway.09:39
wgrantstub: We have a concrete and immediate plan for divorcing the SSO DB, FYI.09:41
stubwgrant: Cool. What is the plan?09:41
wgrantI expect to have it testable by next week.09:41
wgrantstub: SSO becomes an xmlrpc-private client.09:41
wgrantNew method on LP takes an OpenID identifier and returns the data SSO needs.09:42
stubwgrant: And what happens when Launchpad is unavailable. Nobody cares?09:42
wgrantstub: SSO just won't return the extra data (Launchpad ID, timezone and teams).09:42
wgrantThat appears to be acceptable to ISD.09:43
stubCool. The HA aspects where what made it a pain and the current system.09:43
stubTypical for HA, when you meet all the goals you end up with a system you don't want to maintain :-)09:43
wgrantstub: Heh.09:44
wgrantWell, LP is far HAer than it used to be :)09:45
wgrantWe no longer have ludicrous periods of downtime.09:45
stubIf you twist your definition of HA enough, yeah.09:45
wgrantHAer, not HA :)09:45
stubFrom a traditional POV, we gave the finger to HA and instead opted for controlled scheduled downtime.09:46
wgrantOf reasonable duration, importantly.09:46
wgrant90 minutes of auth unavailability is clearly unacceptable.09:46
wgrantA minute, which is what we'll probably be at once this is done, seems OK.09:46
stubWe currently offer 99% uptime I think (99.65% with fastdowntime, plus unscheduled failures). That figure is passed on to any other systems relying on LP in addition to their own downtime.09:48
wgrantstub: How do you figure 99.65%?09:50
wgrantOutages are currently either 90 or 150 seconds.09:50
stub5 minutes per day - what we promise rather than what we actually use.09:51
wgrantAt 90s that's 99.90% for the day09:51
stubNeed some real metrics for the real answer, including the unscheduled outages :)09:52
wgrantMost unscheduled outages hit SSO too, fortunately :)09:52
stubwgrant: How do we know when the bug heat update has completed? Some garbo output to watch for?10:36
wgrantstub: SELECT COUNT(*) FROM bug WHERE heat_last_updated < '2011-02-13';10:36
stubCool. I guess we can drop the heat_last_updated soon. That covered by a bug or kanban task or something?10:37
wgrantMight wait for things to settle before doing that.10:37
stubI'll file another tech-debt bug10:38
stubBug #931987   if you want to claim it10:42
czajkowskiello are uploads not being picked up ?11:10
czajkowskiwgrant:  sorry not sure where to ask that question in here or -ops11:11
wgrantczajkowski: ops, probably.11:11
wgrantIt's possibly related the pgbouncer issue.11:11
czajkowskik thank11:11
tumbleweedwgrant: ta11:13
czajkowskitumbleweed: yo're here11:13
tumbleweedI haven't gc-ed channels in a while...11:14
=== Guest53490 is now known as danilos
=== matsubara-afk is now known as matsubara
czajkowskirick_h: aloha12:18
StevenKrick_h: I note your lack of reply. However, I fixed your combo-rootdir issue in devel.12:27
rick_hStevenK: lack of reply?12:28
rick_hStevenK: I saw that MP, will pull that and try it out thanks!12:28
StevenKrick_h: You said you'd reply to my combo-url mail on the list.12:30
StevenKrick_h: Pull it out? Just merge devel? :-)12:30
rick_hStevenK: ah, yea well my tests came back failing last night12:30
rick_hpart of my reply was going to be an update on the new test templates and such12:30
StevenKAh, kay12:31
rick_hI got a bunch of js timeouts so both failed and I'm working on figuring out wtf this morning12:31
rick_hno landing from me :/12:31
StevenKrick_h: The AJAX log branch is seperate, I'm guessing?12:31
rick_hyea, but that new test came back with js timeout12:31
rick_hso something bigger perhaps? or maybe I just caught a bad ec2 day12:32
StevenKI threw two branches at ec2 today, both passed nicely.12:32
rick_h*sigh* well, I'm peeking and will try to relaunch in a bit12:32
rick_hbut yea, so I didn't reply to the email yet12:33
rick_hI hate it, tests all pass locally peachy, only way to "check" is to toss things at hours of ec2 and cross your fingers12:34
StevenKWelcome to Launchpad. :-/12:34
deryckadeuring, rick_h -- https://plus.google.com/hangouts/extras/talk.google.com/orange-standup14:32
=== Ursinha_ is now known as Guest56696
sinzuijcsackett, Do you have time to review https://code.launchpad.net/~sinzui/launchpad/listify-cached-licences/+merge/9289315:00
jcsackettsinzui: good timing, i have another branch i need to review this morning anyway, so certainly. :-)15:01
jcsackettlooking at yours now.15:01
sinzuijcsackett, fab, I am looking at your first branch now15:01
sinzuijcsackett, replied with a patch15:19
jcsackettsinzui: i have replied to yours as well, with some notes/nitpicks. r=me.15:23
jcsackettsinzui: i will patch mine, thanks for the code. wasn't sure how to test just registering, nor did i think it was strictly necessary. happy to add it though. :-)15:23
rick_hhmm, so if I fire up an ec2 test run and ssh to the box I should be able to find the files and manually fire off test commands while it runs right?15:27
=== al-maisan is now known as almaisan-away
=== salgado is now known as salgado-lunch
rick_hlooks like no, I can't boooo15:54
rick_hderyck: ok, big duh...the 8 tests that fail are the 8 updated ones to look in the build dir16:10
deryckrick_h, aren't they all being updated to look in the build dir?  or just in this branch it's only these 8 that are changed?16:11
rick_hderyck: only those 8 are changed. I'm trying ot see if hte build dir is done via the test runner16:12
rick_hmake is run16:12
rick_hwhich should make the build dir16:12
deryckrick_h, yeah, that's what I was saying this morning; it smells like the files aren't there.16:13
rick_hderyck: yea, the make command is in run_demo_server so wonder if the YUI js tests are run in a different place and the build dir is gone/not there yet16:14
rick_hderyck: so yea, guessing the issue is how the ec2 test script is run. Working on getting my head around it16:14
rick_hbut that makes sense now16:14
=== matsubara is now known as matsubara-lunch
=== salgado-lunch is now known as salgado
=== matsubara-lunch is now known as matsubara
lifelessinline adverts in the twitter stream. Disappointing twitter, disappointing.17:37
=== _mup__ is now known as _mup_
=== maxb is now known as Guest79372
czajkowskiabentley: are you about for a quick question, not urgent , can ping tomorrow it's re https://answers.launchpad.net/launchpad/+question/18492518:01
=== nhandler_ is now known as nhandler
benjiwhen doing a make in a fresh devel checkout I get "ImportError: No module named convoy.meta"; I followed the instructions in Steve's "Convoy is ready for others18:32
benji" email, so I'm wondering what step I left out18:32
rick_hbenji: convoy should be installed via an apt-get upgrade18:33
rick_hit's part of the ppa18:33
benjirick_h: I would have thought the same thing18:34
lifelessrick_h: what do we import convoy for ?18:34
lifelessbenji: what flaour of Ubuntu are you running?18:34
benjioh; I wonder if it's not packaged me18:34
rick_hlifeless: convoy.meta helps walk the list of YUI modules we defined18:34
rick_hso thatit knows how to import it18:34
benjilifeless: an older one ;)18:34
benjilet me look exactly18:34
lifelessbenji: tsk!18:34
benjilifeless: I have a newer VM, let me fire it up and see if it works there18:35
rick_hbenji: ah, ok that makes sense then18:35
lifelessbenji: you know we're meant to upgrade @ beta release each ubuntu release ? :)18:35
benjilifeless: I mostly do, I forgot which VM I was running18:36
rick_hlifeless: do things run well on precise now?18:36
rick_hI've been waiting for all that pgsql stuff to settle before upgrading the laptop, but looking forward to18:36
=== deryck is now known as deryck[lunch]
lifelessrick_h: so what I encourage folk to do, and I do myself, is run my host in $ubuntu-current (usually upgrading around alpha 1), and do dev in an LXC lucid container18:37
rick_hI need to play with lxc I suppose. I've not messed with it at all yet18:37
benjirick_h: it's pretty cool, if a bit young still18:39
benjirick_h: yay, my newer VM works fine18:55
abentleyczajkowski: I wish that were a quick question, but I've looked at it several times and not come up with an answer.18:55
abentleyczajkowski: It looks like it's failing to gpg-sign a revision.18:55
abentleyczajkowski: I don't know why it's failing, but I don't think it should be trying to sign in the first place.18:56
abentleylifeless: this is a CVS import.  We don't gpg-sign those, do we? https://code.launchpad.net/~rpm5/rpm/trunk18:57
lifelessI don't think we sign any imports18:57
lifelessbut if we sign any, I expect we sign all18:57
rick_hbenji: awesome18:58
czajkowskiabentley: ok just confused as I've never seen that kind of thing before again apologise for the question for the coming weeks19:00
lifelessabentley: actually IIRC we did setup signing of imports way back with baz, it was a religion then19:01
lifelessabentley: so I expect our cscvs infrastructure does indeed sign things19:01
abentleylifeless: Maybe I'll pull the successful imports and see if they're signed.19:01
abentleylifeless: 10254 commits not signed19:06
lifelessabentley: from a cvs import ?19:07
lifelesssvn ones are all bzr-svn now, which won't sign as we're not invoking the commit codepath; ditto hg and git.19:07
abentleylifeless: yes, https://code.launchpad.net/~rpm5/rpm/trunk.  62 commits were signed (but with keys my box doesn't know)19:07
lifelessabentley: verra interesting19:08
lifelessabentley: I wonder if the different import slaves have different configs for bzr or something19:08
abentleylifeless: Yes, I wonder too.19:08
abentleylifeless: Also wonder how many of our CVS imports are failing.19:09
=== deryck[lunch] is now known as deryck
=== matsubara_ is now known as matsubara-afk
=== mbarnett` is now known as mbarnett
lifelesswhats the easiest way to get a js repl ?21:33
benjilifeless: FF Ctrl-shift-K21:37
lifelessbenji: ok, and thats bare in a new tab....21:38
lifelessbenji: so I should be less vague; I want to play with handlebars.js21:38
lifelessah, http://tryhandlebarsjs.com/ looks the ticket21:40
benjilifeless: hmm, maybe find a page that already loads it?  Or do some document.createElement("script");21:41
benjior that :)21:41
lifelessabentley: had you seen http://engineering.linkedin.com/frontend/client-side-templating-throwdown-mustache-handlebars-dustjs-and-more in evaluating moustache ? [just curious]21:41
abentleylifeless: no.  Thanks for pointing it out.21:42
=== salgado is now known as salgado-afk
abentleylifeless: "...none of the templating options worked well across client and server unless the server could also execute JavaScript..."21:47
lifelessits an interesting statement, but missing details needed to evaluate it21:49
lifeless... which is a bit disappointing21:51
abentleylifeless: Agreed.21:51
abentleylifeless: I don't agree that Mustache has "clean" syntax.  Am I silly for wanting loops to be distinct from conditionals?21:52
lifelessmmm, I could go either way21:53
lifelessits very pithy as it stands21:53
rick_hlifeless: just create an index.html with handlebars in it. Then just use things like the normal dev tools if you want, but I find just using livereload and that test.html file enough to tinker away21:57
rick_hlifeless:  https://github.com/mockko/livereload/blob/master/README-old.md21:58
rick_hlifeless: another cool js toy area http://jsfiddle.net/22:00
rick_hlifeless: that's cool because you can share a fiddle via link to others, interactive pastebin22:00
wgrantjelmer: Hi, you have a couple of bits of QA.22:12
rick_hStevenK: ping, you see my email to the list today?22:36
StevenKWha? E-mail?22:41
StevenKrick_h: I've been on mumle since I started, e-mail is a secondary concern22:42
StevenKBah. mumble22:42
StevenKrick_h: make build can't run combobuild22:46
StevenKIf you just run make (Which is the 'inplace' target), that will run combobuild22:47
rick_hStevenK: ok, so what's the "right" way for me to get combobuild run in tests?22:54
rick_hStevenK: my issue is I can't ec2 test/land anything because make check only runs make build and combobuild never gets run22:55
rick_hStevenK: if I break out combobuild to be two parts, one that just copies files into build/js/lp and one that uses convoy to generate meta.js can build run combobuild? (the first part)22:55
StevenKrick_h: Yes. I can do that for you today.22:56
rick_hStevenK: ok, if you've got time. I thought I couldn't just run combobuild form make build, but couldn't recall why22:57
rick_hbut yea, all the JS tests now point to the build/js/... for their test files so I need that to be populated for make check22:57
StevenKrick_h: Because convoy won't be installed everywhere, and machines like banana and nutmeg use make build22:58
StevenKAnd gandwana for that matter. I should fix that.22:58
rick_hStevenK: ok, I wasn't sure 100%22:58
rick_hthe other thing would be to break out the meta generation stuff of convoy out but think that'd be making for more problems22:58
StevenKI don't think there is any gain there22:59
rick_hmy only concern with the splitting of meta.js generation is that when we run that, we only want to generate the meta.js for the non-minified files23:00
rick_hso I think what we do now is, copy files, generate meta.js, minify files23:01
rick_hso it may be a pita to have to break that into 3 steps ugh23:01
StevenKYou think? :-)23:02
StevenKI'm happy with 2 steps23:02
rick_hwell I know convoy only looks at the root directory and generates metadata for all files in there23:02
rick_hif we say copy/min files in one step, and generate meta.js with convoy in another23:03
rick_hit'll have dupe metadata info23:03
StevenKPerhaps convoy needs a patch to ignore minified files23:04
rick_hyea, maybe an --exclude regex or something23:04
rick_hnvm, already has an exclude regex for the cmd line version23:05
StevenKSo we can --exclude '-min.js' ?23:05
rick_hwell except I don't think we call it from the cmd line do we? We import extra_metadata?23:06
StevenKutilities/js-deps -n LP_MODULES -s $BUILD_DIR/lp -o $BUILD_DIR/lp/meta.js >/dev/null23:06
StevenKThat is called from combo-rootdir23:07
rick_hright, js-deps so that's got to be updated to pass the args to main() or whatever23:07
StevenKIt already handles this23:07
StevenKRun utilities/js-deps --help23:07
rick_hyea, sorry, main() catches it23:08
StevenKYou're trying to solve problems that don't need solving :-)23:08
rick_hsorry, thoguht it was handled outside of the main we imported23:08
* rick_h is taking a second to trace code :)23:08
rick_hok, well anyway. Sounds like a plan to me. We can keep it two steps if we can make sure it skips the -min.js files23:08
wgrantOnly got Product:+index 99% down by 25%.23:21
wgrantAlthough mean down by 45%, which I guess is better.23:23
lifelessso, mean means all users notice it23:29
lifeless99% means outliers notice it23:29
lifelessif both shift by the same amount, and 99% isn't capped by timeouts, then you removed a constant overhead23:29
wgrantI know.23:29
wgrantAh, there we go, the remaining major win is trivial.23:32
* wgrant fixes.23:32
wgrantlifeless: Do you know what the current stats targets are on prod?23:41
wgrantThat query with massive planning overhead that I found last week is still only 50ms on DF cold, and 10ms hot.23:41
wgrantIncluding planner.23:41
lifelessI can find out if you need23:42
jelmerwgrant: yes, sorry - it's first on my list of things to do tomorrow23:42
lifelessshould we roll it back in the interim ?23:43
jelmerlifeless: roll back my change you mean? it's only just landed23:44
wgrantIt landed a couple of days ago23:44
wgrantThat's two or three LP deployments :)23:44
lifelessjelmer: it's 10 commits back, landed on the 13, its now the 15th, adjusting for tz's its been in trunk for 36 hours23:47
lifelessjelmer: anything in trunk is a pipeline stall for the whole team, QA on it *has* to happen promptly.23:47
lifelessideally the person landing, but failing that anyone with knowledge - and yes, that makes things with specialist knowledge benefit from explicit handoffs for QA.23:48
lifelessjelmer: if you can tell us what you were going to do, we may be able to do it23:48
jelmerlifeless: it only landed on qastaging 21 hours ago according the qa tagger23:48
lifelesseven so, thats plenty of time to have either qa'd or handed off the qa23:49
wgrantlifeless: What's the process for live index creation these days?23:51
wgrantApply before or after landing?23:51
wgrantIt should take about 60ms to create :)23:51
lifelesswgrant: land on qastaging, add and QA there, then prod.23:52
wgrantland on devel, add and QA on qastaging, then prod?23:53
wgrantIt turns out that ProductReleaseFile.productrelease is unindexed.23:53
wgrantSo I guess the table is just pathetically small.23:53
lifeless74K rows23:54
wgrantOh, right, the PRF.23:54
wgrantp-r-f, sorry23:54

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!