/srv/irclogs.ubuntu.com/2011/11/07/#launchpad-dev.txt

wgrantSomeone linked to bug #1234 on the ISO tracker.00:02
wgrantI expect.00:02
wgrantAlso, ubuntuqa uses edge.00:02
lifelessanother bug to file00:08
* StevenK grumbles at TALES being useless.00:46
lifeless.00:59
StevenKlifeless: http://pastebin.ubuntu.com/730574/01:04
lifelesswin01:09
lifelessuhm, nothing jumps out at me01:09
StevenKWhat's annoying me is that I saw it last week, and I fixed it, and I can't remember how.01:14
StevenKwgrant: Do you have any clues from the above traceback?01:15
wgrantNot really.01:16
StevenKNo traceback when not logged in. Curious.01:23
wgrantHardly.01:26
wgrantIt's in all probability the user info in the top right corner.01:26
wgrantThat's what uses link-display-id or whatever it is the most.01:27
StevenKBut factory.makePerson() should set all of that up, surely ...01:27
wgrantWhich layer are you in?01:28
StevenKDatabaseFunctional01:28
StevenKwgrant: Is DatabaseFunctional too low?02:10
wgrantStevenK: shouldn't be.02:27
StevenKwgrant: Same error with LaunchpadFunctional :-(02:32
StevenKOh, damn it, I know what I'm missing02:32
huwshimiIs anyone else getting sporadic "Exception: RabbitMQ server is not running." errors when trying to run 'make run'?02:47
lifelesshuwshimi: not I03:09
lifelesshuwshimi: but make run isn't a common thing for me atm03:09
lifelesshuwshimi: poolie had that recently, but I don't know what was causing it for him03:09
wgrantI use make start and make run frequently.03:09
wgrantNo trouble.03:09
wgrantBut this is lucid.03:09
huwshimihmmm03:10
StevenKAssertionError: queries do not match: 0 != 181703:11
StevenKWhee!03:11
lifelesscan has review ? https://code.launchpad.net/~lifeless/python-oops-twisted/bug-873030/+merge/8142703:11
pooliehaha yes03:11
StevenKAnd 200 SPPH queries, so success03:11
pooliehuwshimi, that's actually launchpad's cute way of saying "rabbitmq server *is* running" :)03:11
lifelessbbiab, -> shopping03:12
pooliei put up a patch to make it give a better message but i think it's not deployed yet03:12
poolielook for a stray rabbit process and kill it03:12
huwshimipoolie: Oh right. That is cute03:12
StevenKpoolie: It doesn't need to be deployed, it just needs to be on devel03:12
poolieapparently our tone of voice is 'extremely sarcastic'03:12
poolieStevenK, does that come from a branch?03:12
StevenKpoolie: lp:launchpad03:13
* poolie goes to look03:13
poolieno, it needs to be fixed in rabbitfixture03:13
StevenKAh03:13
StevenKWhere does that live?03:13
pooliehuwshimi, was that in fact your problem?03:13
wgrantIt's an egg.03:14
poolieif so that nudges me to make sure my thing lands03:14
huwshimipoolie: I'm not sure, I'll try next time it complains03:14
wgrantThe only things that come from branches are bugs.03:14
poolieby 'deployed' i meant 'packaged updated etc'03:14
pooliehuwshimi, if there's an orphaned instance i think it will happen again next time until you do kill it03:14
huwshimipoolie: it hasn't complained for a while03:15
pooliewgrant, ha ha03:15
pooliebut, what are the things in sourcecode/ then?03:15
StevenKpoolie: We have three ways of adding dependencies.03:16
wgrantpoolie: They're bugs.03:16
wgrantpoolie: They shouldn't be branches.03:16
pooliei know this03:16
poolieStevenK, i know there are 3 ways03:16
poolieis this wgrant's way of saying that one of those ways is deprecated?03:16
wgrantOr at least generally discouraged, yes.03:17
wgrant(also, there's a fourth way: including them in the tree)03:17
pooliethat's what you mean by saying they're bugs?03:18
wgrantYes.03:18
StevenKOh, ugh, I didn't know we linked from lib into sourcecode.03:19
wgrantThat's how sourcecode works :)03:19
StevenKThat's disgusting, kill it.03:19
StevenKwgrant: I have a test that works!03:20
StevenKwgrant: It's a bit evil at the moment since it creates 100 BugNominations, but it proves the query count is excessive03:20
StevenKOh, it's even worse. lib/mailman is a copy of sourcecode/mailman03:23
* StevenK tries to work out how to prefetch the entire batch to seed Storm's cache03:25
huwshimiI'm guessing in Launchpad there is no intelligent way to create user icons from there logos or mugshots right?03:30
wgranthuwshimi: No.03:31
huwshimi#$%03:31
wgranthuwshimi: For projects we ask them to upload a separate 14x14 image.03:31
wgrantParticularly since you can't sensibly scale much down to 14x1403:31
huwshimiwell you can, we just don't have the infrastructure03:31
lifelessimage scaling without jaggies is not -all- that simple. We might have been better off asking for a single svg.03:32
wgrantO_o03:33
wgrantnot sure if serious03:33
huwshimiNo,03:33
huwshimithis is ridiculous03:33
huwshimiwhy do we have to overthink everything03:34
huwshimithis the web03:34
huwshimipeople scale images ALL THE TIME03:34
wgrantYes.03:34
huwshimiand they don't have massive scaling SVG mega frameworks, just to do the simplest of things03:34
wgrantUsing SVG here is entirely ridiculous.03:35
wgrantBut I'm not convinced that we can usefully scale normal sized images all the way down to our 14x14 icon format.03:36
lifelessit all depends on what the criteria you're aiming for are. If we are happy with browser scaling, just set attributes on the image.03:36
wgrantAnything below 32x32 tends to be pretty hideous, IME.03:37
lifelessIf we aren't happy with browser scaling, I think its a safe assumption we won't be happy with unattended resizing either.03:37
lifeless(Because browsers keep iterating on their scaling logic to make things look good, so they are pretty solid)03:37
huwshimilifeless: Really and use all the extra bandwidth, you really think that's the best practice here?03:38
huwshimilifeless: And how about background images?03:38
huwshimilifeless: Shall we exclude older browsers from having scaled images too?03:39
lifelesshuwshimi: I don't know - I'm not *setting* the goals here -03:39
lifelesshuwshimi: you'll know much better than I the limitations of the browsers we're dealing with03:39
huwshimilifeless: Right, but this is a solved problem. Let's not invent a new ethos for handling images03:40
lifelesshuwshimi: thats great03:40
lifelesshuwshimi: I didn't realise it was so well solved :)03:41
wgrantHmm? Server-side thumbnailing isn't exactly an uncommon, new concept...03:42
huwshimiwgrant: Thankyou03:42
StevenKPerhaps lifeless is one of our four IE6 users.03:42
lifelesswgrant: not down to 14x1403:43
wgrantI recall implementing it in some of my early web projects in like 2002, and it was probably popular before then and 10-year-old me just didn't realise.03:43
wgrantNo.03:43
wgrantNot down to 14x14.03:43
wgrantI agree that we can't do that.03:43
lifelessthumbnailing is one thing03:43
lifelessbut extremely large, or extremely small scales are damn tricky03:43
huwshimilifeless, wgrant: You seriously think we shouldn't scaled down to ~14 (or 20 or whatever we want for small icon sizes)? You seriously think most people aren't just going to scale it before they upload it anyway (if they're a project)03:44
lifelessheck, Next machines in 1997 happily thumbnailed images in a directory listing, if memory serves.03:44
wgrantIt is a rare website that uses small icons like that.03:44
wgrantWe should probably stop.03:44
wgranthuwshimi: No, I argue that we shouldn't be using 14x14 icons for projects and people.03:45
wgrantI don't think anyone else does.03:45
wgrantThey use more sensible stuff of at least 20x20, from what I've seen.03:45
lifelesshuwshimi: I know that its hard to get a good representation of a logo at 14x14 without manual fixups - e.g. an icon editor03:45
lifelesshuwshimi: I don't know if our users care about that03:46
lifelesshuwshimi: and I don't know if we need 14x14 for any reason other than having it already.03:46
wgrantExactly.03:46
lifelesshuwshimi: I am not about to argue for complexity we don't need : see francis bug report for the cost of complexity on us as a team.03:46
huwshimiright, but that shouldn't stop us from forcing users and ourselves into set sizes (192, 64, 14)03:46
lifelesshuwshimi: one answer, for instance, is to scale by default and allow overrides for folk to fix things up03:47
lifelessstub: https://code.launchpad.net/~lifeless/python-oops-twisted/bug-873030/+merge/81427 (OCR penance time)03:48
lifelesshuwshimi: if your new design will permit a larger micro size, it may make a substantial difference to auto-scaled images [experimentation or citatin needed]03:48
StevenKMy prefetch query doesn't impact the query count :-(03:50
huwshimiUgh, it just turns my job into a pipe-dream, as everything I design requires weeks of work to implement the simplest thing (and would never get priority over other work)03:50
StevenKIt could just be a stupid query03:50
stublifeless: k03:52
lifelessStevenK: might be the cache size as well03:52
wgrantStevenK: dev runs with a storm cache of only 100 objects.03:52
wgrantStevenK: Product uses 1000003:52
wgrantProduction, that is.03:52
lifelesshuwshimi: are there js only scaling libraries? I assume there must be03:53
lifelesshuwshimi: you could scale on the client and upload three things from one js form03:53
lifelesshuwshimi: s/you could/could you/?03:53
StevenKwgrant: Right, so my test might just be too large at the moment03:53
huwshimilifeless: Well there kind of are, but they still require the client to download the full-size image03:53
lifelessStevenK: or just about right03:53
StevenKwgrant: TBH, I'm tempted to just blame my current query03:53
lifelesshuwshimi: I meant in the edit form for the person; do all the prep on the client03:54
StevenKlifeless: I'm creating 100 bug nominations, and then browsing the view with ?batch=100. It seems excessive to me, and I wrote it.03:54
lifelesshuwshimi: that would avoid nearly-all backend changes03:54
huwshimilifeless: Oh, I see. Well there's no javascript version of PIL, Imagemagick etc. no.03:54
lifelesshuwshimi: which is where we are not really prepped to do dynamic stuff03:54
lifelesshuwshimi: we have reasonable tools to run e.g. imagemagick in the backend, but they aren't trivial to use if you're not familiar with them, which IIRC was one of your goals03:56
stublifeless: So you do want a grammar and coding style review, or wait for someone who knows a bit about twisted and wsgi?03:57
lifelessstub: jamesh is eyeballing it03:57
lifelessstub: I hope he's wiring it up to test in the u1 stack03:57
lifelessstub: as much of a review as you want to give would be great.03:57
huwshimilifeless: I'm not sure quite what you mean. It's pretty easy to set up PIL for thumbnailing etc. (it even has .thumbnail())03:57
lifelesshuwshimi: I would expect that to be done in a 'job' straight after the upload succeeds, to avoid timeout issues.03:58
lifelesshuwshimi: which involves a new table; a new config entry to run under the shared job environment, some tests for your code, and a Job definition03:59
StevenKAnd 64 million interfaces03:59
lifelesswe're iterating towards this being lighter still04:00
huwshimilifeless: Sure, or it is often set up as a service when required (e.g. launchpad.net/thumbnail-service/14/14/file.jpg)04:00
lifelessand, pick yourself off the flow, it used to be much worse04:00
lifelesshuwshimi: s/flow/floor/04:00
lifelesshuwshimi: I'd have some scaling worries about a JIT service, though yes we could do one.04:01
stublifeless: what is the number after your name in NEWS?04:01
lifelessstub: did I forget the # ? its a bug number04:01
huwshimilifeless: Right, but anyway, I'll stop talking about this now as it is only a frustrating mental exercise04:01
stublifeless: And the word 'Bug' :)04:02
lifelessstub: not needed ;)04:02
lifelessstub: the rest of the file - and in other projects too  - just does #123404:02
stubYay for confusing04:03
lifelessseen a Debian/changelog recently ? :)04:03
stub'Better than a Debian changelog' is damning with faint praise04:03
stubBut at least with the '#' it won't look like you are giving your body measurements or date of birth04:04
lifeless:)04:04
StevenKHm, my query doesn't help with only 10 bug nominations either, still 197 queries.04:13
lifelessStevenK: is your theory that they *can* come out of cache false?04:13
lifelessStevenK: if the deeper code is doing anything *other* than attribute traversal, you can't optimise by cache loading.04:14
lifelessStevenK: you would need to actually setify the logic (which is a good thing anyhow)04:14
StevenKlifeless: I thought I could cache the bugnominatons themselves in the first case.04:14
StevenKBut it seems that is pointless04:15
StevenKlifeless: So, I'm tempted to declare that cache seeding won't help, but then I'm stuck about what I can do instead04:16
lifelesshave you identified the cause of the problem ?04:16
StevenKI've identified what is causing the masses of SPPH queries.04:17
lifelessgo on04:17
StevenKI'm not sure if that is the cause, but it's one thread to tug on.04:17
wgrantStevenK: This is for latest_published_component?04:19
lifelessStevenK: ok, so can you pastebin the call stack ?04:19
StevenKwgrant: So latest_published_component wants SPPH, and I then I think verifyUpload() does too04:19
wgrantStevenK: How are you preloading it?04:19
wgrantYou'll need to turn it into a cachedproperty.04:19
wgrantAnd populate it manually in a set-based manner.04:20
StevenKwgrant: Currently, I was trying to cache the bug nominations themselves first, but that hasn't changed anything.04:20
wgrantAre there lots of BugNomination queries?04:20
StevenKwgrant: 2 per row04:21
wgrantThat doesn't make much sense.04:21
wgrantDo you have a query log?04:21
StevenKYup04:21
StevenKwgrant: http://people.canonical.com/~stevenk/test_query_log.txt04:22
lifelessnone of those are attribute access queries04:23
wgrantAh, so SPPH isn't really the problem at all.04:23
wgrantThe whole thing is.04:23
lifelessthey are all explicit lookups from procedural code AFAICT in a quick glance04:23
StevenKHeh04:23
lifelesscache cannot help you at all04:23
wgrantWell, the built-in Storm cache can't.04:23
lifeless^storm ...04:23
wgrantcachedproperty can04:23
wgrantSo, you are going to need to radically setify canApprove.04:24
wgrantUnless you want to do it all manually beforehand, which I would discourage.04:24
StevenKRight, so canApprove() is the crux of the problem. I certainly agree.04:25
lifelessso what you need to do is write a set based version of it04:25
wgrants/crux of/04:25
lifelessthat takes many nominations (or something like that) and figures out the answer for all of them.04:25
wgrantExactly.04:25
lifelessmake the existing canApprove forward to that new function04:25
lifelessand finally migrate this code path over to use the new function directly with all of the nominations for the page04:26
StevenKBut canApprove() is hideous :-P04:27
lifelessyes04:27
lifelesstake small bits04:27
lifeless*bites*04:27
StevenKHm, there is a BugNominationSet ...04:27
StevenKSo there are two issues I can see.04:28
StevenK1. My batch in the view is bugtasks.04:28
StevenK2. canApprove() sucks04:28
lifelessthe other alternative is to eliminate nominations; is possible that the new security rules mean they are not useful now.04:29
lifelessso 1) is something that cachedproperty may help with04:29
StevenKOh, sure, eliminating nominations doesn't sound controversial *at all* :-P04:31
StevenKwgrant: Can I beg you for a pre-impl about this new method?04:46
wgrantIt's pretty simple really, but OK.04:47
pooliewgrant, re bug 435905, i wonder if you would agree we should just close it05:03
poolieand keep wrapping roughly where we currently do05:03
poolieuntil we get proper markup05:03
lifeless3K timeouts. Hmm05:04
wgrantlifeless: Network outage overnight.05:04
lifelessstub: so no review results ?05:04
lifelessstub: or just the # is it ?05:04
wgrantpoolie: Do you really mean me?05:05
pooliei do; istr you had some opinion about always using greasemonkey before05:06
poolieand your 'fix the defaults' reminded me of it too05:06
wgrantAh.05:06
wgrantWell, I think the fix for it is markup.05:06
stublifeless: Getting there, distractions distractions05:06
pooliethen i agree05:06
pooliethanks05:06
lifelessstub: ;)05:06
lifelessaieee, gmail, what have you done05:07
wgrantpoolie: The current uploads work mostly. We shouldn't keep hacking it to make various special cases work. We should do what the rest of the world does.05:07
wgrantWe should follow what the rest of the world does for an awful lot of things.05:07
wgrantWe needed to do our own thing 7 years old.05:07
wgrants/old/ago/05:07
wgrantBut it's no longer 7 years ago, and the world has moved on.05:08
poolie'current uploads'?05:08
wgrantEr.05:08
wgrantSorry, multiple conversations.05:08
pooliecurrent css?05:08
wgrantYes.05:08
pooliei agree with the general thing anyhow05:08
pooliei do wonder how much say feature flags or timelines are available externally in other frameworks05:09
lifelessrpm.net05:09
poolie?05:10
poolieit lapsed?05:10
lifelesshuh05:11
lifelesslet me find their current home05:11
pooliedid you mean rpmfind?05:11
wgrantWas that the fork, or the not fork?05:11
wgrantI can't remember.05:11
pooliethe description of the hardware on http://rpmfind.net/ is awesome05:12
pooliethough i bet we have some that is older05:12
lifelesshttp://newrelic.com/05:12
lifelesshuh, they have added python since I last strolled by05:12
lifelesshttp://newrelic.com/docs/python/instrumented-python-packages05:14
pooliemm05:14
pooliethere may even be free versions05:14
poolies//alternatives05:15
lifelesswell, oops-tools & sentry are in that space, though neither is anywhere -near- as advance.05:15
lifelessoops-tools is the most metric enabled open end-to-end one I know of for Python05:16
* stub wonders why magic asynchronous tests are a good idea05:17
lifelessstub: magic?05:19
poolieis there any kind of trick about $PYTHONWARNINGS?05:19
stubtest methods returning deferreds with callbacks that the test runner somehow executes because the test case class has a property set to a particular value05:19
lifelessstub: well, you need *something*05:20
lifelessstub: this has the nice property that its extensible to different async environments in one library (vs e.g. twisted trial)05:20
stubbecause tests being nearly the ultimate in synchronous code require asyncronicity05:21
lifelessyeah, they very much do when the code is callback based05:22
lifelessand the callback engine doesn't guarantee pause/resume/single step facilities05:22
stubbut a helper invoked explicitly would remove the magic05:22
stubAnd would even allow you to test post state without registering these tests as callbacks.05:23
lifelesssee under callback engine guarantees05:24
stubd.addCallback(lambda _: self.assertEqual(expected, self.calls)) would be much more readable as just self.assertEqual(expected, self.calls)05:24
stubThat doesn't mean anything to me.05:25
pooliebut there's a difference ,right05:26
pooliethe assertion is: _eventually_ when this is called, there are N calls05:26
lifelessstub: I agree that it would be nicer if it was single stepped and looked just like regular python05:28
stubI don't see the difference between 'd.addCallback(lambda: blah blah assert blah); return d' and 'self.run_deferred(d); self.assert'05:28
lifelessin principle inlineCallbacks could be used.05:28
huwshimiA review for some lovely person, if possible: https://code.launchpad.net/~huwshimi/launchpad/avatars-everywhere-712894/+merge/8143005:28
stubYer, just the more I see twisted code the more I think that twisted was a bad idea and should never have been shoe horned into Python.05:29
stubCode that makes you go 'blergh' when you haven't gone through the indoctrination process keeps you isolated.05:30
lifelesshuwshimi: I've given it a once over05:35
poolieah, the trick is it's new in 2.7?05:35
lifelesspoolie: what is new in 2.7 ?05:35
poolie$PYTHONWARNINGS05:36
lifelessah, dunno.05:36
poolieadded by one B. Warsaw05:36
nigelb*yawn*, morning05:45
pooliehi nigel05:50
nigelbHey poolie :)05:50
poolieis there some easy way to override buildout and make it use a dependency from a tree, while i'm futzing with it?05:50
pooliei guess replace the egg with a symlink05:50
wgrantpoolie: You should be able to symlink the tree into the root of your LP tree, then add the dir name to the 'develop' setting in buildout.cfg.05:52
lifelesspoolie: bin/buildout buildout:develop=". ../path/to/tree"05:53
lifelesswgrant: no need to symlink05:54
wgrantOh, or that.05:54
wgrantDidn't know that worked.05:54
pooliewow, ok05:56
poolieso those two trees come before everything else?05:57
wgrantdevelop should just be '.' by default... what do you mean?05:58
pooliehuwshimi: whoa!05:58
pooliewgrant, i meant before things pulled in from eggs05:59
pooliehuwshimi, that would be a good thing to add behind a user 'labs' switch05:59
pooliethough i'm not suggesting you block on adding one05:59
wallyworld_stub: hi, i have a sql question for you if you have a moment sometime05:59
huwshimipoolie: Yeah, we'll see how much it breaks launchpad :)05:59
StevenKwgrant: http://pastebin.ubuntu.com/730709/06:00
poolieat least a flag would be good06:00
StevenKwallyworld_: Just ask, it may not be stub that answers.06:00
huwshimipoolie: Wouldn't labs be similar to launchpad-beta-testers?06:00
wallyworld_StevenK: it's a performance related thing06:00
StevenKwallyworld_: Then lifeless or wgrant would probably also have opinions06:01
wallyworld_fair enough. although i didn't want to bug them :-)06:01
wallyworld_but anyway, i think this will be ok, hopefully: https://pastebin.canonical.com/55351/06:01
wallyworld_it is to check whether a given person owns any pillar06:02
wallyworld_and i'm looking to use it amongst other places in a storm validator06:02
wallyworld_so i want it to perform well06:02
wallyworld_there's a similar query for isSecurityContact06:03
pooliehuw, it would be similar with a few differences06:03
poolie- the ui is more obvious than joining/leaving a team06:03
poolie- teams are kind of used for too many things06:03
poolie- we can have more than one concurrent experiment06:04
stubwallyworld_: That query seems very odd.06:04
poolie- we could hook it up to feedback06:04
poolie- oh, users would have a clearer idea what things were experimental06:04
wallyworld_stub: maybe. the unit tests pass :-)06:04
wallyworld_but if it can be improved...06:04
poolieat the moment they just see everything (though i believe they're going to get some kind of systematic indication in future)06:04
wgrantwallyworld_: I'd prefer that it was in a setter, not a storm validator.06:04
stubYes, but selecting the first row from the person table if the EXISTS matches, throwing it away and returning true...06:05
wallyworld_stub: the only reason is use select from person was to keep dtore.find happy06:05
wallyworld_stub: i can constuct sql without any table "select exists ...." but storm complains06:06
wallyworld_i guess i could do a store.execute() with raw sql?06:06
wallyworld_by complains i mean store.find() says "there's no table"06:06
wallyworld_if i try and run something like "select exists (select something where condition)"06:07
wallyworld_wgrant: storm validators are bad?06:08
wallyworld_they seem to be used in the type of thing i'm doing already06:08
wallyworld_eg is_valid_person and friends06:08
wgrantLots of things are already used for lots of types of things :)06:09
wgrantI think Storm validators impede clarity.06:09
wgrantAnd are often used to do very bad things.06:10
wallyworld_i don't agree that they hinder clarity06:10
wallyworld_anything can be misused06:10
wgrantStorm validators become problematic when eg. you want to pass an argument into them.06:11
wgrantFor example, admins might be allowed to override some restrictions.06:11
stubhttps://pastebin.canonical.com/55352/ might work better06:11
* wallyworld_ looks06:11
lifelesshuwshimi: I suspect it will need the eager loading changes I reference done before you can enable the flag :(06:12
lifelesshuwshimi: Its very cool to see it being done06:12
wgrantlifeless, huwshimi: It also has an XSS hole or two.06:12
stubwallyworld_: But what you have is just as fast despite looking a bit WTF?06:12
wallyworld_stub: thanks. i'll use that. although the one i pasted also works, but your is easier to grok i guess.06:13
huwshimilifeless: Oh, sorry I mis-understood06:13
wallyworld_stub: i guess i was trying to avoid the count(*)06:13
stubwallyworld_: I'm not sure of how to shoe horn it into Storm06:13
lifelesshuwshimi: I was probably unclear06:13
huwshimilifeless: It's all good06:13
lifelesshuwshimi: Landing with a flag is fine, I just think its near-certain that enabling it will make things timoeut06:13
huwshimilifeless: Sure06:13
lifelesshuwshimi: so you could, if you think I'm wrong try it and possibly save yourself some time06:13
stubwallyworld_: If you go for yours, change the UNION to UNION ALL06:13
wallyworld_stub: i think because yours does a select from... i can get it in06:13
lifeless(IMBW)06:13
lifelessI don't think I am, in this case.06:14
huwshimilifeless: What's involved in making the eager loading changes?06:14
wallyworld_stub: ok. i'll have a look at it. if it all goes pear shaped, i can always revert to my original sql since the performance is ok06:14
wallyworld_thanks06:14
huwshimilifeless: Is it something I can take a look at?06:15
stubwallyworld_: We are looking at <10ms, so whatever ;)06:15
wallyworld_stub: cool. the perf figure is what i was after. thanks :-)06:15
wallyworld_wgrant: i agree about the argument passing thing. so validators are good for invariant conditions06:15
wgrantwallyworld_: It's *presently* invariant.06:16
huwshimiwgrant: Where are the XSS holes? This is mostly just existing code, but I'm happy to fix it up a bit06:17
wallyworld_yes. i don't see this one changing though06:17
wallyworld_but perhaps it could06:17
lifelesshuwshimi: the key function is the one I mentioned in my review06:17
wgrantStevenK: if reason is not None and not (is_new and getComponentsForUpload)06:17
lifelessgetPrecachedPersonsFromIDs06:17
lifelesswhich calls06:17
lifeless_getPrecachedPersons06:17
lifelessthe related but separate cacheBrandingForPeople already grabs all three sizes06:18
lifelessso you don't need to touch that06:18
huwshimilifeless: Ah I see06:19
wgranthuwshimi: The URL contains user-controlled content (in the filename), and it's now in the page unescaped. cgi.escape is also no good for attribute values -- it doesn't escape quotes by default.06:19
wgranthuwshimi: I recommend that you use structured()06:20
wgrantIn c.l.w.menu06:20
huwshimilifeless: I have to go deal with a whinging baby, but I'll take a look at this tomorrow.06:20
huwshimiwgrant: Thanks, I'll take  a look06:20
StevenKwgrant: http://pastebin.ubuntu.com/730715/06:21
lifelesshuwshimi: cool. Please feature flag it *anyway* because we may well find that this change causes brand new late evaluations06:22
huwshimilifeless: Sure, no problems. Thanks for that06:22
lifelesshuwshimi: I may seem to be being overly cautious here, but believe me, LP is designed to be slow06:22
lifelessso we're fighting all the time06:22
huwshimilifeless: Haha, no I totally understand06:22
poolielifeless,  thanks for the tip about setup.py06:27
pooliethe one in oops-repository was trivially broken06:28
pooliei pushed a fix06:28
lifelessoh thanks06:30
pooliei get a warning about extras_require and install_requires06:38
pooliebut perhaps they're used by one of the other cluster of tools06:38
lifelessyah06:42
lifelessthanks for the review stub06:48
lifelessstub: shall we catchup ?07:15
stublifeless: sure.07:29
* stub boots laptop07:29
poolielifeless, still here?07:31
pooliei thought i would need to dpkg txfixtures so it could be installed on the buildd guests07:32
lifelesspoolie: ah possibly. I don't know enough there sorry :(07:48
pooliepretty sure07:48
poolieat any rate if we're going to split this out as a dependency of buildds it will need to get there somehow07:48
lifelessstub: d/c ?08:35
adeuringgood morning08:54
jelmerdoes anybody know what is up with strawberry (qastaging's code importer) ? It doesn't seem to be processing code imports.09:40
StevenKjelmer: Ah, you're attempting to QA r14243?09:43
StevenKHm, no rvba today.09:44
jelmerStevenK: no, I was hoping to QA bug 48593209:44
StevenKjelmer: Yes, which is linked to stable r14243. :-)09:45
StevenKallenap: Could you try and QA r14234, r14250 and r14260 today at some point?09:45
wgrantjelmer: qastaging had some download-cache issues earlier. Might want to poke a LOSA to poke strawberry.09:46
bigjoolsjelmer: are you able to do some QA please? http://launchpad.net/bugs/48593209:46
StevenKbigjools: He just said he was ...09:46
bigjoolsah, /me reads backscroll09:46
jelmerwgrant, StevenK: I'll check with a LOSA, thanks09:46
StevenKbigjools: Have you bugged rvba for the three revisions he has to do? :-)09:46
bigjoolsStevenK: I see you're on the same jihad as me09:47
lifelessbigjools: what rt should I tweak to add the oopses exchange to the txlpongpoll permissions (on all three sites)09:47
StevenKbigjools: Yes. The deployment report has been Red Squaded.09:47
lifelesss/to add/to get losas to add/09:47
bigjoolslifeless: I'd do a new one09:47
bigjoolslifeless: but failing that, 4810909:48
lifelessnew one done09:50
lifelessallenap: hi; why ez-setup ?09:55
lifelessallenap: oh, I see - why don't we put that in the shared LP download cache ?09:55
allenaplifeless: Otherwise it tries to get it from the network.09:55
allenaplifeless: D'oh, ignore that.09:56
allenaplifeless: I assume you're talking about txlongpoll? I recently added ez_setup to Storm too.09:57
lifelessyes09:57
allenaplifeless: For txlongpoll I want it to be standalone, and not depend on the LP download cache. It's meant for Landscape to use too, though I have no idea if they have plans to do so.09:58
lifelessallenap: mmm sure09:59
lifelessallenap: these things are not actually related.09:59
lifelessallenap: e.g. LP deployment target can assume someone has done bzr checkout ... deployment cache09:59
lifelessallenap: without compromising use of bootstrap + any other deployment cache.09:59
lifelessallenap: separately, how would you feel about us dropping zope.testrunner from it (or at least making subunit.run a supported target, so I can use testrepository :))10:01
allenaplifeless: So I should put ez_setup.py into download-cache/dist then refer to it there in Makefile.10:01
lifelessallenap: I think it would avoid some duplication10:01
allenaplifeless: Very happy about that.10:01
allenaplifeless: I think moving ez_setup.py will make it harder for non-LP folk to use txlongpoll, but perhaps bootstrap.pt DTRT.10:03
allenaplifeless: Also, Launchpad itself has ez_setup.py in it.10:03
allenapBut I guess that's a candidate for moving to the cache.10:03
lifelessallenap: python ./bootstrap.py will download from the internet; only we care about that :)10:03
lifelessallenap: (and other folk that can reasonably use the lp download cache, like landscape)10:03
lifelessallenap: (or setup their own)10:04
lifelessallenap: its only a thought10:04
allenaplifeless: I don't see that it buys much.10:04
lifelessallenap: I fear and loath ez_setup; having it in one place would reduce the amount of evil in the world10:05
lifelessallenap: also I don't want to have to add it to oops, oops-amqp, oops-wsgi, oops-twisted, oops-datedir2amqp, oops-tools, ...10:05
lifelessallenap: (I'm being selfish :P)10:05
lifelessbigjools: rt 48999 if you didn't get a CC from rt10:06
bigjoolsgot it10:06
lifelessallenap: the README claims manual egg location is needed, but it Just Worked for me; are the docs old, or does the LP downloadcache make things easier ?10:09
allenaplifeless: Hehe, okay.10:10
* allenap reads the README10:10
jtvWe're getting silly warnings from the packaging_translations job that it's skipping translations for Lenny.  Anyone mind if I downgrade that to "info"?10:10
jtvbigjools: (I know I said there was no time for me to get into a new item today, but there was only 1 bug to triage :)10:11
allenaplifeless: Ah, yes, the README is slightly wrong, and it should Just Work with the LP download-cache.10:11
allenapIt should read tarball instead of egg.10:11
lifelessallenap: in the options in the twisted plugin10:13
lifelessallenap: how do you not do a short option - pass None instead of "o" ?10:13
allenaplifeless: You mean you want a null OOPS prefix? Sorry if I'm being stupider than usual today, I had a rough night :)10:15
lifelessI'm adding amqp oops support to txlongpoll10:17
lifelesswhich is a bit fugly because its not a twisted publisher, but nevermind that :)10:17
lifelessI am running out of single letters that are not used by twistd10:18
lifelesse.g. -e is encrypted10:18
allenaplifeless: Yes, None seems to do the trick.10:22
lifelesscoolio10:22
lifelessuhm10:22
lifelessWill need your advice on whether to test this or not in ~ 5 minutes10:22
lifelessalso hope you have a better day than night10:22
allenapThanks :)10:33
lifelessallenap: https://code.launchpad.net/~lifeless/txlongpoll/oops-amqp/10:42
lifelessallenap: I have no idea whether this is sanely testable or not.10:43
allenaplifeless: Okay, looking.10:43
lifelessthanks!10:44
lifelessallenap: I'm going to halt()11:03
lifelessallenap: that branch should work as-is, I'm sure you can play with it more to see ;)11:03
allenaplifeless: Okay. There are a couple of syntax errors in there. I'm inclined to say that all it needs is something to verify that publishers are set up correctly after calling setUpOopsHandler.11:05
lifelessah there are aren't there11:05
lifelessinteresting that the plugin isn't tested at all then :)11:05
lifelessallenap: perhaps we should move all this code out of the plugin, make it testable, and just import it from the plugin to activate it11:06
allenaplifeless: Yeah, agreed.11:06
lifelessallenap: would you have the bandwidth to run with this ? I wanted to solve the bits that had changed while you guys were building stuff11:07
allenaplifeless: Yeah, I guess so. I'll need to run it by bigjools.11:07
lifelessif you can, great. if you can't, thats ok too.11:08
bigjoolsyou rang11:08
lifelessbigjools: I put amqp support for oopses in a branch of txlongpoll11:08
bigjoolscool11:08
lifelessbigjools: this, by virtue of syntax errors uncovered txlongpoll's twistd plugin being untested - entirely untested.11:08
lifelessbigjools: (because the test suite still passes)11:08
bigjoolsawesomery11:09
lifelessbigjools: I was asking allenap if he had the bandwidth to fix that (by moving its contents into the main module where its testable), incorporating my otherwise trivial branch as he goes.11:09
lifelessbigjools: he said possibly-but-ask-you11:09
bigjoolsI am ok with it as long as he is11:09
lifelessmore-or-less11:09
allenapYes, I'm good with it. Deal.11:09
lifelesssweet, thanks!11:10
jtvWho's up for a really easy review?  https://code.launchpad.net/~jtv/launchpad/bug-887063/+merge/8143911:16
jelmerjtv: it seems you're just changing from warning to warn?11:18
jelmerjtv: The proposed fix says you're changing to info11:18
jtvOh God I got that wrong didn't I?  Thinko.11:19
jtvThis is why we need reviews even on “trivial” changes.  :)11:19
jelmer:)11:20
jtvFix is being pushed.11:20
jtvDone11:20
jtvjelmer: the diff has updated.  Note there's also a change to the imports; that's from the automatic imports-formatting script.11:24
jtvUnsurprisingly, tests, still pass.11:24
jtvExcept for, very surprisingly, an unrelated test that breaks even on devel.  What is it _now_?11:28
jtvAnd I can't really investigate with my system needing unrelenting restarts.11:29
jelmerjtv: which test?11:29
jtvtest_splits_and_merges11:29
jelmerjtv: r=me, if that failure isn't spurious I'm curious how your change breaks it..11:30
jtvIt's not my change.  I get the failure even in devel.  :(11:30
jtvArgh!  No wait.  I applied the same change in devel.  Something's wrong with me today, clearly.11:31
* jtv reverts & reruns11:31
jtvReverting my change doesn't fix the breakage in devel.  WTF.11:32
jtvProbably a test isolation failure then, I'm guessing.11:32
jtvjelmer: are you getting a failure there?  ./bin/test -vvc lp.translations.tests.test_translationpackagingjob -t test_splits_and_merges11:33
jtvthanks jelmer — I'll file a separate bug for that breaking test, assuming my own branch gets through EC2.11:39
jelmerjtv:   Ran 1 tests with 0 failures and 0 errors in 3.503 seconds.11:39
jtvOneiric?11:39
jelmerlucid11:39
jtvThat may be it then.11:40
jelmer(I'm on precise but running a container with lucid)11:40
jtvAh.11:40
jtvCall for Help: Could someone try this on oneiric?  ./bin/test -c lp.translations.tests.test_translationpackagingjob -t test_splits_and_merges11:40
StevenKjtv: Ran 1 tests with 0 failures and 0 errors in 1.609 seconds.11:44
jtvthanks StevenK11:44
jtvOh, StevenK: what revision is that on?11:45
StevenKjtv: r1426211:45
jtvHmm… on 14261.11:45
jtvWho knows—let's try.11:46
jtvAh no wait, I've got a bus to get onto.  :(11:46
jtvnn folks!11:48
=== dpm is now known as dpm-lunch
benjihttps://dev.launchpad.net/ | On call reviewer: benji | Critical bugtasks: 26612:59
benjiheh12:59
=== benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugtasks: 266
deryckMorning, all.13:51
jelmerArvo, Deryck13:53
jmljelmer: tee hee14:00
jelmerjml: hmm?14:05
jmljelmer: that's a very Australian way of wishing someone a good afternoon.14:05
jelmerah, hmm14:06
* jelmer wonders where he picked that up14:06
=== _mup__ is now known as _mup_
allenapbenji: Fancy a Storm review? It already has a +1, but needs another. https://code.launchpad.net/~allenap/storm/django-disconnection-errors/+merge/8071714:23
benjiallenap: my pleasure14:23
allenapThanks!14:23
benjiallenap: https://code.launchpad.net/~allenap/storm/django-disconnection-errors/+merge/80717 looks good14:52
allenapbenji: Thanks.14:52
=== dpm-lunch is now known as dpm
=== jcsackett_ is now known as jcsackett
bigjoolsjelmer: still around?16:53
jelmerbigjools: jup16:57
bigjoolsjelmer: hurray! did you manage to QA r14243?16:57
jelmerbigjools: no :-( the code importer on qastaging is down and I couldn't find a losa earlier16:58
jelmerbigjools: I see mbarnett is around now, so I'll ask him16:58
bigjoolsjelmer: mbarnett is around16:58
bigjoolsjelmer: great, thanks!16:58
abentleybenji: could you please review https://code.launchpad.net/~abentley/launchpad/yui-navigator/+merge/81470 ?  It's overlength, but that's just from indentation changes.17:05
* mbarnett hides 17:08
timrcbigjools, are there any plans of releasing a 2.0 api for this LTS?17:25
bigjoolstimrc: I have not heard of any17:28
bigjoolsbut we should look at it17:28
timrcbigjools, I don't think we have any pressing needing for it, devel suits us just fine, so it's more out of own personal curiosity ;)17:28
timrcer need... damn you -ing17:30
=== matsubara is now known as matsubara-afk
benjiabentley: I was away for lunch when you asked, but I just finished your branch.18:42
abentleybenji: thanks.18:42
benjimy pleasure18:42
abentleybenji: Updated per your suggestions.18:48
benjiabentley: cool18:49
lifelessmorning19:19
timrchowdy19:19
m4n1shis there a command line tool where I can access bug info from LP19:29
m4n1shand also initiate merge requests?19:29
lifelessthere is a thing called hydrazine19:37
lifelessand bughugger19:38
lifelessand lp-tools19:38
m4n1shthanks19:44
m4n1shlifeless: not in archives?19:57
m4n1shits in a PPA19:57
lifelesssinzui: do you have time for a catch up today ?20:08
sinzuiI do.20:08
sinzuimaybe in 30 minutes when I complete this review20:09
lifelesscoolio20:09
jelmerlifeless: lp-tools is in the archive20:19
jelmers/lifeless/m4n1sh/20:19
m4n1shahh. it is called lptools20:19
jcsacketthey, lifeless! i updated the MP you marked needing fixing. care for a second look? (https://code.launchpad.net/~jcsackett/launchpad/open-delegate-subscription-problems/+merge/81090)20:21
lifelesssure thing20:21
lifelessjcsackett: line 169 in the diff is very suspect20:23
lifelessjcsackett: thats the bit where you delete a Makefile and replace it with a symlink to somewhere on your hard disk.20:25
jcsackettlifeless: yeah, i just saw that.20:25
jcsackettlifeless: fixing that now, sorry for the time waste. :-p20:25
lifelessjcsackett: np, I've reviewed the rest :)20:26
lifelessjcsackett: I have one suggestion to avoid duplication further20:26
jcsackettlifeless: i'm all ears.20:26
lifelessjcsackett: which should be in your mailbox by now :)20:26
lifelessdelegation!20:26
jcsackettlifeless: oh, i do like that. we are likely to be doing that check a fair bit.20:27
jcsackettconsider it done, and thanks. :-)20:27
lifelessde nada20:27
flacostelifeless: hi20:34
flacostelifeless: available for our call?20:34
flacosteearly this week20:34
lifelessI sure am20:35
lifelesssinzui: will in a little bit later be ok ?20:35
lifelesssinzui: if I talk to the boss first :)20:35
lifelessflacoste: skype ?20:36
sinzuilifeless, ping me when you are ready20:37
flacostelifeless: yes, i'm trying a new headset, let's see how it goes20:37
lifelesssinzui: great, will do20:37
bachi abentley, i'm getting an error from bzr lp-submit.  it's the first time i've tried it on a new oneiric machine.  are you familiar with any problems like: http://pastebin.ubuntu.com/731374/20:56
abentleybac: Sorry, I'm not familiar with that.  It looks like a password/launchpadlib issue.20:57
maxbdbus error talking to your gnome-keyring-daemon20:59
maxbThough quite what's broken such that g-k-d is not responding, I don't know21:00
allenaplifeless: lp:~allenap/txlongpoll/oops-amqp is what I did with your branch of the same name.21:22
allenaplifeless: It depends on lp:~allenap/txlongpoll/bootstrap-without-net, and some revisions have been mixed up a bit between those branches. I'll remedy that now.21:23
allenaplifeless: Actually, no, they're not mixed up, I was being paranoid.21:24
lifelesssinzui: ping21:28
sinzuihi lifeless21:28
lifelessallenap: sounds great, do you need anything from me ?21:28
lifelesssinzui: skype ?21:28
* sinzui starts skype21:28
thumperflacoste: hi21:29
flacostehi thumper21:29
thumperflacoste: busy?21:29
allenaplifeless: Nope, I'll get it reviewed tomorrow unless you want to +1 it at some point :)21:29
lifelessallenap: cool; I may, will see how the day pans out21:30
flacostethumper: more or less, general post-UDS catchup21:30
flacostethumper: what can i do for you?21:30
thumperflacoste: can I call you for some questions?21:30
flacostesure, skype21:30
allenaplifeless: Thanks, have a good day :) Night all.21:36
wallyworld___sinzui: jcsackett: you guys around for the standup?22:08
jcsackettoh right, time changed.22:08
* jcsackett gets on mumble22:08
sinzuiwallyworld___, sorry, my call with lifeless got very interesting22:09
wallyworld___np :-)22:09
wgrantsinzui: That's worrying.22:09
=== benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 266
timrc266 critical bug tasks, I guess it may be time for a new "super critical" category :)?22:19
thumperHAHA22:19
timrcThank you, thank you, I'll be here all night22:19
=== wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugtasks: 275
wgranttimrc: It's even better :)22:19
timrcouch22:21
wgrantjcsackett: http://webnumbr.com/launchpad-critical-bugs22:28
wgranthttps://lpstats.canonical.com/graphs/LPProjectCriticalFiled/20101108/20111108/22:31
timrcJuly was a good month for you22:34
wallyworld___sinzui: we are looking to finish the standup, we can talk again tomorrow?22:38
sinzuiwallyworld___, thanks.22:39
wallyworld___np22:39
=== wallyworld___ is now known as wallyworld
allenapwgrant: I have a couple of txlongpoll branches that need reviewing. Do you fancy taking a look? The only catch is that I'm going to bed in a minute so I won't be around to discuss things.23:02
allenapThey are:23:03
allenaphttps://code.launchpad.net/~allenap/txlongpoll/bootstrap-without-net/+merge/8132323:03
allenaphttps://code.launchpad.net/~allenap/txlongpoll/oops-amqp/+merge/8150423:03
wgrantallenap: Sure, I'll do them today.23:05
allenapwgrant: Thank you :)23:05
lifelesstimrc: ubuntu has more ;)23:05
timrclifeless, ever have the urge to mark them all "Won't Fix" ;)?23:06
lifelesstimrc: no, but the rest, yes23:28
lifelessallenap: http://pypi.python.org/pypi/python-subunit23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!