=== salgado is now known as salgado-afk
lifelessI suspect Distribution:+bugtarget-portlet-bugfilters-stats is going to be mondo painful00:04
lifelessalso we need to remove the override on Archive:+index00:04
wgrantHow low is it now?00:05
wgrantThe page, I mean.00:05
wgrantNot the 16000 override.00:05
wgrantKill it.00:05
wgrantYes, but that takes a couple of minutes to load.00:05
wgrantHmm. Maybe I want to look at Person:+contactuser.00:07
wgrantSince its 99% is twice the next worst.00:07
lifelessmean sql00:08
wgrantFor +contactuser?00:08
lifelesshmm, 99th percentile forsql is mising00:08
lifelesswgrant: yes00:08
wgrantIt'll be mail.00:08
lifelesswgrant: its probably mail sending00:08
lifelesswgrant: we would benefit a lot if we had instrumentation on how long sending each email takes00:09
lifelessits been hard getting that evaluated by is (for fairly good reasons) - but it still an open question around where the issues live00:09
lifelessarchive:+index should wait for monday00:10
thumperlifeless: can I get you to mentor wallyworld? https://code.launchpad.net/~thumper/launchpad/blueprint-linked-bug-tasks/+merge/5373400:20
lifelessthumper: I need to pop up to the shops for lunch stuff00:22
lifelessthumper: will review after00:22
thumperlifeless: ok, thanks00:22
huwshimiDoes anyone know of a way to use feature flags in JavaScript?00:30
lifelessexport the value of the flag in the .pt file00:31
lifelessinterpret it in your js00:31
huwshimilifeless: That was my plan, but I was having trouble getting some context values. Maybe I'll try again.00:34
lifelessits been done before :)00:35
lifelesshuwshimi: e.g. showing the render time in the top right00:35
huwshimilifeless: heh no wonder it wasn't working. The feature flag wasn't set. The shame.00:45
lifelesshuwshimi: rotfl00:45
huwshimilifeless: I didn't realise that it didn't keep them between make runs00:46
lifelesshuwshimi: make schema will zerg them from your dev insance00:46
lifelesshuwshimi: 'make run' won't reset them00:47
huwshimilifeless: I didn't think I'd done a make schema... but maybe I had00:47
StevenKhuwshimi: make schema completly resets the db to 'pristine sampledata'00:47
StevenKNow to wash my eyes out for saying 'pristine sampledata'00:47
* thumper is mildly confused by a comment in code00:52
thumpermodel/person.py line 179700:53
mwhudsonthumper: i guess _getPrecachedPersons isn't in the interface?00:54
thumperit doesn't need to be00:54
poolielifeless, i liek your idea about incomplete (bug 737008)00:54
thumperthe self object isn't security wrapped00:54
_mup_Bug #737008: old incomplete bugs are not treated as incomplete <Launchpad itself:Invalid> < https://launchpad.net/bugs/737008 >00:54
thumperonce inside the object00:54
pooliemaybe we should make that bug ask for what you said (as Low)00:54
thumperyou are free of the security proxy on self00:54
lifelesspoolie: care to do that ?00:54
mwhudsonthumper: sure, but if you did getUtility(IPersonSet) you get a proxied object00:55
lifelessthumper: you can use the proxied set and see what happens....00:55
thumpermwhudson: yes you do00:55
thumperbut once you call a method on the object (that is in the interface)00:55
thumperthe self passed through isn't wrapped00:55
thumperotherwise that'd be insane00:56
* thumper is 98% sure of this00:56
thumperlifeless: you back?01:49
thumperanyone know how to clear the storm cache for tests?01:49
lifelessI should have been on leave today01:59
mwhudsonoh yes, the memorial02:00
lifelessI'll talk to flacoste, do a day off next week02:01
lifelessthumper: ^ store answers for you02:03
=== statik_ is now known as statik
thumperlifeless: ta02:17
thumperlifeless: do I need both, or just the second?02:17
lifelessone is deeper than the other02:19
lifelessinvalidate will cause existing objects to be refreshed from the db on next use02:19
lifelessreset will make them unusable and cause errors if you try to carry an object from before the reset forward02:19
thumperanyone remember off the top of their head what the storm equivalent of the sqlobject SQLRelatedJoin is?02:24
poolielifeless, i wonder if you (or mrevell or someone) should post to the blog to explain about the critical/high/low categories02:25
pooliere all the "why is this now Low" comments02:25
lifelessbut there have only been 3 or 402:26
lifelessin several hundred bug updates02:26
lifeless(one bug - the wiki one - had a longer discussion that I'm counting as one)02:26
lifelessI don't think this is really all that interesting to most users of lp02:27
pooliemaybe not02:29
lifelessI think people are responding because they get notified02:34
lifelessand would do so even if they knew that medium is equivalent to low, because the thing they care about is having their change made02:34
thumperthis code is soooooo... bad02:40
* thumper enfixorates02:40
wgrantthumper: Which code?02:43
thumperlike the way to add an attendee, it iterated through all the attendees to see if the person matched02:45
pooliehuwshimi, did you think you fixed the bug where you have to press enter twice to save the tag list?02:46
pooliedoesn't seem fixed for me on lpnet02:46
lifelessone enter selects the tag, the other saves02:47
lifelesshuwshimi: btw02:47
poolieyes, i know02:47
lifelesshuwshimi: try putting a non-typeahead tag into a bug02:47
pooliebut that makes no sense if i've already completely entered a tag02:47
huwshimipoolie: The fix is in lazr-js which I haven't got around to releasing a new version of yet.02:47
lifelesshuwshimi: its not done till its done02:48
poolieok thanks02:48
pooliejust wondered if it was in-progess or failed validation02:48
wgrant     1029  OOPS-1902EE937  BugTask:+nominate02:48
wgrant1029 statements on +nominate!?02:48
pooliesorry just one more thing there, what controls which tags show up in the portlet?02:48
poolieoh, deja vu, this is a caching bug, isn't it?02:49
wgrantProbably, yes.02:49
wgrantSprints are that bad?03:08
wgrantCan I burn bug heat?03:09
lifelesswgrant: no03:09
wgrantDoes anybody use it enough that the performance penalty is worth it?03:10
lifelessthe performance penalty is not intrinsic to having heat03:10
wgrantBut we need to either optimise or destroy.03:11
lifelessdistro are asking for heat improvements03:11
wgrantWell, destruction is the ultimate optimisation :)03:11
thumpergetUtility(IPersonSet).getPrecachedPersonsFromIDs isn't doing what I'd expect03:11
lifelessif we don't deliver the (thing) then its not optimisation anymore03:11
wgrantthumper: list()03:12
lifelessthumper: are you listifying it and passing need_validity=True ?03:12
wgrantthumper: You need to evaluate the result.03:12
thumperwgrant: yeah got that03:12
thumperand I ahve need_validity=True03:12
lifelessso what are you seeing03:12
thumperbut it is still doing the valid query for some of them03:12
thumperand if I add more people03:12
thumperthen it does shit loads more03:12
thumpereven getting some people03:12
thumperI'm wondering if I'm hitting the storm cache limit03:12
lifelesswhere are you doing the preloading ?03:12
lifelessthere is no cache limit03:13
thumpertest says different03:13
wgrantlifeless: Even in tests?03:13
lifelessits much more likely that something is triggering loading and doing a person:fmt before your eager load happens03:13
thumperI'm preloading in Sprint.attendances03:13
* thumper tries something03:13
lifelesswgrant: if you test with getUserBrowser, should be same as prod03:13
lifelessthumper: you're using the matcher for testing this, right ?03:14
thumperand the DatabaseFunctionalLayer03:14
lifelessBrowsersWithQueryLimit or whatever it was03:14
wgrantI wonder.03:23
lifelesswgrant: dangerous03:23
wgrantShould I turn recalculateBugHeat into something that adds a commit hook?03:23
wgrantAs a quick fix for all this heat badness.03:23
lifelesswhat do you mean03:23
wgrantAt the moment any change to a bug calls recalculateBugHeat, which then sets maxheat everywhere and blah.03:24
wgrantSome views call addChange several times.03:24
lifelessI don't know what you should do03:26
lifelessI can share the thoughts I had looking at this previously03:26
wgrantWell, I should move heat updating to a job.03:26
wgrantBut that's a bit hard.03:26
bachi thumper03:27
lifelessupdating the heat in a context should be exactly two single-row queries always.03:27
thumperbac: hi03:27
bacthumper: hey could you re-review my branch?  i addressed all of your issues.03:27
lifelesswgrant: [in the current design/implementation]03:27
thumperbac: diff is updating03:28
bacthumper: despite what i said in the earlier comment, i've removed the line-height altogether03:28
lifelesswgrant: the fact that its now looked like a very low hanging fruit03:28
LPCIBotYippie, build fixed!03:28
LPCIBotProject db-devel build #464: FIXED in 4 hr 44 min: https://hudson.wedontsleep.org/job/db-devel/464/03:28
thumperlifeless: if I reduce the number of people I add to the sprint, I get expected behaviour.  with 30 people, it is fine, with 50 I get some validation queries, with 100 I get people and validation queries03:28
thumperlifeless: so I'm guessing it is a caching issue03:28
lifelessthumper: I've done some pretty large datasets and not see that, but you might be right03:31
lifelessthumper: is it constant regardless of people in launchpad.dev (I use the harness to populate data for that sort of manual test)03:31
lifelesswgrant: does that make sense, or have I been to minimal in my description ?03:32
wgrantlifeless: It makes sense.03:32
wgrantlifeless: I guess it should be reasonably cheap, but it's still going to be somewhat duplicated.03:33
wgrantI'll fix the underlying stuff first.03:33
wgrant(mostly the fact that DSP's pk isn't (distribution, sourcepackagename), so Storm likes to reretrieve it lots)03:33
lifelesswgrant: things like: why do we query 'hottest bug' - if this bug has higher heat, update the cache, if not do nothing unless our heat lowered, and in that case [rare] query highest bug and set if and only if different03:33
wgrantlifeless: Right.03:35
lifelesswgrant: I don't see any reason to move heat to a job03:35
lifelesswgrant: I wouldn't object to it per se, I just don't see anything driving it03:36
thumperlifeless: so where is this storm cache defined?03:38
thumperrendered locally, a sprint with 100 attendees was 30 queries03:38
thumper200 attendees was 30 queries03:38
thumper400 attendees was 10203:38
thumperwondering if I was hitting the limit again03:39
lifelessheh, do we even use stormsugar.py03:39
thumperI was looking for it03:39
thumperbut didn't end up using it03:39
lifelessthumper: grep for StupidCache03:40
StevenKIs that seriously its name?03:40
thumperbac: I think you were too aggressive in your code removal03:40
thumperbac: with no link it should be a <span> and it is no more03:40
wgrantlifeless: Well, for one thing it seems pretty unwise to be locking Ubuntu during every change to an Ubuntu bug.03:43
lifelesswgrant: right, but only *one bug* can cause that03:44
lifelesswgrant: note that all bug updates will take a row lock preventing deletes on ubuntu03:44
wgrantlifeless: We hope.03:44
lifelesswgrant: if there are two bugs fighting for peak heat03:45
lifelesswgrant: we can deal03:45
wgrantStevenK: What are you doing to Jenkins?03:46
StevenKMoving it03:46
StevenKWell, moving its DNS name to lpci.w.o03:46
lifelessdoes bug expiry look a the project group ?03:48
lifelessif so, that would explain why its not working for launchpad03:48
lifelessand there is no ui for setting it on project gourp03:48
wgrantIt doesn't seem to use the project group setting.03:49
StevenKwgrant: It should announce URLs as lpci.w.o anyway. hudson.w.o is now a CNAME.03:50
lifelesswth defines 'fixed elsewhere'03:50
wgrantlifeless: Where have you seen that?03:50
lifelesswe support *per distroseries* bug expiry03:50
wgrantEr, what?03:50
wgrantDo we?03:50
lifelesshttps://lp-oops.canonical.com/oops.py/?oopsid=1857C157#longstatements query 403:50
StevenKThat seems like crack.03:51
StevenKPure crack.03:51
lifelessits mindblowingly insane03:51
wgrantlifeless: Er.03:51
wgrantThat's not per-distroseries bug expiry.03:51
wgrantThat's finding bugs in distroseries in distributions that are expirable.03:51
StevenKSigh, lp-oops DIAF03:52
StevenKIt always misbehaves for me03:52
wgrantAuthentication drops the query string, yes.03:52
lifelesshttps://bugs.launchpad.net/ubuntu/+bugtarget-portlet-bugfilters-stats - thats where the 'fixed elsewhere' thing shows up03:52
lifelesswgrant: thats true, but the query is nutjob noddy anyway03:52
wgrantlifeless: Welcome to Launchpad.03:52
StevenKwgrant: And https://lp-oops.canonical.com/ is the most unhelpful front page EVER03:53
lifelessextra_clauses.append("BugTask.bug IN " "(SELECT DISTINCT bug FROM BugCve)")03:55
lifeless-not win-03:55
huwshimiStevenK: Ouch, whatever server is running that site must be really hurting if that's running in debug mode03:56
wgranthuwshimi: That's devpad.03:57
lifelessthe fixed elsewhere queries are buggy too - they ignore product series bugs03:57
huwshimiwgrant: I'm guessing it doesn't have much ram free.03:58
StevenKMeh, it's only 220MiB into swap04:01
huwshimiwell we probably shouldn't be running it in dev mode anyway04:05
huwshimior rather debug mode04:05
lifelesshuwshimi: why not?04:05
huwshimilifeless: In debug mode Django holds every query in ram04:06
lifelesshuwshimi: since start up?04:06
lifelesshuwshimi: or for a request lifetime?04:06
huwshimilifeless: I think so04:06
StevenKThe only thing that isn't playing so nice with carob is the PPR04:07
lifelessgeneration ?04:07
lifelessyeah, its started failing again04:07
huwshimilifeless: Sorry I mean, I think it is since start up04:08
lifelesshuwshimi: right, well that could be cause for concern, yes.04:08
huwshimilifeless: Let me just check the docs04:08
huwshimilifeless: "It is also important to remember that when running with DEBUG turned on, Django will remember every SQL query it executes. This is useful when you are debugging, but on a production server, it will rapidly consume memory." from: http://docs.djangoproject.com/en/dev/ref/settings/#debug04:09
lifelessmy respect for django just increased04:10
lifelessso, could you please file a bug on oops-tools noting this?04:10
StevenKlifeless: You had some? :-)04:12
lifelessStevenK: its gone so far in one direction that things that make it worse, make it better04:12
StevenKwgrant: Last SPR processed by p-s-c was for edgy.04:13
wgrantStevenK: We are getting somewhere.04:13
StevenKThe OO.o SPRs are taking 1 minute.04:14
huwshimilifeless: We might have to check our other Django deploys too.04:18
huwshimi(I believe we have others)04:19
* wgrant stabs postgres in the eye.04:20
* mwhudson has one too....04:21
mwhudson(but it hardly uses the database so that might be ok)04:21
huwshimilifeless: https://bugs.launchpad.net/oops-tools/+bug/73732704:22
_mup_Bug #737327: django should not be running in debug mode on production, it will eat all the RAM <OOPS Tools:New> < https://launchpad.net/bugs/737327 >04:22
thumperlifeless: and other review poke04:24
* thumper goes to get friday takeaways04:24
lifelessthumper: reviewing04:30
lifelessthumper: I have suggestions fwiw, but nothing mandatory so far04:30
thumperok, I'll look later04:32
* thumper afk04:32
jtvwgrant: do you know where the contents of /srv/launchpad.net/ubuntu-archive/ubuntu-distscopy come from?04:38
wgrantjtv: It's maintained by the script.04:42
wgrantIt keeps two copies of the dists tree around.04:43
wgrantSo it can do reasonably atomic updates.04:43
jtvwgrant: what is "the" script?04:43
wgrant"the script" being cron.publish-ftpmaster04:43
wgrantIt grabs the backed up dists tree ubuntu-distscopy, and moves it to ubuntu/dists.new04:43
wgrantIt then runs the publisher on that.04:44
jtvI see it copying _from_ ubuntu-distscopy… does it write _to_ ubuntu-distscopy further on?04:44
wgrantSee cleanup() near the top.04:44
jtvSo this is a chicken-and-egg situation?04:44
wgrant# Create backup dists folder, if this is the first time.04:45
jtvThat copies _from_ ubuntu-distscopy04:46
wgrantThen it will move the empty folder to dists.new, swap dists and dists.new bring the new dists.new up to date, then move dists.new into ubuntu-distscopy04:46
wgrants/ bring/, bring/04:46
jtvBut publish-distro is failing to write a Release file there04:46
jtvNo, a freshly-created distroseries.04:47
wgrantDoes it have any ComponentSelections?04:47
wgrantYou need ComponentSelections in all your publishable series.04:47
jtvwhat's a component selection?04:47
wgrantOr you'll have no components.04:47
wgrantAnd the publisher will crash.04:47
jtv"Components?  But Andy Tanenbaum told me that Linux was monolithic!"04:47
wgrant(not optimal, but it's better than lucilleconfig)04:47
wgrantBug #67504204:48
_mup_Bug #675042: Release file generation fails for series without components <lp-soyuz> <soyuz-publisher> <Launchpad itself:Triaged> < https://launchpad.net/bugs/675042 >04:48
wgrantDoes that look like what you're seeing?04:48
jtvYes, except it's not in /var/tmp/archive.04:49
wgrantSo, try adding a ComponentSelection for main to your new series.04:49
jtvDoes the test publisher know how to do that for me?04:50
wgrantI doubt it.04:51
wgrantNot even the LOF does.04:51
jtvThe Line Of Flight?04:51
jtvLaunchpad Object Factory.04:51
jtvI'm tempted to fix the bug, but in this case that'd probably be hiding incomplete test coverage for cron.publish-ftpmaster, wouldn't it?04:52
wgrantIt would.04:53
* jtv sings hi-ho, making componentselections we go04:58
wgrantlifeless: Hi.05:04
lifelesswgrant: hi05:06
lifelesshi ho05:06
lifelesshi ho05:06
lifelessits off to work we go?05:06
wgrantHa ha.05:06
wgrantCould you please explain analyze query 106 from https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1902A2028?05:06
wgrantIt is a wonderful 9978ms abomination.05:06
wgrantIt takes about 30s on mawson, and adding a very sensible index brings it down to 2s.05:07
lifelesswhat bug05:07
wgrantBug #276950, probably.05:08
_mup_Bug #276950: DistroSeries:+queue Timeout accepting many packages queue page <lp-soyuz> <queue-page> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/276950 >05:08
wgrantAlthough this is probably only bad on GETs.05:08
lifelessstill running05:08
wgrantI guess the index was not terribly bad when it was created.05:09
lifelessstill going05:10
lifelessI'm going to go fiddle this portlet a bit more and check back in a bit05:10
lifelessok, on the bug05:11
lifelesshot is 600ms05:11
lifelesscold is 201945.17605:12
StevenKOver 2 minutes?05:13
lifelesswgrant: so you're adding distinct there05:13
wgrantOver three minutes.05:13
lifelessto stop redundant rows?05:13
wgrantI presume so.05:13
lifelesslets see05:14
lifeless53 rows05:14
lifeless720 rows without the distinct05:16
lifelesswgrant: so its doing about 10 times the work needed05:18
lifelesswgrant: because the non distinct version is still 600ms05:18
wgrantlifeless: It must think that a full scan is a better idea. It's probably right.05:19
lifelesswgrant: rephrasing it may work05:23
lifelesswgrant: its doing index scans05:24
lifelesswgrant: an index that permits doing distinct inline may make it a lot better, we can ask stub to play with that later05:24
jtvwgrant: I'm adding a makeComponentSelection to the factory.05:29
wgrantjtv: Why?05:29
wgrantthe constructor is fairly trivial.05:29
wgrantThe only bonus is omitting an import.05:29
jtvwgrant: makes it easy to say self.factory.makeComponentSelection(distroseries, "main")05:30
jtvAccepts component, component name, or none.05:31
jtvAlso, it means that yokels such as yours truly won't have to worry about just how trivial they are to create.  :-)05:31
jtvAnd meanwhile I'm on to the next hurdle: missing Sources files05:32
wgrantEven with a component?05:33
jtvIt may be the thing you ran into with my parallelization branch—it's a-f.05:33
jtvWARNING a-f: E: Could not open file /var/tmp/archive/distribution510022/dists.new/distroseries510031/main/source/Sources.bz2.new - open (2: No such file or directory)05:33
wgrantDoes that dir exist?05:34
jtv(Luckily I rigged this test with a "now tar up the temp directory so I can have a look" finale)05:35
jtvoh wait, this is in /var/tmp05:35
jtvIt'd be nice to avoid messing with /var/tmp, wouldn't it?05:36
jtvYuck, crud piling up there!05:36
jtvwgrant: the dists.new directory isn't present05:37
wgrantWhat's in /var/tmp/archive?05:37
wgrantBesides dozens of distribution dirs.05:37
jtvDozens of distribution-like dirs with "-cache," "-misc," "-overrides," "-partner," or "-temp" suffixed to their names.05:39
wgrantWhat evil have to done to cron.publish-ftpmaster? It sounds like it's not moving the backup in.05:40
wgrants/have to/have you/05:40
jtvI made it use a temp directory instead of /srv/launchpad.net, I sabotaged the lockfile usage, and I faked dsync-flist and commercial-compat.sh.  That's about it, really.05:42
wgrantecho "$(date -R): Moving backup dists into place..."05:43
wgrantDoes it do that?05:43
lifelesswgrant: so its about 300ms in the bpr lookup05:44
lifelessand 300-500 (depending on plan) in the bpph filtering05:44
wgrantlifeless: Right.05:44
lifeless210814 rows in bpph05:45
jtvwgrant: yes it does do "Moving backup dists into place…"05:46
wgrantjtv: Then why is dists.new not there? :/05:46
jtvwgrant: well we solved that one by adding the ComponentSelection didn't we?05:47
wgrant16:37:12 < jtv> wgrant: the dists.new directory isn't present05:47
wgrantdists.new's existence is not really related to ComponentSelection.05:48
jtvwgrant: but how does the (non-)existence of dists.new relate to this problem?05:49
jtvTo the Sources problem, I mean.05:49
wgrantOh, I guess it probably moved it back away by the time you checked...05:50
lifelesswgrant: I'd get stub to try that index05:52
wgrantlifeless: Which?05:52
lifelesswgrant: the one you said helped on mawson ?05:52
wgrantAh, right.05:52
wgrantIt still doesn't make it awesome. But I guess we'll see.05:53
lifelesswhats it on btw ?05:53
wgrantbpph(archive, distroseries, status)05:53
wgrantIt cuts the BPPH search time massively. So probably not much help in the hot case.05:53
wgrantEr, distroarchseries, but yeah.05:55
lifelessok, I'm going to stop fiddling05:56
lifelessthis is another case of overnormalisation05:56
wgrantThanks for trying.05:56
lifelessbpr's shouldn't be shared05:56
wgrantFor efficiency they should not.05:56
wgrantFor every other reason they should be.05:56
lifelessor we should do a fact table for this05:57
lifelessrather than chained content tables05:57
* lifeless handwaves05:57
wgrantThis is the sort of situation in which cross-table indices would work well. The underlying table is immutable, so there is no concern there.05:58
wgrantBut yes, we do reasonably urgently need to denormalise a bit.05:58
wgrantWe also need collection preload helpers.06:00
* lifeless sobs at related_projects in person.py06:03
lifelessit aggregates product pillar counts but nothing else06:10
wgrantBetter than nothing.06:10
jtvTime for the buying of the foodstuffs, ja?06:35
=== jtv is now known as jtv-eat
wgrantYay, only two CCW oopses so far today.07:32
lifelesswgrant: false positives still, or actual things-we-need-to-act-on ?07:53
wgrantlifeless: The latter.07:54
lifelesswgrant: excellent07:54
lifelesswgrant: well, excellent that we can tell.07:54
lifelesswhat is the issue?07:54
wgrantWell, they are sort of actual things. Some of our code is crashing, possibly due to broken remote bug trackers.07:56
wgrantBut they are real code bugs.07:56
wgrantFragile parsers, basically.07:57
jtv-eathi henninge, hi rvba08:27
=== jtv-eat is now known as jtv
lifelessI can has review? https://code.launchpad.net/~lifeless/launchpad/bug-711071/+merge/5395608:36
rvbalifeless: Hi Robert, I have two MP which include db changes, but I can't get a hold of stub these days ... maybe you could take a look at these (the db part)?08:48
lifelessrvba: both should be looked at by stub09:01
lifelessI will note though that this09:01
lifeless718- potemplate.distroseries.owner = self.factory.makePerson(09:01
lifeless719+ makePerson = self.factory.makePerson09:01
lifeless720+ potemplate.distroseries.distribution.owner = makePerson(09:01
lifelessis an ugly change09:01
lifelesspotemplate.distroseries.distribution.owner = \09:02
rvbaI'm still trying to change that09:02
lifeless    self.factory.makePerson(password='test')09:02
lifelesswould be better09:02
rvbathe "makePerson = self.factory.makePerson" is only here for not having a really long line09:03
lifelessindeed - see the \ instead09:03
rvbathat was my first move ... but I've been advised the opposite ;-) ... and made the change09:04
lifelesswho by?09:05
lifelessthere was a meme that \ is bad, but that is a broken meme09:05
rvbaJulian, the big guy :-)09:05
rvbaI don't mess why people that tall :-)09:05
lifeless(heck, 80 character limits are bad - we really should make usre of the realities of our development environments)09:06
lifelessrvba: so its neither here nor there09:06
lifelessrvba: I will only say, that seeing that, I would do a driveby fix if I was in the area09:06
henningervba: id do this:09:07
henningedistribution = potemplate.distroseries.distribution09:07
henningedistribution.owner = self.factory.makePerson(...)09:07
henningeHi jtv!09:08
rvbahenninge: this looks like a better option indeed09:08
rvbalifeless: any 'final thought' on this?09:11
lifelessrvba: what henninge suggests is ok09:12
lifelessrvba: however as I say, stub should look at these patches, he should be around AFAIK09:13
rvbalifeless: henninge thanks for the drive by advice :-)09:13
rvbalifeless: I'll ask him about the db part yes09:13
henningervba: np09:14
bigjoolsmorning all09:16
jtvhi bigjools09:17
=== almaisan-away is now known as al-maisan
lifelessbigjools: what are the specs of the machine you run lp tests on ? - and how long does it take?09:28
lifeless[to run everything, that is]09:29
bigjoolsI have 2 machines09:29
bigjoolsquad core Phenom, with 3 very sad cores doing nothing, takes 270 minutes09:29
lifelessthats amd right ?09:29
lifelessbut 4 core - moderately old?09:30
bigjoolsthe "quad core" i5 with SSD takes about 240 IIRC but not run it on there for a while09:30
wgrant270 minutes? Is that with or without Windmill?09:30
bigjoolsit's about 4.5 hours now09:30
wgrantWhen I got my new machine in December, it was slightly under 3 hours.09:30
bigjoolsyes, we have had a massive regression in time09:30
wgrantec2 is back down around 3.509:30
wgrantNot much worse than it used to be.09:31
wgrantSince I killed Windmill.09:31
=== adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: adeuring | https://code.launchpad.net/launchpad-project/+activereviews/
lifelessadeuring: https://code.launchpad.net/~lifeless/launchpad/bug-711071/+merge/5395609:36
adeuringlifeless: I'll look09:36
rvbaadeuring: Hi, I'd appreciate a review of https://code.launchpad.net/~rvb/launchpad/db-dds-diffpage-form/+merge/5391309:48
adeuringrvba: sure, let me just finish a review for Robert09:49
rvbaadeuring: sure09:49
bigjoolswgrant: my last ec2 took 4 hours09:58
wgrantbigjools: Mine take between 3:30 and 3:4510:00
bigjoolslifeless: sorry missed your question, yeah the Phenom is 2 years old now10:02
lifelessI'm considering a new desktop10:03
bigjoolsbut parallel test runs... could breathe new life into it :)10:03
lifelesswill want 2-4GB per core though10:03
bigjoolslifeless: if money is not a problem I'd get an i710:04
lifeless[because of silly overheads]10:04
bigjoolsor perhaps there's better now10:04
lifelessbigjools: I have a gen-1 i7, limited to 6GB memory10:04
bigjoolsthey're not slack :)10:04
lifelessbigjools: runs like a rocket, but nowhere near enough memory to use all the cores well10:04
lifelessnot even for compiling drizzle ;P10:04
bigjoolslifeless: memory contention?10:05
bigjoolsbus ^10:05
lifeless8 g++ instances >> 6GB of ram, for instance10:05
adeuringlifeless: r=me10:08
lifelessadeuring: thanks!10:09
lifelessshould fix 80 timeouts/day10:09
lifelesshuh, thats oddd10:10
lifelessthere is no 'set commit message' widget on https://code.launchpad.net/~lifeless/launchpad/bug-711071/+merge/5395610:10
lifelessor am I crazy?10:10
lifelessthere is now. weird10:12
=== al-maisan is now known as almaisan-away
=== almaisan-away is now known as al-maisan
deryckMorning, everyone.10:57
=== al-maisan is now known as almaisan-away
=== almaisan-away is now known as al-maisan
rvbastub: thanks for your review on my branch (lp:~rvb/launchpad/db-distroseries-migrate-owner-to-registrant) ... a question though:11:03
rvbastub: patch-2208-53-0.sql was adding a new 'registrant' column to distribution. This branch renames the 'owner' column into registrant for distroseries. Could you elaborate on where the conflict is?11:03
stubrvba: The conflict is in my mind then :)11:04
stubI'll change that now11:04
rvbastub: all right thx ;-)11:04
adeuringrvba: r=me11:22
allenapderyck: Javascript tests, lib/lp/registry/javascript/tests/test_milestone_table.html for example, are messed up; the skin appears missing. Applying http://paste.ubuntu.com/582035/ fixes it. I'm wondering if my build changes last week broke this. Do you know much about this?11:24
rvbaadeuring: thx for the review, just pushed the changes you suggested.11:30
adeuringrvba: great, thanks!11:31
rvbastub: thank you for your reviews. I just pushed the modifications you requested.11:31
=== al-maisan is now known as almaisan-away
deryckallenap: I think the CSS was broken because we changed to fetchCSS: false in the test. I never worked out what CSS it wanted because it was just trying the Yahoo CDN with some massive list of CSS.11:37
deryckallenap: so if adding that file link fixes it, awesome!11:37
deryckallenap: we had to set fetchCSS false to better play with Windmill as the test runnner.11:38
allenapderyck: Cool. As long as it wasn't me :) I might @import it from test.css.11:38
deryckallenap: sounds good.11:38
allenapThanks for your help.11:38
derycknp.  Thank you for working on this!11:38
wgrantjml: Your assignment of that bug is encouraging.11:39
jmlwgrant: I'm just looking into it right now. No amazing insights.11:39
wgrantOh :(11:39
wgrantWell, good luck...11:39
jmlwgrant: do you have any clue as to why the comment says "run a job that will not time out first, followed by a job that is sure to time out." when the code seems to only run one job?11:39
stubrvba: Looks fine11:40
wgrantjml: StuckJob is two jobs.11:41
jmlwgrant: ta.11:41
wgrantSee iterReady... the first one has an arg of 1, the second 2. Only when the arg is 2 does it sleep.11:41
jmlyeah. I see now.11:41
wgrantIt is not really sensible.11:42
jmlit seems a bit indirect.11:42
jmlhuh. If I'm reading this correctly, TwistedJobRunner only runs one job at a time.11:47
lifelessnight y'all12:12
bigjoolslifeless: nn - just replying to your ha uploaders email BTW12:13
lifelessbigjools: cool,thanks12:13
bigjoolsnearly done if you want to wait :)12:14
lifelessbigjools: take the time to reply fully12:27
lifelessbigjools: I'm just lining up the ducks to point losas at after we finish the appserver capacity work, so there isn't a panic - we can knock ideas back and forth for maybe a couple of weeks without impacting schedules12:27
* lifeless halts()12:28
bigjoolslifeless: np at all12:29
wgrantjml: No luck?12:32
=== henninge is now known as henninge-lunch
wgrantOr perhaps anti-luck?12:32
jmlwgrant: some success.12:32
jmlwgrant: test_shhh manipulates sys.path on import and doesn't clean it up, which is why the _pythonpath thing doesn't bite us on the full suite run.12:33
wgrantjml: Ahhh.12:33
jmlampoule tries to get the correct sys.path for the "must be available" packages by importing them in the current process and manipulating their path.12:34
wgrantI got that bit.12:34
wgrantIt's a bit odd.12:34
jmlwent down a bit of a rabbit hole assuming it was something to do with zope.testing respawning.12:34
wgrantAs did I, but then I deleted some stuff to prevent that and it was still broken.12:34
wgrantI didn't think that test_shhh might have been doing path evil.12:35
jmlI'm going to change test_timeout to temporarily extend sys.path and fix up the test_shhh thing in another branch.12:38
jml... and file a bug about providing a decent mechanism for doing this with python-fixtures.12:39
gmbadeuring: Are you accepting reviews today?12:49
=== bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: adeuring, bac | https://code.launchpad.net/launchpad-project/+activereviews/
bacgmb: i am if adeuring isn't12:49
adeuringgmb: yes, but i'd like to start my lunch break right now.12:49
gmbadeuring: Ok, no worries.12:49
gmbSo, bac. Hi :).12:50
gmbbac: https://code.launchpad.net/~gmb/launchpad/non-js-muting-bug-734732/+merge/53981 if you please12:50
bacgood morning12:50
=== salgado-afk is now known as salgado
bachi mrevell13:12
mrevellHello bac13:12
* jml sets the test to run 50 times over and takes a brief programmer's break13:13
=== almaisan-away is now known as al-maisan
=== henninge-lunch is now known as henninge
abentleyderyck: I'm trying to use jquery, but I can't figure it out.  The queries in lib/lp/code/windmill/tests/test_recipe_index.py succeed on my page, and they shouldn't, because it's a different page.14:36
abentleyderyck: e.g. self.client.waits.forElement(jquery=u'("div#edit-build_daily a.editicon.sprite.edit")', timeout=5)14:36
deryckok, that's a bit scary.14:37
deryckabentley: you don't get an error with that on your page?14:37
abentleyderyck: No.14:37
abentleyderyck: it's green.14:38
abentleyderyck: I'm using a pdb session to test.14:38
abentleyderyck: meanwhile, $("ul#branch") fails, even though firebug identifies an element that way.14:39
abentleyApparently, the $ makes the difference.14:40
deryckabentley: can you commit the test you have to your branch so I can pull and run it?14:42
abentleyderyck: sure, one sec.14:42
abentleyderyck: pushed to bzr+ssh://bazaar.launchpad.net/~abentley/launchpad/js-translation14:44
deryckabentley: ack.14:44
deryckoh yippee, another Windmill hang!14:44
abentleyderyck: run with bin/test -t test_sharing_details windmill --layer=WindmillLayer14:44
deryckabentley: let me get a bt on my current Windmill hang, and then I'll kill it and look at yours.14:45
abentleyderyck: thanks!14:45
gmbadeuring, bac: I have a minor JS branch for review, if you'd be so kind: https://code.launchpad.net/~gmb/launchpad/fix-js-errors-bug-737557/+merge/5400714:59
bacgmb: i'll do it14:59
adeuringgmb: I'll look14:59
baci win!14:59
adeuringbac:  ok ;)14:59
gmbThanks :)15:00
gmbbac: https://code.launchpad.net/~gmb/launchpad/fix-js-errors-bug-737557/+merge/5400715:00
gmbSorry, forgot I'd pasted it.15:00
deryckabentley: ok, sorry.  got what I need now.  Turning to your branch.15:11
bacsinzui: are you saying you want me to put the line-height: 20 back in?15:11
sinzuibac: no. I do not want that instruction anywhere in the code base15:13
bacsinzui: good -- it's gone15:14
sinzuiSorry about the confusion. I got very distracted yesterday evening. I composed my reply to you, but never sent it15:15
deryckabentley: ok, so this has me terrified of the reliability of jquery selectors now.15:37
abentleyderyck: sorry.15:38
deryckabentley: if you try to click jquery=u'("div#edit-build_daily a.editicon.sprite.edit")' it will fail.  But using waits.forElement or asserts.assertNode, it passes.15:38
deryckabentley: which clearly makes no sense and is wrong15:38
leonardradeuring or bac, can you review https://code.launchpad.net/~leonardr/lazr.restful/operation-sanity-check/+merge/54014 ?15:40
abentleyderyck: so avoid using with those?  I've been plugging away using xpath, and it's... beauty-challenged.15:40
adeuringleonardr: i'll look15:40
deryckabentley: I'm not sure what to recommend at this point.  xpath is equally problematic. You often get false positives the same way, from crafting xpath expressions you don't understand.15:41
leonardradeuring: i'll fill in some background in a comment15:41
leonardrsince the feature is new15:41
adeuringleonardr: great, thanks!15:41
deryckabentley: or fragile tests from poorly crafted xpath.15:42
deryckabentley: and there is no way to hang IDs off this html?  because of the menu stuff.15:42
abentleyderyck: I wonder if any of our intermittant failures are from waits.forElement(jquery= ?15:42
deryckabentley: if it's a failure and not a hang, that's easy to check -- i.e. was it a jquery line?  Hangs don't appear related to this at all.15:43
LPCIBotProject devel build #553: FAILURE in 4 hr 58 min: https://lpci.wedontsleep.org/job/devel/553/15:44
=== al-maisan is now known as almaisan-away
abentleyderyck: Yes, not hangs.  But failing to wait long enough would cause a failure on a later line.15:44
deryckindeed, it's something to look out for.15:45
leonardradeuring: background is added15:45
deryckabentley: but we're only using it in recipe tests at this point, I think.15:45
abentleyderyck: I could use ids, but classes seemed a better approach, because I could consistently use the same value-- class=incomplete, not id=branch-incomplete15:45
deryckabentley: I don't see an issue with specific IDs.  Reuse is not always a virtue when dealing with the DOM and js and js testing.15:46
abentleyThen I can do foo=Y.one('#branch'), foo.one('.incomplete')15:46
abentleyIt just makes it simpler once you've got the checklist node.15:47
deryckbut you can hang id="branch-incomplete" off the element just for the sake of testing.  sure, it's hacked on, but if the test is less fragile.15:48
deryckabentley: I leave it to you, though.  Just warning that xpath is usually a source of trouble at some point.  But maybe this is easy enough xpath.15:49
abentleyderyck: I would rather not have both class=incomplete and id=branch-incomplete.  That's a DRY violation to me.15:49
deryckabentley: sure, but I'm arguing DRY is not that big a deal in HTML. ;) :)15:50
abentleyderyck: But I hadn't realized the testing implications before, so perhaps it's better to change my approach now.15:50
deryckabentley: yeah, HTML is not code. ;)15:50
abentleyderyck: i'm saying if I add ids, I'll remove the classes.  No need for both.'15:51
deryckabentley: but I do agree it's generally better if you can avoid.  less characters, less to download.  but our testing story is weak without the ids.  and you could just use branch-incomplete and not the class.15:51
deryckagreed ^^ :-)15:51
deryckabentley: ok, so I'll lunch now, if you've got everything you need.15:53
abentleyderyck: anyhow, I think I have to use xpath with the picker.15:53
deryckabentley: maybe so.  this is one area where we've had to use it and can't avoid it in the past.15:53
abentleyderyck: cool.  Thanks.  Enjoy lunch.15:54
deryckok, cool.  thanks!15:54
=== deryck is now known as deryck[lunch]
gary_postermrevell, hi.  I'm trying to find the accordion prototyping round 2 version 2 slides16:12
gary_posterI found https://dev.launchpad.net/LEP/BetterBugSubscriptionsAndNotifications/Testing/AccordionPrototypingRound216:12
=== salgado is now known as salgado-lunch
gary_posterwhich has version 216:13
mrevellAh, glad you got it gary_poster16:13
gary_posterbut I can't find version 216:13
gary_posterand https://wiki.canonical.com/Launchpad/UserResearch/BugSubsFeb2011/Round2 points to the wrong location (round 1)16:13
mrevellOh, sorry16:13
mrevellLet me look16:13
gary_posterthank you16:13
gary_postermrevell, found it somehow or other :-)16:16
mrevellgary_poster, I've created a page at https://dev.launchpad.net/LEP/BetterBugSubscriptionsAndNotifications/Testing/AccordionPrototypingRound2/Version2 which links to the 5 slides and I've updated the wiki page16:16
mrevellon Canonical16:16
gary_posterawesome thank you mrevell16:16
mrevellno probs :)16:18
adeuringleonardr: r=me. Thanks for the new checks and for the explanation!16:23
sinzuibac: will you have time to review https://code.launchpad.net/~sinzui/launchpad/tea-and-biscuts/+merge/54041 today?17:08
=== deryck[lunch] is now known as deryck
=== salgado-lunch is now known as salgado
=== adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: bac | https://code.launchpad.net/launchpad-project/+activereviews/
=== beuno is now known as beuno-lunch
abentleyderyck: turns out there's some picker logic in testing.windmill.widgets, so we don't need to use xpath for it.18:01
deryckI forgot about that actually.  Sorry.18:02
abentleyderyck: np18:03
abentleyderyck: how would you test for the absence of a class on a node?  Currently, I'm doing self.client.waits.forElement(xpath='//*[@id="branch"]/*[@class="complete sprite yes"]') to ensure the node is not unseen.18:07
deryckabentley: that looks fine. Are you worried that by not being explict about the absence of unseen that the test will pass but not be correct?18:10
abentleyderyck: I'm just wondering if you would avoid using xpath for that, and if so, how.18:10
deryckabentley: well, I would have mad the ID "branch-complete" or given it some other unique ID, to avoid the xpath.  As we talked about eariler.18:12
abentleyderyck: okay, but if you did that and did forElement(id="branch-complete"), how would you know what the class was?18:13
jcsackettleonardr: ping18:13
leonardrjcsackett, yo18:13
jcsacketthi. do you have a few minutes to mumble or skype about webservice stuff?18:14
deryckabentley: I wouldn't test that in Windmill.  I would do that in the YUI test.  I wouldn't worry much about DOM in the Windmill test. You really should only care the data was saved correctly.18:14
leonardrjcsackett, sure, 1 minuteeeeeee18:15
jcsackettleonardr: thanks.18:15
abentleyderyck: If I don't wait until the dom is updated, how do I know when it's safe to check the model?18:15
deryckabentley: do you not add some class when you remove the unseen class, i.e. changing it to "seen"?18:16
sinzuideryck: all pickers are using the wrong headings. I do not think the instructions should be a heading at all in fact. That will make the text smaller18:16
derycksinzui: agreed.  Just plain on text.18:16
abentleyderyck: No, I don't add a "seen" class.  But I could use the other element, where I add "unseen".18:17
sinzuideryck: I can start this in a hour or so...I was irked by the very add-member issue this morning when testing18:17
deryckabentley: yes, that would work.18:17
abentleyderyck: Okay, how would that work?18:17
abentleyderyck: forElement(id="branch-incomplete") still won't wait until the DOM is updated.18:18
deryckabentley: I'm getting confused.  :-)  I thought you just said you could wait for the unseen class on the other element.  so waits.forElement(blah blah blah about the other element now).  That won't work?18:19
derycksinzui: thanks!  Did something change recently to cause this?18:20
abentleyderyck: I've got lots of elements that have the unseen element, so waits.forElement(class="unseen") will not wait until branch-incomplete gains the class.18:20
leonardrjcsackett: expose_as_webservice_entry()18:20
abentleys/unseen element/unseen class/18:20
sinzuideryck: sort of. Heading sizes were updated to be like the guidelines. but in this specific case, the instructions exceeded the default width of the picker last December when we fixed security in team membership18:21
abentleyderyck: it's fine if the answer is "I would use xpath in that case".18:22
deryckabentley: can you craft an xpath expression or jquery selector?  That would be my suggestion, yes.  Use one of those.18:23
derycksinzui: ah, gotcha.18:23
abentleyderyck: I am already using an xpath expression, so I'll just keep using it, then.18:23
deryckabentley: ok.18:25
abentleyderyck: you said to avoid xpath where possible.  I was just exploring where that's possible.18:28
deryckabentley: yeah, maybe it's not possible here.  I'm always worried when we're checking for the visibility of an element.  But maybe there's no other way to know the page changed.18:29
abentleyderyck: when the page changes, one element becomes visible, the other is hidden, and the href and contents of a link are changed.18:31
deryckabentley: so I'd check for the contents change then.18:34
deryckabentley: textIn18:34
deryckah, crap, that's an assert18:36
abentleyderyck: all the uses I see are client.asserts.assertTextIn.  Is there a waits version?18:36
deryckabentley: yeah, that's what I just noticed.  Let me look18:37
deryckabentley: so that won't work.  Sorry.  You could do waits.forElementProperty to do the classname or href nicer than xpath.18:40
abentleyderyck: Ah, okay.18:40
=== beuno-lunch is now known as beuno
jmlOK. I think I've got this failure reproducing consistently.19:20
jmlTime to make some food and then track the bugger down.19:20
deryckHave a nice weekend, everyone.20:04
bachi jcsackett, where do i see the diff overlay you mention?20:22
bacjcsackett: nm, i found it20:25
jcsackettbac: good thing. i never got the ding i usually get to let me know someone's pinging me. :-)20:31
bacnice branch jcsackett, r=ac20:41
bacer, r=bac20:41
jcsackettthanks, bac.20:42
jcsackettsinzui: standup at 5:30 or at 6, given our other members are enjoying the weekend?20:44
sinzuiWe can do it now if you like20:44
jcsackettsinzui: right now i am at a coffee shop. i'm figuring out when i need to make the short walk back home. :-)20:46
sinzuiPing me when you are ready20:46
jcsackettsinzui: righto.20:46
=== bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews/
leonardrwallyworld, do you know where thumper is? i'd like to do the standup and then take a nap21:07
leonardrunless... oh, wait, it's saturday for oyu, isn't it?21:08
sinzuileonardr: yes it is21:08
sinzuitake a nap21:08
leonardrok, never mind then. nap time! talk to you on monday/tuesday21:08
jcsackettsinzui: i am now in a mumble friendly place. i can do standup whenever you like.21:16
jmlI think ampoule handle timeouts badly21:16
sinzuijcsackett: starting mumble21:16
=== salgado is now known as salgado-afk
LPCIBotYippie, build fixed!22:58
LPCIBotProject devel build #554: FIXED in 5 hr 16 min: https://lpci.wedontsleep.org/job/devel/554/22:58
LPCIBotProject windmill build #72: FAILURE in 1 hr 16 min: https://lpci.wedontsleep.org/job/windmill/72/23:33

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!