gary_posterbac benji danilos gmb: call in 112:29
* benji goes to the utility drawer to find a nine volt battery to satiate this chirping smoke detector.12:41
benjimuch better12:45
gary_postergmb, do you have time to throw a drowning man a line? via Skype? :-)13:37
gmbgary_poster: Can you give me 10 minutes to just grab a drink?13:40
gary_postersure gmb :-)13:40
gmbgary_poster: Cool. I'll ping you shortly.13:40
gary_posterthank you13:40
gmbgary_poster: Ready when you are13:49
bachi benji -- you free to chat?  i've been looking for good ways to screen share, etc but have nothing13:53
gary_postergmb, sorry, missed that.  calling...13:55
bacbenji: skype?13:56
benjisounds good13:56
bacbenji: bin/test -vvt test_mark_duplicate_query13:58
bac bin/test -vvt test_mark_as_duplicate_query13:59
bacbin/test -vvt test_mark_duplicate_query_count14:02
bacbenji: pushed fixes to lp:~bac/launchpad/bug-813322-214:33
bacbenji: and the correct failing test is:  bin/test -vvt test_mark_as_duplicate_query_count14:34
benjibac: looking now14:35
gary_postergmb, someone added something in the template that disables the expander if there are more than 10 bugtasks.  The fields are still calculated, and rendered, but the expander is not drawn.  That could be an easy thing to change.  But...15:00
gary_posterwhat do you do if you actually need to change something for a bugtask in that collection?  Doesn't that seem like a broken behavior--or is there a workaround?15:00
gary_posterIt strikes me that I could improve this by doing the "draw the form on demand" after all.15:00
gary_posterThat would make the big story more functional, and the small and big story faster15:01
=== Ursinha` is now known as Ursinha
=== Ursinha is now known as Guest8410
=== Guest8410 is now known as Ursula
=== Ursula is now known as Ursinha
gary_posterAnd CHR is done!  Happy day.15:13
gmbgary_poster: Sorry, completely missed the fact that Colloquy had disconnected...15:17
* gmb reads backscroll15:18
gary_poster:-) np15:18
gmbgary_poster: Well, bug 1 is an example of this (I was wondering why the expanders went away; this explains it). If you click on any of the edit icons on the bug 1 bugtasks table you get redirected to the bugtask:+editstatus page. (i.e. exactly what used to happen before the expanders).15:20
_mup_Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Confirmed for compscibuntu-bugs> <dylan.NET.Reflection:Invalid> <dylan.NET:Invalid> <EasyPeasy Overview:Invalid by ramvi> <GenOS:In Progress by gen-os> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <LibreOffice:In Progress by bjoern-michaelsen> <Linux Mint:In Progress> <The Linux OS Project:In15:20
_mup_Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Confirmed for compscibuntu-bugs> <dylan.NET.Reflection:Invalid> <dylan.NET:Invalid> <EasyPeasy Overview:Invalid by ramvi> <GenOS:In Progress by gen-os> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <LibreOffice:In Progress by bjoern-michaelsen> <Linux Mint:In Progress> <The Linux OS Project:In15:20
gmbgary_poster: So essentially we degrade nicely already. Changing things so that all the unnecessary work isn't done shouldn't break anything,  I expect.15:20
gary_postergmb, ok so keep current behavior but don't do unnecessary Python work.  cool15:22
gmbgary_poster: Indeed. And we can then (probably) punt on the "Make it nicer and then faster" work if we want to.15:22
gary_posterright gmb.  That sounds like a win for everything except the future :-)15:22
gmbgary_poster: Indeed. And if the future involves the Yellow squad we may yet regret it :)15:23
gary_postercross that bridge then and all that :-)15:23
gary_posterok I'll see if I can pull this off...15:23
gmbgary_poster: Ping me if you need to pull at any weird threads on that code.15:24
benjibac: I've found that just under 90% of the queries are generated by eight or nine repetitions of very similar sets of queries over and over15:24
gary_postercool thanks gmb15:24
bacbenji: skype?15:24
benjibac: http://paste.ubuntu.com/662731/15:26
gary_postergmb, one-line change makes code go from 6 seconds to 2 ;-)15:44
gmbgary_poster: Want me to review it?15:44
gmbOr are you going to do Other Clever Things.15:45
gary_postergmb, yeah thanks.  Lemme see if I get this up quickly--I have stuff I need to clean up.  No, I intend to run away at the mention of Clever Things--or at least walk on a treadmill at a stately pace.15:46
gmbVery wise.15:47
benjibac: it looks like the repeats aren't identical, but they only vary by bug ID15:53
bacbenji: actually i see a lot of *identical* queries15:55
baclet me paste15:55
benjibac: are you logging the statement only or the parameters too, there are only about two dozen unique queries, but the parameters vary15:56
bacbenji: http://pastebin.ubuntu.com/662767/plain/  -- the numeric prefix is the number of times the query was seen15:58
bacbenji: i'm stepping out to get a bite to eat.16:00
benjithat is a lot of dupes; I'm dissapointed that there isn't some sort of caching scheme that keeps those from happening16:00
benjik, I'm going to do the same in a while16:00
bacbenji: would love to talk to you about how we can accomplish that...16:00
benjion the other hand, even if we got rid of all the dupes, that would still be 258 queries16:00
gary_posterthat number is not unheard of16:11
benjibac: (for when you get back), the call to Snapshot on line 1834 is responsible for 7 queries for each duplicate and the next line generates 3 more, so that's 10 queries per duplicate right there16:16
benjithat's in lib/lp/bugs/model/bug.py16:16
benjiwow, then the subscribers that are fired on the notify(event) line trigger 31 more queries (per duplicate bug)16:19
gary_postergmb, https://code.launchpad.net/~gary/launchpad/bug724025/+merge/71076 when you are ready.  I ended up implementing it with a view function rather than a one-line template change so I could add comments and such, but it is still pretty short.16:22
gmbgary_poster: Righto, looking now.16:22
gary_posterthank you16:22
gmbgary_poster: Good stuff. Approved.16:24
gmb\0/ DebBugs works again!16:25
gmbLet joy be unconfined.16:25
gary_poster:-) thanks gmb16:25
gary_posterlunch, and a bank visit, in triumph ;-)16:28
bacbenji: interesting17:27
benjibac: if I skip creating the snapshot and firing the event (either of which may not be practicle) the test is down to 55 queries, of which 22 are updating bug heat17:33
benjione thought would be to defer firing the object modified event until email sending time (or if we have a reasonable way to create jobs, queue up a job for it)17:34
bacbut the notify is what populates the database for the cronjob to later chew on17:36
benjibac: I haven't looked at the subscriber, but I doubt it's very complicated, right?  We could either just have a function that we call that records the fact that we need to fire the duplication event and that the email should be sent (or they could be two different flags, preferably) or we could fire a new event that has a subscriber that does what the aformentioned function would do17:39
benjithis is starting to get into the pattern of "if you know their request will succeed, just record the fact that the user asked for X and tell them it worked and then do X ASAP, but out of band with the web server"17:41
benjiwhich is a good pattern, but not one that we do much of, as far as I am aware17:41
bacbenji: the event -> db -> action chain is pretty convoluted.  i'm not convinced the even mechanism really buys us much.  but, in order to avoid db changes, i worked within that framework17:44
bacbenji: perhaps that was a bad decision17:44
bacif you look at the diff of my branch it'll be clear the work that was done to close the loop.  it was pretty minimal17:45
benjiit's conservative to fire an object modified event when you modify an object, but if none of the subscribers care (and we're not too worried about introducing subscribers that do care in the future) then we can skip that part17:45
bacbenji: but we do care, as the subscriber is the one who stuffs the deferred bug notification into the db18:07
benjibac: right, but that's "our" subscriber, we can just call that code directly (no need for a subscriber) because we know we need it to happen now, the other subscribers may be less important (but I bet we still want them to run eventually)18:09
benjiI'm investigating other things, but a deferred execution model seems to fit this really well.18:09
bacdarn, benji is gone19:21
bacwell, gary_poster, i'll brag to you19:21
gary_posterI like bragging!  go for it!19:21
bacbased on benji's suggestion i removed the Snapshot and the use of the event model19:21
bacqueries down from 467 to 9119:21
gary_posterhah!  wow!19:22
gary_posterand things still work, I take it? ;-)19:22
bacnot sure it *works* but that's a detail19:22
gary_posterbac, but I assume the basics work of what you were doing before, so sounds very good so far :-)19:23
bacyeah, i hope so19:24
baci'll verify it works now19:24
gary_postercool, lemme know19:24
baci think Snapshot may be possessed19:24
benjidaylight come and me wan' go home19:30
bacbased on benji's suggestion i removed the Snapshot and the use of the event model19:31
bacqueries down from 467 to 9119:31
benjiGood news everyone: routers work better with electricity.19:31
benjibac: cool!19:31
gary_posterbenji, bac hasn't said whether it actually works yet though.  He has another branch in which he has deleted the code tree and the query count has dropped even more.19:33
benjibac: while sitting in the twighlight of my unpowered home I dug into the subscribers some more, the main culprit is add_bug_change_notifications19:33
bacyes, i determined postgres is the root of our problem19:34
bacare either of you getting the smoke from the dismal swamp fire?19:34
benjiI can tell you that ZODB doesn't have these problems.  It has *other* problems, but not these.19:34
benjiI haven't been outside since my morning bike ride and I didn't notice any then.19:35
gary_posterno smoke for me19:36
benji"Great Dismal Swamp" is the perfect name for a swamp19:36
gary_posternext to the Really Big Mountain.19:36
benjiwhich overlooks The Especially Wet Lake19:36
gary_posterlol, very nice19:37
benjithese seem like place names from The Princess Bride19:37
benjithe more I look at this, the more I think it's crying out for an async approach; we shoudl be doing all this work to figure out what changed and who cares after the actual request has finished19:43
gary_posterthat's what bac is working towards, in the small19:44
benjihmm, or a long-poll type thing, but we'd still run afoul of the timeout19:44
benjiif we had a simple and reliable way to schedule work for later, I suspect we could fix lots of timeouts and otherwise slow pages with this particular hammer19:45
gary_posterThe job system is the chosen hammer19:45
gary_posterI don't hink it is super simple19:46
gary_posterit's fairly reliable19:46
* bac -> chr20:02
bacgary_poster, benji: my changes *appear* to work when exercised via lp.dev.  proper tests are now the focus.20:09
bacbenji: did you do chr?  there was practically nothing there20:10
benjibac: nope, my power was out at the time and then I forgot20:10
benjiI'm glad there wasn't a backlog20:10
baci am releasing myself on my own recognizance.  be back later.20:17
gary_posterdoesn't sound safe to me.20:32

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!