| StevenK | So, you have a dastardly plan? | 00:17 |
|---|---|---|
| StevenK | wgrant: & | 00:17 |
| * StevenK grumbles at his fingers. | 00:17 | |
| wgrant | StevenK: Yeah, testing it on DF | 00:25 |
| * StevenK resurrects the redux branch | 00:38 | |
| wgrant | StevenK: I have an equivalent that is 1ms rather than 1000ms, but I'm not sure that it is truly equivalent yet. | 00:52 |
| StevenK | wgrant: That was the problem? The query was taking 1 second per notification? | 00:54 |
| wgrant | StevenK: Calculating the structural subscribers for a private bug took 600-1200ms | 00:55 |
| wgrant | StevenK: And because the notification code is crap, it ran that calculation several times for a single operation. | 00:55 |
| wgrant | I have a query which fixes the slowness | 00:55 |
| wgrant | And I might refactor it all later to remove the duplication | 00:55 |
| wgrant | Since four separate bugnotificationrecipient timeouts plagued my last week | 00:56 |
| StevenK | wgrant: Ah, so the real smoking gun is the change in model/structuralsubscription.py | 00:57 |
| wgrant | StevenK: I believe so | 00:58 |
| wgrant | StevenK: Though dupe/etc subs may be similarly afflicted, they weren't important in this case. | 00:58 |
| wgrant | (probably since private bugs with dupes are relatively rare) | 00:58 |
| wgrant | StevenK: I've added the full suite of APGs on DF, so performance testing there should be more accurate. | 01:15 |
| * StevenK stabs django for lying | 01:22 | |
| lifeless | StevenK: oh? | 01:25 |
| StevenK | lifeless: Figured it out. The docs could be clearer about lots of things. | 01:28 |
| * lifeless is no wiser | 01:29 | |
| StevenK | lifeless: Adding commands that manage.py can execute | 01:30 |
| wgrant | Oh | 01:42 |
| wgrant | EXPLAIN ANALYZE gets a bit confused when dealing with UNIONed subqueries | 01:42 |
| wgrant | -> Index Scan using teamparticipation_team_key on public.teamparticipation (cost=0.00..30.16 rows=8 width=8) (actual time=0.016..0.173 rows=56 loops=5) | 01:43 |
| wgrant | Output: public.teamparticipation.id, public.teamparticipation.team, public.teamparticipation.person | 01:43 |
| wgrant | Index Cond: (public.teamparticipation.team = (2794)) | 01:43 |
| wgrant | What it actually means is "Index Cond: (public.teamparticipation.team = [result of UNIONed subquery that happens to start with a literal 2794])" :( | 01:43 |
| StevenK | Hah, helpfl | 01:43 |
| StevenK | *helpful | 01:43 |
| lifeless | wgrant: really ? | 01:45 |
| lifeless | wgrant: can I see the full explain analyze + original query ? | 01:45 |
| wgrant | lifeless: https://pastebin.canonical.com/71501/ | 01:47 |
| wgrant | Observe the last couple of lines of the plan | 01:47 |
| wgrant | And how they make no sense unless it actually means the full UNIONed result | 01:48 |
| wgrant | Note also that the output of the HashAgg is '(2794)' because of that | 01:50 |
| wgrant | (ignore the Hash Semi Join... I'm trying to work out what it's doing there) | 01:52 |
| lifeless | What happens if you replace the select constant as grantee | 01:55 |
| lifeless | with select grantee from (select constant as grantee) | 01:55 |
| lifeless | I wonder if its misplanning entirely | 01:55 |
| lifeless | -> worried | 01:55 |
| wgrant | Nah, it just optimises that away | 01:56 |
| wgrant | The plan seems correct apart from the hash semi join. | 02:02 |
| wgrant | Which duplicates the other subplan, which is largely identical and more efficient | 02:02 |
| wgrant | I don't understand why it thinks it also needs the hash semi join | 02:02 |
| lifeless | 11294 Time Outs | 02:09 |
| lifeless | 6415 / 0 Branch:EntryResource:getMergeProposals | 02:09 |
| lifeless | ^ regression methinks | 02:09 |
| wgrant | lifeless: That's the bug you triaged a week ago | 02:09 |
| lifeless | ah' | 02:09 |
| wgrant | (and yes) | 02:09 |
| lifeless | i'm getting old in my young age. | 02:09 |
| spm | you have a beard. bearded people are old. qed. | 02:12 |
| spm | a 9yo expert in old people told me that. | 02:12 |
| wgrant | spm: He forgot that already | 02:12 |
| wgrant | Due to his age. | 02:12 |
| spm | clearly | 02:12 |
| StevenK | spm: '9yo expert' is a tautology | 02:13 |
| spm | not from the 9yo's perspective. | 02:13 |
| spm | ^^ relativity | 02:13 |
| lifeless | thats special | 02:13 |
| wgrant | StevenK: So, mind if I take over the structsubs branch? | 02:13 |
| StevenK | wgrant: ... to do what? | 02:13 |
| wgrant | StevenK: Make it fast. | 02:14 |
| wgrant | StevenK: I mean the private structsubs branch | 02:14 |
| StevenK | wgrant: If you like | 02:14 |
| wgrant | Not the information type one | 02:14 |
| StevenK | The model code has landed already. | 02:14 |
| StevenK | The UI will happen tomorrow now that get_bug_tags() no longer exists | 02:15 |
| wgrant | Right | 02:15 |
| StevenK | I'm still trying to bend django to my will | 02:15 |
| StevenK | It acts utterly and completly stuffed with :memory: sqlite | 02:16 |
| lifeless | StevenK: :( | 02:22 |
| lifeless | StevenK: it possibly is dropping all the handles to it | 02:22 |
| lifeless | StevenK: if thats the case, use a random /tmp path for the db | 02:22 |
| StevenK | lifeless: Yeah, I'm getting to that point | 02:28 |
| StevenK | lifeless: I thought it was a process thing last Monday -- as in, if I could do syncdb and runserver in one process, it should be fine. It isn't. | 02:35 |
| lifeless | StevenK: sorry for making your life hard | 02:35 |
| lifeless | StevenK: /tmp will be the go | 02:35 |
| StevenK | lifeless: Yeah, I have two choices there. Hard code /tmp/auditor.sql in auditor's config, or forcibly import auditor's settings in the fixture and mktemp | 02:37 |
| lifeless | StevenK: you don't want a hard coded path :) | 02:38 |
| lifeless | StevenK: If I can make a (little awkward but tolerable I think) suggestion. | 02:38 |
| StevenK | Yes I do, it will be EASY | 02:38 |
| StevenK | As opposed to that last thing you suggested | 02:38 |
| lifeless | it will fail horribly when you have concurrency. | 02:39 |
| lifeless | so, in your settings.py, consult an environment variable for the path | 02:39 |
| lifeless | if its not set, use whatever path you were using before the :memory: experinment. | 02:39 |
| lifeless | in the fixture, set a unique path using TemporaryDirectory to get a throwawy subtree. | 02:39 |
| lifeless | and export it via EnvironmentVariable | 02:40 |
| lifeless | (both are fixtures from fixtures) | 02:40 |
| StevenK | The fixture already creates a temp directory for the logfile, so that should be easy | 02:41 |
| lifeless | cool | 02:41 |
| StevenK | lifeless: http://pastebin.ubuntu.com/1131833/ | 02:48 |
| lifeless | cool | 02:50 |
| StevenK | Now to hack the fixture | 02:50 |
| wgrant | There'd better not be a default of a well-known path in /tmp, or I will be unhappy :) | 02:51 |
| StevenK | wgrant: BLEH | 02:52 |
| StevenK | Because debugging via connecting to the DB is for chumps or something. | 02:52 |
| lifeless | wgrant: there will be, but the fixture will override it | 02:53 |
| lifeless | wgrant: and prod deploys will use postgresql. | 02:53 |
| lifeless | wgrant: so suck it up white boy :) | 02:54 |
| wgrant | lifeless: The fact that /tmp exposes a "create item by name" interface is a historical mistake. | 02:54 |
| wgrant | So a path of /tmp/auditor.sql doesn't make sense | 02:55 |
| wgrant | As it's not possible by sensible means to create a file with that path | 02:55 |
| StevenK | wgrant: So I'm supposed to make use of tempfile.mkstemp and just guess? | 02:57 |
| wgrant | Guess? | 02:57 |
| StevenK | Even though the fixture is going to override it and it's just for people wanting to run auditor's tests locally? | 02:57 |
| wgrant | Is any callsite not going to override the default? | 02:58 |
| wgrant | The test suite surely has to | 02:58 |
| wgrant | If nothing uses the default, then the default doesn't need to be there. | 02:58 |
| StevenK | There are three testsuites here -- auditor's, auditorfixture's and LP's. | 02:59 |
| StevenK | The fixture will override the default, so it's fine for the fixture itself and LP. | 02:59 |
| lifeless | the risk is that someone will attack your system by having a symlink to something you care about at that path | 02:59 |
| lifeless | but they can't generate arbitrary contents | 02:59 |
| wgrant | It's not a significant risk here. | 02:59 |
| wgrant | But it's something that people will copy. | 03:00 |
| wgrant | And it provides no value here | 03:00 |
| lifeless | so put big warning messages around it. | 03:00 |
| wgrant | => it should not exist | 03:00 |
| lifeless | wgrant: whats your alternative ? | 03:00 |
| lifeless | wgrant: whenever you find yourself saying 'has no value' please provide an alternative, as an existence proof, for something functionally equivalent. | 03:00 |
| StevenK | wgrant: It provides value for people being able to run auditor's test suite | 03:00 |
| lifeless | wgrant: and yes, I realise the irony that I didn't do that for ORM discussion. | 03:01 |
| lifeless | wgrant: I should have. | 03:01 |
| StevenK | Without having to set environment variable | 03:01 |
| wgrant | lifeless: Well, in order to use the default, I'd have to copy auditor.sql to /tmp/auditor.sql before running auditor. It's surely just as easy to set AUDITOR_SQL before running auditor. | 03:01 |
| wgrant | StevenK: Can't the test suite poke it in other ways? | 03:01 |
| lifeless | wgrant: huh?! | 03:01 |
| wgrant | Having the test suite use a single well-known commonly-conflicted path is not a good idea. | 03:01 |
| StevenK | wgrant: Like I said before, which test suite? | 03:01 |
| lifeless | wgrant: tell you what, to get StevenK moving along. | 03:01 |
| StevenK | There are three. | 03:01 |
| wgrant | StevenK: The one that would use /tmp/auditor.sql | 03:02 |
| wgrant | Any test suite using that is non-concurrent | 03:02 |
| lifeless | wgrant: understand we're *improving* the situation, it *used* to be just /tmp/auditor.sql always. | 03:02 |
| lifeless | wgrant: so file a bug, and StevenK can look at it later. | 03:02 |
| StevenK | Actually, it used to be /home/steven/auditor.sql ... :-( | 03:02 |
| lifeless | StevenK: hah, same issue though. | 03:02 |
| StevenK | Right | 03:03 |
| wgrant | Ah, if it's already worse, then OK :) | 03:07 |
| StevenK | Although, thinking about it, the testsuite doesn't use that, so the default can probably die | 03:11 |
| lifeless | StevenK: one step at a time | 03:12 |
| lifeless | StevenK: you may be right, but why yak shave when you don't need to. | 03:12 |
| StevenK | wgrant: Are you winning at making private structsubs fast? | 04:55 |
| wgrant | StevenK: Yes. | 05:14 |
| wgrant | Trying to also win at not making the queries horrid | 05:15 |
| adeuring | good morning | 07:53 |
| === almaisan-away is now known as al-maisan | ||
| === al-maisan is now known as almaisan-away | ||
| rick_h_ | jcsackett: ping when you get in, quick ? for you | 11:27 |
| === matsubara is now known as matsubara-lunch | ||
| jcsackett | rick_h_: ping. | 13:56 |
| rick_h_ | jcsackett: pong | 13:57 |
| deryck | adeuring, here's my branch, if you don't mind: https://code.launchpad.net/~deryck/launchpad/support-pape-max-auth-age/+merge/117781 | 13:57 |
| rick_h_ | jcsackett: nvm about earilier, don't think it'll work | 13:57 |
| adeuring | deryck: ok | 13:58 |
| jcsackett | rick_h_: ok. | 13:58 |
| jcsackett | rick_h_: hangout still planned for now? | 14:03 |
| rick_h_ | jcsackett: yea, just hangouts not playing nice and starting up for me atm | 14:03 |
| rick_h_ | 5th try to get it to load/start | 14:03 |
| deryck | rick_h_, are we using our url, or you starting a random one? | 14:04 |
| rick_h_ | deryck: I'll try our url next I think | 14:04 |
| rick_h_ | was trying to start a new one with no success | 14:04 |
| deryck | worked for me. | 14:04 |
| === matsubara-lunch is now known as matsubara | ||
| adeuring | deryck: r=me | 14:21 |
| rick_h_ | jcsackett: so I'm going to go think for a bit around lunch. Do you want to get together for a call this afternoon and kind of run through some points? | 15:09 |
| jcsackett | rick_h_: sounds like a plan. have a particular time in mind? | 15:10 |
| rick_h_ | jcsackett: shoot for 3:00? | 15:10 |
| jcsackett | sounds good. | 15:11 |
| rick_h_ | cool, thanks | 15:11 |
| deryck | adeuring, thanks for the review | 15:21 |
| jcsackett | sinzui: ping. | 15:53 |
| sinzui | hello | 15:53 |
| jcsackett | sinzui: free to chat a bit? | 15:53 |
| sinzui | yes | 15:53 |
| jcsackett | fantastic. | 15:53 |
| jcsackett | sinzui: hangout invite sent. | 15:54 |
| === deryck is now known as deryck[lunch] | ||
| === deryck[lunch] is now known as deryck | ||
| rick_h_ | jcsackett: up for hangout? | 18:56 |
| rick_h_ | deryck: will want to put you on deck perhaps after this one as I try to get stuff talked over pre-EOD (heads up) | 18:56 |
| jcsackett | rick_h_: i'm free. | 18:56 |
| rick_h_ | jcsackett: awesome | 18:57 |
| deryck | rick_h_, sure. | 18:57 |
| deryck | rick_h_, I'm done in an hour, since I started early and am at the beach. Just to give you a heads up too ;) | 18:57 |
| rick_h_ | deryck: ok, will try to go quick | 18:57 |
| jcsackett | rick_h_: we can keep ours short so you can talk to deryck. | 18:57 |
| jcsackett | rick_h_: can you send me the hangout invite? there are two of you in my G+ list and i'm not sure which to send the invite too. :P | 18:59 |
| lifeless | deryck: o/ | 19:02 |
| nigelb | mwhudson: welcome back, post-new family member arrival :) | 19:20 |
| mwhudson | nigelb: thanks! | 19:21 |
| lifeless | mwhudson: wb | 19:23 |
| lifeless | mwhudson: everything go smoothly ? | 19:23 |
| mwhudson | lifeless: ended up with a c-section due to baby getting stuck on the way out, but aside from that everything is good | 19:24 |
| lifeless | mwhudson: woo (we did too) | 19:24 |
| mwhudson | ah right | 19:27 |
| mwhudson | it's a pretty intense 45 mins or so between them saying "we need to do a caesar" and getting it done :) | 19:27 |
| mwhudson | especially if everyone's been awake for 24 hours by that point | 19:27 |
| lifeless | mwhudson: I have /no idea/ how long it took | 19:28 |
| lifeless | tis all a blur | 19:28 |
| lifeless | I remember the operating room vividly | 19:28 |
| mwhudson | heh yeah | 19:28 |
| mwhudson | i remember being taken away and forcefully fed some carbs | 19:28 |
| lifeless | before, after or during ? | 19:28 |
| mwhudson | before the op | 19:29 |
| lifeless | interesting | 19:29 |
| lifeless | they didn't do that for me | 19:29 |
| lifeless | perhaps my weight dissuaded them :P | 19:29 |
| rick_h_ | deryck: still got 10min? | 19:31 |
| deryck | rick_h_ sure | 19:31 |
| rick_h_ | deryck: cool, normal hangout url? | 19:31 |
| lifeless | deryck: want to pick a time for a call? | 19:32 |
| deryck | lifeless, yeah. let's do my Thursday afternoon, your Friday morning. if that works for you. | 19:33 |
| lifeless | its clear | 19:33 |
| lifeless | sinzui: did my logic make sense w.r.t. mutually invisible teams in a project? | 19:58 |
| sinzui | lifeless: yes | 20:04 |
| sinzui | I asked Cody if they really use mutually invisible teams in a project? | 20:04 |
| lifeless | maxb: *headdesk* ** 100 | 20:20 |
| maxb | indeed :-/ | 20:25 |
| StevenK | wgrant: http://pastebin.ubuntu.com/1133382/ is the traceback | 22:59 |
| wgrant | StevenK: Something under related_projects is throwing an AttributeError, probably. | 23:11 |
| wgrant | You may want to call it manually | 23:12 |
| wgrant | in a test or harness | 23:12 |
| wgrant | And see what the actual error s | 23:12 |
| wgrant | is | 23:12 |
| StevenK | wgrant: Sorted it out. Did you want to review the MP I just put up? | 23:35 |
| wgrant | StevenK: Am doing so | 23:36 |
| wgrant | StevenK: Just a few things that need fixing. | 23:48 |
| StevenK | wgrant: Thanks. I've never quite figured out how IStoreSelector maps to I{,Master,Slave}Store | 23:51 |
| wgrant | StevenK: DEFAULT_FLAVOR == IStore | 23:52 |
| wgrant | MASTER_FLAVOR = IMasterStore | 23:52 |
| wgrant | SLAVE_FLAVOR = ISlaveStore | 23:52 |
| wgrant | Webapp code should very very very very very rarely use anything other than IStore | 23:52 |
| lifeless | cjwatson: nominations seem like overkill to me | 23:54 |
| lifeless | cjwatson: we had to cripple *who* could nominate because folk just nominated everything. | 23:55 |
| cjwatson | I'm aware of that | 23:55 |
| lifeless | good good | 23:55 |
| cjwatson | Nevertheless our defect analysts etc. need to be able to target bugs, but we don't want them to have full driver privs | 23:56 |
| cjwatson | note target not nominate | 23:56 |
| lifeless | so target is a superset of nominate. | 23:56 |
| cjwatson | I know | 23:56 |
| cjwatson | BugNomination.canApprove => target | 23:56 |
| wgrant | cjwatson: What undesirable privilege does being a series driver convey? | 23:57 |
| lifeless | I am confused; perhaps you can describe the things you /don't/ want the defect analysts to do, and how often they do them today. | 23:57 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!