[01:31] wgrant: https://code.launchpad.net/~stevenk/launchpad/drop-populate-specification-aag/+merge/143627 [01:42] StevenK: Self-review? [01:44] wgrant: Happy to do, wondering if you agree with deciding to not run it against sampledata. [01:45] StevenK: I forget -- will it un-null access_grants if it's public? [01:45] If it will, then it's not a no-op and should be run [01:45] But IIRC it leaves access_grants null in that case, so it will be a no-op [01:49] wgrant: http://pastebin.ubuntu.com/1540007/ [01:50] Right, no-op [02:30] StevenK: Need any help with the model branch? [02:41] wgrant: Yes [02:41] Push branch, show test failures :) [02:41] wgrant: http://pastebin.ubuntu.com/1540093/ is the diff; test failures are http://pastebin.ubuntu.com/1536679/ [02:45] wgrant: I can push the branch if you'd prefer that to a diff. [02:46] StevenK: Have you replicated the Product.active check that was in visible_specification_query? [02:46] Oh god [02:47] That's so not the place for it [02:47] Mmm, it was not unreasonable [02:47] Given that the spec search stuff isn't in a single place, like bugtasksearch is [02:48] Now I will have to revert the tables bit :-( [03:47] My query seems to fail [03:47] It won't return distribution specs :-( [03:48] StevenK: You failed to left join properly [03:49] You need to left join product and then say (product.active IS NULL or product.active) [03:58] StevenK: Did you see danilos' confirmation of my suspicion this morning? [03:58] The deps thing is indeed a regression from the privacy work, so your rewrite will fix it [03:58] I'm not sure if my rewrite is brutal enough for that yet [03:59] Anyway, looking at that next. [04:00] wgrant: And yes, I saw it because Curtis asked me about it. [04:01] Well [04:01] The rewrite involves reverting all the previous changes [04:01] So it will be :) [04:01] It does not [04:01] :( [04:02] There are fair amount of changes which are required, like passing users around [04:02] Ah, of course [04:09] wgrant: https://code.launchpad.net/~stevenk/launchpad/use-specification-aag/+merge/143630 [04:16] StevenK: r=me with a comment, but please check performance on DF first [04:20] wgrant: I think I need a LeftJoin in sharingjob, too [04:21] StevenK: Ah, yes, so you do [04:21] wgrant: So http://pastebin.ubuntu.com/1540213/ [04:21] In fact sharingjob shouldn't be considering product.active at all [04:22] wgrant: I can change it to only append privacy_query[-1], but that's a bit horrid [04:23] Yeah, that's not a solution :) [04:23] I guess do the left join but otherwise leave it as it is [04:24] I can shift out the product stuff out of get_specification_privacy_filter and do it all callsites bar that one, but I like that even less [04:34] wgrant: DF has been patched, and has actually swapped back in enough to render pages. [04:39] wgrant: The main query of blueprints.dogfood.l.n/~ takes 240ms [04:41] The old query on prod is roughly the same, but prod is a lot faster than DF [04:45] For https://blueprints.dogfood.launchpad.net/nova , the query is 10ms [04:46] StevenK: Grab an explain analyze of the slow query [04:47] On DF, or prod? [04:48] DF [04:48] We know that prod's is terrible :) [04:49] wgrant: http://pastebin.ubuntu.com/1540253/ [04:50] wgrant: And the query http://pastebin.ubuntu.com/1540256/ [04:52] StevenK: Right, that looks fine. [04:53] It's terrible, but that's not the fault of your changes [04:53] Heh [04:54] Distribution is quick, but that doesn't involve the new columns at all, Product is like 10ms [04:54] Hm, it should involve the new columns [04:55] Distribution doesn't filter at all [04:55] By access_policy/grants, I mean [04:55] No private distributions, ergo no private blueprints for any distribution. [04:56] :/ [04:56] It just assumes that they're all public [04:56] Doesn't even check [04:56] I can fix that, if you wish [04:57] Which would involve Storm-ifying it [04:57] Which is not a bad thing at all :) [04:57] Ideally we'd unify blueprint search like bug search is [04:58] Well, I can do that too ... [04:58] That's probably a bit more of an effort [04:58] But maybe [05:06] wgrant: http://pastebin.ubuntu.com/1540277/ [05:09] StevenK: orlf [05:09] have you looked at the query log for Person:+specs? [05:09] Yeah, I glanced it [05:10] First it checks if there are any specs [05:10] Then it gets the specs [05:10] Then it counts the specs [05:10] Then it checks if there are any specs [05:10] StevenK: You could get ops to check the new query on prod [05:11] Haha [05:11] Given it's all populated there now [05:11] wgrant: The EXPLAIN or just run the query for timing? [05:11] explain analyze [05:11] but oh god have you tried the old query on DF? [05:11] Your eyes may burn [05:11] Show me? [05:12] 1.4s with tonnes of merge joins [05:12] Haha [05:12] http://paste.ubuntu.com/1540286/ [05:12] The new one could be considered to be a marginal improvement [05:13] Haha [05:13] Did you spot your typo in the name field? [05:13] yes, before I submitted but I'm sick enough that I really couldn't be bothered fixing it [05:14] wgrant: So, specificationsearch, or leave it? [05:14] Depends on what sort of callsites we have [05:31] Bah, is_not_in doesn't exist [05:37] Not(...is_in) [05:37] Yeah [05:37] wgrant: http://pastebin.ubuntu.com/1540315/ works, do you want it in the branch? [05:38] StevenK: Can Specification.completeness_clause now be Stormified? [05:38] Not sure what other non-Storm callsites there would be [05:39] + clauses.append( [05:39] + SQL('Specification.fti @@ ftq(%s)' % quote(constraint))) [05:39] fti_search is a helper for that, IIRC [05:39] wgrant: Yeah, I just switched to that [05:40] Distribution, DistroSeries, Product and ProductSeries want it [05:41] Are they all stormified? [05:41] Sorry, DistroSeries, ProductSeries and ProjectGroup [05:41] And all three are not [05:43] Hee hee [05:43] I'll leave the explanation of that to the reader [05:44] "When we get around to specificationsearch" [05:44] Which I'm getting dangerously close to just starting [05:44] Because this is just pathetic [05:50] Ah [05:50] Not actually exploitable, sadly [05:51] Since permissions for private specs don't seem to be precached [05:51] So you just get a 403 [05:51] (but that also means there's probably a timeout bug on any page that lists lots of private specs) [05:51] StevenK: So, how do you feel about doing specificationsearch? :) [05:52] 'cause what we have today is sort of buggy as shit [05:52] wgrant: So, guess what I started about 5 minutes ago [06:01] StevenK: Bug #1100617 is relevant [06:01] <_mup_> Bug #1100617: Blueprint searches don't precache permission checks

< https://launchpad.net/bugs/1100617 > [08:19] morning [10:40] link to online help on the recipe page seems to have disappeared === stub1 is now known as stub [14:22] benji, gary_poster, bac: So, in a fit of giddiness I tried to get around the only-one-external-link-in-a-card problem; if you have time to look at it, see https://code.launchpad.net/~gmb/lp2kanban/cards2workitems/+merge/143543 for the latest update. If not, I'll just land yesterday's work and make a new branch for the new stuff. [14:23] gmb: cool; I may have time later today to look at it [14:24] * gary_poster on call === slank_away is now known as slank [14:29] benji, Okay; in that case I'll land yesterday's work (I'm hoping to be able to show this off to people in Austin) and make a fresh branch and merge proposal. [14:29] k [14:38] benji, New merge proposal is here for your delectation when you get a chance: https://code.launchpad.net/~gmb/lp2kanban/dont-rely-on-external_system_url/+merge/143700 [14:38] gmb: cool === Ursinha_ is now known as Ursinha === _mup__ is now known as _mup_ === salgado is now known as salgado-lunch === salgado-lunch is now known as salgado [16:47] Hm - is it just me or is lib/lp/soyuz/templates/binarypackagepublishinghistory-listing-detailed.pt dead? I can't actually see a way to get at it, short of URL-hacking to add +listing-details [16:56] Maybe it is there somewhere and I'm just too stupid to find it. [17:00] I'd hardly think you could you of all people cjwatson are stupid ! [17:18] cjwatson, my approach to that if I can't find any real reference to it is to remove the zcml that hooks up the template, run tests, and see what breaks. you're probably right. === yofel_ is now known as yofel === mthaddon` is now known as mthaddon === deryck is now known as deryck[lunch] === matsubara is now known as matsubara-afk [19:30] czajkowski: LP sets a pretty high bar on smarts-needed-to-tell-whats -used :) === deryck[lunch] is now known as deryck === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha === StevenK_ is now known as StevenK [23:13] cjwatson: I think DASBPR:+index used to use it, but hasn't since 3.0 [23:14] It's unused now, and I think the only callsite of its source counterpart is DSSPR:+index [23:14] Everywhere else uses +listing-archive-extra [23:14] IIRC [23:36] Argh, you know unused code is old when bzr log goes back to a Mark commit [23:36] (Not the same thing) [23:37] Heh, yes. [23:37] cjwatson: What was it? [23:38] lib/lp/soyuz/browser/publishedpackage.py - removing it as part of my current branch [23:40] * cjwatson wonders if database/schema/launchpad.html still has a useful purpose [23:40] Aside from indicating that publishedpackage was once "a very large view" [23:41] wgrant: How do I actually get to a BinaryPublishingRecordView in the webapp? I can't come up with a suitable URL [23:43] Huh [23:44] How has publishedpackage survived... [23:44] I deleted some related bits years ago [23:44] But apparently missed the browser stuff [23:44] I guess I didn't think it existed :) [23:44] Ah [23:44] And it doesn't actually mention 'publishedpackage' anywhere in it, so a grep would have failed [23:45] launchpad.html can probably die [23:45] $ wc -l database/schema/launchpad.html [23:45] 30685 database/schema/launchpad.html [23:46] Indeed [23:46] * cjwatson removes :) [23:46] cjwatson: Also, https://launchpad.net/~wgrant/+archive/ppa/+binarypub/25603241/+record-details [23:49] Ah. Are there no more polished paths that use stuff from there? [23:49] This is in aid of trying to figure out which bits of the webapp I need to update to show phased_update_percentage [23:49] cjwatson: That's the only URL, but ZCML could reference those views.