[00:18] wgrant, mumble? [00:21] Project windmill-db-devel build #320: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/320/ [00:31] lifeless: re your service-based Launchpad idea, it would be really nice to eliminate branch scanning and just have a service to query the branches. [00:40] * wallyworld_ needs caffeine stat! [00:41] * thumper passes wallyworld_ the IV [00:56] abentley: In principle yes [00:57] abentley: I wonder if bzr is quite fast enough yet [00:58] abentley: I think there are some things we'd want to have caches for - e.g. 'show me branches that have new work' might want a fast graph query [00:58] a nice thing about a SOA is that you can make that change independent of more or less everything else [00:58] yes indeed! [01:37] Project windmill-devel build #132: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/132/ [01:41] lifeless: I think that it probably is fast enough, considering there's no invocation time. Fast graph traversal is a service you were already going to provide, right? [01:42] lifeless: by invocation, I mean bzrlib import cost. [01:42] abentley: right; I think it depends on the particular bit of bzrlib we're talking about [01:43] abentley: things like extracting a file can still do substantial IO, grabbing a revision should be pretty quick [01:43] abentley: and it will depend on what we're doing whether a particular thing works / doesn't. [01:43] abentley: where it will, I think using the branches directly is a good idea. [01:44] lifeless: The data we currently retrieve from scans doesn't include files, though. It's mostly revision ids and a revision graph. [01:44] abentley: given the complexity of bzrlib, memory footprint etc, I'd be inclined to have a webservice - say loggerhead :) - providing this, and it be backed by the raw branches [01:45] lifeless: Yes, this is what I meant by "a service to query the branches". I didn't mean using the bzr protocol. [01:45] abentley: we may still need scanning to populate a graph cache for graph queries, but it could stop being relevant for viewing the recent commits in the web UI [01:46] abentley: cool [01:46] abentley: did you know loggerhead has a json API already? [01:46] lifeless: You did mention it before, but I hadn't thought about it. [01:46] How complete is it? [01:47] topically, gustavo wants an API to query branch revision tips for principia [01:47] lifeless: providing that service via loggerhead makes sense, *iff* we have a well-behaved, performant loggerhead. [01:47] wants a collection of (branch url, revid) [01:47] wgrant: its minimal; easy to extend though [01:48] True. [01:48] wgrant: we'd need to add something for private branch lookups if appservers do them === mwhudson_ is now known as mwhudson [01:53] Hm. [01:54] Looks like staging updates are working fine, but the code isn't being synced beforehand? [01:57] scripts/update-bzr-version-info.sh [01:57] Creating bzr-version-info.py at revno 10574 [01:57] I bet its the config-manager change on carob [01:57] spm! [02:00] successful-updates.txt shows it updated to 10574 for the second time several hours after a new rev was blessed. [02:00] So I'm pretty sure that's it. [02:00] sinzui: Sorry I missed the call this morning. [02:00] sinzui: Did you make any progress on the picker issue? [02:01] I think we basically need to completely rethink it before we can make any progress. [02:01] wgrant: we had nothing blessed for 5 days ? [02:03] lifeless: 10578 was only bleesed May 22 23:32 [02:03] wgrant: which is still 3 days ago [02:04] Sure. We've had stuff blessed since. [02:04] But potentially only a single update has failed to update the code. [02:04] Since the 2011:05:23 13:21 update could have started before 23:40 [02:06] the update on the 24th [02:06] saw rev 10574 [02:06] on sourcherry [02:07] Right. [02:07] And that's the only failed update we have evidence of. [02:33] cody-somerville: ping [02:33] bug 787765 - can you tell me the figure you see under '79 queries/external actions issued in 0.85 seconds' on the page that was/is failing ? [02:33] <_mup_> Bug #787765: ProductSeries:+index timeouts < https://launchpad.net/bugs/787765 > [02:33] cody-somerville: and can you reproduce it on qastaging [02:41] lifeless, under? [02:42] cody-somerville: what that text shows for you :) [02:42] ah [02:42] '4 queries/external actions issued in 0.14 seconds' [02:42] cody-somerville: timeout though, right ? [02:42] lifeless, yup [02:42] lifeless, And it does load on qastaging. [02:42] cody-somerville: can you get a profile on qastaging ? [02:44] oh wait, my bad. Was loading wrong milestone on qastaging. [02:44] It timesout on qastaging as well [02:44] cody-somerville: can you add me as a driver on qastaging? [02:46] lifeless, Sure. FYI, the project page is timing out on qastaging.launchpad.net as well but not every time. [02:46] (https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1971QASTAGING19) [02:47] lifeless, Done. Added 'lifeless' to oem-solutions-releng team on qastaging. [02:47] 16 0 7.5976 0.1703 lp.bugs.browser.structuralsubscription:431(expose_user_administered_teams_to_js [02:49] cody-somerville: you're in lots of teams aren't you ;) === StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256 [02:50] lifeless, 225 [02:50] (indirect + direct) [02:51] do you know how many you have admin access over ? [02:53] lifeless, Administrator of 15, Owner of 17. [02:53] (cut, grep, and wc for the win!) [02:53] heh [02:53] that probably fails the indirect test [02:53] anyhow [02:54] ah, true, I could have used grep to instead of wc [02:54] *to count [02:55] And could have used a regular expression instead of cut to make sure I was looking at the right field [02:55] * StevenK hands cody-somerville a 'pointless pipe' award [02:55] Project windmill-db-devel build #321: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/321/ [02:57] wallyworld_: How's your picker stuff going? [02:59] wgrant: been tied up doing other stuff - revising popup field name branch, doing a feature flag branch for disclosure, seperating out a separate branch for the irc nick formatter etc [02:59] wallyworld_: Ah, right. [02:59] wallyworld_: It seems to me that we're not going to get very far by treating the search/rank/affiliation issues separately. [02:59] wow, slow [02:59] 190530 0 3.2106 1.3876 +storm.properties:51(__get__) [03:00] We have a set of requirements and seem to be jumping into designing these three things separately. [03:00] Without actually really thinking about how they will work together. [03:00] cody-somerville: so this is easy fixed [03:00] wgrant: did you know there's a FormattersAPI class which is supposed to handle escaping of invalid html ids. but it doesn't ensure uniqueness [03:00] cody-somerville: care to put the patch up? [03:00] wallyworld_: I didn't! [03:00] wallyworld_: That's somewhat unpleasant. [03:00] its probably the root cause for a half dozen timeouts [03:01] wgrant: curtis mentioned it in reviewing the mp [03:02] wgrant: with the picker stuff, i think we can do alliliation separately from search/rank. affiliation will be expensive, but we will only need to do it for each batch of 6 or whatever results [03:02] potentially expensive [03:02] wallyworld_: I don't think that's the case. [03:03] wallyworld_: We need to be able to filter by affiliation, or at least rank by it. [03:03] Search for "william" on a Launchpad bugtask. [03:03] That's clearly wrong. [03:03] hmmm. that will complicate things for sure [03:03] Exactly. [03:03] lifeless, Ugh, sure! :-) [03:04] The search at the moment is useless. [03:04] It finds the 100 most relevant results, then orders them by displayname. [03:04] we will need to consider denormalising the data model in that case to make affiliation easy to get [03:04] Possibly, but we first need to consider what we *want*. [03:04] agreed [03:04] Do we want ranking, do we want filtering, or what? [03:04] no question there [03:04] wgrant: that's what the product team needs to feed us - requirements :-) [03:05] cody-somerville: we're doing 64K __eq__ calls [03:08] wallyworld_: Are those requirements, or design? [03:08] Can haz review from someone? [03:09] wallyworld_: The only real requirement is that it returns "good" results. :) [03:09] +13/-0 [03:09] wgrant: wallyworld_: can I be of assistance? [03:09] requirements. ie what we need to deliver as opposed to how to do it [03:09] wallyworld_: This is part-way between requirements and design, IMO. [03:10] lifeless: we are pondering the new search behaviour of the picker [03:10] Because currently it is useless unless you type the full username or displayname. [03:10] wgrant: not sure i agree :-) i think we need to be clear about what we need to deliver, not how to do it [03:11] wallyworld_: You have a branch to stop using Person.fti, right? [03:11] wgrant: yes, in ec2 now [03:11] wallyworld_: Have you tested how that works in practice? [03:11] wallyworld_: I think it will eliminate displayname matching. [03:11] Which would be a very bad thing. [03:11] wgrant: well, i ran it locally and tried a couple of searches with and without the fti removed and they appear to behave the same [03:12] wallyworld_: I don't see where it does a displayname search. [03:12] Apart from fti (which is name, displayname) [03:12] i can't recall if i checked display name matching. i'll have another look [03:12] to be sure [03:13] But the query is obscene, so it's a bit hard to tell :/ [03:13] * wgrant tries. [03:13] lifeless, I've got to take the garbage out and then I need to head to bed but I'll push a branch with a patch tomorrow morning if someone else hasn't gotten around to it. Kudos for looking into this. :-) [03:16] cody-somerville: I've finished capturing my learnings [03:16] cody-somerville: if you need a hand, just shout [03:16] * cody-somerville nods. [03:16] wallyworld_: So, back to affiliation for a sec: yes, it's easy to just show the affiliation of the current six, but that doesn't help if I have to scroll through 10 non-deterministic batches to find the right person :/ [03:17] (yes, it is actually non-deterministic, it seems :/) [03:17] wgrant: wallyworld_: is there a LEP ? [03:17] TrustedPickers [03:17] wgrant: yes, i agree that we may want to sort by affilliation. that is not something i was previously considering [03:17] https://dev.launchpad.net/LEP/TrustedPickers [03:18] Ahh, https://dev.launchpad.net/LEP/TrustedPickersUseCases doesn't seem to be linked from anywhere. [03:18] But is useful. [03:19] It has a usecase like the one I raised [03:19] StevenK: Can I revert your changes on DF? [03:19] Looks like it's just that merge from yesterday. [03:20] what defines affilate? === jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256 [03:21] I'm worried about the use of irc handles, maybe I'm cracked [03:21] lifeless: Probably membership in a team related to the project. [03:21] But then we may well also want a Canonical affiliation, so that falls apart. [03:27] wgrant: Sure [03:28] lifeless: getFeatureFlag. i want to add a variant which returns a user specified default value if features.getFlag() returns None. i'd like to add this to lp.services.features.__init__. but i think you disagree about doing that there? [03:28] jtv: Can haz review? [03:28] StevenK: when I finish my current one, sure [03:28] wallyworld_: why do you want to do that ? [03:28] wallyworld_: what problem are you solving? [03:29] StevenK: Thanks. [03:29] lifeless: i have an int feature flag and i don't want to do what was done in archive.py [03:29] limit = getFeatureFlag(FEATURE_FLAG_MAX_SYNCHRONOUS_SYNCS) [03:29] try: [03:29] limit = int(limit) [03:29] except: [03:29] limit = 100 [03:29] wallyworld_: you'll still need that code [03:30] wallyworld_: because crap might be entered in [03:30] not if i can specify a default as in myflag = getFeatureFlag('flagname', 1) [03:30] wallyworld_: I'm totally happy with folk layering convenience functions; I'm unhappy with folk embedding schemas in it [03:30] wallyworld_: thats not sufficient to remove the try:except: [03:31] sure, i'd do the try except inside the getFeatureFlag [03:31] and maybe name the function getIntFeatureFlag [03:31] wallyworld_: I think thats a bad idea because it hides the ability to tell whats going on, and the casting you want isn't related to feature flags [03:32] wallyworld_: I think you want a safe_int(value, default=0) function [03:32] hmmm. i think it is because we specify the type of feature flag - boolean, string, int etc - so there should be relevant methods to get flags cast to the expected type [03:33] safe_int works for me [03:33] wallyworld_: That type is just documentation. [03:33] well, the type is crufty and I'm not 100% on board with it being there [03:33] but I support folk doing what they think makes sense ;) [03:34] not sure why you dislike it. it serves a good purpose [03:34] can i add safe_int to features __init__ ? [03:34] wallyworld_: I don't think it belongs in features [03:34] theres probably an implementation in the tree already [03:35] in a util or misc or helpers or some such module alrady [03:35] wallyworld_: i dislike it because typing drives the requirements -way- up for very little win. [03:35] fair enough. will look. i wasn't going to call it safe_int if it were in feaures init :-) [03:36] i can't yet bring myself agree with that :-) [03:36] wallyworld_: I'm totally in favour of the documentation we have [03:37] i like the doco too. but if it makes sense to define a flag as an int, we should easily be able to treat it as such [03:37] s/treat/use [03:37] wallyworld_: You may want to kill that ec2 instance before it lands. [03:38] wallyworld_: Try searching for "william grant" in the assignee picker on https://bugs.dogfood.launchpad.net/launchpad/+bug/1234 [03:38] <_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts < https://launchpad.net/bugs/1234 > [03:39] wgrant: sure. i had retested yet. was on my todo :-) display name matching broken then i assume [03:39] ? [03:39] wallyworld_: Yeah :( [03:40] I'm not sure why it returns what it does, but it's certainly not right! [03:40] wgrant: bollocks. i'll look at modifying the query to specifcally match displayname [03:40] wallyworld_: We probably still need FTI over that. [03:40] wallyworld_: And Person.fti is already over just name and displayname. [03:41] So we probably want an index on Person.fti instead. [03:41] (which dev and DF already have, but production doesn't) [03:41] really? [03:41] why not production? [03:41] Because production lacks it. [03:41] It only exists on DF because I created it yesterday totest :) [03:42] No idea why :/ [03:42] we can add that now, losas are around [03:42] True. [03:42] are we sure it wouldn't be better just to say (pseudo) 'xxxx where name like ? or displayname like ?' [03:42] wgrant: how big is it on df ? [03:43] lifeless: Most will only have three terms, so it should be small, but let's see. [03:43] \di+ person_fti [03:43] wallyworld_: there is no index on display name [03:43] doesn't seem worth the cost of creating and maintaining a fti over just two simple string fields? [03:44] 49MB [03:44] wallyworld_: fred bloggs should match fred J bloggs [03:44] wallyworld_: But how do you propose to do a full-text search of displayname? [03:44] Without an FTI index? [03:44] Right, what lifeless said. [03:44] wgrant: from the perspective of a user doing a search [03:44] wallyworld_: and foo%bar style searches are terribly slow [03:44] lifeless: Hm? [03:45] fair enough. [03:45] wgrant: wildcard searches are not indexed [03:45] wgrant: literal prefixes sometimes use indices [03:46] ok on the LEP [03:46] Must point 1 [03:46] I think is over broad [03:46] I would say that the information which lead to a *hit* must be shown [03:46] which is not the same as showing all the possible things we search on on every hit [03:47] secondly, point 2: will require a new cached data structure [03:47] Agreed [03:48] wgrant: agreed on 1 or 2 or both ? [03:48] I don't agree that 2 needs a new datastructure, unless we want to search/rank on it, which we probably do. [03:49] wgrant: ' previous bug assignee' [03:49] wgrant: + Ubuntu [03:49] Ah, missed that. [03:49] Ubuntu? [03:49] Ubuntu scope searches [03:49] What about them? [03:49] 400K bugs to consult to find previous bug assignees [03:49] Right. [03:50] I hadn't considered the assignee case. [03:50] I suspect we want to expose whatever thing we have an let folk delete inappropriate things [03:50] but thats a different discussion [03:51] lifeless: perhaps a case for a new service :-) [03:51] perhaps [03:51] I think it would fit well in a services model [03:52] lifeless: Right, if we have assignees then we very probably want to permit manual administration. [03:52] yep, especially since there will likely need to be a (denormalised) data model to go with it [03:53] lifeless: Shall we hijack a LOSA to 'CREATE INDEX person_fti ON person USING gist (fti);'? [03:53] lifeless: btw, found intOrZero() and friends in canonical.lp.helpers.py [03:53] wallyworld_: hahahahaha :> [03:53] canonical.lp buuuuuuuuuurn [03:54] I've been trying to remove that, with little to no success [03:54] so how confident are we that creating that fti index will solve the picker timeouts? [03:55] StevenK: well, seems like i'm about to add another bit of code that uses it :-) [03:55] wallyworld_: The queries execute in <3s on DF, and that's DF. [03:55] wallyworld_: DIE [03:55] wallyworld_: There's no other significant explanation for its slowness. [03:55] wallyworld_: there may be other causes of slow, but we know that that is a cause [03:55] besides all the unions etc [03:55] jtv: Still reviewing? [03:55] Yes [03:55] wallyworld_: unions are not prima facie causes of slowness [03:55] wallyworld_: Unions aren't slow unless the queries are slow. [03:56] jtv: How large is this branch? :-) [03:56] StevenK: it's not the size that matters [03:56] it's the distractions of people going "hey jtv are you still reviewing that branch?" [03:56] The emailaddress search seems to be the slowest now. [03:57] i agree about unions not being slow per se. i was more commenting on the number of queries. and when you consider our batcher does a count(*) and then a select it's doubly bad [03:58] Yes, our batcher is stupid :( [03:58] I really must finish that upgrade [03:58] Yeah. [03:58] and in postgres a count(*) is just as bad as a select i'm told so in effect we are executing our queries twice [03:58] The FTI is 100ms on DF. Still slow, but not timeoutfully so. [03:59] wallyworld_: yes, thats true. [03:59] MVCC FTW [03:59] i wonder if we can't fix that? [03:59] its mostly fixed but not followed through on [04:00] ah, the non exact count stuff? [04:00] three things [04:00] using query estimates - function is in the tree [04:01] can you point to the function? [04:01] making 'last' links cheap (batchnav fixed, need to merge into the tree and start updating collections to use it) [04:02] and lastly making arbitrary offsets cheap (same code as point 2) [04:02] oh [04:02] and turning off estimates entirely for collections where we can do that [04:03] wallyworld_: its in lib/canonical/database I think [04:03] ta [04:03] estimateRowCount [04:04] basically parses EXPLAIN [04:05] that's nice assuming it works well [04:05] so one thing you can do is estimate and if the count is < 100 execute the count [04:05] on the assumption we've appropriate indices for small datasets to be answered well [04:08] wgrant: wallyworld_: on the LEP then [04:08] I was about to say. [04:08] what should I do with my feedback - leave it with you, add to the LEP ? [04:11] i haven't digested the comments yet. we can add them and discuss with the product team etc [04:11] It seems slightly bad that we have a picker deliverable in 1.5 weeks but we don't actually know what we're doing. [04:12] lifeless: with the ff int thing, even if i use intOrZero(getFeatureFlag('foo')), that will be used all over the place and will pollute the code with stuff StevenK is trying to get rid of. sure wish i could just add a single helper to features __init__ and use that.... [04:12] wallyworld_: what stuff is stevenk trying to get rid of ? [04:13] the helpers.py i think he said [04:13] wallyworld_: and why do you need to use it all over the place? [04:13] wallyworld_: totally unrelated efforts [04:13] wallyworld_: canonical.lp as a place I think rather than helpers.py specifically, I think [04:13] because anywhere we want to see the value if the ff in order hide a block of code, we need to do that test [04:14] wallyworld_: I'm confused, if its an int you're not using it for hiding stuff [04:14] spiv: ah ok. makes more sense [04:14] wallyworld_: for hiding stuff just use [04:14] if getFeatureFlag('foo'): [04:14] lifeless: we will be. we will be incrementing the int each time a 2 week sprint is done [04:15] so pickers first [04:15] please start from the beginning [04:15] we say "if ffvalue == 1" then show the picker stuff [04:16] This sounds like a very strange use of flags. [04:16] what does this do for you that 'if getFeatureFlag('newpicker'): does not ? [04:16] Exactly. [04:16] ok. so we want to use the one feature flag value [04:16] Why? [04:16] why ? [04:16] feature flag name i mean [04:16] why? [04:16] What benefit does that provide? [04:16] Besides no fewer LOSA requests and less flexibility? [04:16] so that we don't have to keep adding new flags each 2 weeks [04:16] wallyworld_: flags are free [04:17] Adding a new flag is not expensive. [04:17] wallyworld_: /by design/ [04:17] wallyworld_: you don't even need to document them. [04:17] You just use them. [04:17] wallyworld_: and for ephemeral ones I wouldn't bother documenting. [04:17] And then ask an admin to add them. [04:17] You'd have to ask an admin to change the int anyway. [04:17] So no admin time saved. [04:17] No coding time saved. [04:17] Much flexibility lost. [04:17] Transparency lost. [04:17] and you'd be stuffed if you were not ready to flick the switch for one part but were for another [04:18] StevenK: moving on to yours… it's the copy-into-derived-series branch you want reviewed? [04:18] wgrant: we can talk tomorrow at teh standup. sinzuiand i discussed it today and sort of thought using an enum style flag was going to be ok [04:18] wallyworld_: using a flag as an enum is fine [04:18] wallyworld_: I don't think this is an enum situation though, do you ? [04:19] wallyworld_: what problem are you trying to solve? [04:19] It sounds like you're trying to put a whole lot of features as different stages on a single flag. [04:19] not having to change flags.py each time we incrementally deliver a new disclosure feature [04:19] You don't have to change it, but changing it is roughly two lines. [04:20] wallyworld_: so you don't ever change flags.py when you deliver a feature [04:20] wallyworld_: you change flags.py when you *start* a feature. [04:20] wallyworld_: *if* you chooes to change it. [04:20] lifeless: yes agreed. and since we are starting a new feature of disclosure each 2 weeks, that's a lot of editing to flags.py [04:21] wallyworld_: I don't understand that [04:21] wallyworld_: its *literally* 2 lines, with no tests needed. [04:22] *And* it's optional, but encouraged. [04:22] lifeless: and if every sqaud added two lines each time, it will bloat. but i guess we remove old entries when a feature is delivered [04:22] wallyworld_: yes [04:22] wallyworld_: just imagine the help text you'd need to describe 8 weeks of work loaded onto one flag [04:23] '1 means pickers, 2 means bug unsubscribe on mistakes, 3 means ...' [04:23] wallyworld_: I wanted the help text to be federated across the codebase [04:23] wallyworld_: but that got lost in translation [04:24] ok. you win :-) [04:24] wallyworld_: KISS wins [04:25] i didn't realise adding to flags.py was optional but i sort of think it should be mandatory [04:26] hell no [04:26] :) [04:26] the -whole- point of these things is to be ultra lightweight [04:26] why? it adds necessary doco [04:26] so devs can know wtf is going on [04:26] it adds doco [04:26] necessary is a matter of perspective [04:27] not all things need prose to explain them. [04:27] things that will only last a few days certainly don't [04:27] so how else can one quickly get an overview of what ff are used? i like being able to go lp.net/+feature-info [04:28] sure, ones that only last a few days re different [04:28] so +feature-rules shows the live rules [04:28] +feature-info includes all features evaluated during the appservers lifetime [04:28] wallyworld_: Flags that have been queried but are undocumented are listed there. [04:28] StevenK: you're done [04:29] wallyworld_: wgrant: on the LEP : I'll leave my feedback with the two of you. Does it help at all with your planning, or are you still in a state of stuck ? [04:30] lifeless: I think we're going to be stuck for at least another couple of days. [04:30] lifeless: We have been given usecases and that's it. [04:30] lifeless: i've got enough to do today. we need to talk to curtis though [04:31] wgrant: I'd be delighted to help - as a sounding board or whatever - if you like. [04:31] sinzui is still doing bug triage though :) so I know he's arund [04:31] Am I needed? [04:32] wgrant: do we have the business rules for deciding affillation documented? that would be a good start [04:32] sinzui: so, bug 696973 - what are we doing to satisfy the multi-tenant case? [04:32] <_mup_> Bug #696973: There are no roles that can see all private artifacts in (public or private) projects < https://launchpad.net/bugs/696973 > [04:32] sinzui: (two proprietary organisations working on one public project) [04:32]