| lifeless | wgrant: storebuildinfo, really? | 00:32 |
|---|---|---|
| lifeless | (bug 828017) | 00:32 |
| _mup_ | Bug #828017: UnparsableDependencies raised by retry-depwait script <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/828017 > | 00:32 |
| lifeless | wgrant: for my education, what makes this a dup?> | 00:32 |
| wgrant | lifeless: Check the referenced builds. | 00:33 |
| wgrant | lifeless: They don't have a builder, or datefinished, or... | 00:33 |
| lifeless | wgrant: are you saying that if they did, it wouldn't raise that oops ? | 00:33 |
| StevenK | It would not, since the information would be set. | 00:34 |
| wgrant | lifeless: The error is raised when the dependencies are unparseable, which means either lp-buildd is buggy and returning bad data, or LP isn't handling it properly. | 00:34 |
| wgrant | In this case, storeBuildInfo isn't being committed. | 00:34 |
| wgrant | None of the metadata is stored. | 00:34 |
| lifeless | ok, so they don't have crap in their headers, for instance. | 00:34 |
| wgrant | No. | 00:34 |
| wgrant | The data is just None. | 00:34 |
| huwshimi | OK, JS hackers, what's the Launchpad way of moving the JS in this template into its own file? http://bazaar.launchpad.net/~huwshimi/launchpad/tag-cloud-removal-709009/view/head:/lib/lp/bugs/templates/bugtarget-portlet-bugtags.pt | 00:40 |
| * wgrant nominates wallyworld_ | 00:41 | |
| huwshimi | I assume I can just move it into a file in lib/lp/bugs/javascript | 00:41 |
| * wallyworld_ looks | 00:41 | |
| huwshimi | but I want to make sure it gets namespaced properly and uses whatever global stuff we have | 00:41 |
| wallyworld_ | huwshimi: much of the js for the bugtask page lives in bugtask_index.js but you may want to use a new namespace for this | 00:43 |
| huwshimi | wallyworld_: What would you recommend? | 00:43 |
| wallyworld_ | huwshimi: simply have the on domready in the tales call the method you have moved to the new namespace | 00:43 |
| wallyworld_ | perhaps lp.bug.bugtask.tagcloud | 00:44 |
| wallyworld_ | but i'm not the best at making up names | 00:44 |
| wallyworld_ | it depends on if you think there may be more stuff to add which would warrant a new namespace | 00:45 |
| wallyworld_ | i think that lp.bugs.bugtask_index has too much stuff at the moment | 00:45 |
| wallyworld_ | might be good to have the tag cloud stuff separate | 00:46 |
| wallyworld_ | typo, should be lp.bugs.bugtask.tagcloud, or even lp.bugs.tagcloud | 00:46 |
| huwshimi | wallyworld_: Where abouts is that tales domready? | 00:46 |
| wallyworld_ | huwshimi: the existing js embedded in the tales have a domready call | 00:47 |
| wallyworld_ | just replace the js in there with a call to the new namespaced method | 00:47 |
| wallyworld_ | huwshimi: do you see what i mean? | 00:48 |
| huwshimi | wallyworld_: I know what you mean, I'm just not sure which file it is in | 00:49 |
| wallyworld_ | huwshimi: the one from your link | 00:49 |
| wallyworld_ | bugtarget-portlet-bugtasks.pt | 00:49 |
| huwshimi | wallyworld_: Ah right. | 00:51 |
| huwshimi | wallyworld_: Is there a way I can modify things so I can remove the javascript from that page completely? | 00:51 |
| wallyworld_ | huwshimi: the on domready callback has to go somewhere | 00:52 |
| wallyworld_ | there's an argument that it belongs with the template | 00:52 |
| wallyworld_ | but it could also go on the template for the index page which contains the tag cloud i guess | 00:53 |
| wallyworld_ | so if it is removed from the template for the tag cloud, it would need to be put separately wherever the tag cloud is used | 00:53 |
| wallyworld_ | i guess the tag cloud is only used in the one place? | 00:54 |
| wallyworld_ | on the bug task index page | 00:54 |
| huwshimi | wallyworld_: At some stage we will be moving to having no javascript in our templates | 00:56 |
| wallyworld_ | \o/ | 00:56 |
| huwshimi | wallyworld_: As I was editing this code already I thought I would try and move this out of the template now | 00:56 |
| wallyworld_ | huwshimi: so as far as i know, we don't yet have a well defined plan to deal with that | 00:56 |
| huwshimi | wallyworld_: I guess really it needs to happen with a proper js loader | 00:56 |
| wallyworld_ | yep | 00:57 |
| huwshimi | wallyworld_: maybe I'll leave this for now then | 00:57 |
| huwshimi | wallyworld_: Well actually I'll move it into its own file, but leave the call | 00:57 |
| wallyworld_ | huwshimi: the best you can do, afaik, is to just have the on domready stub in the template and the core business logic elsewhere | 00:57 |
| wallyworld_ | for now | 00:57 |
| huwshimi | wallyworld_: Yep, ok that will set it up for the future complete removal. Thanks :) | 00:58 |
| wallyworld_ | i've used this exact pattern where i need to be able to invoke the js in an xhr callback | 00:58 |
| wallyworld_ | as well as the page loaf | 00:58 |
| lifeless | benji: still around? I'm going to guess not ... | 00:58 |
| wallyworld_ | huwshimi: np. good luck with it | 00:58 |
| huwshimi | wallyworld_: Thanks | 00:58 |
| lifeless | wallyworld_: hi; did you see my ping earlier ? | 01:17 |
| lifeless | thumper: number of possible reviews - performance or visualisation ? | 01:18 |
| wallyworld_ | lifeless: ah yes, thanks. i tested it and yes it does do the right thing | 01:18 |
| thumper | lifeless: yes | 01:18 |
| thumper | lifeless: both | 01:18 |
| lifeless | thumper: so consider https://code.launchpad.net/launchpad-project/+activereviews | 01:18 |
| lifeless | thumper: or ubuntu/+activereviews | 01:19 |
| lifeless | thumper: for a developer in either project, thats going to be a large fraction - maybe 80 or 90 percent - of what we'd be showing on their ~/+activereviews, simply due to the activity of the projects | 01:19 |
| lifeless | thumper: or do you think it would be a much smaller fraction ? | 01:19 |
| thumper | approved merges from mid 2009 is a bit sad | 01:20 |
| lifeless | it is | 01:20 |
| thumper | lifeless: I have no metrics | 01:20 |
| thumper | so I can't make any real call on this | 01:20 |
| lifeless | one way to get some is to implement it, flagged, and see | 01:20 |
| thumper | sure | 01:21 |
| thumper | at the time of development we didn't have feature flags | 01:21 |
| lifeless | yeah, they make things easier :) | 01:21 |
| thumper | and I chose the solution that was less likely to explode with timeouts | 01:21 |
| thumper | but feel free to "fix" it | 01:21 |
| lifeless | :) I might scratch at the itch | 01:22 |
| lifeless | jml's request for -a- page is reasonable, the main thing I didn't understand is why that shouldn't be on the existing per user page | 01:22 |
| lifeless | since the data volume is ~the same either way. | 01:23 |
| StevenK | lifeless: And how do you determine a person is involved in the project? | 01:24 |
| lifeless | StevenK: current heuristic of being a reviewer for the branch | 01:24 |
| lifeless | StevenK: can iterate on that, obviously. | 01:24 |
| lifeless | StevenK: As a dashboard for reviews, +activereviews is pretty good. The only issue at the moment, for a single dev, is having to visit one per project | 01:25 |
| StevenK | Indeed. | 01:25 |
| lifeless | consider that looking at 'requested reviews I can do' for all projects in LP is equivalent to just seeing those you are a reviewer for - because the list is empty when you aren't an oficial reviewer. | 01:26 |
| huwshimi | wallyworld_: Do you mind doing a quick once over of what I did? https://code.launchpad.net/~huwshimi/launchpad/tag-cloud-removal-709009/+merge/81689 | 01:28 |
| * wallyworld_ looks | 01:28 | |
| wallyworld_ | huwshimi: for testability, you want to avoid Y.io | 01:33 |
| wallyworld_ | you want: lp_client = new Y.lp.client.Launchpad(); | 01:33 |
| wallyworld_ | lp_client.io_provider(xxx) | 01:34 |
| wallyworld_ | a bit pseudo ish | 01:34 |
| wallyworld_ | but the above allows a mock io provider to be used for tests | 01:34 |
| huwshimi | wallyworld_: Ah, I didn't do that part, but this is a good opportunity to fix it | 01:34 |
| wallyworld_ | see existing stuff in bugtask_index.js or elsewhere | 01:34 |
| huwshimi | wallyworld_: Thanks, I'll take a look | 01:35 |
| wallyworld_ | huwshimi: also, set('innerHTML') i think we are trying to avoid where possible, can't recall exactly, but setContext() may work | 01:35 |
| wallyworld_ | huwshimi: or new_node = Y.Node.create(xxxx); old_node.replace(new_node) | 01:36 |
| wallyworld_ | or something like that | 01:36 |
| huwshimi | wallyworld_: Sure, I'll fix that up too | 01:36 |
| wallyworld_ | i know that the server may generate only legit html and all that, but it is a potential security hole | 01:37 |
| wallyworld_ | other than that, looks ok | 01:37 |
| wallyworld_ | huwshimi: in case you don't notice it, the lp_client construction is usually inside a setup type function so the client is only constructed once | 01:38 |
| huwshimi | wallyworld_: Yeah ok thanks | 01:39 |
| wallyworld_ | good luck :-) there should be several examples to cargo cult. let me know if you are stuck | 01:39 |
| huwshimi | wallyworld_: No problems, will do | 01:39 |
| huwshimi | wallyworld_: I think it should be fine in the setup function I already have there. | 01:44 |
| wallyworld_ | the lp_client construction? yes, so long as setup is only called once which i guess it will be | 01:45 |
| wallyworld_ | you will want to make your setup function take a config dict as an argument so you can pass in the io_provider stub for tests | 01:46 |
| huwshimi | wallyworld_: Oh ok, in that case would it be easier to have its own function? | 01:47 |
| huwshimi | wallyworld_: Easier for testing that is | 01:47 |
| huwshimi | wallyworld_: I guess it would be | 01:48 |
| wallyworld_ | huwshimi: perhaps. getClient(config) or something | 01:48 |
| wallyworld_ | i think setup_lp_client(config) is also used, which assigns a module scoped lp_client variable | 01:49 |
| wallyworld_ | then the code can just use lp_client and assume it's been all set up | 01:49 |
| wallyworld_ | and the test would call setup_lp_client() with the correct config to pass in the io+provider | 01:50 |
| wallyworld_ | whereas the prod code would call with no confif | 01:50 |
| huwshimi | wallyworld_: Should that setup function check to see if a lp_client already exists and if so do nothing? | 02:01 |
| wallyworld_ | huwshimi: yes | 02:01 |
| huwshimi | wallyworld_: Ah good | 02:01 |
| wallyworld_ | so var lp_client = null; somewhere in the module, and then the setup function sets it | 02:02 |
| huwshimi | wallyworld_: Yup | 02:03 |
| huwshimi | wallyworld_: Well, "var client;" is enough right? | 02:04 |
| huwshimi | erm | 02:04 |
| huwshimi | var lp_client; | 02:04 |
| wallyworld_ | i prefer to be explicit, but your call | 02:04 |
| huwshimi | wallyworld_: I don't mind. I'd prefer to be consistent | 02:04 |
| wallyworld_ | sure. i'm not sure what our coding standard says for that to be honest | 02:05 |
| huwshimi | wallyworld_: let me check | 02:05 |
| huwshimi | wallyworld_: Most of these files seem to just declare the variable name | 02:05 |
| wallyworld_ | sounds good | 02:06 |
| huwshimi | wallyworld_: The docs don't seem to say though | 02:08 |
| wallyworld_ | as you say, just use what's gone before :-) | 02:08 |
| huwshimi | wallyworld_: Ok, here's where I'm at: https://code.launchpad.net/~huwshimi/launchpad/tag-cloud-removal-709009/+merge/81689 | 02:27 |
| * wallyworld_ looks | 02:28 | |
| wallyworld_ | huwshimi: i have led you astray a bit i'm sorry. if all you are doing is making a straight Y.io call and don't need the lp_client extras, you just do: | 02:35 |
| wallyworld_ | var io_provider = Y.lp.client.get_configured_io_provider(config) | 02:35 |
| wallyworld_ | where config is what's passed into your setup | 02:35 |
| wallyworld_ | it is either undefined or a dict with an io_provider key | 02:35 |
| wallyworld_ | if you do need the full lp_client, you do lp_client = new Y.lp.client.Launchpad(config); | 02:37 |
| wallyworld_ | so in your current code, you are missing the config param for constructing the lp_client, but as per the above, you may not need that | 02:37 |
| lifeless | \o/ no oopses on asuka since the 2nd (all via amqp) | 02:38 |
| huwshimi | lifeless: Oh, that config parameter did exist, I lost it at some point | 02:38 |
| huwshimi | lifeless: sorry that was meant of wallyworld_ | 02:38 |
| huwshimi | wallyworld_: Or maybe it only ever existed in the code in my brain | 02:39 |
| wallyworld_ | huwshimi: perhaps :-) | 02:39 |
| lifeless | older for tellurium. | 02:39 |
| wallyworld_ | so if you want, you can just get yourself the io_provider | 02:39 |
| wallyworld_ | good news about the oopses | 02:39 |
| huwshimi | wallyworld_: So this is how it should look? http://paste.ubuntu.com/733801/ | 02:41 |
| * wallyworld_ looks | 02:46 | |
| wallyworld_ | huwshimi: that looks ok to me. your test setup then just calls setup_io_provider(). if io_provider is only used the once in setup_taglist, you may consider ditching the separate setup | 02:48 |
| huwshimi | wallyworld_: yeah ok | 02:48 |
| wallyworld_ | so make setup_taglist take the config param | 02:49 |
| huwshimi | wallyworld_: ok sure | 02:50 |
| wallyworld_ | huwshimi: sorry about the false start before, i'm used to needed the whole lp_client object | 02:51 |
| huwshimi | wallyworld_: No, all good, I've learnt some things | 02:51 |
| wallyworld_ | excellent | 02:51 |
| wallyworld_ | the mock io provider we have is pretty good | 02:51 |
| wallyworld_ | lp.app.javascript.testing.mockio.js | 02:52 |
| huwshimi | wallyworld_: Should I still have a check for if a io_provider already exists? | 02:54 |
| huwshimi | wallyworld_: Or is that really only important for the lp_client | 02:54 |
| wallyworld_ | huwshimi: it doesn't really matter in this case because it's just grabbing a value from a dict | 02:56 |
| huwshimi | wallyworld_: ok sure | 02:57 |
| wgrant | lifeless: bugsummary is pretty impossible to modify :/ | 03:01 |
| wgrant | lifeless: There's like 5 patches, and they weren't flattened into trusted.sql. | 03:01 |
| wgrant | And they aren't in order :*( | 03:01 |
| huwshimi | wallyworld_: Ok, hopefully this is what you meant :) https://code.launchpad.net/~huwshimi/launchpad/tag-cloud-removal-709009/+merge/81689 | 03:08 |
| lifeless | wgrant: sorry :) - twitch. | 03:08 |
| lifeless | hey, anyone know if there is a good 'touched recently' field on question ? | 03:08 |
| wgrant | I guess we need a new baseline schema soon anyway. | 03:12 |
| wgrant | lifeless: you probably have to take the max of all the date fields :/ | 03:13 |
| lifeless | just query response I think will do | 03:15 |
| lifeless | they seem to combine to cover it | 03:15 |
| wgrant | They even cover answers? | 03:16 |
| lifeless | last-response | 03:16 |
| lifeless | last-response + last-query forms the rule for who needs to act next | 03:16 |
| wgrant | Right, but what about a proposed answer? | 03:17 |
| lifeless | arguably doesn't need to cover messages - they are scanned separately | 03:17 |
| wgrant | Erm, what does it cover, then? | 03:17 |
| wgrant | Is datecreated OK? | 03:17 |
| lifeless | whiteboard, title description | 03:17 |
| wgrant | Oh, whiteboard. | 03:17 |
| wgrant | Blah. | 03:17 |
| lifeless | so datecreated isn't enough | 03:17 |
| lifeless | down to 5s query | 03:18 |
| lifeless | big improvement | 03:18 |
| lifeless | (for 1 week) | 03:18 |
| lifeless | seq scanning question | 03:21 |
| wallyworld_ | huwshimi: i would reverse the config and io_config names, otherwise ok i think. the setup config can contain more than just io_provider. and io_config seems better suited for use in the io call | 03:27 |
| huwshimi | wallyworld_: Ok, no problems | 03:28 |
| lifeless | 500ms with two simple indices | 03:30 |
| lifeless | OTOH I could just limit those to their containing scope. | 03:30 |
| lifeless | probably easier I think. | 03:30 |
| wgrant | lifeless: For a full week? | 03:30 |
| lifeless | yes | 03:31 |
| huwshimi | wallyworld_: OK, done, https://code.launchpad.net/~huwshimi/launchpad/tag-cloud-removal-709009/+merge/81689 | 03:34 |
| * wallyworld_ looks | 03:34 | |
| wgrant | jtv: For arch-parallel a-f, see shelf 21 on DF. | 03:36 |
| wallyworld_ | huwshimi: one more improvement - use start and end callbacks in to show/hide spinner | 03:37 |
| wallyworld_ | start and end callbacks in io_config | 03:37 |
| wallyworld_ | that way you don't need to call hide in success handler | 03:37 |
| wallyworld_ | and again in failure callback | 03:37 |
| wallyworld_ | use something like "end: Y.bind(namespace.hideSpinner, namespace)" | 03:38 |
| huwshimi | wallyworld_: What does the bind give me that just doing "end: namespace.hide_spinner" doesn't? | 03:43 |
| wallyworld_ | huwshimi: ah, sorry. i referenced an example where i had to pass a parameter to the hide api call, and i think bind is required then | 03:44 |
| huwshimi | wallyworld_: Ah right | 03:44 |
| huwshimi | np | 03:44 |
| wallyworld_ | the js tests will ensure that it all works in any case | 03:44 |
| StevenK | I'd like to return count(archive.id) in a SELECT query, but only if it's > 1 | 03:46 |
| StevenK | Oh, wait, they will not disappear. | 03:46 |
| wgrant | StevenK: HAVING | 03:46 |
| StevenK | So I need to filter on state | 03:46 |
| wgrant | SELECT owner, COUNT(*) FROM archive WHERE status = 0 GROUP BY owner HAVING COUNT(*) > 1 | 03:47 |
| StevenK | wgrant: Can I do select count(...) AS foo .... HAVING foo > 1 ? | 03:47 |
| wgrant | Probably. | 03:48 |
| * StevenK tries | 03:48 | |
| StevenK | Probably not, that is. | 03:48 |
| lifeless | ok | 03:49 |
| lifeless | wgrant: what did you want me to look at ? | 03:49 |
| lifeless | now I have my query for oops pruning sorted, I'll look at your thing | 03:49 |
| wgrant | lifeless: Thanks. | 03:49 |
| wgrant | lifeless: So, http://paste.ubuntu.com/732768/ is the suggested undenormalisation. | 03:50 |
| wgrant | lifeless: It's only ~10% slower for Ubuntu. | 03:50 |
| wgrant | But 2.5x slower for ubuntuone-servers, because this ends up doing a seqscan on apg. | 03:50 |
| wgrant | Not sure why. | 03:50 |
| lifeless | I guess I need to regen sample data | 03:51 |
| lifeless | ah itsi nthe bin | 03:51 |
| wgrant | The paste contains 4 queries at the end. | 03:51 |
| wgrant | The third is fastest. | 03:51 |
| StevenK | Sample data is in the bin? | 03:51 |
| wgrant | It's ~600ms for Ubuntu, vs 1200-1500ms for the other three. | 03:51 |
| wgrant | But it still does a seqscan. | 03:51 |
| wgrant | And is 120msish for ubuntuone-servers, rather than the old denormed 30-40ms version. | 03:52 |
| lifeless | the old version was slow on ubuntu ? | 03:52 |
| lifeless | or is this just data that the oldver was better overall | 03:52 |
| lifeless | ? | 03:52 |
| wgrant | Your version was just under 600ms for Ubuntu on DF. | 03:52 |
| wgrant | The new one is a little over. | 03:52 |
| lifeless | whats changed in the schema ? | 03:53 |
| wgrant | APG.{artifact,policy} are mutually exclusive, and APA.policy exists. | 03:53 |
| wgrant | So each artifact's policy is stored on APA, instead of many times in APG. | 03:53 |
| lifeless | ah yes | 03:54 |
| lifeless | so this is a normalisation | 03:54 |
| lifeless | not a denorm :P | 03:54 |
| wgrant | undenorm, I said. | 03:54 |
| wgrant | It's not normalising, it's reverting the denormalisation :) | 03:54 |
| lifeless | double negative parsing fail | 03:54 |
| wgrant | Heh | 03:55 |
| lifeless | ok, which query sucks again ? | 03:55 |
| wgrant | The four at the end all do the same thing. The third is fastest. | 03:56 |
| wgrant | One CTE and a couple of subselects. | 03:56 |
| wgrant | Try against product 9514 to see the greater slowness. | 03:58 |
| wgrant | But the plans are the same. | 03:58 |
| wgrant | Er, the plans for distro 1 and product 9514, that is. | 03:58 |
| lifeless | wgrant: plan 3, product 9514 - 114ms | 04:08 |
| lifeless | plan 4 65ms | 04:09 |
| wgrant | Bah, really? | 04:09 |
| wgrant | Indeed, 30% faster. | 04:12 |
| wgrant | But slower for Ubuntu. | 04:12 |
| lifeless | http://paste.ubuntu.com/733853/ | 04:12 |
| lifeless | thats the plans you had | 04:13 |
| wgrant | #3 there has a different context (ubuntuone-servers rather than ubuntu), but yes. | 04:14 |
| lifeless | bah | 04:14 |
| jtv | wgrant: do you mean that parallel a-f has been sitting around as uncompleted work? That would be a serious waste. | 04:14 |
| lifeless | wgrant: 410ms for #3 on ubuntu | 04:14 |
| wgrant | jtv: Well, we don't really know how much of a win it is. | 04:14 |
| lifeless | wgrant: do you want the plan ? | 04:15 |
| wgrant | lifeless: That's 650ms on DF. #4 is 1.1s. So it is behaving somewhat differently to qastaging :/ | 04:15 |
| wgrant | lifeless: Not really. | 04:15 |
| lifeless | wgrant: try raising your stats | 04:15 |
| jtv | wgrant: thing is, it won't be at all hard to make the MF manage a sensibly-sized pool of processes. So if 2 parallel processes is a win but 5 aren't, we can do 2. | 04:15 |
| huwshimi | wallyworld_: I made the change to start/end and also changed the show show/hide to using classes instead of styles. As the show/hide are one liners do you think they should still be in separate functions? | 04:16 |
| huwshimi | wallyworld_: https://code.launchpad.net/~huwshimi/launchpad/tag-cloud-removal-709009/+merge/81689 | 04:16 |
| wgrant | lifeless: Mmm, the slow bits of the plan aren't really different | 04:18 |
| lifeless | wgrant: those were all hot btw | 04:19 |
| wgrant | The plan for #3 on 9514 is identical, and performance is almost the same. But it still hsa the seq scan, which seems unfortunate. | 04:20 |
| wgrant | But if it's only 410ms for Ubuntu then I guess that's not too bad... | 04:20 |
| lifeless | wgrant: 123557 rows | 04:21 |
| lifeless | wgrant: 16ms to seq scan | 04:21 |
| lifeless | wgrant: (just to read the rows once through, no processing) | 04:21 |
| wgrant | lifeless: How'd you measure that? SELECT *? | 04:22 |
| lifeless | wgrant: yes | 04:23 |
| lifeless | 59785 rows are ubuntu | 04:23 |
| lifeless | wgrant: thats 50% - seqscan is always going to be fastest ;) | 04:23 |
| wgrant | That takes 160ms on DF. | 04:24 |
| wgrant | Hmm. | 04:24 |
| wgrant | But that can't be right, so maybe not. | 04:24 |
| wgrant | lifeless: Yeah. | 04:24 |
| lifeless | http://paste.ubuntu.com/733866/ | 04:24 |
| wgrant | But U1 and other products will be smaller. | 04:24 |
| lifeless | sure | 04:24 |
| lifeless | 9514 was the 'slow' one right ? | 04:24 |
| wgrant | 100ms instead of 40ms, yes. | 04:24 |
| wgrant | 10294 is LP, which may be interesting. | 04:25 |
| lifeless | 4K grants for 10294 and 9514 each | 04:26 |
| lifeless | sorry, 60K for ubuntu, not 600K | 04:26 |
| lifeless | my eyes, me eyes, my eyes | 04:26 |
| lifeless | wgrant: still 5% of a table with narrow rows will hit ~ all pages unless its clustered on the selector | 04:29 |
| wgrant | True. | 04:30 |
| wgrant | It's also only a 9MB table and should be nice and hot. | 04:30 |
| lifeless | wgrant: you can defeat the seqscan by unioning | 04:35 |
| lifeless | if you want | 04:35 |
| wgrant | Yeah, but that's really slow for Ubuntu, at least on DF. | 04:36 |
| StevenK | distroseries-edit-datereleased => devel [OK] (up for 4:39:27.158866) | 04:36 |
| StevenK | Oh, come on, ec2. | 04:36 |
| lifeless | 800ms here | 04:36 |
| lifeless | wgrant: 180ms in | 04:38 |
| lifeless | -> Index Scan using accesspolicygrant__artifact__person__key on accesspolicygrant (cost=0.00..7.14 rows=2 width=8) (actual time=0.005..0.006 rows=2 loops=33978) | 04:38 |
| lifeless | Index Cond: (pg_temp_43.accesspolicygrant.artifact = accesspolicyartifact.id) | 04:38 |
| lifeless | rm, something like that | 04:38 |
| StevenK | # XXX cprov 2007-02-22 bug=87098: | 04:38 |
| StevenK | # Since we only allow one PPA for each user, | 04:38 |
| StevenK | Ha. Hahaha. Hahahahahahaahahahahahaha | 04:38 |
| poolie | hm | 04:39 |
| poolie | so the buildd package can be installed pretty easily as a dpkg on developer systems | 04:39 |
| wgrant | lifeless: OK, 800ms is better than I get, but still 100% slower than #3 | 04:39 |
| poolie | at the moment it automatically starts a buildd | 04:39 |
| poolie | i suspect this is not very safe or desirable | 04:39 |
| wgrant | poolie: It's a nice easy remote root vulnerability, right. | 04:40 |
| poolie | i'm inclined to handle it by splitting off an "actually run it" package | 04:40 |
| poolie | which can be installed on the real buildds | 04:40 |
| poolie | with that still called launchpad-buildd | 04:40 |
| poolie | and then eg launchpad-buildd-bin or python-lpbuildd | 04:40 |
| poolie | or maybe both | 04:40 |
| poolie | that has all the stuff that will be developer dependencies | 04:40 |
| wgrant | Mmm. | 04:41 |
| wgrant | I'd prefer not to have launchpad-buildd in any readily accessible apt repo. | 04:41 |
| poolie | :) | 04:41 |
| poolie | someone will run it | 04:41 |
| wgrant | Yes. | 04:41 |
| poolie | well, i could take this course and just not upload it | 04:41 |
| wgrant | Huh. | 04:41 |
| wgrant | Speaking of remote root vulnerabilities. | 04:42 |
| wgrant | lolpython | 04:42 |
| poolie | the other way these things tend to be handled is to have the daemon off by default and an 'are you really sure' in /etc/default/something | 04:42 |
| poolie | ? | 04:42 |
| wgrant | yaml.load isn't safe. It loads pickles. | 04:42 |
| wgrant | What a sensible default. | 04:42 |
| poolie | seriously? | 04:43 |
| wgrant | Yeah. | 04:43 |
| wgrant | You have to use yaml.safe_load for that. | 04:43 |
| poolie | as a kind of dwim if you feed it a pickle? | 04:43 |
| wgrant | Not sure. I guess they define a way for YAML to contain pickles. | 04:44 |
| jtv | Say wgrant, how do I dput a package onto a PPA on dogfood? | 04:45 |
| StevenK | wallyworld_: What happened to TPG? | 04:45 |
| wallyworld_ | ? | 04:45 |
| poolie | wgrant, not pickles as such as far as i can see | 04:45 |
| poolie | but you can call python objects from inside it | 04:46 |
| StevenK | wallyworld_: "rev.aanet.com.au" | 04:46 |
| wgrant | Yeah, not actual pickles. | 04:46 |
| wgrant | Its own implementation. | 04:46 |
| wgrant | But still. | 04:46 |
| poolie | it's similar to doing eval | 04:46 |
| wgrant | Who could possibly think that was a reasonable idea. | 04:46 |
| wallyworld_ | StevenK: oh, i've relocated today to a different place, change of scenery | 04:46 |
| poolie | anyhow | 04:46 |
| poolie | > off by default and an 'are you really sure' in /etc/default/something | 04:46 |
| poolie | the main drawback is | 04:46 |
| wgrant | jtv: [df] | 04:47 |
| wgrant | method = ftp | 04:47 |
| wgrant | fqdn = upload.dogfood.launchpad.net:21 | 04:47 |
| wgrant | incoming = %(df)s | 04:47 |
| wgrant | login = anonymous | 04:47 |
| poolie | losas need to turn it on i guess | 04:47 |
| wgrant | jtv: dput df:ubuntu something_source.changes | 04:47 |
| poolie | and, some people will turn it on and get in to trouble | 04:47 |
| wgrant | poolie: And it breaks the automated installation. | 04:47 |
| lifeless | wgrant: plan three is 640ms | 04:47 |
| StevenK | wgrant: That's to ubuntu, not a PPA | 04:47 |
| jtv | wgrant: so no mention of the PPA at all? | 04:47 |
| wgrant | lifeless: How fast was your query? | 04:47 |
| lifeless | wgrant: uhm, variable. just got a 450 | 04:47 |
| poolie | so... | 04:47 |
| wgrant | jtv: Ah, for a PPA. Use a PPA path. | 04:47 |
| lifeless | wgrant: lots of noise | 04:47 |
| wgrant | lifeless: Er, your query from a couple of days ago, that is. | 04:48 |
| wgrant | I can't remember. | 04:48 |
| wgrant | Was 300ms or something. | 04:48 |
| lifeless | oh right | 04:48 |
| lifeless | but that was old schema wasn't it ? | 04:48 |
| wgrant | It was. | 04:48 |
| jtv | wgrant: is a PPA path something like “ppa:user/ppa”, without changes for dogfood? | 04:48 |
| wgrant | jtv: The FTP path is ~user/ppa/distribution | 04:49 |
| wgrant | So df:~user/ppa/distribution | 04:49 |
| jtv | Oh | 04:49 |
| jtv | Thanks | 04:49 |
| wgrant | lifeless: But I would like to get a real performance comparison between the two schemas. | 04:52 |
| wgrant | lifeless: The queries in the denormed one are already slow. | 04:52 |
| wgrant | StevenK: Erm. | 04:54 |
| StevenK | wgrant: Hm? | 04:54 |
| wgrant | Ah, I see. | 04:55 |
| wgrant | Just somewhat bemused at the misspelt instance variables your rev added. | 04:56 |
| wgrant | but there was actually a reason for it, and the misspelling wasn't yours. | 04:56 |
| StevenK | Oh? | 04:56 |
| jtv | stub: any luck with Q/A for bug 826653? | 04:56 |
| _mup_ | Bug #826653: preflight check could report patches to be applied <fastdowntime-later> <qa-needstesting> <Launchpad itself:Fix Committed by stub> < https://launchpad.net/bugs/826653 > | 04:56 |
| stub | Needed to wait for a staging update... lets have a look | 04:57 |
| wgrant | StevenK: "derivative" is not spelt "derivitive", and it's sort of odd to set the instance variables in setUpFields. | 04:57 |
| stub | Probably should avoid a rollout today since we are doing the slony upgrade btw. | 04:57 |
| wgrant | Do we have a GSA scheduled to do the package swap? :) | 04:58 |
| jtv | stub: that's a shame—we could roll out all 14 available revisions once your Q/A on this is done. | 05:01 |
| stub | qa-ok | 05:01 |
| stub | wgrant: I thankfully don't have to juggle those details now ;) | 05:01 |
| wgrant | stub: Oh? | 05:02 |
| jtv | thanks stub | 05:02 |
| stub | Other people schedule the time | 05:02 |
| wgrant | Well, as long as someone knew a GSA was required... | 05:03 |
| StevenK | stub: What should I do with https://code.launchpad.net/~stevenk/launchpad/db-packageupload-archive-index/+merge/79903 ? | 05:03 |
| poolie | biab | 05:04 |
| wgrant | lifeless: So, do you have a feeling as to which one is better? Your numbers are probably better than mine. Is the denorm worth it? | 05:06 |
| stub | StevenK: Land it? | 05:07 |
| StevenK | stub: Last time I spoke to you, you wanted me to hold off a few days. It's fine to land and have the indexes dropped? | 05:07 |
| wgrant | st[a-z]+/i: Doesn't that drop the index, which we were going to do once we determined it was unused? | 05:07 |
| StevenK | Hahaa | 05:07 |
| stub | StevenK: I'll look | 05:07 |
| stub | I recall now | 05:08 |
| lifeless | wgrant: still fiddling | 05:08 |
| lifeless | wgrant: the full is trivial - 4ms | 05:08 |
| wgrant | lifeless: Ah, k | 05:08 |
| StevenK | stub: And sorry for dropping the ball :-( | 05:08 |
| wgrant | lifeless: Huh? | 05:08 |
| lifeless | wgrant: seeing if I can get a better restricted result | 05:08 |
| wgrant | Oh. | 05:08 |
| wgrant | Right. | 05:08 |
| wgrant | Yes, there aren't many full observers. | 05:08 |
| lifeless | wgrant: so I'm fiddling without the full case | 05:08 |
| wgrant | It's possibly beneficial to split the table, but let's see what you find. | 05:08 |
| lifeless | wgrant: denorming the sort key onto apg might be good | 05:10 |
| lifeless | if you want to make it fly | 05:10 |
| wgrant | lifeless: It would be. | 05:10 |
| wgrant | But, uh, let's not go there... | 05:10 |
| wgrant | yet, at least. | 05:10 |
| stub | So packageupload.signing_key is never used. But probably should leave that. | 05:12 |
| wgrant | stub: The index? | 05:13 |
| wgrant | Hmmm. | 05:13 |
| wgrant | Odd. | 05:13 |
| wgrant | It should be. | 05:13 |
| stub | package_copy_job_idx is not used much | 05:13 |
| stub | yes | 05:13 |
| wgrant | package_copy_job_idx? | 05:13 |
| wgrant | What's that? | 05:13 |
| wgrant | It doesn't match our usual format, and I can't see it on packagecopyjob | 05:14 |
| stub | id_distroseries_archive is good to drop - it is unused | 05:14 |
| wgrant | Oh. | 05:14 |
| wgrant | On PackageUpload. i see. | 05:14 |
| StevenK | stub: There are not many PCJs floating around | 05:14 |
| StevenK | wgrant and I have a plan to make more | 05:14 |
| stub | distroseries_status_idx is still used, but I suspect those uses will switch to the other index if it is dropped | 05:14 |
| wgrant | id_distroseries__archive seems like a completely bad idea. | 05:15 |
| lifeless | wgrant: denorming the sort key onto grant is insufficient | 05:16 |
| lifeless | wgrant: these are all under 1s | 05:16 |
| lifeless | wgrant: and all doing full set ops, so sustainable | 05:16 |
| stub | http://paste.ubuntu.com/733885/ is before, http://paste.ubuntu.com/733886/ is after if anyone wants to see the raw stats | 05:16 |
| lifeless | wgrant: my recommendation is this: | 05:16 |
| stub | Timeframe was around when I last discussed this with StevenK :-) | 05:16 |
| StevenK | packageupload__signing_key__idx | 05:16 |
| lifeless | - A) go with this normalised schema | 05:16 |
| StevenK | That's totally unused, should we just drop it? | 05:16 |
| lifeless | - B) if its too slow create a fact table from it (like bugsummary) that can answer in a few ms. | 05:17 |
| wgrant | Sounds good. | 05:17 |
| wgrant | Thanks. | 05:17 |
| stub | StevenK: If inserts or updates are not too slow, leave the signingkey index. It is good to always have indexes on foreign key references to avoid surprises. | 05:17 |
| StevenK | stub: Fair enough, if you're happy dropping the two indices, let's land it, then | 05:18 |
| stub | If there are still queries looking for distroseries without an archive, are they broken? | 05:20 |
| StevenK | Yes | 05:20 |
| wgrant | Hmm. | 05:20 |
| StevenK | Since they'll conflate PPAs and the primary archive | 05:20 |
| wgrant | Except possibly Distribution:+ppas. | 05:20 |
| wgrant | Let's see what queries that does. | 05:20 |
| StevenK | stub: I thought we seperated columns by __ in indicies, so I'm surprised by id_distroseries... | 05:22 |
| stub | We still got 70k queries hitting the distro__status index. If we drop it, the alternatives are the archive__distro__status index and the distro index. Neither would be good if there is a valid use for queriying all archives for a distro,status | 05:22 |
| wgrant | StevenK: It's a typo. | 05:22 |
| stub | StevenK: Old naming conventions | 05:22 |
| StevenK | Ah | 05:22 |
| stub | and that would be a typo anyway | 05:22 |
| StevenK | Heh | 05:22 |
| stub | Possibly that is there to support a foreign key reference? Or maybe a failed attempt to optimize a query. | 05:23 |
| stub | c/is/was/ | 05:23 |
| StevenK | Conjecture, hearsay ... :-) | 05:23 |
| StevenK | But it looks like that index has been there for a long long while | 05:24 |
| wgrant | No. | 05:24 |
| wgrant | It's new. | 05:24 |
| wgrant | IIRC lifeless added it. | 05:24 |
| wgrant | To optimise DistroSeries:+queue for NEW/ACCEPTED. | 05:25 |
| wgrant | I don't see how it could possibly work, though :) | 05:25 |
| StevenK | Haha | 05:25 |
| lifeless | which one? | 05:25 |
| wgrant | "packageupload__id_distroseries__archive__idx" btree (id, distroseries, archive) WHERE status = ANY (ARRAY[0, 1]) | 05:25 |
| wgrant | Primary key first in the index... not entirely useful. | 05:25 |
| wgrant | I guess that packageupload(status) would do the same thing. | 05:26 |
| wgrant | -- Partial index for queue processing view : the lopsided data makes this | 05:26 |
| wgrant | -- necessary (millions of rows matching the archive, 100's of rows match the | 05:26 |
| lifeless | made by me, landed by you | 05:26 |
| wgrant | -- query). | 05:26 |
| wgrant | Really? | 05:27 |
| wgrant | No. | 05:27 |
| lifeless | check annotate :P | 05:27 |
| StevenK | But it's +queue, it's for a tiny amount of archives | 05:27 |
| wgrant | committer: Robert Collins <robert@canonical.com> | 05:27 |
| wgrant | Create a partial index to support queue processing queries more efficiently. | 05:27 |
| lifeless | you put the db revno in it | 05:27 |
| wgrant | committer: William Grant <william.grant@canonical.com> | 05:27 |
| wgrant | branch nick: fix-patch-57 | 05:27 |
| wgrant | Live-deployed patches need a non-zero patch number. Production is already fixed. | 05:27 |
| lifeless | ah | 05:27 |
| lifeless | ok | 05:27 |
| lifeless | anyhow, if its not used, remove it; it would have been added based on the queries seen at the time | 05:28 |
| wgrant | StevenK: Yeah, but as the comment says it's very lopsided. | 05:28 |
| wgrant | lifeless: It probably is still useful. | 05:28 |
| wgrant | But it's excessive. | 05:28 |
| lifeless | note that a sort by id can trigger that index. | 05:28 |
| stub | It hasn't been hit once in a few weeks. We don't need it. | 05:28 |
| StevenK | stub: So, that's a +1 to toss that branch at db-devel? | 05:29 |
| stub | I'm interested in if we really should drop the distroseries__status__idx index. | 05:29 |
| wgrant | Distribution:+ppas uses SPPH, not PU. | 05:29 |
| wgrant | It would be nice if we could find out what was using that index. | 05:30 |
| stub | So it seems that if there are still queries not using archive as a filter on packageupload, they are broken? That would also explain why the id_distroseries index is never hit. | 05:30 |
| stub | ignore that | 05:30 |
| stub | So it seems that if there are still queries not using archive as a filter on packageupload, they are broken? | 05:31 |
| wgrant | stub: Well, or the data is sufficiently skewed (archive 1 still has more than 50% of the table, probably) that the planner decides using archive is not useful. | 05:31 |
| stub | It would still use the archive__distroseries__status index I think | 05:32 |
| lifeless | what I've seen happen a couple of times is that transforming the query very slightly will cause indices not to be used | 05:32 |
| wgrant | Very slightly :( | 05:32 |
| lifeless | so it may just be a minor change during derived distros | 05:33 |
| lifeless | stopped it being used | 05:33 |
| wgrant | There have been no changes of that sort, really. | 05:33 |
| wgrant | But maybe. | 05:33 |
| stub | If inserts and updates are not causing us grief, I'm happy enough to leave the index in there. PG seems to still like using it for whatever reason. | 05:33 |
| StevenK | Right, so just id_distroseries dies | 05:34 |
| stub | Yup | 05:35 |
| StevenK | stub: That's landed. r11140 on db-devel | 05:57 |
| poolie | so if i want to get /usr/share/launchpad-buildd onto the pythonpath when launchpad runs, what would be a clean way to do that....? | 06:01 |
| poolie | maybe just in the makefile | 06:05 |
| lifeless | poolie: buildout | 06:13 |
| poolie | buildout.cfg | 06:13 |
| poolie | yeah got it | 06:13 |
| lifeless | poolie: I'm curious why you say this isn't really python | 06:14 |
| poolie | it contains perl :-P | 06:14 |
| wgrant | It's not a Python package. | 06:15 |
| poolie | and the package as a whole just seems oriented towards being a dpkg not an egg | 06:15 |
| wgrant | It's an application. | 06:15 |
| wgrant | Which happens to use an internal Python package. | 06:15 |
| poolie | obviously python things can contain arbitrary stuff | 06:15 |
| poolie | right | 06:15 |
| lifeless | so, why do you want it on the python path then ? | 06:15 |
| poolie | i want the python stuff on the path | 06:16 |
| wgrant | poolie: I would just put it in sourcecode/ for now. | 06:16 |
| wgrant | Rather than disabling it by default and not mangling /etc/sudoers and etc. | 06:16 |
| poolie | i could split it into one part distributed as an egg and one part as a dpkg | 06:16 |
| poolie | wgrant, i think i'm going to go for two dpkgs | 06:16 |
| wgrant | :/ | 06:17 |
| poolie | why? | 06:17 |
| poolie | lifeless semi-vetoed sourcecode | 06:17 |
| wgrant | And I am mostly-vetoing dpkg. | 06:17 |
| lifeless | why? | 06:17 |
| wgrant | It's not a package that should be available to anyone. | 06:17 |
| wgrant | It's terribly dangerous. It is badly behaved. | 06:18 |
| lifeless | what should we use instead? | 06:18 |
| wgrant | sourcecode. | 06:18 |
| wgrant | I suspect. | 06:18 |
| wgrant | Until we have a proper fake version. | 06:18 |
| poolie | are my changes making it any more dangerous? | 06:19 |
| poolie | people could run it today and get into trouble | 06:19 |
| wgrant | You're proposing putting the package into a public archive and encouraging its installation. | 06:19 |
| poolie | i'm happy to keep the part that actually starts the daemon out of a public archive | 06:20 |
| wgrant | How? | 06:20 |
| poolie | not uploading it? | 06:20 |
| poolie | uploading a poisoned highly-versioned package that does nothing? | 06:21 |
| wgrant | Having the source package sometimes build the evil package and sometimes not sounds like a recipe for trouble. | 06:21 |
| wgrant | This is presumably only a temporary measure. | 06:21 |
| wgrant | LP shouldn't need the buildd tree. | 06:22 |
| poolie | perhaps we can detect if the package is running on a 'real' buildd and abort otherwise? | 06:22 |
| poolie | wgrant, i'm trying to make lp depend only on the parts it actually needs to test | 06:22 |
| poolie | which don't include anything running as root | 06:22 |
| poolie | i guess we could do a further split of the 'actually really run it as root' scripts into a separate source tree | 06:23 |
| wgrant | lifeless: What is the policy on test fakes these days? | 06:23 |
| lifeless | requires for new microservices | 06:24 |
| lifeless | this is in a grey area | 06:24 |
| lifeless | I would vastly prefer one | 06:24 |
| lifeless | but split out without one is better than not split out | 06:25 |
| poolie | ... | 06:25 |
| lifeless | and poolie has a record of continuing to tug on things | 06:25 |
| poolie | so, meaning what exactly? | 06:25 |
| poolie | launchpad tests do not need the actual buildd implementation at all? | 06:25 |
| lifeless | ELOCAL, bbs | 06:25 |
| wgrant | poolie: They shouldn't. | 06:25 |
| poolie | i would prefer that | 06:25 |
| wgrant | poolie: Why would they? | 06:25 |
| wgrant | They need something that looks like it. | 06:26 |
| wgrant | They clearly don't need the actual thing, since it can only run as root. | 06:26 |
| poolie | there are a couple of answers | 06:26 |
| wgrant | We care that it speaks the protocol correctly. | 06:26 |
| poolie | one is to reduce risk while splitting things i want to keep as much test coverage as i can at least during the transition | 06:26 |
| poolie | because there are no good automated overall integration tests | 06:27 |
| wgrant | Right. | 06:27 |
| poolie | so... | 06:27 |
| poolie | i guess i could look at just redoing those tests now | 06:27 |
| wgrant | I suggest that the cheapest thing to do right now, until we have a strategy to move forward, is to use sourcecode. I don't like it either, but I think it's the best thing for now. | 06:27 |
| poolie | your main concern being that people will blithely install the buildd package and kill themselves | 06:28 |
| wgrant | And LP doesn't really like using packages for Pythonish deps. | 06:28 |
| wgrant | And lp-buildd doesn't make sense as an egg. | 06:29 |
| wgrant | And lp-buildd deployment is fragile and obscure enough that I'd really really not like to change it right now by splitting the packages up. | 06:29 |
| poolie | perhaps we should just look at the tests that do depend on it | 06:29 |
| poolie | and see if they can easily be changed right now | 06:29 |
| wgrant | Possibly, but we'd probably need to rewrite significant volumes of tests on both sides. | 06:30 |
| wgrant | Worth a try, though, | 06:30 |
| wgrant | jtv's been in this area lately. | 06:30 |
| wgrant | Hi jtv. | 06:30 |
| jtv | I deny everything. | 06:30 |
| poolie | so the connections are | 06:30 |
| poolie | BuilddSlaveTestSetup (a fixture) | 06:31 |
| wgrant | jtv: You've been dealing with buildd integration and tests thereof lately, right? | 06:31 |
| poolie | some things in translations that use generate_translations_templates and also write to the lp db | 06:31 |
| jtv | One particular aspect. | 06:31 |
| poolie | and the codehost | 06:31 |
| poolie | the utility 'pottery-generate-intltool' | 06:32 |
| poolie | and that's about it | 06:32 |
| poolie | so the buildmaster things could perhaps be satisfied with a fake | 06:32 |
| poolie | the translations tests really look like integration tests to do with how code inside the buildd runs | 06:33 |
| jtv | Everything related to buildd looks like integration tests. :/ | 06:33 |
| poolie | i guess they need to be split into tests that cover the two respective sides of the handoff | 06:33 |
| jtv | The buildd infrastructure isn't easily split into “sides” of communication, unfortunately. | 06:34 |
| jtv | (Also, there are at least three sides) | 06:34 |
| wgrant | jtv: Master and slave? | 06:34 |
| wgrant | What's the third side? | 06:34 |
| jtv | Librarian. | 06:34 |
| poolie | and codehosting too i think | 06:34 |
| wgrant | Mm, slightly. | 06:34 |
| lifeless | ideally three sets - in-lp, in-buildd, and a smattering of end to end tests run on jenkins [note that we don't have these today, because we don't run the LP test suite as root... | 06:34 |
| wgrant | But the librarian and codehosting deps from the buildd are very weak and easily fakeable. | 06:35 |
| lifeless | the more network fakes we build the easier it gets | 06:35 |
| lifeless | not to be trite | 06:35 |
| poolie | yeah | 06:36 |
| jtv | But we have same very nasty issues to deal with there, such as “what if interopation with another builder aborts changes done in the context of your builder while you wait for the librarian?” This is what I' trying to deal with at the moment. | 06:36 |
| poolie | i just don't want one ginormo commit/deploy, since it will be hard to debug | 06:36 |
| lifeless | jtv: that doesn't depend on the builder implementation nor the librarian implementation in any way that a fake can't simply simulate | 06:36 |
| jtv | IMHO we need loads of unit test coverage built first, followed by a streamlining of the integration tests. | 06:36 |
| jtv | Another problem compounding the situation though is that the whole buildmaster architecture urgently needs to be cleaned up. | 06:37 |
| poolie | well | 06:38 |
| wgrant | poolie: Right, I want small steps. | 06:38 |
| wgrant | The easiest way to get the split done now is to use sourcecode. Then we can start severing connections, writing fakes, etc. And eventually drop the dep. | 06:39 |
| wgrant | Rather than breaking deployments by reworking the package into a form that is usable in an ugly way until we have fakes. | 06:39 |
| poolie | can you explain why using a deb will be breaking deployments, ugly, etc | 06:40 |
| wgrant | Well, nobody knows how lp-buildd deployments work now. | 06:40 |
| wgrant | The package mangles sudoers itself in ways that interact unobviously with puppet. | 06:41 |
| wgrant | etc. | 06:41 |
| wgrant | I don't like our chances of changing the package without breaking deployments in various ways. | 06:41 |
| lifeless | another sequence you could use | 06:41 |
| lifeless | is to write fakes in-lace | 06:41 |
| lifeless | get it all copacetic | 06:42 |
| lifeless | then remove the code | 06:42 |
| poolie | fwiw i just now have lp passing its tests using the deb version of lpbuildd | 06:42 |
| lifeless | that said, we have qastaging / staging to test deploys on | 06:42 |
| poolie | what is in-lace? | 06:42 |
| lifeless | in-place | 06:42 |
| wgrant | lifeless: Hahahaha | 06:43 |
| poolie | in which place? | 06:43 |
| lifeless | so I'm not scared of undetected breakage hitting production | 06:43 |
| wgrant | We saw how well staging testing worked last time... | 06:43 |
| wgrant | poolie: In the LP tree. | 06:43 |
| lifeless | wgrant: once we used it properly it seemed to work ok | 06:43 |
| lifeless | wgrant: or did I miss something? | 06:43 |
| wgrant | lifeless: Do we know what went wrong? | 06:43 |
| wgrant | We broke prod last week. | 06:43 |
| wgrant | Because staging QA didn't. | 06:43 |
| lifeless | yes, AIUI we didn't actually end-to-end things the first time around | 06:44 |
| lifeless | IMBW | 06:44 |
| wgrant | We did. | 06:44 |
| wgrant | But the builder was unbroken. | 06:44 |
| lifeless | what was broken then ? | 06:44 |
| wgrant | Quite how it was unbroken I don't think we know. | 06:44 |
| poolie | lifeless, so you're saying, keep lp's own copy of the code | 06:44 |
| poolie | and gradually rewrite the tests to not need a builder process? | 06:44 |
| poolie | we could | 06:44 |
| lifeless | poolie: well, we want a builder process | 06:45 |
| lifeless | poolie: but that should be a network fake rather than a live instance | 06:45 |
| lifeless | poolie: https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements#Test_fake | 06:45 |
| poolie | riht | 06:46 |
| poolie | hm | 06:47 |
| lifeless | I'm saying that separating the trees (into lp and a live buildd with included test fake tree) could be the last step, rather than the first. | 06:47 |
| poolie | aside from the coupling to the deployment environment, i don't think testing against the actual buildd code is all that bad | 06:47 |
| poolie | right | 06:47 |
| poolie | however, i have actually done it in the other order | 06:48 |
| poolie | so i'd kind of rather not abandon it unless there's actually something unavoidably bad | 06:48 |
| poolie | it is meant to be a cleanup so i do want it to be something we're happy is at least incrementally cleaner | 06:49 |
| lifeless | I'm fine with either order; I don't see a package in the LP PPA as terrible, particularly if you're going to keep poking at it - in fact, by getting closer to how we deploy (e.g. if we hint to lamont we want the same package and get his support), we may reduce friction | 06:52 |
| poolie | i hope so | 06:52 |
| lifeless | poolie: I only offer up the other order for consideration | 06:52 |
| poolie | what do you mean by 'same package'? | 06:52 |
| poolie | same binary, without a cat rebuild? | 06:52 |
| lifeless | same build process | 06:53 |
| lifeless | cat would rebuild, always does | 06:53 |
| lifeless | but no hidden magic | 06:53 |
| poolie | afaik that is what we did in the last deploy | 06:53 |
| poolie | or perhaps it's just sufficiently hidden magic :) | 06:53 |
| poolie | woo | 06:53 |
| poolie | everything likely to have broken now seems to be passing | 06:54 |
| poolie | so wgrant can i make you tolerably happy if i use the dpkg dependency but | 07:01 |
| poolie | - split off the 'kill me now' bits into a separate package which is not in ppa:launchpad | 07:01 |
| lifeless | man, TestOopsPrune is a littttle braindead | 07:01 |
| poolie | - file bugs against the tests that depend on importing from it | 07:02 |
| poolie | - perhaps try to rig the init script so it will start without disruption on the buildds but not on anyone else's machine | 07:02 |
| wgrant | Perhaps. | 07:03 |
| lifeless | it writes the same oops into recent and old directories, and then tests that its only deleted from some. | 07:04 |
| lifeless | win | 07:04 |
| lifeless | . | 07:04 |
| wgrant | Hah | 07:05 |
| poolie | wgrant, so just to be clear, the two things are | 07:11 |
| poolie | - it will be horrible if people start offering buildds on untrusted networks | 07:11 |
| poolie | - generally we don't like python in debs- | 07:11 |
| poolie | (but we maybe like sourcecode less, depending who you ask) | 07:11 |
| poolie | is that correct? | 07:14 |
| poolie | well, https://code.launchpad.net/~mbp/launchpad/800295-delete-buildd/+merge/81815 | 07:22 |
| lifeless | wgrant: StevenK: jtv: whats a good mixin interface that product and distribution already inherit, to add a referenced_oopses(start, end) method to | 07:27 |
| lifeless | 'There is None' is a valid if sad answer | 07:27 |
| poolie | ok i'd better go | 07:30 |
| poolie | may be back later | 07:30 |
| lifeless | wgrant: we were talking about the API the other day | 07:55 |
| lifeless | wgrant: what decorators will I need for findReferencedOOPS(start_date, end_date), do you think ? | 07:55 |
| lifeless | wgrant: operation_parameters, export_read_operation, I presume | 07:56 |
| wgrant | lifeless: Yep. | 07:56 |
| wgrant | And operation_for_version | 07:56 |
| wgrant | But that should be it. | 07:56 |
| wgrant | Just leave the return type undeclared. | 07:56 |
| lifeless | thanks | 07:57 |
| lifeless | now, I'd like a sanity check on something | 07:57 |
| lifeless | I plan to ignore security and scan everything, on a public anonymous API | 07:57 |
| lifeless | but return *only* the OOPS strings themselves. | 07:57 |
| lifeless | I'm assessing our regexps as tight enough to not disclose anything | 07:57 |
| lifeless | Am I wrong? | 07:57 |
| wgrant | Well. | 07:58 |
| wgrant | That's a good question. | 07:59 |
| wgrant | I don't currently like the way we permit whitespace. | 07:59 |
| wgrant | But at most it can reveal one word. | 07:59 |
| lifeless | do we actually linebreak OOPS-bar ? | 08:01 |
| lifeless | perhaps we should make that not break, and tighten the refex | 08:01 |
| wgrant | That's what I was thinking. | 08:02 |
| lifeless | or | 08:02 |
| lifeless | perhaps we should tighten the regex and file a low bug about linebreaking that | 08:02 |
| wgrant | Even better. | 08:02 |
| lifeless | if someone writes oops-tradesecrethere in a bug, we may disclose something; I think thats a fairly low risk | 08:03 |
| stub | oops-i-did-it-again | 08:03 |
| lifeless | will pick up'i' :P | 08:03 |
| stub | Today Thailand celebrates Loy Kratong to pay respect to the spirit of the waters. | 08:08 |
| wgrant | Heh | 08:08 |
| nigelb | Irony right there. | 08:09 |
| nigelb | stub: You're in a safe area I hope? | 08:09 |
| stub | I'll be last to go | 08:09 |
| nigelb | :) | 08:09 |
| lifeless | jtv: where were you putting your 'object implements interface' tests ? | 08:13 |
| lifeless | jtv: I'm adding a mixin interface to product and distribution; I went looking in lib/lp/registry/tests/test_product.py first | 08:13 |
| lifeless | jtv: ah, I found it | 08:19 |
| lifeless | jtv: I was searching for verifyObject :) | 08:20 |
| mrevell | Hello | 09:00 |
| adeuring | good morning | 09:00 |
| StevenK | And AWS add a second US West region. | 09:18 |
| StevenK | No, this isn't confusing AT ALL. | 09:18 |
| nigelb | StevenK: Actually, there are 4 US locations :) | 09:29 |
| nigelb | Virginia, Portland (new), California, and a Govt location which is only for US folks. | 09:30 |
| nigelb | Morning bigjools! | 09:31 |
| mrevell | ha | 09:31 |
| nigelb | Hrm, did I scare him away? | 09:31 |
| nigelb | :D | 09:31 |
| allenap | Good morning everyone. Who knows about PPPAs? | 09:51 |
| wgrant | allenap: What's up? | 09:52 |
| allenap | wgrant: I can see a PPPA, and so can an esteemed colleague of mine, but get 401 when adding the suggested entries to sources.list. I noticed there's no user auth garbage in the URLs it suggests. | 09:53 |
| wgrant | allenap: Those pages aren't meant to be used. | 09:54 |
| wgrant | They were invisible until someone did something 18 months ago. | 09:54 |
| wgrant | Use https://launchpad.net/~/+archivesubscriptions | 09:55 |
| wgrant | instead | 09:55 |
| allenap | wgrant: Ah ha, okay. So the PPPA needs to be granted to the user. | 09:55 |
| wgrant | The URLs will probably magically become sensible once you click the "View" link on +archivesubscriptions. | 09:55 |
| wgrant | That generates your unique access token. | 09:55 |
| wgrant | Hah | 09:55 |
| wgrant | And the CSS is broken, so you can actually see which of the View links are normal, and which are magical. | 09:55 |
| allenap | wgrant: Thanks, that's great. | 09:57 |
| nigelb | wgrant: Magical? | 09:57 |
| wgrant | nigelb: JavaScript which requests an access token. | 10:03 |
| nigelb | ah. | 10:04 |
| nigelb | I don't see any of those. but then I only have one PPA listed there. | 10:04 |
| jml | is it true that LP has instant oops availability? | 11:28 |
| wgrant | jml: On staging and qastaging, but not on production yet. | 11:28 |
| wgrant | We don't have the rabbitmq metrics completely done yet. | 11:28 |
| jml | wgrant: ah right, thanks. | 11:28 |
| wgrant | It all works fine, but we're not putting rabbit into production until we have good monitoring and such. | 11:29 |
| jml | +1 | 11:29 |
| jml | is there a live feed of oopses too? | 11:30 |
| wgrant | Not as yet. | 11:30 |
| jml | or do you have to know the code before you can get to them? | 11:30 |
| wgrant | But oops-tools is hackable now, so I expect we'll see some nice stuff soon. | 11:30 |
| wgrant | And it's all split out into non-LP-specific projects, so others can use it if they so desire. | 11:30 |
| wgrant | lifeless has done amazing work to that end. | 11:30 |
| jml | yeah, that's why I'm asking, as it happens. | 11:31 |
| lifeless | there is an index on date I think | 11:33 |
| lifeless | so we can do an API to get latest N | 11:33 |
| lifeless | and expose that in LP itself | 11:33 |
| lifeless | or add it to the oops-tools console | 11:34 |
| lifeless | I've also just got the OK from jane to LGPL the plumbing | 11:34 |
| wgrant | Ah, excellent. | 11:34 |
| wgrant | That'll make things a bit more sensible. | 11:34 |
| lifeless | jml: lp:python-oops-tools if you want to go looking | 11:37 |
| jml | lifeless: thanks | 11:38 |
| lifeless | I am considering adding an explicit oops-reference table to LP | 11:39 |
| lifeless | to let us have references into the separate oops-tools service | 11:39 |
| lifeless | this might make the crashdump project better | 11:39 |
| stub | lifeless: Any reason to have the oops-reference table in lp rather than in a database controlled by the oops service? | 11:52 |
| adeuring | allenap: are you doing reviews today? | 11:58 |
| lifeless | stub: yes, because LP knows what things hold references | 12:00 |
| lifeless | stub: both ends may want to know actually; e.g. oops-tools knows 'referenced by url, url, url, ...' and lp knows 'bug X references oops, oops, oop', question Y references oop, oops, oops etc | 12:00 |
| lifeless | stub: this is a new thought still getting assembled though; ask me tomorrow I may have a totally different idea | 12:01 |
| allenap | adeuring: I am. | 12:04 |
| === allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugtasks: 275 | ||
| allenap | adeuring: Though I probably have enough on my plate already :-/ | 12:04 |
| allenap | adeuring: I am out much of the afternoon, working late this evening instead. | 12:04 |
| adeuring | allenap: ah, ok. If you have some spare time nevertheless, could you have a llook at this mp: https://code.launchpad.net/~adeuring/launchpad/banner-for-beta-features-2/+merge/81833 ? | 12:05 |
| allenap | adeuring: Sure :) | 12:05 |
| stub | lifeless: LP doesn't really know, it holds a heap of information that needs to be scanned and processed. The way we do it at the moment is not scalable. If we picked up references at insert time we no longer need to scan. But where to store those references? Just as easy to send a message to rabbit as it is to store the reference in a db table. | 12:26 |
| stub | lifeless: And that way the service that stores the OOPSes doesn't need to reach into the Launchpad database to determine what can stay or what can go. | 12:26 |
| === jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugtasks: 275 | ||
| rvba | Could you guys tell me how to create a "mirrored" branch? I'd like to answer a question in #lp but I can't find how to change a branch type myself (https://help.launchpad.net/Code/MirroredBranches) | 14:02 |
| rvba | jcsackett: I'm pinging you since you've OCR ;). Do you have any idea? how to created a mirrored branch? (See my question above). The doc in https://help.launchpad.net/Code/MirroredBranches really seems outdated… | 14:05 |
| rvba | s/you've/you're/ | 14:05 |
| rvba | s/created/create/ | 14:06 |
| rvba | phew | 14:06 |
| nigelb | rvba: on the code.launchpad.net/project page click on "Import a Branch" I think. | 14:06 |
| nigelb | where project is whichever project it needs to be associated with. | 14:07 |
| rvba | nigelb: but what about if you want to have a branch in ~me/+junk/my_mirrored_branch ? | 14:07 |
| nigelb | rvba: I think mirrors aer associated with project not +junk. *checks* | 14:07 |
| rvba | That's not what https://help.launchpad.net/Code/MirroredBranches says but this might be outdated… | 14:08 |
| nigelb | Oh. Hrm. Interesting | 14:08 |
| nigelb | Imported branches don't have a Mirrored status anyway. | 14:09 |
| rvba | I need a code specialist… abentley maybe? Would you know the answer to my question above? | 14:10 |
| abentley | rvba: I've never needed to, but... | 14:11 |
| rvba | abentley: I suppose the right way to do that is to properly create a project and have the branch mirrored in that project… | 14:12 |
| rvba | But that's not what https://help.launchpad.net/Code/MirroredBranches says. | 14:12 |
| abentley | rvba: It looks like you go to the pillar page and click "Import a branch". | 14:12 |
| rvba | abentley: I'm sorry but what is the pillar page (you have to know that I've been trapped inside Soyuz for a few months, I just escaped very recently). | 14:14 |
| abentley | rvba: https://code.launchpad.net/launchpad/ for example | 14:15 |
| nigelb | Trapped inside soyuz. Oh dear. | 14:15 |
| rvba | abentley: ok, so no mirrored branch in /~me/+junk then? | 14:15 |
| abentley | rvba: And probably anywhere else branches are listed. | 14:15 |
| abentley | rvba: You might want to ask someone on the Bazaar team. They're the last ones who touched this functionality. | 14:16 |
| rvba | abentley: ok I'll do that, thanks for the help. | 14:17 |
| deryck | jcsackett, are you reviewing adeuring's js branch? | 15:04 |
| jcsackett | deryck: yeah, i was just looking through it. | 15:04 |
| deryck | jcsackett, ok, cool. Thanks. | 15:04 |
| jcsackett | was about to tell adeuring i had left some questions on it. | 15:04 |
| jcsackett | it seems a bit redundant now, but. adeuring, i left some comments on your MP. :-) | 15:05 |
| deryck | heh | 15:06 |
| adeuring | jcsackett: I deleted the redundant stuff (and thanks for the review) | 15:15 |
| jcsackett | thanks, adeuring. r=me. | 15:30 |
| adeuring | jcsackett: thanks! | 15:31 |
| === matsubara is now known as matsubara-lunch | ||
| === beuno is now known as beuno-lunch | ||
| mrevell | danhg, We should aim to talk about bug 888596 before the end of the day; if not, tomorrow. | 16:21 |
| _mup_ | Bug #888596: Instructions for generating gpg keys are not provided for Unity <Launchpad itself:Triaged by danhg> < https://launchpad.net/bugs/888596 > | 16:21 |
| danhg | I'm looking over it now | 16:30 |
| abentley | allenap, jcsackett: could you please review https://code.launchpad.net/~abentley/launchpad/back-button-support/+merge/81875 ? | 16:43 |
| === beuno-lunch is now known as beuno | ||
| === matsubara-lunch is now known as matsubara | ||
| jcsackett | abentley: looking now. | 17:13 |
| === deryck is now known as deryck[lunch] | ||
| jcsackett | abentley: r=me. looks fine. | 17:21 |
| abentley | jcsackett: thanks. | 17:21 |
| === salgado is now known as salgado-lunch | ||
| sinzui | jcsackett, do you have time to mumble? | 17:52 |
| mrevell | Night all | 17:59 |
| abentley | deryck[lunch]: I'm up for our chat anytime. | 18:00 |
| === deryck[lunch] is now known as deryck | ||
| deryck | abentley, ok, cool. Give me just a bit more and I'll ping you. | 18:02 |
| jcsackett | sinzui: sure. one moment. | 18:07 |
| deryck | abentley, I can chat now, if you're still free. | 18:16 |
| abentley | deryck: let's go! | 18:16 |
| === salgado-lunch is now known as salgado | ||
| jcsackett | bac: r=me on your lplib MP. | 18:18 |
| bac | just saw that, thanks jcsackett | 18:23 |
| jcsackett | sinzui: i have to run out for a few moments; i'll ping you to mumble when i return. | 18:34 |
| jcsackett | sinzui: i am again free to mumble whenever you are. | 19:08 |
| deryck | jcsackett, are you reviewing still? | 19:33 |
| jcsackett | deryck: i am. have a branch for me? | 19:33 |
| deryck | jcsackett, indeed! https://code.launchpad.net/~deryck/launchpad/toggle-bug-fields-config-widget/+merge/81900 | 19:33 |
| deryck | jcsackett, more of my widgets for new bug listings work. | 19:34 |
| jcsackett | deryck: looking at it now. | 19:34 |
| deryck | jcsackett, awesome, thanks! | 19:34 |
| jcsackett | deryck: r=me. looks like nice work. | 20:06 |
| jcsackett | and it just taught me how the getter/setter attr stuff in YUI works. :-P | 20:06 |
| deryck | jcsackett, awesome, thanks! | 20:06 |
| deryck | heh | 20:06 |
| deryck | good deal then. | 20:06 |
| jcsackett | indeed. :-) | 20:06 |
| deryck | it's pretty expressive when you get you're head around it. | 20:07 |
| jcsackett | yeah, it certainly seems pretty cool. | 20:07 |
| cr3 | hi folks, there are broken links to the Twisted Coding Standard and CodingStyle for Zope 3 on https://dev.launchpad.net/PythonStyleGuide | 20:11 |
| cr3 | I suspect the first should point to: http://twistedmatrix.com/trac/browser/trunk/doc/core/development/policy/coding-standard.xhtml?format=raw | 20:11 |
| cr3 | and the second to: http://wiki.zope.org/zope3/CodingStyle | 20:11 |
| cr3 | not knowing what the original links looked like, these are only googled guesses | 20:11 |
| lifeless | sounds good to me | 20:14 |
| lifeless | cr3: so, did you want to talk ? | 20:14 |
| cr3 | lifeless: sure, I was mostly concerned about solving the problem of synchronizing user data not only internally but for community projects as well. do you think it would be reasonable to expose both a restful and xmlrpc interface, ie would this just cause more confusion? | 20:16 |
| lifeless | I think thats premature generalisation then. | 20:17 |
| lifeless | very easy to overbuild and underdeliver that way | 20:17 |
| cr3 | lifeless: as you said, concentrating on technology at this point might be premature but I feel there's already a strong preference, maybe because of precedence, for xmlrpc rather than restful, maybe because of launchpadlib experience. | 20:17 |
| lifeless | we only truely know our requirements => and we're wrong most of the time :) | 20:17 |
| cr3 | lifeless: good argument, I'm already convinced :) | 20:18 |
| lifeless | ok :) | 20:18 |
| lifeless | so - a LEP soon ? | 20:18 |
| lifeless | based on what *result tracker* needs | 20:18 |
| cr3 | lifeless: onto the next concern then: I'm not sure when I can deliver a LEP so I hope nobody else might be depending on exposing such a microservice, ie I wouldn't want to block anyone or duplicate their effort | 20:18 |
| lifeless | I will work with you to refine things and make tech choices once the needs are really understood | 20:19 |
| lifeless | cr3: writing the LEP doesn't imply implementing | 20:19 |
| lifeless | cr3: its a discussion point for decision making | 20:19 |
| lifeless | cr3: And I promise you, if/when someone else needs something similar, I won't queue them up behind you | 20:19 |
| cr3 | lifeless: ok, lets say for Monday then, any later would just be procrastination :) | 20:20 |
| lifeless | cool | 20:20 |
| lifeless | I shall be around :) | 20:20 |
| cr3 | broken links in PythonStyleGuide fixed, enjoy! | 20:24 |
| * cr3 thinks that wikis should have a broken link crawler running periodically | 20:24 | |
| lifeless | I think they break enough on their own | 20:27 |
| === matsubara is now known as matsubara-afk | ||
| abentley | deryck: could you please review https://code.launchpad.net/~abentley/launchpad/dynamic-listing-robustness/+merge/81904 ? | 20:47 |
| deryck | abentley, I can. I'm about to go offline to switch work locations, but can do it when I return. | 20:47 |
| abentley | deryck: Sure. | 20:47 |
| jcsackett | abentley: just looked at your most recently put up MP. it is r=me. | 20:56 |
| abentley | jcsackett: thanks. | 20:56 |
| lifeless | allenap: would voice help ?> | 20:57 |
| lifeless | allenap: we may be talking past each other | 20:57 |
| allenap | lifeless: Yeah, let's do that. | 20:58 |
| allenap | Skype okay? | 20:58 |
| lifeless | allenap: now? Or next week? now is fine for me | 20:58 |
| lifeless | skype is fine | 20:58 |
| cr3 | huwshimi, danhg: for the meeting, mumble or just irc? | 20:59 |
| huwshimi | cr3: Hi, lets do voice if we can. | 20:59 |
| allenap | lifeless: Yeah, now is good. | 20:59 |
| huwshimi | danhg: Do you have mumble set up? | 20:59 |
| lifeless | allenap: calling | 21:01 |
| allenap | lifeless: Skype has crashed :-/ | 21:01 |
| lifeless | hah! win :( | 21:01 |
| cr3 | huwshimi: worst case, I can probably try to conference both of you in by phone | 21:02 |
| huwshimi | cr3: Lets mumble for now and if that doesn't work for Dan we'll try something else. | 21:03 |
| cr3 | huwshimi: http://results-tracker.ubuntu.com | 21:04 |
| cr3 | huwshimi: http://results-tracker.ubuntu.com/ubuntu/oneiric/+testruns/2183 | 21:09 |
| deryck | abentley, hey, I see jcsackett beat me to the review. | 21:38 |
| abentley | deryck: yeah. He's feisty. | 21:38 |
| deryck | indeed | 21:38 |
| jcsackett | deryck, abentley: i just like keeping the queue clean. :-P | 21:43 |
| lifeless | abentley: hi | 21:57 |
| lifeless | abentley: why do you want to precache batches ? | 21:57 |
| abentley | lifeless: I want to make the user experience fluid. | 21:58 |
| lifeless | abentley: I don't think its a good use of money to compute results that will only rarely be used. | 21:58 |
| abentley | lifeless: I don't follow. | 21:59 |
| lifeless | abentley: there are what, 10 adjacent batches - first, last, next, prev, and 6 or so different sorts. | 22:00 |
| lifeless | abentley: the user can only follow one of those, any results for the other 9 will be wasted. | 22:00 |
| huwshimi | deryck: Just running a bit late. 5 mins? | 22:00 |
| deryck | huwshimi, ok. abentley and possibly adeuring will join us. mumble ok? | 22:00 |
| huwshimi | deryck: Fine | 22:00 |
| abentley | lifeless: No, they are not wasted just because they are not used immediately. | 22:00 |
| abentley | lifeless: The various sorts are always the first batch, no matter what batch you're on. | 22:01 |
| abentley | lifeless: So they apply for the life of the page. | 22:02 |
| lifeless | if the user doesn't change to all of those sorts, then the results do not benefit that user, right ? | 22:02 |
| abentley | lifeless: yes. | 22:02 |
| lifeless | We are running short of server resources as it is; I think we need a strong justification for doing speculative calculations like this - particularly while we have timeouts doing bug search batches | 22:03 |
| lifeless | there is also the complication that if you don't serialise the requests you can easily DOS our serers | 22:03 |
| lifeless | and if you do serialise, then you need to make sure you're still responsive when the user clicks on a batch that you haven't received a result for yet. | 22:03 |
| deryck | huwshimi, abentley -- let's meet in Orange room. | 22:04 |
| abentley | lifeless: If money is an issue, exactly how much money are we talking about? We can't compute the tradeoff without knowing that. | 22:04 |
| huwshimi | deryck: No problems, I'll be a minute | 22:04 |
| abentley | lifeless: if 10 requests can DOS our servers, doesn't that indicate a problem with our infrastructure? | 22:06 |
| lifeless | abentley: can I ask what is unfluid about the experience today ? | 22:06 |
| lifeless | . | 22:06 |
| lifeless | (sorry, my adsl is terrible at the moment) | 22:06 |
| lifeless | . | 22:06 |
| lifeless | tcp, please come back to me | 22:06 |
| abentley | lifeless: on a call. | 22:07 |
| lifeless | . | 22:07 |
| lifeless | . | 22:07 |
| lifeless | abentley: we have 64 concurrent service points, which can support all our users today. If you make bug search - our single most common operation - use 10 times the resources, we'll have to expand the cluster. | 22:08 |
| lifeless | abentley: how much? I don't know. | 22:08 |
| lifeless | abentley: but server scaling isn't cheap, and we should strive for efficiency. I don't see calculating results speculatively as being very efficient; I'm sorry but I'm going to ask that this not land - as a similar thing in the bug subscription area was backed out. | 22:10 |
| huwshimi | http://people.canonical.com/~huwshimi/status_colours.png | 22:17 |
| === jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 275 | ||
| sinzui | wallyworld_, wgrant, StevenK: http://www.youtube.com/results?search_query=most+interesting+man+in+the+world&aq=0&oq=most+inter | 22:58 |
| lifeless | bbiabm, dropping lynne up the street | 23:04 |
| === wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugtasks: 275 | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
