=== lifeless [n=robertc@ppp245-86.static.internode.on.net] has joined #launchpad-meeting === ddaa [n=ddaa@nor75-18-82-241-238-155.fbx.proxad.net] has joined #launchpad-meeting === danilos [n=danilo@cable-89-216-150-41.dynamic.sbb.co.yu] has joined #launchpad-meeting === kiko [n=kiko@200-171-140-32.dsl.telesp.net.br] has joined #launchpad-meeting [01:46] oy === carlos [n=carlos@18.Red-88-16-32.dynamicIP.rima-tde.net] has joined #launchpad-meeting [01:46] hi [01:47] yo === stub [n=stub@ppp-58.8.16.239.revip2.asianet.co.th] has joined #launchpad-meeting [01:47] stub! [01:47] thanks for joining in [01:47] so for the +translate timeouts [01:47] thanks stub [01:47] stub: how do you feel? [01:47] there are some approaches we can take [01:47] ok. Had a nice nap. [01:48] 1. we currently issue a large number of suggestion queries per page; I could change the code to issue only 4 [01:49] 2. we could change the loading of suggestions to come in via ajax. one question is whether to do it as N small hits or as the 4 queries (as I propose in 1.) [01:49] will we sacrifice detail for 1? like not listing which are "used in..." etc? [01:49] 3. we could do the database refactoring and then revisit the issue again. [01:49] or is it only about 4 BIG queries? [01:49] no, danilos -- equivalent queries, but doing them in one shot instead of 10 [01:50] kiko: ok, but how much will that help? stub? [01:50] kiko: will it work if a user selects 100 queries? [01:50] you mean 100 translations? [01:50] yeah, it will [01:50] it will issue only 4 queries [01:50] kiko: yeah, sorry [01:50] one for each type of suggestion [01:50] Issuing fewer, large queries is generally better than many small. [01:50] (or are they 5? I forget. it's a small number) [01:50] But we need to time them to be sure. [01:50] I think it will help some, but not too much [01:50] kiko: me too, I think it's just 4 [01:51] I would like to do both [01:51] stub, kiko: I am more thinking on order of magnitude -- I am aware it will help, but will it help enough? [01:51] optimise the DB + move it to reduce the amount of queries [01:51] it's basically an "id IN (...)" query instead of N "id = X" queries [01:51] even if just the optimisation kills the timeouts [01:51] danilos, it won't help enough, no. it will mitigate the problem. [01:51] carlos ++ [01:51] kiko: ok, so I am all for both, just like carlos [01:52] the only reason for doing 1. before 3. is because I don't know when you guys can get to 3. [01:52] I'm really unhappy with all the pain being inflicted on SteveA and I on rosetta 1.0 [01:52] If we are going to trim the tables down as specced, we might as well do it first to avoid needing to redo work. [01:52] and I want to make sure the distractions stop so that you can concentrate on those items. [01:52] Also any benchmarking on the old schema will be irrelevant with the new schema [01:52] stub, nobody's going to have time before 1.0 -- rosetta's in bad shape [01:53] stub: it involves changing a lot of existing code, so no go at this time, I think [01:53] ok. So 3 isn't an option. [01:53] stub, I don't think you have much time either.. [01:53] kiko: the only problem with DB optimisations is the migration process [01:53] carlos, and updating the code. [01:53] not really, we are talking about more than 24 hours to do the migration.... [01:53] at least that was our guess at all hands [01:53] carlos: well, each and every query, class usage (ok, we can mitigate that by providing same interfaces in code), etc. [01:54] we cannot block the DB for 24 hours... [01:54] carlos: ah, right, but I guess we can block rosetta input for 24 hours [01:54] danilos: and the rest of launchpad [01:54] then do the migration on carbon or something [01:55] hum... stub, any ideas? [01:55] carlos, 24 hours of migration downtime is manageable, but taking weeks to do the code update is not so much [01:55] Nothing that will be aailable in the short term. [01:55] It might not be 24 hours - again, we won't know until we have benchmarked it. [01:55] danilos, carlos, stub: we could disable rosetta for that period and survive. [01:55] ok... btw, stub, will you be considering 8.2 upgrade soon? [01:55] kiko: without distractions, I'm sure I would migrate code in a couple of working days [01:56] carlos, a "couple". :-) [01:56] kiko: We can't do that yet. It isn't implemented. Same with read only launchpad. [01:56] tests shouldn't be an issue... [01:56] kiko: 2 [01:56] stub, I mean, we could disable translations.launchpad.net and put a page there. [01:56] stub, and turn off the cronjobs [01:56] kiko: no, I already talked with stuart about it, and the migration would lock person table too [01:56] Hmm... is it all on the one domain now? Then we could. [01:56] and that will lock the rest of launchpad [01:57] carlos, oh. the person table?! [01:57] carlos: I could work around that by dropping the constraint during migration. [01:57] can't we have a read-only copy of person table? (we won't be changing it) [01:57] kiko: we are moving that reference between tables [01:58] stub: if you feel confident about that, then it's fine for me to shutdown rosetta for the migration, that makes the process much more easy [01:58] yeah. [01:58] we can only run into problems with accounts being merged into anothers while we do the migration, right? [01:58] that's what I was thinking [01:58] danilos: That is a good point. [01:58] danilos, we can disable merging, good point. [01:59] that should be easy too [01:59] it's work, I know [01:59] at least more easy than playing with triggers and trying to do it on a live system... [01:59] We still need to test this though - we need to have disabled anything that looks at the rosetta tables, and I'm guessing we have portlets around that want to do that. [01:59] but it's at least not launchpad downtime for an undetermined time [01:59] mmmm [01:59] anyway, so, our plan is? go with 1 (optimize translate to have constant number of queries), then when 1.0 rolls out, go on with db optimizations as well [01:59] stub, can't they go on querying the old tables? or will that be a disaster? [02:00] danilos, there was the ajax solution. how do people feel about that? [02:00] kiko: more work for "jumpy" page, imho [02:00] That would work I guess. [02:00] i.e. suggestions popping up as they're loaded [02:00] expanding the page, etc. [02:00] it's true [02:00] carlos, is that how your patch behaved? [02:01] I think it also makes it harder for us to tell what the end-user experience is [02:01] if we use clickety-drop-downs, then that won't be very discoverable, imho [02:01] right [02:01] kiko: I think it was hidden and when the user selected it [02:01] we expand and load the suggestions [02:01] kiko: if non ajax solution works well, I would prefer it [02:02] I think this is basically just hiding the problem [02:02] danilos, well, mitigating the problem [02:02] though, it might make sense for any *complete* translations [02:02] i.e. if a message is translated, don't show the suggestions [02:02] that might even turn out to be better UI, imo [02:03] anyway, we should do that *after* the optimisation [02:03] (don't show == use ajax to show them) [02:03] well, I think these are mutually exclusive [02:04] almost, anyway [02:04] i.e. ajax still means separate queries for each message [02:05] carlos, yeah, you think so? [02:05] danilos, well, maybe not, but it's more complex if so. [02:05] though, if we go with above of showing suggestions by default only for untranslated messages [02:05] well, I guess that we can optimise it, and later add the ajax part as an extra feature [02:06] both would turn out useful [02:06] carlos: agreed [02:06] I know will not be 100% reused and will require more changes [02:06] ok. [02:06] so fine, leave +translate with me for the time being. I will refactor the queries and then we'll see how it looks [02:06] ok [02:06] hopefully 1.0 will be in better shape by then [02:06] thanks. [02:06] now, another db related thing: search? [02:07] do we only want google-search at this time, or should we go with db search as well? [02:07] db search may require db upgrade at the very least (for partial indexes) [02:08] I think we should keep going with the search feature [02:08] hum, maybe not [02:08] http://www.postgresql.org/docs/8.0/static/indexes-partial.html [02:08] google-search for now will be enough [02:08] as planned [02:08] after 1.0 you can come back to proper searching [02:08] I was under the impression that this required 8.2 at least [02:09] yup [02:09] IF you surprise me and deliver 1.0 without me getting fired [02:09] kiko: sure, search was a post 1.0 feature [02:09] okay. [02:09] carlos, please make an effort to get TR landed so I can do browsing next week, ok? [02:10] kiko: I will do it before leaving, don't worry [02:10] thanks [02:11] danilos: so, let's defer that for post 1.0 [02:11] carlos: right [02:14] kiko: anything else? [02:15] carlos, danilos: no. thanks a lot! [02:15] thanks for joining too stub === kiko [n=kiko@200-171-140-32.dsl.telesp.net.br] has left #launchpad-meeting ["Left] [02:15] thanks dudes! [02:15] yup, especially stub, hope you get better soon === stub coughs pitifully [02:15] :) === carlos [n=carlos@18.Red-88-16-32.dynamicIP.rima-tde.net] has left #launchpad-meeting ["Ex-Chat"] === lifeless [n=robertc@ppp245-86.static.internode.on.net] has joined #launchpad-meeting === lifeless [n=robertc@ppp245-86.static.internode.on.net] has joined #launchpad-meeting