[01:46] <kiko> oy
[01:46] <carlos> hi
[01:47] <danilos> yo
[01:47] <kiko> stub!
[01:47] <kiko> thanks for joining in
[01:47] <kiko> so for the +translate timeouts
[01:47] <danilos> thanks stub
[01:47] <carlos> stub: how do you feel?
[01:47] <kiko> there are some approaches we can take
[01:47] <stub> ok. Had a nice nap.
[01:48] <kiko> 1. we currently issue a large number of suggestion queries per page; I could change the code to issue only 4
[01:49] <kiko> 2. we could change the loading of suggestions to come in via ajax. one question is whether to do it as N small hits or as the 4 queries (as I propose in 1.)
[01:49] <danilos> will we sacrifice detail for 1? like not listing which are "used in..." etc?
[01:49] <kiko> 3. we could do the database refactoring and then revisit the issue again.
[01:49] <danilos> or is it only about 4 BIG queries?
[01:49] <kiko> no, danilos -- equivalent queries, but doing them in one shot instead of 10
[01:50] <danilos> kiko: ok, but how much will that help? stub?
[01:50] <carlos> kiko: will it work if a user selects 100 queries?
[01:50] <kiko> you mean 100 translations?
[01:50] <kiko> yeah, it will
[01:50] <kiko> it will issue only 4 queries
[01:50] <carlos> kiko: yeah, sorry
[01:50] <kiko> one for each type of suggestion
[01:50] <stub> Issuing fewer, large queries is generally better than many small.
[01:50] <kiko> (or are they 5? I forget. it's a small number)
[01:50] <stub> But we need to time them to be sure.
[01:50] <kiko> I think it will help some, but not too much
[01:50] <carlos> kiko: me too, I think it's just 4
[01:51] <carlos> I would like to do both
[01:51] <danilos> stub, kiko: I am more thinking on order of magnitude -- I am aware it will help, but will it help enough?
[01:51] <carlos> optimise the DB + move it to reduce the amount of queries
[01:51] <kiko> it's basically an "id IN (...)" query instead of N "id = X"  queries
[01:51] <carlos> even if just the optimisation kills the timeouts
[01:51] <kiko> danilos, it won't help enough, no. it will mitigate the problem.
[01:51] <danilos> carlos ++
[01:51] <danilos> kiko: ok, so I am all for both, just like carlos
[01:52] <kiko> the only reason for doing 1. before 3. is because I don't know when you guys can get to 3.
[01:52] <kiko> I'm really unhappy with all the pain being inflicted on SteveA and I on rosetta 1.0
[01:52] <stub> If we are going to trim the tables down as specced, we might as well do it first to avoid needing to redo work.
[01:52] <kiko> and I want to make sure the distractions stop so that you can concentrate on those items.
[01:52] <stub> Also any benchmarking on the old schema will be irrelevant with the new schema
[01:52] <kiko> stub, nobody's going to have time before 1.0 -- rosetta's in bad shape
[01:53] <danilos> stub: it involves changing a lot of existing code, so no go at this time, I think
[01:53] <stub> ok. So 3 isn't an option.
[01:53] <kiko> stub, I don't think you have much time either..
[01:53] <carlos> kiko: the only problem with DB optimisations is the migration process
[01:53] <kiko> carlos, and updating the code.
[01:53] <carlos> not really, we are talking about more than 24 hours to do the migration....
[01:53] <carlos> at least that was our guess at all hands
[01:53] <danilos> carlos: well, each and every query, class usage (ok, we can mitigate that by providing same interfaces in code), etc.
[01:54] <carlos> we cannot block the DB for 24 hours...
[01:54] <danilos> carlos: ah, right, but I guess we can block rosetta input for 24 hours
[01:54] <carlos> danilos: and the rest of launchpad
[01:54] <danilos> then do the migration on carbon or something
[01:55] <danilos> hum... stub, any ideas?
[01:55] <kiko> carlos, 24 hours of migration downtime is manageable, but taking weeks to do the code update is not so much
[01:55] <stub> Nothing that will be aailable in the short term.
[01:55] <stub> It might not be 24 hours - again, we won't know until we have benchmarked it.
[01:55] <kiko> danilos, carlos, stub: we could disable rosetta for that period and survive.
[01:55] <danilos> ok... btw, stub, will you be considering 8.2 upgrade soon?
[01:55] <carlos> kiko: without distractions, I'm sure I would migrate code in a couple of working days 
[01:56] <kiko> carlos, a "couple". :-)
[01:56] <stub> kiko: We can't do that yet. It isn't implemented. Same with read only launchpad.
[01:56] <carlos> tests shouldn't be an issue...
[01:56] <carlos> kiko: 2
[01:56] <kiko> stub, I mean, we could disable translations.launchpad.net and put a page there.
[01:56] <kiko> stub, and turn off the cronjobs
[01:56] <carlos> kiko: no, I already talked with stuart about it, and the migration would lock person table too
[01:56] <stub> Hmm... is it all on the one domain now? Then we could.
[01:56] <carlos> and that will lock the rest of launchpad
[01:57] <kiko> carlos, oh. the person table?!
[01:57] <stub> carlos: I could work around that by dropping the constraint during migration.
[01:57] <danilos> can't we have a read-only copy of person table? (we won't be changing it)
[01:57] <carlos> kiko: we are moving that reference between tables
[01:58] <carlos> stub: if you feel confident about that, then it's fine for me to shutdown rosetta for the migration, that makes the process much more easy
[01:58] <kiko> yeah.
[01:58] <danilos> we can only run into problems with accounts being merged into anothers while we do the migration, right?
[01:58] <kiko> that's what I was thinking
[01:58] <stub> danilos: That is a good point.
[01:58] <kiko> danilos, we can disable merging, good point.
[01:59] <carlos> that should be easy too
[01:59] <kiko> it's work, I know
[01:59] <carlos> at least more easy than playing with triggers and trying to do it on a live system...
[01:59] <stub> We still need to test this though - we need to have disabled anything that looks at the rosetta tables, and I'm guessing we have portlets around that want to do that.
[01:59] <kiko> but it's at least not launchpad downtime for an undetermined time
[01:59] <kiko> mmmm
[01:59] <danilos> anyway, so, our plan is? go with 1 (optimize translate to have constant number of queries), then when 1.0 rolls out, go on with db optimizations as well
[01:59] <kiko> stub, can't they go on querying the old tables? or will that be a disaster?
[02:00] <kiko> danilos, there was the ajax solution. how do people feel about that?
[02:00] <danilos> kiko: more work for "jumpy" page, imho
[02:00] <stub> That would work I guess.
[02:00] <danilos> i.e. suggestions popping up as they're loaded
[02:00] <danilos> expanding the page, etc.
[02:00] <kiko> it's true
[02:00] <kiko> carlos, is that how your patch behaved?
[02:01] <kiko> I think it also makes it harder for us to tell what the end-user experience is
[02:01] <danilos> if we use clickety-drop-downs, then that won't be very discoverable, imho
[02:01] <danilos> right
[02:01] <carlos> kiko: I think it was hidden and when the user selected it
[02:01] <carlos> we expand and load the suggestions
[02:01] <carlos> kiko: if non ajax solution works well, I would prefer it
[02:02] <danilos> I think this is basically just hiding the problem
[02:02] <kiko> danilos, well, mitigating the problem
[02:02] <danilos> though, it might make sense for any *complete* translations
[02:02] <danilos> i.e. if a message is translated, don't show the suggestions
[02:02] <danilos> that might even turn out to be better UI, imo
[02:03] <carlos> anyway, we should do that *after* the optimisation
[02:03] <danilos> (don't show == use ajax to show them)
[02:03] <danilos> well, I think these are mutually exclusive
[02:04] <danilos> almost, anyway
[02:04] <danilos> i.e. ajax still means separate queries for each message
[02:05] <kiko> carlos, yeah, you think so?
[02:05] <kiko> danilos, well, maybe not, but it's more complex if so.
[02:05] <danilos> though, if we go with above of showing suggestions by default only for untranslated messages
[02:05] <carlos> well, I guess that we can optimise it, and later add the ajax part as an extra feature
[02:06] <danilos> both would turn out useful
[02:06] <danilos> carlos: agreed
[02:06] <carlos> I know will not be 100% reused and will require more changes
[02:06] <kiko> ok.
[02:06] <kiko> so fine, leave +translate with me for the time being. I will refactor the queries and then we'll see how it looks
[02:06] <danilos> ok
[02:06] <kiko> hopefully 1.0 will be in better shape by then
[02:06] <kiko> thanks.
[02:06] <danilos> now, another db related thing: search?
[02:07] <danilos> do we only want google-search at this time, or should we go with db search as well?
[02:07] <danilos> db search may require db upgrade at the very least (for partial indexes)
[02:08] <carlos> I think we should keep going with the search feature
[02:08] <danilos> hum, maybe not
[02:08] <danilos> http://www.postgresql.org/docs/8.0/static/indexes-partial.html
[02:08] <kiko> google-search for now will be enough
[02:08] <carlos> as planned
[02:08] <kiko> after 1.0 you can come back to proper searching
[02:08] <danilos> I was under the impression that this required 8.2 at least
[02:09] <stub> yup
[02:09] <kiko> IF you surprise me and deliver 1.0 without me getting fired
[02:09] <carlos> kiko: sure, search was a post 1.0 feature
[02:09] <kiko> okay.
[02:09] <kiko> carlos, please make an effort to get TR landed so I can do browsing next week, ok?
[02:10] <carlos> kiko: I will do it before leaving, don't worry
[02:10] <kiko> thanks
[02:11] <carlos> danilos: so, let's defer that for post 1.0 
[02:11] <danilos> carlos: right
[02:14] <carlos> kiko: anything else?
[02:15] <kiko> carlos, danilos: no. thanks a lot! 
[02:15] <kiko> thanks for joining too stub 
[02:15] <carlos> thanks dudes!
[02:15] <danilos> yup, especially stub, hope you get better soon
[02:15] <danilos> :)