/srv/irclogs.ubuntu.com/2006/08/10/#launchpad-meeting.txt

=== ubuntulog [i=ubuntulo@trider-g7.fabbione.net] has joined #launchpad-meeting
=== Starting logfile irclogs/launchpad-meeting.log
-ChanServ(ChanServ@services.)- You do not have channel operator access to [#canonical-ops] 09:20
=== ubuntulog [i=ubuntulo@ubuntu/bot/ubuntulog] has joined #launchpad-meeting
=== #canonical-ops is desynced from zelazny.freenode.net at 09:21am
=== ddaa [n=ddaa@nor75-18-82-241-238-155.fbx.proxad.net] has joined #launchpad-meeting
=== stub [n=stub@82.109.136.116] has joined #launchpad-meeting
=== carlos [n=carlos@13.Red-88-16-33.dynamicIP.rima-tde.net] has joined #launchpad-meeting
carloshi12:15
SteveAhi12:15
=== danilos [n=danilo@82.117.204.58] has joined #launchpad-meeting
SteveAhi12:15
daniloshi12:15
SteveAso, stub and I were just talking about the downtime required to run the rosetta updates12:16
danilosyeah12:16
carlosSteveA: do we have info about when is launchpad less used ?12:16
SteveAstub tells me that with the current code, we're talking about many hours' downtime12:16
SteveAI have a few comments about this that we should consider12:16
SteveA1. we don't want to take down all of launchpad for N hours in order to do rosetta updates12:17
carlosbetween 2 and 4 hours12:17
SteveA2. is it not possible to do the updates incrementally?12:17
SteveAabout 1,12:17
stub2-4 hours per run, or 2-4 hours for both runs?12:17
carlosstub: per run12:17
SteveAif we need to do lots of rosetta database work, we can take down just rosetta,12:18
carlosthat's on staging12:18
carlosand it depends on the load of the server, because sometimes is done in 2 hours and other times in 3 hours and a half12:18
SteveAby changing the authorization adapters to deny all rosetta edits12:18
SteveAand changing the "permission denied" page to say about rosetta being under maintenance12:18
SteveAso, that's one way of managing this12:18
SteveAso, the rest of launchpad (shipit, bugs, etc.) can stay running12:18
danilosI think that might be the best way out12:19
SteveAbut12:19
carlosyeah, at least for this initial run12:19
carlosnext time, for edgy + 112:19
carloswill be rosetta and soyuz12:19
SteveAlet's talk about why this needs to be done all in one go12:19
daniloson the other hand, it will take longer that way, no?12:19
SteveAwhy can't we run it a little at a time12:19
SteveAwith pauses in between?12:19
SteveAfor example, one package at a time12:20
stubI think Steve means one package at a time with commits in between12:20
carloshmm12:20
daniloswell, it was designed in such a way that it can be run that way, since it also supports incremental updates; carlos, isn't that so?12:20
carlosyes, it is12:20
carlosnot per package12:20
carlosbut per table12:20
carlosdoing it per package will require to add some extra code12:20
carlosthe problem of doing it per table12:20
carlosis that at some point people will start seeing some inconsistent data12:21
carlosbut I don't think they could actually break anything12:21
daniloswell, then the per-package seems easier and safer12:21
carloswe don't point to edgy translations so the amount of people looking at that data would be really small12:21
carlosdanilos: but also much more slow12:22
SteveAthat's okay12:22
SteveAit's okay for it to take a week12:22
stubyup12:22
SteveAif it means no interruptions in service12:22
daniloswell, but it will lock only a package at a time12:22
carlosSteveA: but it will only be useful for this two runs we need now12:22
danilosand each will be locked for like a really small amount of time12:22
SteveAif one package is locked for 30 minutes, once12:22
SteveAthen that's fine12:22
carlosedgy + 1 will not need that approach, right?12:22
stubIt will lock the tables it is inserting into as normal, but the locks will be for a much shorter period and probably unnoticible.12:22
SteveA3 minutes is even better than 3012:23
danilosyeah, I think it will be far less than 30 minutes12:23
SteveAbut this is a much better approach for our service overall12:23
stubIf it takes 30 mins to do the inserts for one package, or even 1 minute, then I don't know if this approach is doable on the live system.12:23
carlosoo.org would take more than 5 minutes...12:23
carlosit's a monster...12:23
SteveAso, we could do the large packages in a maintenance period12:24
SteveAwe need to find out how long an average-sized package will take12:24
carlosSteveA: hmmm12:24
SteveAotherwise, look for a smaller granularity of commit still12:24
carlosI really prefer if we close rosetta12:24
carlosthis change is going to delay edgy translations again12:25
carlosand I'm already getting lots of complains about it12:25
carlosthe changes we are talking about would take more than a couple of days between code changes, testing on staging, code reviews and landing on production12:25
carlos+ the time to deploy it12:25
carlosso that would be end of next week in the best case12:26
daniloswell, this is actually true, so it all depends on the timing12:27
SteveAstu and i agree that we can take rosetta down12:27
SteveAif you guys can write some security code that12:27
SteveAhas a launchpad.conf option12:28
SteveAso that when this option is set12:28
SteveApermission is denied to everything except for launchpad.Admins12:28
carloseven for the pages that are read only ?12:29
SteveAand to either12:29
carlosI mean, the ones that doesn't need to be logged in12:29
SteveA1. raise a custom subclass of ForbiddenAttribute or whatever12:29
SteveAor 2. customize the forbidden oops page (and authorization required, or whatever)12:30
SteveAwe need to do this for every page that allows writing to rosetta12:30
danilosso, carlos, what do you think about this?12:30
SteveAwe'll test on staging12:30
SteveAby locking the rosetta tables12:30
carlosif it's just for teh pages that allow writing12:30
SteveAand checking that we can't lock up launchpad12:31
SteveAjust for edit pages12:31
carlosthe first part is quite easy12:31
SteveAI mean, we could do this in the forms instead12:31
SteveAinstead of the security code12:31
carlosthe second part... is the first time I do it so I'm not sure which option is better12:31
SteveAif there are only a few rosetta forms that allow updating rosetta tables12:31
SteveAthat might be easier12:31
=== mpool [n=mbp@ozlabs.tip.net.au] has left #launchpad-meeting []
carlosthere should be 4 or 5 different forms12:32
SteveAraise a RosettaDownForMaintenance error12:32
daniloswell, we also need to block imports, no?12:32
SteveAyes12:32
SteveAand do an error page for RosettaDownForMaintenance12:32
carlosdanilos: that's just a matter of disable the cron job12:32
SteveAwe'll be making the tables read only to the launchpad user anyway12:32
SteveAso the database-level stuff will be okay12:32
SteveAbut we want to give nice errors and not OOPSes12:32
danilosok, just taking notes of everything12:33
carlosok12:33
danilosso carlos12:35
daniloswhat do you want us to do? rosetta downtime, right?12:35
SteveAjust to check something12:36
SteveAwhen we were in london together12:36
SteveAwe talked about updating the stuff into a temporary table12:36
SteveAand then in a separate transaction, or separate translactions, moving them across to the real tables12:36
SteveAis that the approach you've taken?12:36
danilosmostly, I think carlos put the temp table creation into the same transaction (so it doesn't get out of sync)12:37
SteveAum12:37
SteveAthat kinda obviates the point12:37
carlosdanilos: yes, I would prefer to do the downtime12:38
danilostemp table is reading only, so it can probably be moved out without too much problems12:38
daniloscarlos: ok, then lets do that as painlessly as possible12:38
daniloscarlos: btw, didn't we get any improvements with temp tables? do we have indexes in there?12:39
stubSo build all the inserts into tables. Then, a few rows at a time, insert some data into the production tables, delete from the temporary store, and commit.12:39
carlosdanilos: indexes helped a bit12:39
carlosbut not too much12:39
carlosSteveA, stub: I guess that it could make us to lose some translations for the migration (anyone added after the temporary table is created)12:40
stubcarlos: That would have happened anyway, as the migration script would be using a snapshot of the database at the time the migration was started12:41
carlosstub: well, if rosetta is closed12:41
danilosstub: it gets a bit more complicated because of references and us updating table-by-table12:41
carlosno new translations will be added, right?12:41
carlosalso, we should block any import from poimport script for edgy or as soon as the potemplate table is filled, entries from the queue will start to be importing12:43
carlosgiving us conflicts with the temporary table12:43
carlosWhat problem do you see If we close Rosetta ?12:47
carlosI think that's doable and in the future, that feature would be interesting  in case of a performance problem like the ones we had in the past so the rest of launchpad is not also affected12:48
daniloswell, we want to go for that anyway12:50
carlosIn that case, I think that closing rosetta two days12:53
carlosone per migration12:53
carlosis not a big problem, if we choose the right time when users are not doing many translations12:53
SteveAstu and i are discussing various things...12:56
carlosok12:59
SteveAok01:02
SteveAhere's the plan01:02
SteveAstu will try the update code on a clone of production, on the production server01:03
SteveAturning off indexes and shit like that01:03
SteveAwe'll get an accurate estimate of the amount of downtime01:04
danilosok01:04
SteveAmeanwhile01:04
SteveAwe can assume it will take at least 2 hrs01:04
carlosSteveA: should I start working on the 'disable Rosetta' feature ?01:04
SteveAso you guys need to add a switch to launchpad.conf01:05
carlosdanilos: I think I should work on it as I'm just switching tasks and you already have a lot of open things01:05
SteveAthat causes rosetta edit views etc. to raise RosettaDownForMaintenance when the view is viewed01:05
SteveAas well as edited01:05
SteveAand an error view for RosettaDownForMaintenance01:05
SteveAand we need to test this on staging, by stu turning off write access to rosetta tables01:06
carlossounds good to me01:06
carlosstub: the script should be run with '-d ubuntu -r dapper' the first time and the next one with '-d ubuntu -r edgy'01:07
daniloscarlos: sure, be my guest :)01:07
carlosthe first one will migrate translations from breezy to dapper and the next one will copy all translations from dapper to edgy01:07
=== salgado [n=salgado@200-171-140-32.dsl.telesp.net.br] has joined #launchpad-meeting
=== flacoste [n=francis@modemcable207.210-200-24.mc.videotron.ca] has joined #launchpad-meeting
flacostehi SteveA04:30
SteveAhi salgado 04:30
SteveAhi flacoste 04:30
salgadohey!04:30
flacostei think the spec you were looking for was https://launchpad.canonical.com/LaunchpadI18n04:30
flacosteactually, that we should have looked for04:30
=== SteveA waits for stub
=== stub [n=stub@82.109.136.116] has joined #launchpad-meeting
SteveAhttps://launchpad.net/products/launchpad-support-tracker/+spec/localized-support04:30
SteveAhttps://launchpad.canonical.com/LaunchpadI18n04:30
SteveAfirst, let's talk about the overall point of LocalizeSupportRequests04:31
salgadoI pasted that on #canonical, didn't I?04:31
salgadookay04:31
SteveA    *04:31
SteveA      Grandma Lucille, who doesn't speak English, was navigating on the web until she reached a flash page, whose content she can't see because the flash plugin is not installed. She'll then click on "Help -> Get Help Online ..." and make a support request in Launchpad.04:31
SteveA    *04:31
SteveA      Malba Tahan, a very nice guy who knows a lot about Ubuntu is willing to share his knowledge by answering some support requests. He goes to the Support facet of Ubuntu and look for open requests. He sees a list of requests written in one of his preferred languages, pick up a randon one in Arabic and answers it.04:31
SteveA04:31
SteveAthese are use-cases from the spec04:31
SteveAso, we have requirements around identifying which support requests are written in a particular language04:32
SteveAand of allowing people to file support requests in their own language04:32
SteveAis that right, as overall goals?04:32
salgadoright04:32
SteveAso, the support tracker needs to be able to track various things about what language the request is made in04:32
flacosteexactly04:32
SteveAand that's self-contained in the support tracker application04:32
SteveAI totally trust you guys to do a good job there04:33
flacostethat was the plan there04:33
SteveAwhat I'm concerned about is how this touches other parts of launchpad04:33
salgadowhat would these various things about the language be?04:33
SteveAand that a good plan for the support tracker may cause problems elsewhere04:33
flacostethe spec also covers what we needed form the launchpad infrastructure04:33
SteveAfirst of all, some overall concerns04:34
SteveA - if we localize a page element that appears on the support pages, but also elsewhere04:34
SteveA   then we get partially localized pages in other parts of the application04:35
flacoste(that was covered indirectly in the spec)04:35
salgadowas it?04:36
flacosteworkaround) Translate strings for support. Display a message when going to untranslated section of the site.Mark template as translated or not. Force English when using an untranslated template.04:36
SteveAand then, all the unresolved issues in https://launchpad.canonical.com/LaunchpadI18n04:36
SteveAas far as managing risk goes...04:38
SteveAi think we should split the support tracker goals up into two specs04:39
SteveA1. making the support tracker support multiple languages of support issues, but with its UI remaining untranslated04:39
SteveA2. translating the launchpad UI, with the first parts to be translated being the support-tracker and the login workflow04:40
SteveAto make 1. a 1.0 goal, but not 2.04:40
SteveAbecause 2. touches so much stuff, and puts an unexpected burden on the infrastructure work04:40
SteveAthat doesn't mean we can't get to 2. for 1.0, but it means that we aren't committing to it04:41
SteveAwhat do you all think?04:41
salgadoI think that'd be quite awful, usability-wise, but I can understand it04:41
flacostei agree that posting a question in my native language while the UI is in English doesn't look too good, but I can understand the idea of doing this two-steps04:43
salgadookay, now we need to convince mark of that, right?04:43
SteveAI'd really plan this as three specs04:43
SteveA - make the support tracker work in multiple languages for the support requests, inc. data model changes etc.04:43
stubLast time we were looking at this, it seemed like the i18n would be postponed indefinitely. To participate in open source development, you needed to have at least basic English skills. I'm unsure if other parts of Launchpad will ever want to be localized, and it will be unfortunate if there is fallout in the rest of Launchpad to support localized support request tracking.04:44
SteveA - make launchpad support internationalization04:44
SteveA - make the support tracker and login / registration system internationalized and localized 04:44
SteveAwe have a dependency relationship between these specs04:44
SteveAtwo of the specs are in the domain of logins and support tracking04:44
SteveAone is in the domain of infrastrcuture04:44
SteveAnow, in terms of planning 1.0 features04:45
SteveAas leader of the infrastructure team, I'm being asked to support a new 1.0 requirement04:45
SteveAso we need to look at resources for that04:45
stubThe spec mentions searching in particular. Our search infrastructure does not support searching in non-English languages. It might work by accident in many cases, but there will be glitches we cannot solve without features being implemented upstream.04:45
SteveAnow, I can imagine a work-around of sorts04:46
SteveAthat is, to ignore the login / signup parts04:46
SteveAbecuase people manage to log in and sign up to get shipit CDs todasy04:46
SteveAso i expect they *can* do so for support requests04:46
flacostestub: from a developer point of view you are right, but not from an Ubuntu user point of view04:46
SteveAand to localize just the body of support pages04:47
SteveAusing the language the user is interacting with the support system04:47
flacosteSteveA: what do you think of the 'Mark template as l10n idea'?04:47
flacosteto prevent half-balked localization in the other part of the system04:47
SteveAI think that's an interesting idea to solve the problem of having a rendered page consistently translated or not04:47
SteveAthat's just one part of the whole infrastructural issue04:48
flacosteindeed04:48
SteveAwhat I would ask for next is this:04:48
SteveA - split up the specs into the three I mentioned04:48
SteveA - take into account the existing spec about i18n for the infrastructural aspects04:48
SteveA - write into that spec the minimum necessary to deploy an internationalized / localized support tracker04:49
SteveAthen we can come back and look closely at what we end up with04:49
SteveAand decide if this is do-able for 1.004:49
salgadosounds good to me04:51
salgadoI can do that04:51
stubCool04:51
SteveAok04:51
flacosteflacoste: side note: in your infrastructure spec you forgot to mention how to handle string interpolation04:52
SteveAI think the support-only first part can be done immediately04:52
SteveAI mean, not blocked04:52
stubThe search issue I mentioned should be listed on the spec. We may need to contract upstream if searching in different languages is particularly bad. 04:52
SteveAwe should have an idea of what languages we're targetting too04:53
stubI suspect the search results will get worse the further the language gets from the standard roman alphabet.04:53
flacostestub: there is UTF-8 support in 8.1 or 8.2 i think04:53
stubflacoste: tsearch2 has utf8 support, but it still is language dependant for stemming etc.04:53
stub(in 8.2, which doesn't exist yet, or CVS head)04:53
flacostestub: indeed, but there is a 8.1 backport of the utf8 stuff i think04:54
stubAnd all our helpers force queries to ascii only, so that will all need to be rewritten.04:54
stub(because of the existing tsearch2 limitation)04:54
flacostestub: ideally, we would also have to add support to the fti to index tickets based on their language04:55
SteveAmy gut says we really should try to internationalize launchpad within the next 6 weeks04:55
SteveAum04:55
SteveAi mean04:55
SteveAmy gut says we really should not try to internationalize launchpad within the next 6 weeks04:55
flacosteSteveA: yes, a list of supported language would be nice - who can decide on that?04:56
SteveAI can see it being a lot of rushed effort04:56
SteveAwith flaky results because it is rushed04:56
flacostestub: (OT: i have another fti question for you after the meeting)04:56
SteveAstu may have time to help on this04:56
SteveAbut we need to look into this04:57
SteveAthanks salgado and flacoste !05:02
SteveAsalgado: please tell me when we should look at the specs again05:02
flacosteTo summarize:05:02
flacoste- salgado will merge the infrastructure part of LocalizedSupportRequest into LaunchpadI18n05:02
flacoste- salgado will move login workflow related changes to a separate spec05:02
flacoste- Infrastructure will review these specs to see if LaunchpadI18n and LaunchpadLoginL10n can be targetted for 1.005:02
flacoste- add string interpolation example to LaunchpadI18n (flacoste)05:02
flacosteOther unassigned issues:05:02
flacoste- need to add search issue to LaunchpadI18n05:02
flacoste- determine list of languages to target05:02
flacosteshould I mail the list with this?05:03
SteveAalso05:03
SteveAwe should have one spec for LocalizedSupportRequest05:03
SteveAand one spec for LocalizedSupportTracker05:03
salgadoyeah, it will be kept, I assumed05:03
SteveAso that we can represent the dependencies well05:03
salgadoah, right05:03
SteveAthanks for summarizing flacoste 05:04
flacostemy pleasure05:04
flacosteshould I mail this summary of our discussion to the list (after including SteveA update)05:04
flacoste?05:04
SteveAmailing the launchpad list will be good.  it needs a bit of an introduction as well, of course05:04
flacosteno problem05:04
SteveAwhere my points for that are05:04
SteveAthat it is good to make the support tracker localized05:05
SteveAit is in a way a special part of launchpad05:05
SteveAin that it is used by end-users who don't necessarily speak english well05:05
SteveAin a way this is like shipit05:05
SteveAit is unlike rosetta (translators always speak english)05:05
SteveAand unlike malone05:06
SteveAand unlike the spec tracker05:06
SteveA(programmers always speak englilsh)05:06
SteveAI generalize a little05:06
SteveAbut these are generally true05:06
SteveAwe must take seriously the task of internationalizing launchpad, because it has an effect on various areas of what we do05:06
SteveAincluding05:06
SteveA - code reviews (reviewers need to know what to check for and understand the mechanisms that will be appearing throughout the code)05:07
SteveA - test running (what locale / language do we run tests in ? )05:07
SteveA - rollouts (how do we get PO files / generate POT files for doing new rollouts? )05:07
SteveA - web request infrastructure (is the standard stuff in zope3 good enough for our purposes)05:08
SteveA - cookie / database language preferences (new UI, new database work, new infrastructure)05:08
SteveA - searching infrastructure (maybe doesn't cope with languages well)05:08
SteveA - checking the current stuff about internationalizing python in zope, zcml, page templates05:09
=== flacoste will put these points in his message
SteveA - having consistently translated pages (maybe using francis' ideas)05:10
SteveA   (but that implies changes infrastructure to support it)05:10
SteveA05:11
SteveAthere may be a middle-ground05:11
SteveAof just internationalizing certain page, like the support tracker pages05:11
SteveAand not worrying about having a full experience for support users05:11
SteveAoh, also05:11
SteveA - internationalizing email we send out05:11
SteveA    like for the sign-up stuff05:11
flacosteand the ticket notifications05:12
SteveAright05:12
SteveAwe should have a standard way of doing that05:12
=== flacoste isn't sure if we covered that in the spec
SteveAand really, that's a spec in itself05:12
SteveA"internationalized email"05:12
SteveAas a management thing, we need to decide about doing all this in a planned and well-thought-out way05:12
SteveAor rushing it to completion over the next 6 weeks, fixing issues as we come across them05:13
flacoste(am i  to understand that 1.0 is scheduled for in 6 weeks?)05:13
SteveAand dealing with the fall-out later.   I'm keen on avoiding a rush.05:13
SteveAsomething like that05:13
SteveA1 oct or so i think05:13
SteveAand people have vacations too05:14
SteveA05:14
SteveAI think that's all05:14
SteveAany comments ?05:14
flacostenot from me05:15
=== sivang expects lots of RTL issues for RTL languags.
sivangso make sure generilzation is made to include those languages for good support just as the latin ones.05:16
sivang(for example, emails need be RTL, input fields and the whole ballgame)05:16
SteveAdepends if RTL languages are in the list of ones we're focusing on05:16
sivangtrue05:17
sivangprobably better start with the latin ones, which would incur less fallout and more quick fixable issues I suppose , and only then advance further.05:18
flacosteSteveA: i added a MessageId interpolation example to LaunchpadI18n05:24
SteveAgreat05:24
flacostewe have a bug with notification currently for those strings, its bug 5498705:25
flacostethe bug report include a fix, I can update the tests and submit the fix for review if you care, otherwise it can wait05:25
SteveAstub should look at that05:28
SteveAI'll assign it to him05:28
flacostegreat05:29
=== flacoste [n=francis@modemcable207.210-200-24.mc.videotron.ca] has left #launchpad-meeting ["Bye"]
=== danilos[gone] [n=danilo@82.117.204.8] has joined #launchpad-meeting

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!