=== bigjools-afk is now known as bigjools === mrevell is now known as mrevell-lunch === SteveA_ is now known as SteveA === mrevell-lunch is now known as mrevell [13:55] * gmb runs to get a drink [13:59] #startmeeting [13:59] Meeting started at 13:59. The chair is kiko. [13:59] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [14:00] hello engineers of the third millenium [14:00] welcome to another fantastic weekly meeting [14:00] kiko: thank you [14:00] hosted by your perpetually unstable chairman [14:00] kiko o cruel [14:01] me [14:01] (that means kiko the humble in portuguese) [14:01] where's my agenda [14:01] * kiko shows raw text [14:01] this is a RAW agenda [14:01] * Hobbsee fears [14:02] * Hobbsee covers eyes [14:02] so who is here this bright morning [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] not me [14:02] me [14:02] me === salgado_ is now known as salgado [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] me [14:02] keep em coming [14:02] me [14:02] me [14:03] where's this new chap, maris [14:03] ah there you ae [14:03] * flacoste welcomes vednis [14:03] me [14:03] are too [14:03] SteveA, we have no stew. [14:03] but now we do! [14:03] me [14:03] That was nice timing. [14:03] Slow clock sorry [14:03] very good [14:04] me [14:04] hello sprinters [14:04] thanks for making it [14:04] this meeting is not the same without sprinters [14:04] so [14:04] [TOPIC] agenda! [14:04] New Topic: agenda! [14:04] * Roll call [14:04] * Agenda [14:04] * Next meeting [14:04] * Actions from last meeting [14:04] * Oops report (Matsubara) [14:04] * Critical Bugs (Rinchen) [14:04] * Bug tags [14:04] * Operations report (mthaddon) [14:04] * DBA report (stub) [14:04] * Sysadmin requests (Rinchen) [14:04] * New packages required (salgado) [14:04] * A top user-affecting issue (mrevell) [14:04] * kiko doesn't fuck up the paste! 10 points [14:04] hurrah! [14:04] that knocks off two topics in one go [14:04] heh :) [14:05] kiko, your language is terrible today :) [14:05] [TOPIC] behold a pale horse!! next meeting time [14:05] New Topic: behold a pale horse!! next meeting time [14:05] very good [14:05] so Rinchen the Ill has been attempting to conspire to change the meeting time [14:05] the proposed meeting time is 18:00 UTC [14:06] this is in order to allow Tim Penhey the Kiwi to participate [14:06] primarily [14:06] I think it's really important that all team leads be here [14:06] and His Team of Antipodians [14:06] hi abentley [14:06] does aaron know about this meeting? [14:06] I believe Rinchen has contacted most people more seriously affected by this change [14:07] Not me [14:07] SteveA, that's probably a question for his manager and super-manager [14:07] stub: you commented in the last meeting [14:07] * kiko hands edwinGrub some internet [14:07] All our managers are super-managers [14:07] is anyone surprised or taken aback by 18:00 UTC as the new meeting time, apart from stub? [14:07] * SteveA basks in the flattery [14:08] speak now or hold your peace until next thursday [14:08] jamesh is not here [14:08] I did not know about it [14:08] I'd prefer to have it earlier once we hit summer time again [14:08] until then, I am completely fine with it [14:08] sure, we can consider changing it again in the future [14:08] very well [14:08] i'd prefer to have it earlier, since i have a class on thursday evenings [14:09] that's not as serious as the kiwis' needs, though [14:09] yeah, to me it's a big deal that tim can't come [14:09] 3am meetings are fun! [14:09] intellectronica, chat with me about it after the meeting, maybe we can arrange something [14:09] [AGREED] 18:00 UTC next thursday, next meeting [14:09] AGREED received: 18:00 UTC next thursday, next meeting [14:09] thanks MootBot the Redundant [14:09] kiko: I assume this is set for the future meetings as well? [14:09] kiko: Someone should update the team calendar. Want me to take care of that? [14:09] (until discussed and changed again) [14:09] If we are changing the time, we could consider changing the day too (although Thursday is now a good day for me) [14:10] gmb, thanks! [14:10] [ACTION] gmb to update the team calendar and the meeting agenda [14:10] ACTION received: gmb to update the team calendar and the meeting agenda [14:10] [14:10] danilos, yes, that's correct. [14:10] Boys and their toys... [14:10] but we can change it again [14:10] that's the beauty of decisions [14:10] [TOPIC] Actions from last meeting [14:10] New Topic: Actions from last meeting [14:10] so the actions were: [14:11] * kiko to add the lp-dependencies question to the MeetingAgenda template [14:11] I did that, but somebody killed it? [14:11] * kiko curses [14:11] oh, sorry, I did [14:11] * New packages required (salgado) [14:11] 10 points for doing it, then -10 points for not noticing [14:11] Rinchen and kiko have inflicted the 18:00 time on us, so done too [14:12] SteveA and stub: staging sorted, I believe? [14:12] and mrevell and mpt? [14:12] * Rinchen and kiko to figure out how to manage lp-bzr's absence from the meeting; consider rotation or time-shift [14:12] * SteveA and stub to discuss options for staging DB and report back [14:12] * mrevell and mpt to discuss https://launchpad.net/bugs/185486 [14:12] Launchpad bug 185486 in launchpad "Merge account option should be on profile page" [Undecided,New] [14:12] * kiko discovers how to paste in X [14:12] We discussed it briefly [14:12] I don't remember the conclusion [14:13] mpt: We concluded, I believe, that you'd look at a better way of handling it. [14:13] we're putting staging on carbon [14:13] ah yes [14:13] GOOD [14:13] yup [14:13] I will need to solve it while rejigging the person page [14:13] kiko: staging only sorted in the short term, although that might depend on the langpack db [14:13] mpt: as part of the Actions menu changes, I think. [14:13] the short term is what I'm after [14:13] thanks stub [14:13] * mpt assigns it to self [14:13] thanks SteveA [14:13] SteveA, mthaddon says there's an ETA of ~ 1.5 weeks on that [14:13] confirmed? [14:14] kiko, confirmed [14:14] [AGREED] staging to carbon (and in 1.5 weeks or so) [14:14] AGREED received: staging to carbon (and in 1.5 weeks or so) [14:14] or sooner, I hope :) [14:14] mpt, anything more to add to the subject? [14:14] stub: what's going on with langpack db on carbon (if anything)? [14:15] * kiko clicks tongue [14:15] danilos: I don't know [14:15] [TOPIC] * Oops report (Matsubara) [14:15] New Topic: * Oops report (Matsubara) [14:15] do it mats [14:15] Today's oops report is about bugs 187389, 157606, 185706, 44834 [14:15] Bug 187389 on http://launchpad.net/bugs/187389 is private [14:15] Yesterday I asked flacoste to take a look at bug 187389. It might interest [14:15] people in the SC team as well. [14:15] Bug 187389 on http://launchpad.net/bugs/187389 is private [14:15] I found out about it while trying to reproduce bug 157606. Someone from the Bugs team would like to take that one? [14:15] Launchpad bug 157606 in malone "IntegrityError with unknown milestone when changing bug's project" [Medium,Confirmed] https://launchpad.net/bugs/157606 [14:15] kiko, I just assigned the bug to myself. [14:16] for 1.2.4. [14:16] mpt, thanks. [14:16] sinzu1: you ok with targetting bug 185706 to 1.2.2? [14:16] matsubara, ah, because of the tickcount thing [14:16] * sinzu1 thinks [14:16] does that bug really need to be private? [14:16] stub, I've added 3 more OOPSes to 44834. Please follow up in the report if that sheds any light there. [14:16] Launchpad bug 185706 in launchpad "OOPS in +editemails form if email address field is empty" [Undecided,New] https://launchpad.net/bugs/185706 [14:17] matsubara: I'll commit to 1.2.2 [14:17] hmm, we keep getting problems with the +editemails form [14:17] thanks sinzu1 [14:17] sinzu1, flacoste: what's up with that? [14:17] kiko: we'll look it up [14:17] flaky validator is what this one looks like [14:18] very well [14:18] kiko: bad validation [14:18] [AGREED} sinzui agrees to https://launchpad.net/bugs/185706 for 1.2.2 [14:18] AGREED received: [AGREED} sinzui agrees to https://launchpad.net/bugs/185706 for 1.2.2 [14:18] Launchpad bug 185706 in launchpad "OOPS in +editemails form if email address field is empty" [Undecided,Confirmed] [14:18] someone from Bugs? [14:18] oh you don't need the angles, hmm [14:18] matsubara: you can assign bug 157606 to me [14:18] Launchpad bug 157606 in malone "IntegrityError with unknown milestone when changing bug's project" [Medium,Confirmed] https://launchpad.net/bugs/157606 [14:18] Thanks BjornT [14:18] flacoste, what's the decision on 187389? [14:19] AGREED matsubara: you can assign bug 157606 to me [14:19] hmm [14:19] [AGREED matsubara: you can assign bug 157606 to me [14:19] flacoste: we'll also look that one up [14:19] [AGREED] matsubara: you can assign bug 157606 to me [14:19] AGREED received: matsubara: you can assign bug 157606 to me [14:19] kiko: ^^^ [14:19] weird. [14:19] thanks flacoste [14:19] got it alredy [14:19] already, even [14:19] [AGREED] flacoste, to take 187389 for 1.2.2 too [14:19] AGREED received: flacoste, to take 187389 for 1.2.2 too [14:20] matsubara, what else? I thought I saw some weird OOPSes in yesterday's report. [14:20] Bug 44834 [14:20] Bug 44834 on http://launchpad.net/bugs/44834 is private [14:20] kiko: which one? [14:20] matsubara, can't look them up now though [14:20] I asked stub to take a look in the weird ones from yesterday's [14:21] I don't know if it's the same, but it's a SQLObjectNotFound error [14:21] right [14:21] very good [14:21] [TOPIC] * Critical Bugs (Rinchen) [14:21] New Topic: * Critical Bugs (Rinchen) [14:21] jtv also took a look since the same bug hit on TrannslationMessage [14:22] yeah, I saw that [14:22] something about doesn't exist in the database, right? [14:22] no critical bugs from Rinchen [14:22] great [14:22] yes kiko [14:22] [TOPIC] * Bug tags [14:22] New Topic: * Bug tags [14:22] matsubara: Not sure if I'm best for that - looking like an SQLObject issue, possibly caching. [14:23] no tags proposed [14:23] so it's getting to that time [14:23] where tom tells all [14:23] stub: who'd be the best person to look at it? jamesh perhaps? [14:23] * Operations report (mthaddon) [14:23] In the process of migrating the staging DB to carbon, demo to jubany and the langpack DB to asuka [14:23] IS is working on the timing of testing for new production DB - I'll update everyone as soon as I know the details [14:23] Did anyone have a chance to look at the backtrace I posted to the list? [14:23] That's it from me unless there are any questions [14:24] stub: which bug is that? [14:24] mthaddon, from which appserver is that BT? [14:24] mthaddon, remind us about the backtrace. [14:24] Jubany should not be running demo [14:24] ah, the crash [14:24] core [14:24] whatever [14:24] salgado, it's from gandwana - not sure if it was lpnet3 or 4 [14:24] Jubany is the production db and shouldn't be running anything else [14:24] SteveA: bug 44834 [14:24] Bug 44834 on http://launchpad.net/bugs/44834 is private [14:24] agreed with stub [14:24] I think jubany can run demo [14:24] kiko, https://pastebin.canonical.com/2139/ [14:25] I'm interested in a rationale for why it shouldn't [14:25] matsubara, for bug 44834 to be properly debugged we'd need access to the DB logs. [14:25] it can, but it shouldn't. Sysadmin 101 stuff. [14:25] Bug 44834 on http://launchpad.net/bugs/44834 is private [14:25] I'm still not hearing a rationale [14:25] mthaddon: what are the reasons to move langpack to asuka? jtv, have you been involved in these discussions? [14:26] danilos: see the mailing list [14:26] carlos: ok, thanks [14:26] stub, my understanding is it's not a perfect solution, but better than having staging updates on carbon affect demo [14:26] danilos: they need some disk space in carbon [14:26] danilos, because we want to migrate staging DB to carbon, but we don't have enough disk space for that and langpack as well [14:26] carlos: ok, but do we need langpack db if we can't use it for performance testing? [14:26] carlos, danilos, jtv: asuka isn't a speed demon [14:27] kiko: exactly my point [14:27] oh that was unfortunate [14:27] [TOPIC] * DBA report (stub) [14:27] kiko: ;-) [14:27] New Topic: * DBA report (stub) [14:27] maybe he was writing it [14:27] kiko: op abuse, no? :) [14:27] yeah but I can't kick myself as host now [14:27] grumble [14:27] * Hobbsee can, if you like... [14:28] Because that is the most critical system we have. You just don't stick other stuff on there. If anyone things 'launchpad is slow today', we have to shut demo down to remove that variable. [14:28] danilos: good point... although right now we worry more about being able to debug problems than performance. Anyway, jtv is going to discuss alternatives with Steve and kiko outside this meeting [14:28] carlos: ok, just making sure we're on top of it [14:28] carlos: right. The big problem is that staging is too slow. [14:28] I can put more rationale together after the meeting. [14:28] very well [14:28] * DBA report (stub) [14:29] anything of note and concern stub? [14:29] But I am strongly opposed to putting any unnecessary services on jubany and I'm fairly sure the IS group would back me on that. [14:29] I'd prefer we measure the effect of demo, and base a decision on measurement rather than just "nothing should run there" [14:29] SteveA: It is a production system. If we want to experiment on it, then you have been feeding people the wrong message the last year. [14:30] I've got most of a spec splitting Person into Person and Account, which is the first part of creating a high availability authentication service for things that need it, such as Landscape. [14:30] I'm unsure of the status of the new database hardware evaluation. [14:30] Nothing else to report at the moment. [14:30] thanks stub [14:30] great news on person versus account [14:30] I will be your #1 fan when that's done [14:30] btw, thanks also for helping me prepare the query [14:30] for the SUBOPTIMAL distro upstream bugs report [14:31] anyone else have anything to thank stub specially for? [14:31] very well [14:31] the thing I don't get is that demo is meant to run exactly the same code as lpnet, just with a different database, for a small set of users evaluating it [14:31] demo is not for doing anything experimental on. [14:31] if we're doing experiments on 'demo', they need to stop [14:32] SteveA: It is taking resources on our bottleneck. [14:32] SteveA, I think it's more the fact that there is load interaction from running two instances [14:32] stub: the APIs guys should read through your spec on splitting account and person [14:32] Even if the load is fine for the first day or week or month even, it an screw up at any time. [14:32] stub: please send leonardr a copy [14:32] yes please [14:33] I'll put the wiki link on the mailing list when I've filled in some remaining sections. [14:33] how many of edwinGru do we need today? [14:33] [TOPIC] * Sysadmin requests (Rinchen) [14:33] New Topic: * Sysadmin requests (Rinchen) [14:33] joey's sick [14:33] so am I but I don't get any better so I'll handle this topic [14:34] who has RT requests [14:34] * EdwinGru develops the secret to cloning [14:34] and wants them addressed [14:34] kiko: I have request 29852, which is about mail to launchpad-feedback bouncing [14:34] bigjools, did we get the package you wanted backported? [14:34] mrevell, so I don't get that. I have seen mail getting through there regularly [14:34] wasn't that a one-off fluke? [14:34] kiko: lamont installed his first version in mawson and it worked very well when I tested it [14:34] As of yesterday, I was still seeing some mail bounces and it'd be nice if the IS guys could take a look. [14:35] kiko: some mail is getting through [14:35] bigjools, that's great news! lamont is the man [14:35] kiko: but it appears other mail is bouncing. [14:35] he sure is [14:35] mrevell, how does it bounce? can I see a bounce message? [14:35] bigjools, communicate to him my thanks ** 2 [14:35] will do [14:35] kiko: Sure, just a sec. It's complainging of a timeout [14:35] he also helped me figure out my postfix mess [14:36] okay [14:36] any other RTs? [14:36] [TOPIC] * New packages required (salgado) [14:36] New Topic: * New packages required (salgado) [14:36] kiko: https://pastebin.canonical.com/2168/ [14:36] if any of the branches you're working on right now depends on any library which is not part of the launchpad-dependencies package, come talk to me ASAP. [14:36] anybody have library requests? [14:36] 5 [14:36] 4 [14:36] 3 [14:37] 0 [14:37] very good! [14:37] that didn't even hurt [14:37] [TOPIC] * A top user-affecting issue (mrevell) [14:37] New Topic: * A top user-affecting issue (mrevell) [14:37] hi [14:37] hi [14:37] I've got a couple this week. [14:37] eww [14:37] :) [14:37] The main issue I've seen reported this week has been translation timeouts, although I've seen that jtv has responded to reports on launchpad-users. [14:38] jtv: is there a particular answer that you'd recommend for members of the Launchpad team to say if they're asked in #launchpad or other venues about translations timeouts? [14:38] There was a spike last weekend. [14:38] jtv, spike in use too? what do the server logs say? [14:38] jtv: language pack generation? [14:38] I have some recommendations that I repeat as necessary. [14:38] yeah, not very comforting ones though [14:38] danilos: possible [14:38] so [14:38] question for you [14:38] carlos once upon a time wrote an AJAX suggestion fetcher [14:39] what do we think of going back to that? [14:39] kiko: Yes, we do think of that. [14:39] kiko: sinzui has promised to look at that in later cycles. [14:39] why sinzu1? [14:39] kiko, jtv: I think we could try to profile current code again (it changed a lot with the new DB refactoring) before going for AJAX [14:39] kiko: I think we can instead hardcode only the handful of messages (like "File", "Open") which cause the problem [14:39] kiko: because we have thousands of rows for those [14:40] danilos, hmmm, that's an interesting thought [14:40] hardcode where? [14:40] a special table? [14:40] kiko: yeah, possibly [14:40] hardcode or cache [14:40] interesting [14:40] kiko: I was asked. I have some experience with that kind of problem. I have not hacked on it (I have some high priorities at this time) [14:40] INTERESTING IDEA!!! [14:40] kiko: these are only *external* suggestions, so it's not important to be 100% correct [14:41] danilos, +10 for a great new interesting idea [14:41] SteveA: hard code which ones we decide to cache [14:41] I had not considered it before [14:41] so two more topics before the bell strikes [14:41] [TOPIC] * Future of the "ui" bug tag (mpt) [14:41] New Topic: * Future of the "ui" bug tag (mpt) [14:42] says that the "ui" tag is for "A bug that suggests the user interface is confusing or otherwise difficult to use" [14:42] flacoste: can we talk over https://devpad.canonical.com/~matsubara/oops.cgi/2008-01-30/B1498 in a bit? [14:42] so it does mpt [14:42] but shows that it's been used for many bugs that happen to involve the UI at all [14:42] SteveA: sure, after our call maybe [14:42] regardless of whether it's confusing or difficult [14:42] mpt, okay [14:43] mpt, why don't we have a confusing tag or something? [14:43] So, we either need to reinforce the current meaning, change the meaning, or retire the tag [14:43] change the meaning [14:43] what kiko suggests [14:43] ui is for ui [14:43] confusing is for confusing [14:43] I'd change the meaning to mean "ui-affecting" which is a useful thing [14:43] I don't think a "ui-affecting" tag would be that useful, because it would apply to ~1/3 of all Launchpad bugs [14:44] that's okay though [14:44] and because I can't think of a use case for it [14:44]