=== mrevell is now known as mrevell-lunch === salgado-afk is now known as salgado === gmb_ is now known as gmb === mrevell-lunch is now known as mrevell [16:50] its my party and i parallel park if i want to with Hobbsee D: [16:51] gah. [16:51] * Hobbsee goes to hunt for some staff [16:51] Hobbsee, the queue page working for you? [16:51] kiko-afk: sec, going and hunting for stafff. [16:52] i will tell you everyday for the rest of my life what Hobbsee did to me [16:52] i will make sure you will be reported Hobbsee [16:52] let them review this [16:52] what you did to me [16:52] i knew i liked having ops here. [16:53] what's parallel parking? [16:54] he's whinging because he got banned for trolling [16:54] I wish I could ban trolls on launchpad [16:55] i wish i could get freenode to do what htey say they will, too. [17:01] man, i hate being a woman in tech sometimes. [17:03] Hobbsee, there's a long series of articles on CACM about this recently [17:03] kiko-afk: oh? [17:03] yeah, are you a subscriber? [17:04] nope - no idea what it is [17:05] the communications of the acm? === kiko-afk is now known as kiko [17:21] kiko: i haven't, no. [17:22] kiko: and i can't test the queue stuff now, we've released. [17:22] presumably i'll be able to test toolchain-ey bits, if someone leaves it for me [17:22] Hobbsee, well, you will be able to soon, anyway. :) [17:22] kiko: neat :) [17:22] kiko: why? [17:22] because II will open, I'm hoping? :) [17:24] oh right [17:24] sure, and hopefully i won't have gone mad by then. [17:27] kiko: anywya, where would i find the cacm? [17:27] oh, my bad. [17:28] Hobbsee, check out acm.org [17:28] kiko: yeah, found that much, sorry [17:28] Hobbsee, didn't you do compsci? [17:29] kiko: i'm doing computing, yes. not compsci as such. === salgado is now known as salgado-lunch === Hobbsee is now known as foo___ === foo___ is now known as Hobbsee === salgado-lunch is now known as salgado [18:57] * Rinchen checks for mootbot [18:58] * Rinchen gets a drink [18:59] AND ACTION! [18:59] me [18:59] my [18:59] me [19:00] \#startmeeting [19:00] #startmeeting [19:00] Meeting started at 20:00. The chair is Rinchen. [19:00] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [19:00] Welcome to this week's Launchpad development meeting. For the next 45 minutes or so, we'll be coordinating Launchpad development. [19:00] me [19:00] [TOPIC] Roll Call [19:00] New Topic: Roll Call [19:00] me [19:00] me [19:00] e [19:00] me [19:00] me [19:00] moo [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:00] m e [19:00] me [19:00] moink [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:01] releases team is here... others? [19:01] thumper? [19:01] me [19:01] me [19:01] me [19:01] I believe thumper has a holiday today [19:01] mthaddon ? [19:01] me [19:01] ah right [19:01] jtv? [19:01] so it is [19:01] me [19:01] me [19:01] me [19:01] So I think the remainder of bzrlp (all two of us) are here [19:01] barry? [19:02] me [19:02] oh sorry [19:02] * barry already me'd [19:02] me [19:02] ok then [19:02] me [19:02] [TOPIC] Agenda [19:02] New Topic: Agenda [19:02] * Next meeting [19:02] * Actions from last meeting [19:02] * Oops report (Matsubara) [19:02] * Critical Bugs (Rinchen) [19:02] * Bug tags [19:02] * Operations report (mthaddon/herb) [19:02] * DBA report (stub) [19:02] * Sysadmin requests (Rinchen) [19:02] * New packages required (salgado) [19:02] * A top user-affecting issue (mrevell) [19:02] * Doc Team report (mrevell) [19:02] * Community procedure for unplanned downtime (mpt) [19:02] me [19:02] * some interesting topic (kiko) [19:03] * kiko chuckles [19:03] [TOPIC] * Next meeting [19:03] New Topic: * Next meeting [19:03] same time, same place, next week [19:03] yeah, let's have a next meeting. [19:03] any obstacles present with that? :-) [19:03] next thursday is a brazilian national holiday, btw [19:04] those are the best meeting days!! [19:04] there's a public holiday next thursday for me [19:04] but I think I'll be around. thinking about swapping it [19:04] ok then.. [19:04] I won't be here too. [19:04] same here .. [19:05] it's May 1st .. a public holiday in germany [19:05] [AGREED] meeting next week, same time, station - apologies: brazil, BjornT, al-maisan [19:05] AGREED received: meeting next week, same time, station - apologies: brazil, BjornT, al-maisan [19:05] I will not be here either. This is my last meeting as a Launchpad member [19:05] carlos: we will miss you! [19:05] it's a holiyday for me too (thanks al-mainsan :) [19:05] Rinchen: I'm not in br ;) [19:06] actually, i think May 1st is a national holiday everywhere except for Canada, US (UK?) [19:06] carlos: ! [19:06] statik: "next time, don't miss" [19:06] * mpt hugs carlos [19:06] * carlos will miss working with you too... [19:06] * Rinchen sniffles. [19:06] cpro1, it doesn't matter -- you follow brazilian holidays unless you explicitly request to change [19:06] May 1st is not a UK holiday. [19:06] carlos, you'll have to joins us for the meeting [19:06] flacoste: Not for UK We have it 5th May === cpro1 is now known as cprov [19:06] * barry will take a holiday if flacoste will [19:06] [AGREED] germany off too [19:06] AGREED received: germany off too [19:06] gmb: i think the uk moves it to the next monday [19:06] [TOPIC] Actions from last meeting [19:06] New Topic: Actions from last meeting [19:06] intellectronica: Yep. [19:06] all of my actions from last week were completed. [19:06] [TOPIC] Oops report (Matsubara) [19:06] New Topic: Oops report (Matsubara) [19:06] Rinchen: I will try to ;-) but will miss the funny 'me' part :-) [19:07] carlos, you can still "me" :-) [19:07] Today's oops report is about bugs 220251, 221072, 221047 [19:07] kiko: I've changed my address, it should be enough to follow local holidays, isn't it obvious ? [19:07] Rinchen: thanks :-P [19:07] Launchpad bug 220251 in malone "Calculating subscribers for bugs with multiple tasks is expensive" [High,Fix committed] https://launchpad.net/bugs/220251 [19:07] Bug 221072 on http://launchpad.net/bugs/221072 is private [19:07] salgado, should #220251 importance be higher? is it not causing inconvenience for marilize? [19:07] cprov, nope [19:07] The project group milestone is broken on edge. abel already fixed and the branch is on the pqm queue. (#221047) [19:07] matsubara, the actual bug fix won't help marilize [19:07] Rinchen, next you'll be saying he can still fix bugs ;-) [19:07] Thanks BjornT and kiko for fixing the timeouts on the bug page (#220251) [19:07] matsubara, what we need to do is run a couple DELETES in production [19:08] cprov, you need to explicitly say you want to change to observe other holidays [19:08] matsubara: a fix for 220251 has landed (and another fix to that too) [19:08] mpt, I'm glad you caught that ;-) [19:08] cprov: watch out for queens day next week. depending how centrally you live, it may be impossible to work. [19:08] cprov: it's basically like carnival. [19:08] kiko: it's not time to discuss it. [19:08] intellectronica: right, this is more like a thank you notice :-) [19:08] SteveA: yes, I was warned :) [19:09] salgado: is that the db query in the LPProductionStatus page? [19:09] SteveA: to not go to Amsterdam, basically. [19:09] matsubara, it is [19:09] salgado: ok, thanks then. [19:09] Rinchen: I'm done here. [19:09] thanks matsubara [19:09] thanks everyone [19:09] [TOPIC] Critical Bugs (Rinchen) [19:09] New Topic: Critical Bugs (Rinchen) [19:10] same two [19:10] [LINK] https://bugs.edge.launchpad.net/launchpad/+bug/206654 [19:10] This is the memory issue. flacoste, SteveA. Any updated status? [19:10] LINK received: https://bugs.edge.launchpad.net/launchpad/+bug/206654 [19:10] Rinchen: yes [19:10] the other is for thumper and mhw who are not here [19:10] [LINK] https://bugs.edge.launchpad.net/launchpad-bazaar/+bug/118625 [19:10] LINK received: https://bugs.edge.launchpad.net/launchpad-bazaar/+bug/118625 [19:10] the issue is with +bugs-text [19:10] Launchpad bug 118625 in launchpad-bazaar "codebrowse sometimes hangs" [Critical,Confirmed] [19:10] so I'll skp that [19:10] and I'm working on a work-around [19:10] ultimately, this view should be rewritten to not use SQLObject [19:11] but raw SQL [19:11] let's not mention particular URLs here :-) [19:11] or this funcitonality moved to API where everything will be batched [19:11] that's it [19:12] Rinchen: the result, operationally, is that from tomorrow or so, we should be back to having reliably stable app servers [19:12] flacoste, great thanks. Looking forward to seeing the results. [19:12] fantastic, thanks. [19:12] [TOPIC] Bug tags [19:12] New Topic: Bug tags [19:12] there are no proposed bug tags [19:12] [TOPIC] Operations report (mthaddon/herb) [19:12] New Topic: Operations report (mthaddon/herb) [19:12] Cherry picks since last meeting: 2008-04-18: r6113 & r6110, 2008-04-21: loggerhead r156, launchpad-loggerhead r29 and r6141 [19:12] Currently two cherry picks pending. Barring any objections we'll hold off until tomorrow at the earliest for them due to hardy. [19:12] Bug #206654 continues to be an issue, though it appears that flacoste has had a breakthrough. [19:12] That's it from me and Tom unless there are any questions. [19:12] Bug 206654 on http://launchpad.net/bugs/206654 is private [19:13] thanks herb [19:13] [TOPIC] DBA report (stub) [19:13] New Topic: DBA report (stub) [19:13] I think all the DB reviews are done finally for this cycle. If you are not sure of the status of your patch, email me. I need to talk with bigjools re: archive permissions because I can't understand his email... [19:13] The DB server seems to be coping quite happily with the current load. This is good. This should also be the highest load we see for quite some time as we work on ways of distributing the load (unless someone manages to remove bottlenecks in the other systems ;) ) [19:13] Done. [19:13] herb, there are actually 3 cherry picks pending, no? [19:14] stub: talk to me after this meeting [19:14] salgado, well the 3rd hasn't been approved [19:14] salgado, sorry, I should have clarified. [19:14] herb, right, good point. can you approve it kiko? [19:14] thanks stub [19:14] any questions for stub before I move on? [19:15] salgado, yeah, mats just asked me [19:15] salgado, I thought I had approved it. maybe my wiki edit conflicted and I didn't see that happen. it's approved. [19:15] kiko, he asked for approval for the pending SQL. there's a cherry pick there as well [19:15] [TOPIC] Sysadmin requests (Rinchen) [19:15] New Topic: Sysadmin requests (Rinchen) [19:15] Hi! Is anyone blocked on an RT or have any that are becoming urgent? I'm aware of the usual ones. [19:16] salgado, that I hadn't seen [19:16] * Rinchen waves at bigjools. [19:16] * bigjools waves back [19:16] BjornT, question for you: does mthaddon know that we did a CB yesterday? [19:16] I do, yes [19:17] [TOPIC] New packages required (salgado) [19:17] New Topic: New packages required (salgado) [19:17] kiko, BjornT I sent an email to losas yesterday after the CB [19:17] if any of the branches you're working on right now depends on any library which is not part of the launchpad-dependencies package, come talk to me ASAP. [19:17] jtv, hows that xmlproc code change coming? [19:17] Rinchen: I was just trying to squeeze it into PQM, thanks [19:17] * Rinchen wants to know when he can unpin that package [19:17] herb, one concern I have is that BjornT hasn't landed the fix on RF yet? [19:17] jtv, awesome, thanks. [19:18] kiko: i haven't? [19:18] kiko, I think it's in process, or landing... [19:18] BjornT, okay, need to refresh that mailbox then :) [19:18] kiko: r6151 [19:18] [TOPIC] A top user-affecting issue (mrevell) [19:18] New Topic: A top user-affecting issue (mrevell) [19:18] hi [19:18] Happy Hardy release day! [19:18] The user-affecting issue I'd like to highlight today is that impact that interest in the new Ubuntu release appears to have had on sites that rely on Launchpad for authentication. [19:18] I know this happens only twice a year but I'd like to acknowledge the slow-down in a post to lp-users and explain its cause. Is there some good news I can offer in that message? [19:19] kiko: i had to resubmit it, since the first one failed due to a spurious test failure... [19:19] BjornT, known spurious test failure? :) [19:19] mrevell: the cause is some old software written when launchpad and ubuntu had far fewer users [19:19] mrevell: the good news is that we're replacing it with new software that works much better [19:20] thanks SteveA :) [19:20] so this will be the last such slow-down [19:20] great to hear, thanks for that. [19:20] Thanks Rinchen [19:20] [TOPIC] Doc Team report (mrevell) [19:20] New Topic: Doc Team report (mrevell) [19:20] This week please give me your feedback on the proposed tour layouts. I've posted new drafts to the list today. [19:21] I know not everyone is directly interested in this sort of thing but I'm interested in everyone's view. [19:21] mrevell, has the community members of the doc team been active lately? [19:21] s/has/have [19:22] Rinchen: We haven't had much interest lately and I need to restoke interest after our 1.2.4 release (i.e. when I'm clear to do that) [19:22] s/clear/more free [19:22] ok, thanks [19:22] I'd also be grateful if you could give a few moments thought on the podcast idea. [19:22] thanks Rinchen [19:22] [TOPIC] Community procedure for unplanned downtime (mpt) [19:22] New Topic: Community procedure for unplanned downtime (mpt) [19:22] hi [19:22] mrevell, I really like the new tour layouts, btw [19:22] Last Sunday, Launchpad had major availability problems for an extended period [19:23] kiko: That's great to hear. If you have any specific feedback I'll give it to Gavin in my call with him tomorrow [19:23] There were non-Canonical people in #launchpad who noticed, but they didn't know what to do. [19:23] I started a mailing list thread about this on Sunday, but it hasn't come to any conclusion [19:23] mrevell, I'll try getting email before I trvel [19:23] thanks kiko [19:23] So we still need to decide: [19:24] * Is it necessary for us to publish some way of getting in touch with admins when Launchpad is down? [19:24] - If so, what should that be? [19:24] - And if not, how will we prevent such downtime happening in future? [19:24] Is there anyone in Montreal accessible by our users? [19:24] mpt, when we were down, what happened when loading the site? [19:24] mpt: Presumably help.lp.net was down as well? [19:24] kiko: HTTP Timeouts [19:24] kiko: basically, the site was totally unresponsive [19:24] not even a 503 page [19:25] zaz a bummer [19:25] or 50x page [19:25] yeah [19:25] kiko, as I understand it, pages were *occasionally* available, but extremely slowly if at all. [19:25] put something on the #launchpad topic [19:25] it was caused by the problem we just fixed [19:25] or on help.launchpad.net [19:25] ken [19:25] or are about to fix [19:25] kiko, who are you suggesting do this? [19:25] however, I don't think we need to set up anything special about this [19:25] Is there anything fancy we can do DNS-wise that would redirect people to an off-site holding page in the case of this sort of outage? [19:25] mrevell: no [19:26] mpt, well, that's my questions for you [19:26] the issue is this [19:26] I'm not suggesting much [19:26] we have monitoring systems [19:26] think it's a bit of a non-issue [19:26] tbh [19:26] and they tell us when launchpad is not functioning [19:26] kiko, people in #launchpad did change the /topic, but that's all they *could* do. [19:26] on Sunday, no one was available to respond to the monitoring systems [19:26] mpt, I mean put in the /topic "call 1800LAUNCHPAD" etc [19:26] the solution is that we have people available to respond to the monitoring systems [19:27] and I'm talking to the information systems people in Canonical about that. [19:27] the monitoring systems detected the outage at the same time that people on the channel did. [19:28] As a normal user, if the service went down, I'd like the warm fuzzies to know someone was looking at it. I believe we should have weekend response to monitoring alerts. [19:28] I would also be very thankful if someone posted a time when they think the problem might be fixed [19:28] (as a normal user) [19:28] Rinchen: perhaps we should add that to the DealingwithCrisis policy/doc [19:28] Rinchen, posted /where/? [19:29] Rinchen: if the monitoring system had been responded to on Sunday, the problem would have been fixed within 5 mins. [19:29] kiko, dunno. Somewhere where I would naturally see it. [19:29] the issue as I see it is that the site is down and there's no fallback place to look [19:29] Rinchen: I mean, we should add that the part of the procedure is to notify in public places that people are looking at the issue [19:29] kiko, right [19:29] Rinchen, that's the key problem, so unless we can answer that, all bets are off [19:29] kiko, that rarely happens though [19:29] stop, please [19:29] we're talking about a problem that was fixed within minutes of an appropriate person looking into it [19:29] SteveA, will you post to launchpad@ when your discussions with IS have a conclusion? [19:30] the issue was the lack of appropriate people on that sunday [19:30] and I'm talking with people about how we address that. [19:30] enough with 1800LAUNCHPAD ;-) [19:30] SteveA, it was a non-proposal, in case you missed that :) [19:30] hmmm [19:31] I have more to say about this but I won't now. Moving on... [19:31] We have UI for planned extended downtime. We don't have UI for unplanned extended downtime, but we may not need it. [19:31] SteveA, if you could post the results of that IS discussion please as mpt suggested [19:31] sure [19:31] thanks SteveA [19:31] thanks SteveA [19:31] mpt, ok to move on? [19:31] yep [19:31] [TOPIC] kiko on vacation (Kiko) [19:31] New Topic: kiko on vacation (Kiko) [19:32] ah, right. [19:32] someone call guiness [19:32] I'm on vacation tomorrow through tuesday [19:32] you can call me and I might have phone coverage [19:32] but look to steve and joey for RCs [19:32] 2 days! [19:32] Rinchen, to order beer, or break a record? [19:32] I've looked at the PQM queue [19:32] mthaddon, in this case, both :-) [19:32] and there doesn't seem to be much at risk there [19:32] does anyone have a high-risk, important or otherwise urgent landing that can't miss this rollout? [19:32] I'd like to remind folks to clear out the pending-reviews queue if you haven't already [19:33] * mthaddon calls guinness [19:33] besides flacoste [19:33] kiko: yes, I am waiting for flacoste to land his restricted librarian stuff so I can land my P3A stuff [19:33] and his libby patch for julian [19:33] so I hear [19:33] P<3A [19:34] silence. that's a good thing I think [19:34] okay [19:34] thanks kiko [19:34] you didn't speak [19:34] so I won't make any exception for your branch! [19:34] :) [19:35] er right. mrevell where's what's new? :-) [19:35] * Rinchen laughs. [19:35] Rinchen: Need to speak to you about that. [19:35] sure, let's do it right after this if you can [19:35] thanks all [19:35] Thank you all for attending this week's Launchpad Developer Meeting. See the channel topic for the location of the logs. [19:35] yep, np [19:35] #endmeeting [19:35] Meeting finished at 20:35. [19:35] Rinchen: thanks! [19:35] thanks Rinchen ! [19:35] this one was fast! [19:35] thanks thanks [19:35] thanks Rinchen [19:35] We're early! [19:35] we're setting records here bub [19:35] thanks ! [19:36] more meat, less fat [19:36] call Guinness again === salgado is now known as salgado-afk