=== Ursinha is now known as Ursinha-afk === bac` is now known as bac [17:00] me? [17:01] moo [17:01] yoo [17:01] hahaha [17:01] oinc [17:01] farmiville meets launchpad [17:02] * rockstar [17:02] me [17:02] me [17:02] argh [17:02] #startmeeting [17:02] Meeting started at 11:02. The chair is Ursinha. [17:02] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [17:02] me [17:02] sigh [17:02] me [17:02] [TOPIC] Roll Call [17:02] Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! [17:02] New Topic: Roll Call [17:02] me [17:02] meeeeee [17:02] me [17:02] me [17:02] sinzui is here several times [17:03] bigjools, [17:03] hi [17:03] * bigjools excuses himself from the meeting [17:03] UDS... [17:03] I guess we don't have a bugs' person here [17:03] bigjools, I have one question for you in the oops section, should be easy for you :) [17:04] ok [17:04] [TOPIC] Agenda [17:04] * Actions from last meeting [17:04] * Oops report & Critical Bugs & Broken scripts [17:04] * Operations report (mthaddon/Chex/spm/mbarnett) [17:04] * DBA report (stub) [17:04] New Topic: Agenda [17:04] * QA stats of the week [17:04] * Proposed items [17:04] [TOPIC] * Actions from last meeting [17:04] New Topic: * Actions from last meeting [17:04] * Ursinha to triage lp-qa-tools bugs [17:04] * done. [17:04] * matsubara to talk to foundations about OOPS-1585G1243 [17:04] * Talked to Gary about it and filed https://launchpad.net/bugs/579987 [17:04] * adeuring to follow up on calculate_bug_heat script failure [17:04] * matsubara to chase landing for bug 574493 [17:04] * Landed and CP'ed [17:04] * matsubara to chase landing for bug 575426 [17:04] * Landed and CP'ed [17:04] * matsubara to ask for the DBA report [17:04] * emailed stub asking for it [17:04] * all QA contacts to take orphaned commits' lists to their teams to check if were left untested items behind because they were orphaned commits [17:04] bot is sleeping [17:04] my stuff is ok, so is matsubara's [17:05] adeuring is not here to follow up on his item [17:05] [action] Ursinha to chase adeuring abour calculate_bug_heat failing script [17:05] ACTION received: Ursinha to chase adeuring abour calculate_bug_heat failing script [17:05] about the orphaned commits, we can discuss that in the QA section [17:06] moving on [17:06] [TOPIC] * Oops report & Critical Bugs & Broken scripts [17:06] New Topic: * Oops report & Critical Bugs & Broken scripts [17:06] One bug and one oops for code team, and one oops to soyuz [17:06] the code bug is bug 579004, triaged high and targetted to 10.05, seems to affect only staging [17:06] fine [17:06] the oops: rockstar, any ideas? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1593S580 [17:06] https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1593EC1021, NotOneError seems to be bug 523346, already fix released, but I still see that happening [17:06] bigjools, is that the same thing? ^ [17:07] Ursinha: gah [17:07] Ursinha, I'll file a bug. [17:07] it's related [17:08] thanks rockstar [17:08] bigjools, may I file another bug? [17:08] Ursinha: yes please, and mention that other one [17:08] bigjools, sure [17:08] thanks [17:08] [action] Ursinha to file a bug for OOPS-1593EC1021, related to bug 523346 [17:08] ACTION received: Ursinha to file a bug for OOPS-1593EC1021, related to bug 523346 [17:09] thanks rockstar and bigjools, I love you both [17:09] moving on [17:09] calculate_bug_heat and upgrade_branches were the last scripts to fail [17:09] rockstar, do you know about the upgrade_branches? is that related to code as it seems? [17:09] heh :) [17:09] Ursinha, yes. That script is my baby. [17:09] rockstar, oh, it seems it's been crying on his last runs [17:10] or its :) [17:10] Ursinha, I'll look into it. [17:10] thanks rockstar [17:10] much appreciated [17:10] Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/523346) [17:10] https://lp-oops.canonical.com/oops.py/?oopsid=1593EC1021 [17:10] Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/523346) [17:10] https://lp-oops.canonical.com/oops.py/?oopsid=1593EC1021 [17:10] Ursinha, the conjugated form was correct, since it's a conjugation of "it" and "has" [17:10] [action] rockstar to look into upgrade_branches failures [17:10] ACTION received: rockstar to look into upgrade_branches failures [17:10] rockstar, I was referring to s/his/its/ ;) [17:10] s/;)/:) [17:11] thanks :) [17:11] we have three critical bugs, two already fix committed: bug 574142 and bug 574493 [17:11] Launchpad bug 574142 in launchpad-registry "Could not adapt ProductSeries to IBranchTarget" [Critical,Fix committed] https://launchpad.net/bugs/574142 [17:11] Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/574493) [17:11] gary_poster, sinzui, are those two going to be or were CPed? [17:11] there's one in progress, code, bug 578331, last comment was danilos' [17:11] how is that? [17:11] Launchpad bug 578331 in launchpad-code "exporting to bzr seems broken since a few days" [Critical,In progress] https://launchpad.net/bugs/578331 [17:11] i think it's fix committed now? [17:11] Ursinha, it's on production-stable, needs CPing [17:12] oh, so mwhudson is here [17:12] oops [17:12] camper :) [17:12] my cover is busted [17:12] https://lpbuildbot.canonical.com/builders/prod_lp/builds/62 [17:12] Ursinha, it is fix released, I updated it [17:12] now [17:12] Ursinha, I'm remebering only one critical and it has been CP'd--the log-out one [17:12] thanks sinzui [17:12] gary_poster, that one seems private, let me see [17:13] Ursinha, 574493 was CP'ed [17:13] bug 574493, right? [17:13] Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/574493) [17:13] that's CP'd [17:13] gary_poster, yes [17:13] cool [17:13] matsubara, gary_poster, so, it's fix released :) could you mark it as such, please? [17:13] or I can do that :) [17:13] will do [17:13] thank you [17:14] danilos, are you taking care of that CP? [17:15] I didn't mark as fix released before because of gary's comment saying the fix was CP'd but not landed on devel. I was waiting for it to reach devel a go through the usual process [17:15] s/a/and/ [17:15] Ursinha, will do [17:15] matsubara, ack. It happened early this morning [17:15] matsubara, that's ok, I was just wondering if they could be closed now [17:15] danilos, thanks [17:15] [action] danilos to take care of CPing bug 578331 fix [17:15] ACTION received: danilos to take care of CPing bug 578331 fix [17:15] Launchpad bug 578331 in launchpad-code "exporting to bzr seems broken since a few days" [Critical,Fix committed] https://launchpad.net/bugs/578331 [17:16] good [17:16] [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) [17:16] New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) [17:16] Chex, stage is all yours [17:16] hello everyone, here is this weeks LOSA report: [17:16] - Applied LP security fix for Bug# 578914 [17:16] : Rolled out in security mode, applying CP's to codebrowse & codehost, and cowboy edge. [17:16] - Bug 531071: app servers leaking memory [17:16] : We have a few dump-logs now ( LPIncidentLog ) , has anyone looked at or could look at these to debug the issues here? [17:16] Launchpad bug 531071 in launchpad-foundations "lpnet app servers are leaking memory again" [High,Triaged] https://launchpad.net/bugs/531071 [17:16] - LP incidents of note: : Codebrowse fixed from hourly restarts after last week rollout, but still needs restarting, daily, or more. : ftp/poppy service crashed a few times this week, Bug 414482 [17:17] Launchpad bug 414482 in soyuz "upload daemon poppy dies often" [Medium,Triaged] https://launchpad.net/bugs/414482 [17:17] : Apr-11: CP 9339 App Servers & codehost/crowberry [17:17] : Apr-12: CP 9341 to germanium/cocoplum [17:17] : Apr-12: CP 9342 to lpnet/cocoplum & loganberry-bzrsyncd [17:18] questions/comments on the report? [17:18] I guess that's May, or an abbreviation for something like "Anonymous Public Report" :) [17:18] wow, yes sorry [17:18] .. s/Apr/May/ my mistake. [17:18] Chex, should we try to get bigjools to up the priority on poppy bug? [17:19] Chex, if that's bigjools territory at all? [17:19] it is [17:19] but it's a bug in zope [17:19] so we're working around that by writing a new sftp server :) [17:20] what a workaround :) [17:20] woohoo, simple workarounds :) [17:20] probably late for this, but I'm always happy to try and apply pressure withing Zope community if necessary [17:21] good to know, thanks [17:21] ok, anyone else? [17:22] bug 531071 maybe? [17:22] Launchpad bug 531071 in launchpad-foundations "lpnet app servers are leaking memory again" [High,Triaged] https://launchpad.net/bugs/531071 [17:22] gary_poster, ^ [17:22] this is already targeted to 10.04 [17:22] we now have some meliae outputs as of the past week. I'll be investigating them soon. [17:22] Ursinha: gary_poster: we have crash dumps of what was requested now on the app servers, if we can work with someone who wants to look at this [17:23] s/already/still/ [17:23] gary_poster: ah fair enough, thank you [17:23] Chex: (I assume they are the same ones you are talking about--spm sent us some emails) [17:23] gary_poster, may I target that to this cycle? [17:24] gary_poster: yes I think so, what I saw in the incident logs was in spm [17:24] 's timeframe, so. [17:24] Chex: cool [17:24] Ursinha: too late ;-) but thanks [17:24] haha, thanks gary_poster :) [17:24] moving on then :) [17:24] [TOPIC] * DBA report (stub) [17:24] [action] send email to stub about the dba report of the week [17:24] New Topic: * DBA report (stub) [17:24] ACTION received: send email to stub about the dba report of the week [17:24] [TOPIC] * QA stats of the week [17:24] New Topic: * QA stats of the week [17:24] Untested items per team: [17:24] Code: 3 [17:24] Foundations: 2 [17:24] Registry: 0 [17:24] Bugs: 3 [17:24] Translations: 2 [17:24] Soyuz: 3 [17:24] Strategy: 0 [17:25] Untested == QA? [17:25] duh [17:25] yes [17:25] :) [17:25] one thing I notice is that since we started using tags, we've been having less untested items at once [17:25] which is cool [17:25] :) [17:26] Untriaged bugs per project: [17:26] agreed [17:26] malone: 17 [17:26] launchpad-buildd: 12 [17:26] lp-dev-utils: 9 [17:26] launchpadlib: 9 [17:26] launchpad-dev-moin-theme: 6 [17:26] launchpad-code: 5 [17:26] launchpad-foundations: 3 [17:26] soyuz: 3 [17:26] launchpad-help-moin-theme: 3 [17:26] lpbuildbot: 2 [17:26] trac-launchpad-migrator: 2 [17:26] launchpad-news-wordpress-theme: 2 [17:26] launchpad-cscvs: 2 [17:26] launchpad: 1 [17:26] launchpad-web: 1 [17:26] launchpad-loggerhead: 1 [17:26] launchpad-documentation: 1 [17:26] tickcount: 1 [17:27] we're not that bad triaging bugs as well [17:27] about the orphaned commits [17:28] last week I asked the QA contacts to check with their teams all OCs that are yours, to see if we're leaving untested things behind [17:28] sinzui answered me, what about the others? [17:28] I suck. I forgot to bring that up with my team [17:28] https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.04-devel and https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.04-db-devel [17:29] rockstar, and you? [17:29] We talked about it. I don't know if anyone did anything about it. [17:29] I will bring up the branch-link-to-bug issue in the next reviewers meeting. Many contributors do not do this [17:30] Ursinha, I totally don't remember the discussion, which means that I suck [17:30] :) [17:30] We could make ec2 land require a linked bug [17:30] c'mon people :) [17:30] sinzui, guess the problem is that not all commits really require bugs [17:30] Oh no, I think we did talk about it, and determined that there were no orphaned commits. [17:30] What about refactorings? [17:30] and not everyone uses ec2 [17:30] right [17:30] ah, hello wgrant :) [17:30] However, we're also not linking bugs to branches in many cases. [17:31] Ursinha: Yeah, I'm not asleep for once :P [17:31] pqm submit certain is not [17:31] wgrant, :P [17:31] Ursinha, we haven't really been tracking QA in the bugtracker - We've been using our Kanban board. [17:31] if you use --fixes=lp:NNNN when you commit, then when you push the branch to LP, it'll be linked automatically [17:31] rockstar, I see. this is not good, but we fall again in the kanban board/lp integration [17:31] problem [17:32] and the tagging script will find out which bug was fixed by that commit, even if it's not mentioned in the commit message [17:32] Ursinha, well, we have two tools essentially doing the same thing. [17:32] rockstar, also what matsubara said [17:32] I still use the milestone because I am tracking every registry related blueprint and bug that I may be required to land a RC for [17:32] Ursinha, fwiw, I've looked through the orphan commits now and there is only one by a translations dev and that one was qa-ok :) [17:33] I don't know what everyone else agrees, but I think we should maintain lp updated so when we integrate that with kanban we'll all be used to the lp dance [17:33] thanks danilos [17:33] Every contributor or even developer from another team gets their in progress bugs targets to a milestone and I verify the number of fix committed is the same number as qa-ok [17:33] Ursinha, asking us to keep two duplicate tools in sync manually seems crazy. [17:34] Foundations often has multiple commits as we pursue a larger bug [17:34] rockstar, but is that hard to use --fixes or link a bug to a branch? [17:34] danilos, the registry orphaned commits were for contributor branches that we landed and for testfix [17:34] Ursinha, no, but I don't want to create a kanban task AND a bug for something I'm about to do. [17:34] sinzui, I count adiroiban as a translations developer as well :) [17:34] And then move them both through their own separate processes. [17:34] We also have kanban cards for our tasks [17:34] I can't use --fixes if there isn't a bug... [17:35] rockstar, I thought that the whole team had agreed on using lp as qa tracker [17:35] He applied to be an lp engineer. He should be counted on someone's team [17:35] did I get this wrong? [17:35] Ursinha, maybe, but the code team isn't doing it that way, and we've found that the kanban board has been the tool we've preferred. [17:35] I guess that needs to be discussed then [17:36] Ursinha, we get our stuff QA'd, so it's not like things are falling through the cracks, but it doesn't have great visibility. [17:36] if you're preferring another tool, we should discuss the problems [17:36] Ursinha, the problem is that we have two tools... [17:36] Ursinha: do I understand correctly that you are saying that orphan commits are bad? [17:36] rockstar, imo the problem is that we don't have kanban integration [17:36] Ursinha, rockstar: it's a recognized problem, and flacoste promises us we'll do something about it (two tools for the same job) [17:36] because of the QA-through-LP initiative? [17:36] gary_poster, only if they are really orphaned [17:36] Ursinha, let's move on [17:37] at least that's my take on it [17:37] gary_poster, not necessarily bad, we just don't want those falling through the cracks [17:37] gary_poster, in the sense that it should have been qaed and it isn't [17:37] what matsubara said [17:37] ...and if we use LP for qa then how do we know? [17:37] right [17:37] if you use LP as the QA tracker, then we have tools to help remind you that some QA is needed [17:38] I think this is a matter for discussion, but maybe not here. mailing list? [17:38] doesn't matter if it doesn't have a bug when it doesn't need to have, the problem is not having a bug when it should [17:38] agreed gary_poster [17:38] cool [17:38] if you start landing OCs, then the tools can't help remind you that QA needs to be done [17:38] so, in this meantime, I'd like to ask you to once in a while take a look in the OC list [17:39] and see if any important things are missing [17:39] https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.05-devel and https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.05-db-devel this cycle [17:39] anyone else want to say something? [17:39] moving on :) [17:39] [TOPIC] * Proposed items [17:39] Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. [17:39] New Topic: * Proposed items [17:39] #endmeeting [17:39] Meeting finished at 11:39. [17:40] errrrr bad paste :) [17:40] but we didn't have any other proposed items, so, nevermind :) [17:40] :-) [17:40] thanks Ursinha [17:40] thank you [17:40] thanks everyone for this week's meeting :) [17:41] cheers === gary_poster is now known as gary-lunch === gary-lunch is now known as gary_poster === Edwin-lunch is now known as EdwinGrubbs === matsubara is now known as matsubara-afk === Ursinha is now known as Ursinha-afk