[15:00] #startmeeting [15:00] Meeting started at 09:00. The chair is matsubara. [15:00] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [15:00] Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. [15:00] [TOPIC] Roll Call [15:00] New Topic: Roll Call [15:00] Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! [15:00] moi [15:00] apologies from stub and Ursinha [15:00] me [15:00] and gary too [15:01] rockstar, Chex, sinzui: hi [15:01] danilos, h [15:01] danilos, hi [15:01] matsubara: It is a national holiday for the US members [15:01] me [15:01] oh, my clock is delayed still [15:02] sinzui, right, I guess rockstar is not joining then. aren't you taking the holiday? [15:02] matsubara: ah, that's why bigjools isn't around... I forgot. [15:02] He took a day holiday as everyone else in the states was on holidays. [15:02] matsubara: I am but I happen to be here and I can answer questions [15:02] noodles775, I see. thanks for covering for him :-) [15:03] today's meeting should be quick [15:03] let's move on [15:03] [TOPIC] Agenda [15:03] New Topic: Agenda [15:03] * Actions from last meeting [15:03] * Oops report & Critical Bugs & Broken scripts [15:03] * Operations report (mthaddon/Chex/spm/mbarnett) [15:03] * DBA report (stub) [15:03] * Proposed items [15:03] [TOPIC] * Actions from last meeting [15:03] New Topic: * Actions from last meeting [15:03] * Ursinha to send one email to lp list explaining the qa-tags experiment [15:03] * matsubara to chase someone from code team about bug 480000 [15:03] * matsubara to chase code people about code script failures (create-merge-proposals, branch puller and update branches) [15:03] * matsubara to ask someone from code about bug 485318 [15:03] * Chex to talk to thumper about the multiple git import failures on the importd [15:03] Launchpad bug 480000 in launchpad-code "OOPS deleting a branch" [Undecided,New] https://launchpad.net/bugs/480000 [15:03] Launchpad bug 485318 in launchpad-foundations "POSTToNonCanonicalURL error using bazaar client" [Undecided,New] https://launchpad.net/bugs/485318 [15:04] I didn't talk to code people about all the issues. [15:04] [action] * Ursinha to send one email to lp list explaining the qa-tags experiment [15:04] ACTION received: * Ursinha to send one email to lp list explaining the qa-tags experiment [15:04] [action] * matsubara to chase someone from code team about bug 480000 [15:04] ACTION received: * matsubara to chase someone from code team about bug 480000 [15:04] [action] * matsubara to chase code people about code script failures (create-merge-proposals, branch puller and update branches) [15:04] ACTION received: * matsubara to chase code people about code script failures (create-merge-proposals, branch puller and update branches) [15:04] [action] * matsubara to ask someone from code about bug 485318 [15:04] ACTION received: * matsubara to ask someone from code about bug 485318 [15:04] Launchpad bug 485318 in launchpad-foundations "POSTToNonCanonicalURL error using bazaar client" [Undecided,New] https://launchpad.net/bugs/485318 [15:05] Chex, did you talk to thumper about the git import failures? === salgado is now known as salgado-lunch [15:06] matsubara: hi, I pinged him in my morning today, but we are time-zone challenged, sorry I didnt get back to him sooner on that [15:07] [action] Chex to follow up with thumper about the multiple git import failures on the importd [15:07] ACTION received: Chex to follow up with thumper about the multiple git import failures on the importd [15:08] thanks Chex [15:08] [TOPIC] * Oops report & Critical Bugs & Broken scripts [15:08] New Topic: * Oops report & Critical Bugs & Broken scripts [15:08] one bug for sinzui: https://bugs.edge.launchpad.net/launchpad-registry/+bug/488762 [15:08] Launchpad bug 488762 in launchpad-registry "SnapShot OOPS editing team with 13000 members" [High,Triaged] [15:08] which you already triaged [15:09] :-) [15:09] of course [15:09] thanks sinzui [15:09] and another one for danilos: https://bugs.edge.launchpad.net/rosetta/+bug/488765 [15:09] Launchpad bug 488765 in rosetta "OOPS accessing translations page for a source package" [Undecided,New] [15:09] and another OOPS-1420ED1047 for danilos which I haven't filed a bug for yet [15:09] https://lp-oops.canonical.com/oops.py/?oopsid=1420ED1047 [15:09] matsubara, looking [15:09] danilos, can you take a look at those and triage accordingly please? [15:10] matsubara, sure, thanks [15:10] [action] matsubara to file a bug for OOPS-1420ED1047 [15:10] https://lp-oops.canonical.com/oops.py/?oopsid=1420ED1047 [15:10] ACTION received: matsubara to file a bug for OOPS-1420ED1047 [15:11] https://lp-oops.canonical.com/oops.py/?oopsid=1420ED1047 [15:11] danilos, I'll let you know once I have a bug for that one [15:11] thanks danilos and sinzui [15:11] we have 2 critical bugs, one is fix committed and another one is in progress [15:11] so all good [15:11] matsubara, I think I've seen that one already (https://lp-oops.canonical.com/oops.py/?oopsid=1420ED1047) [15:11] we had 2 script failures since last week [15:12] one in checkwatches which gmb already took care of [15:12] and another one in distributionmirror-prober [15:12] sinzui, does that one ^ belongs to registry? [15:13] matsubara, yes, that's bug 484368 [15:13] Launchpad bug 484368 in rosetta "LocationError: 'top_projects_and_packages_to_translate'" [High,Triaged] https://launchpad.net/bugs/484368 [15:13] danilos, thanks [15:14] matsubara: I will take the mirror prober [15:14] [action] sinzui to investigate failure on the mirror prober (The script 'distributionmirror-prober' didn't run on 'loganberry' between 2009-11-23 06:07:04 and 2009-11-23 12:07:04 (last seen 2009-11-23 04:54:10.444057)) [15:14] ACTION received: sinzui to investigate failure on the mirror prober (The script 'distributionmirror-prober' didn't run on 'loganberry' between 2009-11-23 06:07:04 and 2009-11-23 12:07:04 (last seen 2009-11-23 04:54:10.444057)) [15:14] thanks sinzui [15:15] I think that's all for this section [15:15] let's move on [15:15] [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) [15:15] New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) [15:16] hello everyone, quick report today [15:16] - Python 2.5 upgrade: Update on how testing is going on Python 2.5? [15:16] - next LP rollout: Scheduled for Wed 02 Dec, 2009 at 2200 UTC. [15:17] - LP incidents of note: [15:17] ; CP 8653 rolled out to loganberry and bzrsyncd [15:17] ; continued Codebrowse restarts [15:17] and thats our report for today. Any questions?? [15:19] Chex, I'm not sure about the python2.5 upgrade status. That's a question for Gary, but he's on holiday today. do you want me to chase that for you? [15:20] matsubara: yes that would be great if you could, please [15:20] [action] matsubar to ask gary about python2.5 update and get back to losas [15:20] ACTION received: matsubar to ask gary about python2.5 update and get back to losas [15:21] I can't type my own name :-( [15:21] thanks Chex! [15:21] matsufail [15:21] hehe [15:22] Chex, btw, there's some progress regarding the codebrowser restarts. Max seems to have identified some bugs and likely some code will land soon [15:23] Chex, mthaddon: there is an email thread going on in the list about it if you are interested in the details [15:23] let's move on [15:23] [TOPIC] * DBA report (stub) [15:23] New Topic: * DBA report (stub) [15:23] relaying for stub [15:23] I've got the first part of LP standalone DB working (build, configure and control a launchpad specific PostgreSQL). [15:23] In DB land we have some more graphs detailing database load for scripts - no surprises so far. [15:23] sanitized db generation is still underway. Last run was 5 days before I killed it and optimized the slow bit. [15:23] Staging restorations were slow due to a very inefficient restore procedure (we had to change due to production replication setup changes). This process has been optimized and a restoration currently underway. [15:24] matsubara: nothing stands out from the DB user CPU usage graphs? [15:24] matsubara: we're seeing a lot of load increase on wildcherry and would really like to get to the bottom of it [15:25] mthaddon, I haven't seen the graphs, I'm relaying the report stub sent to me. I'll pass your question on to him and ask stub to contact you. is that ok? [15:25] ah right [15:26] yeah, of course [15:26] [action] matsubara to ask stub to contact losas about load increase on wildcherry [15:26] ACTION received: matsubara to ask stub to contact losas about load increase on wildcherry [15:27] ok, last one [15:27] [TOPIC] * Proposed items [15:27] there are no proposed items [15:27] New Topic: * Proposed items [15:27] which brings this meeting to an end, unless someone have any questions? [15:29] Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. [15:29] #endmeeting [15:29] Meeting finished at 09:29. === matsubara is now known as matsubara-lunch === salgado-lunch is now known as salgado === matsubara-lunch is now known as matsubara === salgado is now known as salgado-afk === matsubara is now known as matsubara-afk === mwhudson_ is now known as mwhudson