[15:59]  * Ursinha pokes MootBot 
[16:00] <matsubara> #startmeeting
[16:00] <MootBot> Meeting started at 10:00. The chair is matsubara.
[16:00] <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
[16:00] <matsubara> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues.
[16:00] <matsubara> [TOPIC] Roll Call
[16:00] <MootBot> New Topic:  Roll Call
[16:00] <gary_poster> me
[16:00] <mrjazzcat> me
[16:00] <Ursinha> me me
[16:00] <matsubara> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us!
[16:01] <matsubara> sinzui, Chex, rockstar, bigjools, allenap, danilos: hi
[16:01] <bigjools> \o
[16:01] <allenap> me
[16:02] <rockstar> hi
[16:02] <sinzui> me
[16:02] <Chex> hello
[16:03] <danilos> me
[16:03] <matsubara> ok, everyone is here.
[16:03] <matsubara> [TOPIC] Agenda
[16:03] <MootBot> New Topic:  Agenda
[16:03] <matsubara>  * Actions from last meeting
[16:03] <matsubara>  * Oops report & Critical Bugs & Broken scripts
[16:03] <matsubara>  * Operations report (mthaddon/Chex/spm/mbarnett)
[16:03] <matsubara>  * DBA report (stub)
[16:03] <matsubara>  * Proposed items
[16:03] <matsubara> [TOPIC] * Actions from last meeting
[16:03] <MootBot> New Topic:  * Actions from last meeting
[16:03] <matsubara>  * matsubara to discuss with foundations bug 403281 and set importance on it
[16:03] <matsubara>         * discussed with Gary and it's now triaged and an importance set.
[16:03] <matsubara>     * rockstar to triage bug 531687
[16:03] <matsubara>     * matsubara to discuss bug 531965 with foundations as well
[16:03] <matsubara>         * discussed with Gary and it's now triaged and an importance set.
[16:03] <matsubara>     * rockstar to investigate failures on mpcreationjobs and sendbranchmail scripts
[16:03] <matsubara>         * <rockstar> matsubara, abentley is already figuring out what's going on with those scripts.
[16:03] <matsubara>     * rockstar to investigate Out-Of-Memory error: https://pastebin.canonical.com/28737/ and get back to losas about it
[16:03] <matsubara>     * matsubara and/or Ursinha to add a count of untriaged bugs per team to the oops section
[16:03] <matsubara> I didn't add the count of untriaged bugs per team to the oops section yet
[16:04] <matsubara> [action] * matsubara and/or Ursinha to add a count of untriaged bugs per team to the oops section
[16:04] <MootBot> ACTION received:  * matsubara and/or Ursinha to add a count of untriaged bugs per team to the oops section
[16:04] <Ursinha> yes sir
[16:04] <matsubara> rockstar, any news about the OOM error? did you get back to losas about it?
[16:05] <matsubara> I see that 531687 was properly triaged and an importance set. thanks for that rockstar
[16:05] <rockstar> matsubara, others on my team are looking into it.  We think it's probably just another scalability issue.
[16:05] <matsubara> cool. thanks rockstar
[16:06] <matsubara> [TOPIC] * Oops report & Critical Bugs & Broken scripts
[16:06] <MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
[16:06] <matsubara> I have 3 bugs for today, 2 of them were already dealt with
[16:06] <matsubara> bug 535071 still need some foundations love
[16:06] <matsubara> gary_poster, can you take a look ^?
[16:07] <matsubara> fwiw, the other two are bugs 535078, 535672
[16:07] <gary_poster> looking (how did I miss this? :-/ )
[16:08] <matsubara> thanks gary_poster
[16:08] <matsubara> we have 4 critical bugs, 3 fix committed and 1 in progress.
[16:09] <bigjools>  /o\
[16:09] <matsubara> for the failing scripts
[16:09] <matsubara> we have two failing frequently: productreleasefinder and oops-pruner
[16:09] <gary_poster> heh
[16:10] <Ursinha> ubottu: there, there
[16:10] <matsubara> I replied to one of the oops pruner failure email but didn't get a reply yet.
[16:10] <matsubara> Chex, could you take a look in the oops pruner script logs and check why it's failing?
[16:10] <Chex> matsubara: yes sure, I can look at that
[16:10] <matsubara> Chex, please reply to one of the failures emails with the traceback or file a bug
[16:11] <matsubara> [action] Chex to verify why oops-pruner is failing and reply to the failure email with more information.
[16:11] <MootBot> ACTION received:  Chex to verify why oops-pruner is failing and reply to the failure email with more information.
[16:11] <matsubara> sinzui, how about the prf script?
[16:11] <matsubara> been failing on a daily basis for the past week
[16:11] <sinzui> It is always running 2 hours late
[16:11] <sinzui> It always runs, just late
[16:12] <sinzui> I think we need to consider changing the threshold, or changing the order
[16:12] <matsubara> sinzui, so we should adjust the monitoring script to take that into account, shouldn't we?
[16:12] <matsubara> [action] sinzui to work with losas to set the increase threshold for the product release finder monitoring script.
[16:12] <MootBot> ACTION received:  sinzui to work with losas to set the increase threshold for the product release finder monitoring script.
[16:13] <matsubara> I think that's all for today's oops/scripts/critical section. thanks everyone
[16:13] <matsubara> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
[16:13] <MootBot> New Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)
[16:14] <Chex> hello everyone, here is this weeks report:
[16:14] <Chex> - We are in the process of working on migrating to using SSO run by ISD branded for LP (open-ID single signon) in
[16:14] <Chex>         Launchpad.
[16:14] <Chex> - The LP DBs on one of the slave DB servers may need to be rebuilt/repacked as we're getting short on
[16:14] <Chex>         space on that server.
[16:14] <Chex> - LP incidents of note:
[16:14] <Chex>         ; LP Cherry-picks: CP to crowberry/codehost; revno 9082
[16:14] <Chex>         ; There was a buildbot error that stopped merging from db-devel to db-stable for a few days, which has
[16:14] <Chex>                 been fixed.
[16:15] <Chex> and thats all for us.  Any questions/comments??
[16:16] <matsubara> Chex, I take you're working with stub to have the db rebuilding done?
[16:17] <Chex> oh, and the buildbot merge error seems to be happening again, so this still needs to be investigated.
[16:17] <Chex> matsubara: yes I believe we will work with stub on the db rebuild.
[16:17] <matsubara> rockstar, can you take that? that's a build engineer thing, right?
[16:19] <rockstar> matsubara, I'm not build engineer anymore.
[16:20] <matsubara> rockstar, oh, who's the build engineer for this cycle?
[16:20] <rockstar> matsubara, I don't think there is one.
[16:20] <matsubara> I guess it'll fallback to foundations then. gary_poster can you work with losas re: the buildbot issue above?
[16:21] <gary_poster> matsubara: yes
[16:21] <matsubara> cool. thanks.
[16:21] <matsubara> [TOPIC] * DBA report (stub)
[16:21] <MootBot> New Topic:  * DBA report (stub)
[16:21] <matsubara> sorry, forgot to thank Chex for his report. thank you Chex!
[16:22] <matsubara> dba report was sent to the list. any question please follow up there
[16:22] <matsubara> [TOPIC] * Proposed items
[16:22] <MootBot> New Topic:  * Proposed items
[16:22] <matsubara> no new proposed items
[16:22] <matsubara> and I guess that's all for today folks. anything else before I close?
[16:23] <Chex> matsubara: no problem!
[16:23] <matsubara> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
[16:23] <matsubara> #endmeeting
[16:23] <MootBot> Meeting finished at 10:23.
[16:24] <Ursinha> matsubara: thanks japanese
[16:26] <bigjools> Ursinha: *chinese*
[16:26] <bigjools> :)
[16:26] <Ursinha> bigjools: lol
[16:27] <matsubara> last I hear, I was vietnamese. Must be the tan :-)
[16:27] <matsubara> heard
[16:27] <Ursinha> vietnamese
[16:27] <Ursinha> the tan
[16:27] <Ursinha> rofl