=== mrevell is now known as mrevell-dejeuner
=== bigjools-afk is now known as bigjools_
=== mrevell-dejeuner is now known as mrevell
=== danilo_ is now known as danilos
=== danilos is now known as danilo-afk2
[16:00] <matsubara> #startmeeting
[16:00] <MootBot> Meeting started at 10:00. The chair is matsubara.
[16:00] <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
[16:00] <matsubara> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues.
[16:00] <matsubara> [TOPIC] Roll Call
[16:00] <MootBot> New Topic:  Roll Call
[16:00] <rockstar> ni!
[16:00] <allenap> me
[16:00] <mbarnett> whee!
[16:00] <Chex> hello
[16:00] <matsubara> Apologies from Ursinha and Stuart
[16:01] <matsubara> gary_poster, bigjools_, danilo-afk2: ho
[16:01] <matsubara> s/ho/hi/
[16:01] <matsubara> sinzui, hi
[16:01] <gary_poster> mo, me
[16:01] <sinzui> me
[16:02] <matsubara> ok, soyuz and translations are awol. let's move on and they can join later
[16:02] <matsubara> [TOPIC] Agenda
[16:02] <MootBot> New Topic:  Agenda
[16:02] <matsubara>  * Actions from last meeting
[16:02] <matsubara>  * Oops report & Critical Bugs & Broken scripts
[16:02] <matsubara>  * Operations report (mthaddon/Chex/spm/mbarnett)
[16:02] <matsubara>  * DBA report (stub)
[16:02] <matsubara>  * Proposed items
[16:02] <matsubara> [TOPIC] * Actions from last meeting
[16:02] <MootBot> New Topic:  * Actions from last meeting
[16:02] <matsubara>     * barry to continue debug on bug 403606 after finishing 3.0 UI stuff
[16:02] <matsubara>     * intellectronica to take a look or find someone to take a look on bug 408738 after finishing 3.0 UI stuff
[16:02] <matsubara>     * chex to trawl logs and add request that caused 500 error to bug 422960
[16:02] <matsubara>         * spm added more info to the bug report
[16:02] <matsubara>     * ursinha to file a bug for OOPS-1345G2533 and coordinate a fix with gary/stub
[16:02] <matsubara>         * Filed bug 427397
[16:02] <ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
[16:03] <matsubara>     * matsubara to trawl logs related to high load on edge yesterday and ping Chex about it
[16:03] <ubottu> Launchpad bug 408738 in malone "OOPS when rendering bug activity" [High,Triaged] https://launchpad.net/bugs/408738
[16:03] <ubottu> Launchpad bug 422960 in launchpad-foundations "appear to be failing to record oops for all +translate HTTP 503 errors" [Undecided,New] https://launchpad.net/bugs/422960
[16:03] <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1345G2533
[16:03] <ubottu> Launchpad bug 427397 in launchpad-foundations "Search triggering error in tsearch query again" [Low,Triaged] https://launchpad.net/bugs/427397
[16:03] <sinzui> matsubara: I looked into Bug  403606. Still no progress. This needs barry and a losa on staging I think
[16:03] <matsubara> [action]  * matsubara to trawl logs related to high load on edge yesterday and ping Chex about it
[16:03] <MootBot> ACTION received:   * matsubara to trawl logs related to high load on edge yesterday and ping Chex about it
[16:04] <matsubara> i still have to do that
[16:04] <matsubara> sinzui, shall I re-add the action item? I'm assuming barry is still busy with 3.0 stuff?
=== bigjools_ is now known as bigjools
[16:04] <bigjools> me
[16:05] <matsubara> hi bigjools, welcome
[16:05] <bigjools> sorry I'm late
[16:05] <sinzui> yes. I expect him to have time next week
[16:05] <matsubara> cool. thanks
[16:05] <matsubara> [action]  * barry to continue debug on bug 403606 after finishing 3.0 UI stuff
[16:05] <MootBot> ACTION received:   * barry to continue debug on bug 403606 after finishing 3.0 UI stuff
[16:05] <ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
[16:05] <allenap> matsubara: None of the Bugs team has time right now to address bug 408738 sadly; we're all maxed out. I expect we won't get to this until after 3.0.
[16:05] <ubottu> Launchpad bug 408738 in malone "OOPS when rendering bug activity" [High,Triaged] https://launchpad.net/bugs/408738
[16:06] <matsubara> allenap, ok. I'll retarget to 3.1.10 then. do you want to keep the action item as reminder or should I remove it/
[16:07] <allenap> matsubara: Remove it. If it's in the milestone it's reminder enough :)
[16:07] <allenap> Thanks.
[16:07] <matsubara> cool. thanks allenap
[16:07] <matsubara> let's move on
[16:07] <matsubara> [TOPIC] * Oops report & Critical Bugs & Broken scripts
[16:07] <MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
[16:08] <matsubara> sinzui, can you take a look at https://bugs.edge.launchpad.net/launchpad-registry/+bug/429802 ?
[16:08] <ubottu> Launchpad bug 429802 in launchpad-registry "Merged teams showing up in the web ui" [Undecided,Triaged]
[16:09] <matsubara> sinzui, and assign/schedule it please
[16:09] <matsubara> there was a bunch of DisconnectionErrors last week
[16:10] <matsubara> which might be related to the outage we had
[16:10] <sinzui> matsubara: We know what we are going to do with that. It the unassignment scheduler for deactivated and suspected accounts
[16:11] <matsubara> sinzui, what's that?
[16:12] <sinzui> There are several manifestations on this problem. the solution is a nightly cronjob that cleans up the teriary data left behind after the account was changed.
[16:13] <matsubara> I see.
[16:14] <matsubara> sinzui, looks like you have everything sorted :-)
[16:14] <sinzui> matsubara: This will fix every where you see a deactivated person/team in a list
[16:14] <matsubara> cool. so probably something for 3.1.10?
[16:15] <sinzui> matsubara: yes, but then we started the release and redesign so I did no do it.
[16:15] <sinzui> matsubara: I will create a spec today for it. I think 3.1.10 or 3.1.11
[16:15] <matsubara> sinzui, thank you
[16:16] <matsubara> On the critical bugs front, we have only one, which is fix committed
[16:17] <matsubara> and on the scripts failures front
[16:17] <matsubara> we have a bunch which failed this week :-(
[16:17] <matsubara> the rosetta-poimport issue, which danilo is taking care of
[16:18] <matsubara> checkwatches also hung and spm/gmb debugged and replied. it's all good now
[16:18] <matsubara> process-pending-packagediffs which julian and cprov took care of with help from steve
[16:19] <matsubara> and we now have a rosetta-approve-imports which is failing
[16:19] <matsubara> probably related to the poimport failure
[16:19] <matsubara> since danilo is not here, I'll ask him later
[16:19] <danilo-afk2> matsubara, poimport has been fix released, approve imports unrelated to rosetta-approve-imports, and the latter should be fixed as well by now
[16:19] <matsubara> [action] matsubara to ask danilo about rosetta-approve-imports failures
[16:19] <MootBot> ACTION received:  matsubara to ask danilo about rosetta-approve-imports failures
[16:19] <bigjools> fyi process-pending-packagediffs are caused by a bug in "diff" iteself!
[16:20] <matsubara> hi danilo-afk2
[16:20] <danilo-afk2> hi, henninge was supposed to be here for the meeting, I am supposed to be afk
[16:20] <matsubara> thanks for the update. it'd be awesome if you could reply to the Scripts failed to run: loganberry:rosetta-approve-imports email sent to the list					
[16:20] <matsubara> danilo-afk2, I pinged both
[16:21] <danilo-afk2> matsubara, I got sms from him that he forgot about it and went out, sorry for the lack of participation from translations, /me goes afk again
[16:21] <matsubara> danilo-afk2, hmm ok. thanks
[16:22] <matsubara> I think that's all for this section
[16:22] <matsubara> thanks everyone
[16:22] <matsubara> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
[16:22] <MootBot> New Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)
[16:22] <Chex> hi everyone, here is our report for this week:
[16:23] <Chex> - staging buildbot is now setup so we can test new  versions of buildbot in a sandbox
[16:23] <Chex> - there have been ongoing discussions & plans to make codebrowse more reliable this week
[16:23] <Chex> - poimport query caused Production DB disk issue on 11-Sep.  Bug: #408718 was applied to production on
[16:23] <Chex> 15-Sep to fix the issue, poimport process was re-enabled after the fix applied.
[16:24] <Chex> anyone have any questions?
[16:25]  * mbarnett chirps
[16:26]  * gary_poster notices the sound of the wind in the trees...
[16:26] <gary_poster> or maybe that's the air conditioning...
[16:27] <mthaddon> as always, losa concerns knock everyone over with their excitement levels
[16:27] <mbarnett> nice work chex, you broke the meeting!
[16:27] <bigjools> or put them to sleep
[16:27] <Chex> sorry all... :/
[16:28] <gary_poster> yay losas for staging buildbot!
[16:28] <gary_poster> matsubara: :-D ?
[16:28] <matsubara> guess not. thanks Chex
[16:28] <matsubara> [TOPIC] * DBA report (stub)
[16:28] <matsubara> nothing to report about the DB
[16:28] <MootBot> New Topic:  * DBA report (stub)
[16:28] <matsubara> [TOPIC] * Proposed items
[16:28] <MootBot> New Topic:  * Proposed items
[16:28] <matsubara> no proposed items
[16:28] <matsubara> anything else before I close?
[16:28] <matsubara> 5
[16:28] <matsubara> 4
[16:28] <matsubara> 3
[16:28] <matsubara> 2
[16:28] <mthaddon> 1.5
[16:28] <matsubara> 1
[16:28] <matsubara> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
[16:28] <matsubara> #endmeeting
[16:28] <MootBot> Meeting finished at 10:28.
[16:29] <gary_poster> thanks matsubara
[16:29] <mbarnett> victory is ours!
=== salgado is now known as salgado-lunch
=== BjornT_ is now known as BjornT
=== salgado-lunch is now known as salgado
=== EdwinGrubbs is now known as Edwin-lunch
=== Edwin-lunch is now known as EdwinGrubbs
=== EdwinGrubbs is now known as Edwin-lunch
=== Edwin-lunch is now known as Edwin-afk
=== Edwin-afk is now known as EdwinGrubbs
=== cprov is now known as cprov-afk
=== salgado is now known as salgado-afk