=== Peng_ is now known as Peng__
=== Peng is now known as Peng_
=== Peng__ is now known as Peng
=== noodles775_ is now known as noodles775
=== _thumper_ is now known as thumper
=== matsubara-afk is now known as matsubara
=== matsubara is now known as matsubara-afk
=== mrevell is now known as mrevell-lunch
=== mrevell-lunch is now known as mrevell
UrsinhaMootBot, hi14:38
MootBotMeeting started at 10:01. The chair is Ursinha.15:01
MootBotCommands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]15:01
UrsinhaWelcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues.15:01
Ursinha[TOPIC] Roll Call15:01
MootBotNew Topic:  Roll Call15:01
Ursinhaapologies from matsubara, he's unwell today15:01
UrsinhaNot on the Launchpad Dev team? Welcome! Come "me" with the rest of us!15:01
* Ursinha is happy, karmic koala is out :)15:02
Ursinharockstar, hi15:02
Ursinhahi noodles775, are you replacing bigjools today?15:02
Ursinhastub, rockstar are missing15:03
Ursinhabigjools as well15:03
Ursinhalets move on, they should join us soon15:03
Ursinha(or not :P)15:03
Ursinha[TOPIC] Agenda15:03
MootBotNew Topic:  Agenda15:03
Ursinha * Actions from last meeting15:03
Ursinha * Oops report & Critical Bugs & Broken scripts15:03
Ursinha * Operations report (mthaddon/Chex/spm/mbarnett)15:03
Ursinha * DBA report (stub)15:03
Ursinha * Proposed items15:03
Ursinhadanilos, go rest man, I can do that for us :)15:04
danilosUrsinha, ok, thanks!15:04
Ursinha[TOPIC] * Actions from last meeting15:04
MootBotNew Topic:  * Actions from last meeting15:04
Ursinha * gary to talk to leonardr about cherry picking lazr.restful updates on lpnet for bug 33199015:04
ubottuLaunchpad bug 331990 in launchpad-foundations "The inline editor widget reports a JSON error when saving non-ASCII characters" [High,Fix committed] https://launchpad.net/bugs/33199015:04
Ursinha * allenap and matsubara to talk about the timeouts on bug pages15:04
Ursinha * matsubara to talk to stub about the DBA report when he gets back15:04
Ursinhaallenap, did you? :)15:04
Ursinhagary_poster, ^15:04
gary_postermine: I've been trying to land the pertinent branch for several days, encountering complications and my own mistakes.  I'm trying today too.15:05
Ursinhathanks gary_poster15:05
allenapUrsinha: A bit :) There was some discussion about the +index page on the list, but, iirc, it didn't come to any conclusions other than "yes, we'd like to fix it".15:05
Ursinha[action] gary_poster to try to land branch that cherry picks lazr.restful updates on lpnet for bug 33199015:05
MootBotACTION received:  gary_poster to try to land branch that cherry picks lazr.restful updates on lpnet for bug 33199015:06
ubottuLaunchpad bug 331990 in launchpad-foundations "The inline editor widget reports a JSON error when saving non-ASCII characters" [High,Fix committed] https://launchpad.net/bugs/33199015:06
Ursinhaallenap, ok, right15:06
Ursinhaallenap, BugTask:+index is not the main offender, though it's in the top ten :P15:07
Ursinhaallenap, I'll talk with matsubara to ask him if he took notes somewhere15:07
Ursinha[action] ask matsubara about his outstanding items from last meeting15:08
MootBotACTION received:  ask matsubara about his outstanding items from last meeting15:08
Ursinhahi stub :)15:08
allenapUrsinha: Surely that's the top offender from bugs though?15:08
Ursinhaallenap, I see DistroSeries:+index15:08
allenapUrsinha: That's Soyuz then :) Not going near that :)15:09
Ursinhaallenap, hehe15:09
Ursinhait's already being fixed by registry, iirc15:09
sinzuiWe landed a fix for DistroSeries:+index but the change is not on edge yet15:09
Ursinha[TOPIC] * Oops report & Critical Bugs & Broken scripts15:09
MootBotNew Topic:  * Oops report & Critical Bugs & Broken scripts15:09
allenapCool. Apologies to Soyuz for tarring them with the Registry brush.15:09
Ursinhaallenap, about bug https://bugs.edge.launchpad.net/malone/+bug/453203, is there something we can do?15:10
Ursinhasinzui, is https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1396EA55 related to https://bugs.edge.launchpad.net/launchpad-registry/+bug/462742? If not I'll be happy to file a bug15:10
ubottuLaunchpad bug 453203 in apport "UnicodeDecodeError in +filebug: unexpected code byte" [Undecided,New]15:10
UrsinhaI've already asked henninge to triage https://bugs.edge.launchpad.net/rosetta/+bug/46289115:10
ubottuLaunchpad bug 462742 in launchpad-registry "OOPS getting a project index page as an anonymous user" [High,In progress]15:10
ubottuLaunchpad bug 462891 in rosetta "TraversalError on +export" [Undecided,New]15:10
Ursinha[action] Ursinha to ask translations to triage bug 46289115:10
MootBotACTION received:  Ursinha to ask translations to triage bug 46289115:10
ubottuLaunchpad bug 462891 in rosetta "TraversalError on +export" [Undecided,New] https://launchpad.net/bugs/46289115:10
sinzuiUrsinha: that fix was landed by intellectronica yesterday15:10
Ursinhasinzui, you told me that yesterday, but is that oops related?15:11
sinzuiUrsinha: yes it is the bug subscription link15:11
henningesinzui: actually, this is oopsing on trying to access preferredemails/email.15:12
sinzuiUrsinha: sorry. I am wonrg15:12
allenapUrsinha: I opened a task against apport in Ubuntu. I can follow up with pitti, see if he has ideas. Also, if I have time I'll try to find a way to get the apport data when we next see this error.15:12
* Ursinha reverts the change in the bug15:12
=== noodles775_ is now known as noodles775
Ursinhaallenap, that would be much appreciated, thanks15:13
Ursinha[action] allenap to follow up with pitti about bug 45320315:13
MootBotACTION received:  allenap to follow up with pitti about bug 45320315:13
ubottuLaunchpad bug 453203 in apport "UnicodeDecodeError in +filebug: unexpected code byte" [Undecided,New] https://launchpad.net/bugs/45320315:13
Ursinhasinzui, want me to file a bug?15:14
sinzuiI will15:15
Ursinhaokay then15:15
Ursinhathanks sinzui15:15
Ursinhaand allenap and henninge15:15
Ursinhahenninge, you can triage bug 462891 if you have a moment :)15:16
ubottuLaunchpad bug 462891 in rosetta "TraversalError on +export" [Undecided,New] https://launchpad.net/bugs/46289115:16
Ursinhaall three critical bugs are already fix committed, good15:16
Ursinhathe jscheck failure is being handled by BjornT, good15:16
henningeUrsinha: I am on it.15:16
Ursinhahenninge, thank you  :) ice_cream++15:16
Ursinhamoving on15:17
Ursinha[TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)15:17
MootBotNew Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)15:17
Ursinhahi Chex15:17
Chexhere is our report for this week15:17
henningeUrsinha: yeah!15:17
Chex- LP Ship-it progress in time for Ubuntu release-day:15:17
Chex        ; New LP app-servers are running15:17
Chex        ; New LP front-end web servers are running15:17
Chex        ; New LP Auth DB server has been switched to and is running15:17
Chex        ; We have a spare LP server available in case of needed extra performance in high loads.15:17
Chex- LP Incidents of note:15:17
Chex        ; CP 8536 to lpnet,bzrsyncd15:17
Chex        ; cowboy config patch to switch auth db master to sweetcherry15:17
Chex        ; CP 8539 to bzrsyncd, crowberry15:17
Chexdoes anyone have any questions??15:18
stubhackberry going boom might be an incident of note.15:18
UrsinhaChex, nice to hear we're ready to the ubuntu frenesy :)15:18
Chexstub: oh ok, neglected to put that on the list15:19
* stub frenzies15:20
Ursinhastub, you can comment on that in your section, that is now :)15:20
Ursinhaany questions for Chex?15:20
sinzuiUrsinha: that is an interesting oops, the create_milestone error only affects an anonymous user on edge.15:20
Ursinhasinzui, why is it interesting?15:21
sinzuiHow often is someone anonymous on edge?15:21
Ursinhasinzui, sure!15:22
Ursinhafollowing a link, maybe?15:22
Ursinhathis is indeed weird15:22
Ursinhawell, lets move on just for the sake of the meeting15:23
Ursinha[TOPIC] * DBA report (stub)15:23
MootBotNew Topic:  * DBA report (stub)15:23
stubWe lost one of the slave databases earlier this week (~05:20 UTC Tuesday). The server load went through the roof. PostgreSQL and shells where still responding despite the load peaking at 94. Processes were not responding to signals.15:23
stubIt turned out one of the drive arrays had hung, so writes were blocking. I'm not sure if this had end user visible results, as the queries made to the slave a read only and likely would have been successfully served from RAM.15:23
stubThe slave was removed from replication as it could not be immediately investigated by IS (not important enough to wake anyone). It was added back into replication the next day.15:23
stubStaging database updates are currently disabled, as fallout from15:23
stubmoving the authdb replication set master to a new database. A fix for Bug #461800 is up for review which is part of this. I still need to work more on the restoration scripts after this lands.15:23
ubottuLaunchpad bug 461800 in launchpad-foundations "new-slave.py no longer works" [High,In progress] https://launchpad.net/bugs/46180015:23
stubThe new database dedicated to the login service (launchpad_prod_4, lpmain slave and authdb master) is running fine. With this and the new appservers we should hopefully weather release day.15:23
Ursinhastub, fixing 461800 will make possible to reenable database updates on staging?15:24
stubIts part of it, not the whole picture15:25
stubThat allows us to build the slave. I still need to build the original db from a pair of dumps instead of a single dump.15:25
UrsinhaI wonder how and if disabled database updates can affect QA15:25
stubGenerally not adversely.15:26
Ursinhastub, do you have an estimated time to finish that?15:27
stubI want to have them working again by Monday. It depends if anything explodes from the release I guess.15:28
Ursinhastub, right, good15:28
Ursinhadoes anyone have something for stub?15:28
Ursinhaall right15:28
UrsinhaChex, a question, do you know when will PQM close?15:29
Ursinhaperhaps the release manage knows it better, noodles775, hi :)15:29
noodles775Ursinha, Chex: according to the docs, 00:00 UTC on Monday.15:30
Ursinhanoodles775, on Monday, right15:30
Ursinhathanks :)15:30
ChexUrsinha: ok, good to know then15:30
Ursinha[TOPIC] * Proposed items15:31
MootBotNew Topic:  * Proposed items15:31
UrsinhaI have something15:31
Ursinhamore of a reminder :)15:31
Ursinhadon't forget about QAing your stuff, it's now or never for next release! :) Needing help, ping the QA-person-in-charge and we'll be glad to help15:31
Ursinhadoes anyone else have something?15:31
noodles775Quite a few teams with > 15 items waiting for QA:15:31
MootBotLINK received:  http://people.canonical.com/~lpqateam/test-plan-report-3.1.10.html15:31
=== salgado is now known as salgado-afk
Ursinhanoodles775, translations doesn't count, we're trying to use lp as QA tool, tagging bugs as qa-needstesting, qa-ok and so on15:33
UrsinhaI'll send one email to the list explaining that today15:33
noodles775Ursinha: great, thanks.15:33
Ursinha[action] Ursinha to send one email to lp list explaining the qa-tags experiment15:33
MootBotACTION received:  Ursinha to send one email to lp list explaining the qa-tags experiment15:33
Ursinhathanks for the reminder noodles775 :)15:33
Ursinharight, anyone else?15:33
UrsinhaThank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.15:34
MootBotMeeting finished at 10:34.15:34
Ursinhathanks everyone15:34
gary_posterthanks Ursinha15:34
Ursinhayou all rock15:34
Ursinhagary_poster, :)15:34
=== salgado-afk is now known as salgado
=== rockstar is now known as rockstar-afk
=== salgado is now known as salgado-afk
=== rockstar-afk is now known as rockstar

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!