| === Peng_ is now known as Peng__ | ||
| === Peng is now known as Peng_ | ||
| === Peng__ is now known as Peng | ||
| === noodles775_ is now known as noodles775 | ||
| === _thumper_ is now known as thumper | ||
| === matsubara-afk is now known as matsubara | ||
| === matsubara is now known as matsubara-afk | ||
| === mrevell is now known as mrevell-lunch | ||
| === mrevell-lunch is now known as mrevell | ||
| Ursinha | MootBot, hi | 14:38 |
|---|---|---|
| Ursinha | #startmeeting | 15:01 |
| MootBot | Meeting started at 10:01. The chair is Ursinha. | 15:01 |
| MootBot | Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] | 15:01 |
| Ursinha | Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. | 15:01 |
| Ursinha | [TOPIC] Roll Call | 15:01 |
| MootBot | New Topic: Roll Call | 15:01 |
| Ursinha | apologies from matsubara, he's unwell today | 15:01 |
| allenap | me | 15:01 |
| sinzui | me | 15:01 |
| Ursinha | Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! | 15:01 |
| Ursinha | me! | 15:01 |
| Chex | salut | 15:01 |
| * Ursinha is happy, karmic koala is out :) | 15:02 | |
| Ursinha | rockstar, hi | 15:02 |
| gary_poster | ,e | 15:02 |
| gary_poster | me | 15:02 |
| Ursinha | hi noodles775, are you replacing bigjools today? | 15:02 |
| Ursinha | stub, rockstar are missing | 15:03 |
| Ursinha | bigjools as well | 15:03 |
| Ursinha | lets move on, they should join us soon | 15:03 |
| Ursinha | (or not :P) | 15:03 |
| Ursinha | [TOPIC] Agenda | 15:03 |
| MootBot | New Topic: Agenda | 15:03 |
| Ursinha | * Actions from last meeting | 15:03 |
| Ursinha | * Oops report & Critical Bugs & Broken scripts | 15:03 |
| Ursinha | * Operations report (mthaddon/Chex/spm/mbarnett) | 15:03 |
| rockstar | ni | 15:03 |
| Ursinha | * DBA report (stub) | 15:03 |
| danilos | me | 15:03 |
| Ursinha | * Proposed items | 15:03 |
| Ursinha | danilos, go rest man, I can do that for us :) | 15:04 |
| danilos | Ursinha, ok, thanks! | 15:04 |
| Ursinha | [TOPIC] * Actions from last meeting | 15:04 |
| MootBot | New Topic: * Actions from last meeting | 15:04 |
| Ursinha | * gary to talk to leonardr about cherry picking lazr.restful updates on lpnet for bug 331990 | 15:04 |
| ubottu | Launchpad bug 331990 in launchpad-foundations "The inline editor widget reports a JSON error when saving non-ASCII characters" [High,Fix committed] https://launchpad.net/bugs/331990 | 15:04 |
| Ursinha | * allenap and matsubara to talk about the timeouts on bug pages | 15:04 |
| Ursinha | * matsubara to talk to stub about the DBA report when he gets back | 15:04 |
| Ursinha | allenap, did you? :) | 15:04 |
| Ursinha | gary_poster, ^ | 15:04 |
| gary_poster | mine: I've been trying to land the pertinent branch for several days, encountering complications and my own mistakes. I'm trying today too. | 15:05 |
| Ursinha | thanks gary_poster | 15:05 |
| allenap | Ursinha: A bit :) There was some discussion about the +index page on the list, but, iirc, it didn't come to any conclusions other than "yes, we'd like to fix it". | 15:05 |
| Ursinha | [action] gary_poster to try to land branch that cherry picks lazr.restful updates on lpnet for bug 331990 | 15:05 |
| MootBot | ACTION received: gary_poster to try to land branch that cherry picks lazr.restful updates on lpnet for bug 331990 | 15:06 |
| ubottu | Launchpad bug 331990 in launchpad-foundations "The inline editor widget reports a JSON error when saving non-ASCII characters" [High,Fix committed] https://launchpad.net/bugs/331990 | 15:06 |
| Ursinha | allenap, ok, right | 15:06 |
| Ursinha | allenap, BugTask:+index is not the main offender, though it's in the top ten :P | 15:07 |
| Ursinha | allenap, I'll talk with matsubara to ask him if he took notes somewhere | 15:07 |
| Ursinha | [action] ask matsubara about his outstanding items from last meeting | 15:08 |
| MootBot | ACTION received: ask matsubara about his outstanding items from last meeting | 15:08 |
| Ursinha | hi stub :) | 15:08 |
| allenap | Ursinha: Surely that's the top offender from bugs though? | 15:08 |
| Ursinha | allenap, I see DistroSeries:+index | 15:08 |
| allenap | Ursinha: That's Soyuz then :) Not going near that :) | 15:09 |
| Ursinha | allenap, hehe | 15:09 |
| Ursinha | it's already being fixed by registry, iirc | 15:09 |
| sinzui | We landed a fix for DistroSeries:+index but the change is not on edge yet | 15:09 |
| Ursinha | ha! | 15:09 |
| Ursinha | :) | 15:09 |
| Ursinha | [TOPIC] * Oops report & Critical Bugs & Broken scripts | 15:09 |
| MootBot | New Topic: * Oops report & Critical Bugs & Broken scripts | 15:09 |
| allenap | Cool. Apologies to Soyuz for tarring them with the Registry brush. | 15:09 |
| Ursinha | allenap, about bug https://bugs.edge.launchpad.net/malone/+bug/453203, is there something we can do? | 15:10 |
| Ursinha | sinzui, is https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1396EA55 related to https://bugs.edge.launchpad.net/launchpad-registry/+bug/462742? If not I'll be happy to file a bug | 15:10 |
| ubottu | Launchpad bug 453203 in apport "UnicodeDecodeError in +filebug: unexpected code byte" [Undecided,New] | 15:10 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1396EA55 | 15:10 |
| Ursinha | I've already asked henninge to triage https://bugs.edge.launchpad.net/rosetta/+bug/462891 | 15:10 |
| ubottu | Launchpad bug 462742 in launchpad-registry "OOPS getting a project index page as an anonymous user" [High,In progress] | 15:10 |
| ubottu | Launchpad bug 462891 in rosetta "TraversalError on +export" [Undecided,New] | 15:10 |
| Ursinha | [action] Ursinha to ask translations to triage bug 462891 | 15:10 |
| MootBot | ACTION received: Ursinha to ask translations to triage bug 462891 | 15:10 |
| ubottu | Launchpad bug 462891 in rosetta "TraversalError on +export" [Undecided,New] https://launchpad.net/bugs/462891 | 15:10 |
| sinzui | Ursinha: that fix was landed by intellectronica yesterday | 15:10 |
| Ursinha | sinzui, you told me that yesterday, but is that oops related? | 15:11 |
| sinzui | Ursinha: yes it is the bug subscription link | 15:11 |
| henninge | sinzui: actually, this is oopsing on trying to access preferredemails/email. | 15:12 |
| sinzui | Ursinha: sorry. I am wonrg | 15:12 |
| allenap | Ursinha: I opened a task against apport in Ubuntu. I can follow up with pitti, see if he has ideas. Also, if I have time I'll try to find a way to get the apport data when we next see this error. | 15:12 |
| Ursinha | oops | 15:12 |
| * Ursinha reverts the change in the bug | 15:12 | |
| === noodles775_ is now known as noodles775 | ||
| Ursinha | allenap, that would be much appreciated, thanks | 15:13 |
| Ursinha | [action] allenap to follow up with pitti about bug 453203 | 15:13 |
| MootBot | ACTION received: allenap to follow up with pitti about bug 453203 | 15:13 |
| ubottu | Launchpad bug 453203 in apport "UnicodeDecodeError in +filebug: unexpected code byte" [Undecided,New] https://launchpad.net/bugs/453203 | 15:13 |
| Ursinha | sinzui, want me to file a bug? | 15:14 |
| sinzui | I will | 15:15 |
| Ursinha | okay then | 15:15 |
| Ursinha | thanks sinzui | 15:15 |
| Ursinha | and allenap and henninge | 15:15 |
| Ursinha | henninge, you can triage bug 462891 if you have a moment :) | 15:16 |
| ubottu | Launchpad bug 462891 in rosetta "TraversalError on +export" [Undecided,New] https://launchpad.net/bugs/462891 | 15:16 |
| Ursinha | well | 15:16 |
| Ursinha | all three critical bugs are already fix committed, good | 15:16 |
| Ursinha | the jscheck failure is being handled by BjornT, good | 15:16 |
| henninge | Ursinha: I am on it. | 15:16 |
| Ursinha | henninge, thank you :) ice_cream++ | 15:16 |
| Ursinha | moving on | 15:17 |
| Ursinha | [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) | 15:17 |
| MootBot | New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) | 15:17 |
| Ursinha | hi Chex | 15:17 |
| Ursinha | :) | 15:17 |
| Chex | hello | 15:17 |
| Chex | here is our report for this week | 15:17 |
| henninge | Ursinha: yeah! | 15:17 |
| Chex | - LP Ship-it progress in time for Ubuntu release-day: | 15:17 |
| Chex | ; New LP app-servers are running | 15:17 |
| Chex | ; New LP front-end web servers are running | 15:17 |
| Chex | ; New LP Auth DB server has been switched to and is running | 15:17 |
| Chex | ; We have a spare LP server available in case of needed extra performance in high loads. | 15:17 |
| Chex | - LP Incidents of note: | 15:17 |
| Chex | ; CP 8536 to lpnet,bzrsyncd | 15:17 |
| Chex | ; cowboy config patch to switch auth db master to sweetcherry | 15:17 |
| Chex | ; CP 8539 to bzrsyncd, crowberry | 15:17 |
| Chex | does anyone have any questions?? | 15:18 |
| stub | hackberry going boom might be an incident of note. | 15:18 |
| Ursinha | Chex, nice to hear we're ready to the ubuntu frenesy :) | 15:18 |
| Chex | stub: oh ok, neglected to put that on the list | 15:19 |
| Ursinha | *frenzy | 15:19 |
| * stub frenzies | 15:20 | |
| Ursinha | lol | 15:20 |
| Ursinha | stub, you can comment on that in your section, that is now :) | 15:20 |
| Ursinha | any questions for Chex? | 15:20 |
| sinzui | Ursinha: that is an interesting oops, the create_milestone error only affects an anonymous user on edge. | 15:20 |
| Ursinha | sinzui, why is it interesting? | 15:21 |
| sinzui | How often is someone anonymous on edge? | 15:21 |
| Ursinha | sinzui, sure! | 15:22 |
| Ursinha | indeed | 15:22 |
| Ursinha | following a link, maybe? | 15:22 |
| Ursinha | this is indeed weird | 15:22 |
| Ursinha | well, lets move on just for the sake of the meeting | 15:23 |
| Ursinha | [TOPIC] * DBA report (stub) | 15:23 |
| MootBot | New Topic: * DBA report (stub) | 15:23 |
| stub | We lost one of the slave databases earlier this week (~05:20 UTC Tuesday). The server load went through the roof. PostgreSQL and shells where still responding despite the load peaking at 94. Processes were not responding to signals. | 15:23 |
| stub | It turned out one of the drive arrays had hung, so writes were blocking. I'm not sure if this had end user visible results, as the queries made to the slave a read only and likely would have been successfully served from RAM. | 15:23 |
| stub | The slave was removed from replication as it could not be immediately investigated by IS (not important enough to wake anyone). It was added back into replication the next day. | 15:23 |
| stub | Staging database updates are currently disabled, as fallout from | 15:23 |
| stub | moving the authdb replication set master to a new database. A fix for Bug #461800 is up for review which is part of this. I still need to work more on the restoration scripts after this lands. | 15:23 |
| ubottu | Launchpad bug 461800 in launchpad-foundations "new-slave.py no longer works" [High,In progress] https://launchpad.net/bugs/461800 | 15:23 |
| stub | The new database dedicated to the login service (launchpad_prod_4, lpmain slave and authdb master) is running fine. With this and the new appservers we should hopefully weather release day. | 15:23 |
| stub | oot. | 15:23 |
| Ursinha | stub, fixing 461800 will make possible to reenable database updates on staging? | 15:24 |
| stub | Its part of it, not the whole picture | 15:25 |
| stub | That allows us to build the slave. I still need to build the original db from a pair of dumps instead of a single dump. | 15:25 |
| Ursinha | I wonder how and if disabled database updates can affect QA | 15:25 |
| stub | Generally not adversely. | 15:26 |
| Ursinha | right | 15:26 |
| Ursinha | stub, do you have an estimated time to finish that? | 15:27 |
| stub | I want to have them working again by Monday. It depends if anything explodes from the release I guess. | 15:28 |
| Ursinha | stub, right, good | 15:28 |
| Ursinha | thanks | 15:28 |
| Ursinha | does anyone have something for stub? | 15:28 |
| Ursinha | all right | 15:28 |
| Ursinha | Chex, a question, do you know when will PQM close? | 15:29 |
| Ursinha | perhaps the release manage knows it better, noodles775, hi :) | 15:29 |
| Ursinha | *manager | 15:29 |
| noodles775 | Ursinha, Chex: according to the docs, 00:00 UTC on Monday. | 15:30 |
| Ursinha | noodles775, on Monday, right | 15:30 |
| Ursinha | thanks :) | 15:30 |
| Chex | Ursinha: ok, good to know then | 15:30 |
| Ursinha | [TOPIC] * Proposed items | 15:31 |
| MootBot | New Topic: * Proposed items | 15:31 |
| Ursinha | I have something | 15:31 |
| Ursinha | more of a reminder :) | 15:31 |
| Ursinha | don't forget about QAing your stuff, it's now or never for next release! :) Needing help, ping the QA-person-in-charge and we'll be glad to help | 15:31 |
| Ursinha | does anyone else have something? | 15:31 |
| noodles775 | Quite a few teams with > 15 items waiting for QA: | 15:31 |
| noodles775 | http://people.canonical.com/~lpqateam/test-plan-report-3.1.10.html | 15:31 |
| MootBot | LINK received: http://people.canonical.com/~lpqateam/test-plan-report-3.1.10.html | 15:31 |
| === salgado is now known as salgado-afk | ||
| Ursinha | noodles775, translations doesn't count, we're trying to use lp as QA tool, tagging bugs as qa-needstesting, qa-ok and so on | 15:33 |
| Ursinha | I'll send one email to the list explaining that today | 15:33 |
| noodles775 | Ursinha: great, thanks. | 15:33 |
| Ursinha | [action] Ursinha to send one email to lp list explaining the qa-tags experiment | 15:33 |
| MootBot | ACTION received: Ursinha to send one email to lp list explaining the qa-tags experiment | 15:33 |
| Ursinha | thanks for the reminder noodles775 :) | 15:33 |
| Ursinha | right, anyone else? | 15:33 |
| Ursinha | good! | 15:34 |
| gary_poster | :-) | 15:34 |
| Ursinha | Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. | 15:34 |
| Ursinha | #endmeeting | 15:34 |
| MootBot | Meeting finished at 10:34. | 15:34 |
| Ursinha | thanks everyone | 15:34 |
| gary_poster | thanks Ursinha | 15:34 |
| Ursinha | you all rock | 15:34 |
| Ursinha | gary_poster, :) | 15:34 |
| === salgado-afk is now known as salgado | ||
| === rockstar is now known as rockstar-afk | ||
| === salgado is now known as salgado-afk | ||
| === rockstar-afk is now known as rockstar | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!