| === mrevell is now known as mrevell-lunch | ||
| === mrevell-lunch is now known as mrevell | ||
| === salgado is now known as salgado-lunch | ||
| === salgado-lunch is now known as salgado | ||
| matsubara | #startmeeting | 16:00 |
|---|---|---|
| MootBot | Meeting started at 10:00. The chair is matsubara. | 16:00 |
| MootBot | Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] | 16:00 |
| matsubara | Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. | 16:00 |
| matsubara | [TOPIC] Roll Call | 16:00 |
| MootBot | New Topic: Roll Call | 16:00 |
| matsubara | Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! | 16:00 |
| Ursinha | me! | 16:00 |
| allenap | me | 16:00 |
| henninge | me | 16:00 |
| al-maisan | me | 16:01 |
| matsubara | Chex, rockstar: hi | 16:01 |
| matsubara | bac, can you stand in for sinzui? | 16:02 |
| Chex | matsubara: hello | 16:02 |
| bac | matsubara: gladly | 16:02 |
| bac | me | 16:02 |
| Ursinha | :) | 16:02 |
| matsubara | thanks bac | 16:02 |
| jtv | me | 16:02 |
| Ursinha | oh, a special guest :P | 16:02 |
| matsubara | I'm standing in for gary | 16:02 |
| matsubara | hi jtv | 16:02 |
| jtv | hi | 16:02 |
| matsubara | we have 3/4 of the translations team around | 16:02 |
| matsubara | pretty cool | 16:02 |
| jtv | with all the oopses going on, we thought we'd look in just in case | 16:03 |
| * Ursinha hi-fives henninge and jtv | 16:03 | |
| * jtv hi-fives back | 16:03 | |
| matsubara | ok, let's get this started | 16:03 |
| Ursinha | yay | 16:03 |
| matsubara | [TOPIC] Agenda | 16:03 |
| MootBot | New Topic: Agenda | 16:03 |
| * henninge ducks | 16:03 | |
| matsubara | * Actions from last meeting | 16:03 |
| matsubara | * Oops report & Critical Bugs & Broken scripts | 16:03 |
| matsubara | * Operations report (mthaddon/Chex/spm/mbarnett) | 16:03 |
| matsubara | * DBA report (stub) | 16:03 |
| matsubara | * Proposed items | 16:03 |
| matsubara | [TOPIC] * Actions from last meeting | 16:03 |
| MootBot | New Topic: * Actions from last meeting | 16:03 |
| matsubara | * add brian to the list of attendees in the MeetingAgenda page | 16:04 |
| matsubara | * done | 16:04 |
| matsubara | * allenap to dig the master bug of OOPS-1474EA771 | 16:04 |
| matsubara | * done. it's #508302 | 16:04 |
| matsubara | * rockstar to take a look in OOPS-1480CMP1 | 16:04 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1474EA771 | 16:04 |
| matsubara | * matsubara to email Tim about https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1488EA174 | 16:04 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1480CMP1 | 16:04 |
| matsubara | * matsubara to talk to leonard about https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1488EA884 | 16:04 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1488EA174 | 16:04 |
| matsubara | * matsubara to talk to salgado about More non-informational disconnectionerrors https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1489J147 | 16:04 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1488EA884 | 16:04 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1489J147 | 16:04 |
| matsubara | * matsubara to talk to stub or gary about InternalError after ther rollout https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1489C1094 | 16:04 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1489C1094 | 16:04 |
| matsubara | * https://launchpad.net/bugs/514267 | 16:04 |
| Ursinha | oh dear | 16:04 |
| matsubara | * matsubara to fix bug link on OOPS-1474EA771 to point to bug 508302 | 16:04 |
| ubottu | Ubuntu bug 514267 in launchpad-foundations "InternalError on clusters under busy load" [Critical,Triaged] | 16:04 |
| matsubara | * done | 16:04 |
| matsubara | * sinzui to investigate failure on send-person-notifications and reply to the list with his findings | 16:04 |
| ubottu | Launchpad bug 508302 in malone "NotImplementedError OOPS when reporting a bug" [High,Fix committed] https://launchpad.net/bugs/508302 | 16:04 |
| matsubara | * https://bugs.edge.launchpad.net/launchpad-registry/+bug/514149 | 16:04 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1474EA771 | 16:04 |
| matsubara | * rockstar to raise the importance of ensuring sufficient monitoring of this part (i.e. branch updates emails failing to be delivered) of code-hosting by thumper | 16:04 |
| ubottu | Ubuntu bug 514149 in launchpad-registry "send-person-notifications.py is broken" [High,Fix released] | 16:04 |
| matsubara | * Gary talk about "too many different kinds of moving parts" in our architecture. If we have fewer moving parts then we can institute more uniform nagios-like-checks. | 16:04 |
| matsubara | * matsubara to update oops-tools to know about the new lp appserver | 16:04 |
| matsubara | * done. | 16:04 |
| matsubara | hmm ok, rockstar is not around | 16:04 |
| matsubara | I haven't talked to leonard nor emailed team about those oopses | 16:04 |
| matsubara | [action] * matsubara to email Tim about https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1488EA174 and * matsubara to talk to leonard about https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1488EA884 | 16:05 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1488EA174 | 16:05 |
| MootBot | ACTION received: * matsubara to email Tim about https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1488EA174 and * matsubara to talk to leonard about https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1488EA884 | 16:05 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1488EA884 | 16:05 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1488EA174 | 16:05 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1488EA884 | 16:05 |
| matsubara | [action] * rockstar to take a look in OOPS-1480CMP1 | 16:05 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1480CMP1 | 16:05 |
| MootBot | ACTION received: * rockstar to take a look in OOPS-1480CMP1 | 16:05 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1480CMP1 | 16:05 |
| matsubara | I guess that's it | 16:06 |
| matsubara | [TOPIC] * Oops report & Critical Bugs & Broken scripts | 16:06 |
| MootBot | New Topic: * Oops report & Critical Bugs & Broken scripts | 16:06 |
| Ursinha | okay, so | 16:06 |
| Ursinha | the alarming bug is bug 514267 | 16:06 |
| ubottu | Launchpad bug 514267 in launchpad-foundations "InternalError on clusters under busy load" [Critical,Triaged] https://launchpad.net/bugs/514267 | 16:06 |
| Ursinha | apparently it's the reason of about 6k InternalError oopses only today in lpnet | 16:07 |
| matsubara | stub is on it, discussion ongoing in the internal channel | 16:08 |
| Ursinha | matsubara: okay, notes in that bug are welcome :) | 16:08 |
| Ursinha | matsubara: thanks | 16:08 |
| matsubara | feel free to add them Ursinha | 16:08 |
| Ursinha | matsubara: I don't know what's being discussed :) | 16:08 |
| Ursinha | anyway, I guess this is most critical thing | 16:09 |
| Ursinha | we have six critical bugs, two in progress | 16:10 |
| matsubara | no other oopses worth mentioning? | 16:10 |
| Ursinha | matsubara: not that already have bugs/are being worked on, afaik | 16:10 |
| matsubara | ok | 16:10 |
| Ursinha | matsubara: bug 511567 is triaged and marked to 10.01 | 16:11 |
| ubottu | Launchpad bug 511567 in launchpad-foundations "Can't remove authorised oauth tokens" [Critical,Triaged] https://launchpad.net/bugs/511567 | 16:11 |
| Ursinha | is that really critical? | 16:11 |
| matsubara | 10.01 should be closed by now, so I guess it should be 10.02 | 16:11 |
| Ursinha | matsubara: right, but is that really critical? | 16:12 |
| matsubara | don't know I asked in the bug report to Curtis who set it as critical | 16:12 |
| Ursinha | bug 514461 is code, in progress, have to check with rockstar how's that | 16:12 |
| ubottu | Launchpad bug 514461 in launchpad-code "scan_branches.py screams and no one is listening" [Critical,In progress] https://launchpad.net/bugs/514461 | 16:12 |
| Ursinha | [action] Ursinha to talk to rockstar about bug 514461 | 16:13 |
| Ursinha | [action] Ursinha to talk to mwhudson about bug 513412 | 16:13 |
| ubottu | Bug 513412 on http://launchpad.net/bugs/513412 is private | 16:13 |
| Ursinha | and, finally, bug 515473 is assigned to curtis and in progress | 16:14 |
| ubottu | Launchpad bug 515473 in soyuz "ppa description contains smal links" [Critical,In progress] https://launchpad.net/bugs/515473 | 16:14 |
| Ursinha | bac: do you know the progress on that one? | 16:14 |
| bac | i do not | 16:15 |
| Ursinha | bac: can you ask him, if possible, please? :) | 16:15 |
| bac | looking into it | 16:15 |
| Ursinha | bac: thanks | 16:15 |
| Ursinha | bug 508025 is only waiting for a test commit so it can be closed | 16:16 |
| ubottu | Launchpad bug 508025 in lpbuildbot "Need production-devel builder" [Critical,In progress] https://launchpad.net/bugs/508025 | 16:16 |
| Ursinha | end of critical bugs | 16:17 |
| Ursinha | now for something completely different | 16:17 |
| Ursinha | script process-pending-packagediffs has been failing lately, is that soyuz? al-maisan? | 16:17 |
| al-maisan | al-maisan: I have a branch that fixes these failure | 16:18 |
| === henninge_ is now known as henninge | ||
| al-maisan | it has been reviewed and I am revising it in accordance with the review comments | 16:18 |
| Ursinha | that's great al-maisan, I'll reply in the failure mail | 16:19 |
| Ursinha | al-maisan: thanks for the info | 16:19 |
| Ursinha | matsubara: you can move on | 16:19 |
| al-maisan | you are welcome | 16:19 |
| matsubara | thanks Ursinha | 16:19 |
| matsubara | [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) | 16:19 |
| MootBot | New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) | 16:19 |
| rockstar | me | 16:20 |
| Ursinha | rockstar: good morning :) | 16:20 |
| Chex | hi everyone | 16:20 |
| rockstar | Ursinha, :) | 16:20 |
| Chex | here is the LOSA report for this week: | 16:20 |
| Chex | - LP Outage Today @1400UTC: LP is back running again after a outage; there was an issue with the new | 16:20 |
| Chex | DB replication lag check not working properly. We are working to see why this happened and | 16:21 |
| Chex | resolve. | 16:21 |
| Chex | - LP incidents of note: | 16:21 |
| Chex | ; LP Cherry-picks: rolled out 8940 to Lpnet on 29-Jan | 16:21 |
| Chex | Does anyone have any comments or questions? | 16:21 |
| matsubara | thanks Chex | 16:22 |
| matsubara | [TOPIC] * DBA report (stub) | 16:22 |
| MootBot | New Topic: * DBA report (stub) | 16:22 |
| matsubara | I'll ask stub to send the report to the list | 16:22 |
| matsubara | [action] matsubara to ask stub to send the dba report to the list | 16:23 |
| MootBot | ACTION received: matsubara to ask stub to send the dba report to the list | 16:23 |
| matsubara | [TOPIC] * Proposed items | 16:23 |
| MootBot | New Topic: * Proposed items | 16:23 |
| matsubara | no proposed items | 16:23 |
| matsubara | ok, since we have some time | 16:23 |
| matsubara | * rockstar to take a look in OOPS-1480CMP1 | 16:23 |
| ubottu | https://lp-oops.canonical.com/oops.py/?oopsid=1480CMP1 | 16:23 |
| matsubara | rockstar, that one is in the actions from last meeting topic for quite some week now | 16:24 |
| matsubara | did you have time to look into that? | 16:24 |
| rockstar | matsubara, I'll take a look. | 16:26 |
| Ursinha | thanks rockstar | 16:28 |
| matsubara | rockstar, I thought you were looking and would report back here... | 16:28 |
| matsubara | shall I close the meeting then? | 16:28 |
| rockstar | matsubara, that's the first I've seen that oops. I'm not sure how I missed it from other meetings. | 16:29 |
| matsubara | by missing the meeting? :-) | 16:29 |
| matsubara | rockstar, anyway, thanks for looking into that. if it helps I'll file a place holder bug and you can fill in the details once you know more about it | 16:30 |
| matsubara | thanks everyone | 16:30 |
| rockstar | matsubara, sure. | 16:30 |
| Ursinha | thanks matsubara | 16:30 |
| matsubara | Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. | 16:30 |
| matsubara | #endmeeting | 16:30 |
| MootBot | Meeting finished at 10:30. | 16:30 |
| bac | Ursinha: sinzui reports bug 515473 has been reviewed and will be on ec2 for landing shortly | 16:32 |
| ubottu | Launchpad bug 515473 in soyuz "ppa description contains spam links" [Critical,In progress] https://launchpad.net/bugs/515473 | 16:32 |
| Ursinha | thanks a lot bac | 16:32 |
| Ursinha | bac: do you know which branch fixes that bug? it's not linked | 16:34 |
| * bac looks | 16:34 | |
| matsubara | rockstar, filed https://bugs.edge.launchpad.net/launchpad-code/+bug/517126 | 16:35 |
| ubottu | Ubuntu bug 517126 in launchpad-code "BzrCheckError raised creating a merge proposal" [Undecided,New] | 16:35 |
| bac | i have linked it to the bug now, ursinha | 16:35 |
| Ursinha | thaaanks bac :) | 16:35 |
| Ursinha | rockstar: are you working on bug 514461? | 16:43 |
| ubottu | Launchpad bug 514461 in launchpad-code "scan_branches.py screams and no one is listening" [Critical,In progress] https://launchpad.net/bugs/514461 | 16:43 |
| rockstar | Ursinha, yes, I have been trying to get it landed for a week. If it got into production-stable, then I'm going to have it CP'd today. | 16:44 |
| Ursinha | rockstar: cool! can you tell the world about it in the bug by adding a comment? :) | 16:45 |
| rockstar | Ursinha, sure. | 16:45 |
| Ursinha | rockstar: thanks | 16:45 |
| === matsubara is now known as matsubara-afk | ||
| === salgado is now known as salgado-afk | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!