=== salgado is now known as salgado-lunch === salgado-lunch is now known as salgado [16:01] #startmeeting [16:01] Meeting started at 10:01. The chair is Ursinha. [16:01] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [16:01] Happy new year! [16:01] me [16:01] Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. [16:01] [TOPIC] Roll Call [16:01] New Topic: Roll Call [16:01] Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! [16:01] me [16:01] me [16:01] hehe :) [16:01] me [16:02] stub isn't here, that's kinda fine [16:02] rockstar, hi [16:02] ni! [16:02] :) [16:02] me [16:02] Chex, hello [16:02] I'm keeping one eye on the meeting, but not really here :P [16:03] jtv, thanks :) [16:03] let [16:03] let's go then [16:03] [TOPIC] Agenda [16:03] New Topic: Agenda [16:03] * Actions from last meeting [16:03] * Oops report & Critical Bugs & Broken scripts [16:03] * Operations report (mthaddon/Chex/spm/mbarnett) [16:03] * DBA report (stub) [16:03] * Proposed items [16:03] [TOPIC] * Actions from last meeting [16:03] New Topic: * Actions from last meeting [16:03] * Chex to file a bug about the multiple git import failures on the importd [16:03] * Ursinha to try to come up with an older working version of oops-tools (non oops-db version) [16:03] https://lp-oops.canonical.com/oops.py/?oopsid=db [16:03] * Done, it's running on devpad and generating the partial summaries since before holidays. Used to generate the daily summaries as well when the db version fails for any reason. [16:03] * Ursinha to send one email to stub about the db report [16:03] * Sent. stub sent the report. [16:04] here.. sorry [16:04] Chex, hey :) [16:04] Chex, so, did you file that bug? [16:04] Ursinha: yes I did, let me get the bug number for you [16:05] thanks Chex [16:05] I'll keep the ball rolling while Chex finds the bug number :) [16:05] [TOPIC] * Oops report & Critical Bugs & Broken scripts [16:06] New Topic: * Oops report & Critical Bugs & Broken scripts [16:06] the failing scripts are being handled already, right jtv? :) [16:06] Ursinha: yes, I got distracted for a while by something about the database, but back on it now [16:06] jtv, haha thanks :) [16:06] and, we had a bunch of DisconnectionErrors on 4th, on staging, and are having them now on lpnet and edge [16:06] I've filed a bug for them, bug 504291, jtv kindly did his analysis [16:06] salgado, could you take this to foundations, please? [16:06] Launchpad bug 504291 in launchpad-foundations "DisconnectionErrors (already disconnected) happening again" [Undecided,New] https://launchpad.net/bugs/504291 [16:07] again!? [16:07] yeah, I'll have a look at it [16:07] salgado, unfortunately :( [16:07] another issue that needs to be investigated by foundations is why is staging not being updated [16:07] mthaddon filed a bug with the error [16:08] bug 504300 [16:08] Ursinha, do you have the number? [16:08] Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300 [16:08] salgado, sorry, was searching the tab [16:08] Ursinha: I filed a bug, and was immediately redirected to this as a duplicate bug: bug #486076. [16:08] Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076 [16:08] right Chex, thanks [16:09] I'll take a look [16:09] Ursinha: which, when I look at it, I am not sure directly addresses our issue, but its a start, I think. [16:09] [action] salgado to take bugs 504300 and 504291 to foundations discussion [16:09] ACTION received: salgado to take bugs 504300 and 504291 to foundations discussion [16:09] Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300 [16:09] Launchpad bug 504291 in launchpad-foundations "DisconnectionErrors (already disconnected) happening again" [Undecided,New] https://launchpad.net/bugs/504291 [16:10] salgado, is that ok for you? [16:10] yep [16:10] right, thanks [16:11] Chex, right, do you think it's not a duplicate? maybe a code guy could enlighten us [16:11] rockstar, ^ [16:11] Ursinha: I was under the impression that if the code tree was very large and big, it overwhelmed the import process, with memory usage.. that doesn't seem like a memory leak to me, but I could be mistaken about that. [16:12] rockstar doesn't love us anymore [16:13] Chex, we can probably sleuth whether or not it's a duplicate. I'd suggest filing the bug. [16:13] Ursinha: here is the orginal bug I filed: Bug #497945 : Large git imports into bzr tree uses lots of memory [16:13] Launchpad bug 497945 in bzr "Large git imports into bzr tree uses lots of memory (dup-of: 486076)" [Undecided,New] https://launchpad.net/bugs/497945 [16:13] Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076 [16:13] rockstar, ^ [16:14] Ursinha: I can discuss this offline with you and/or rockstar later, not to hold up the meeting [16:14] * rockstar agrees [16:14] Chex, thanks :) [16:14] rockstar, thanks :) [16:14] moving on then [16:14] These kinds of bugs are harder to track down, but we know they exist. [16:15] [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) [16:15] New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) [16:15] here is our report for this week, folks: [16:15] - LP edge updates broke today (CSS broken) because make build was relying [16:15] on python2.4, salgado is helping to fix this issue now. [16:15] - LP staging updates are broken at the moment, Bug 504300 has been filed [16:15] addressing this issue. [16:15] Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300 [16:16] - LP seeing intermittent 500s on shipit servers, we are [16:16] investigating that more to see exactly what is causing those. [16:16] - LP incidents of note: [16:16] ; LP Cherry-picks: revno 8809 rolled out to lpnet 30-Dec, [16:16] revno 8810 rollout to lpnet 07-Jan; [16:16] ; Crowberry upgraded from 8Gb to 32GB RAM 23-Dec [16:16] and thats our report for this week, any questions/comments? [16:16] not here [16:16] anyone? [16:16] [action] Chex to discuss with rockstar about bug 497945 [16:16] ACTION received: Chex to discuss with rockstar about bug 497945 [16:16] Launchpad bug 497945 in bzr "Large git imports into bzr tree uses lots of memory (dup-of: 486076)" [Undecided,New] https://launchpad.net/bugs/497945 [16:16] Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076 [16:17] all right [16:17] thanks Chex! [16:17] [TOPIC] * DBA report (stub) [16:17] New Topic: * DBA report (stub) [16:17] sent by email to the list [16:17] [TOPIC] * Proposed items [16:17] New Topic: * Proposed items [16:17] anyone? [16:17] Ursinha: your welcome, thanks.. [16:17] no items proposed in the meeting agenda page [16:18] 5 [16:18] 4 [16:18] 3 [16:18] 2 [16:18] 1 [16:18] Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. [16:18] #endmeeting [16:18] Meeting finished at 10:18. [16:18] thanks everyone [16:19] Thanks Ursinha [16:20] thanks Ursinha === EdwinGrubbs is now known as Edwin-lunch === salgado is now known as salgado-afk === Edwin-lunch is now known as EdwinGrubbs