=== Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha [17:00] #startmeeting [17:00] Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. [17:00] Meeting started at 11:00. The chair is Ursinha. [17:00] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [17:00] [TOPIC] Roll Call [17:00] Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! [17:00] New Topic: Roll Call [17:00] me [17:00] me [17:00] me [17:00] me [17:01] Chex, sinzui, rockstar, hello [17:01] * rockstar [17:01] me [17:01] Chex and sinzui are missing [17:01] me [17:02] :) [17:02] only sinzui [17:02] let's move on, try to break rockstar's last week record [17:02] :) [17:02] [TOPIC] Agenda [17:02] * Actions from last meeting [17:02] * Oops report & Critical Bugs & Broken scripts [17:02] * Operations report (mthaddon/Chex/spm/mbarnett) [17:02] * DBA report (stub) [17:02] New Topic: Agenda [17:02] * QA stats of the week [17:02] * Proposed items [17:02] [TOPIC] * Actions from last meeting [17:02] New Topic: * Actions from last meeting [17:02] None [17:02] [TOPIC] * Oops report & Critical Bugs & Broken scripts [17:02] New Topic: * Oops report & Critical Bugs & Broken scripts [17:02] only one bug, soyuz [17:03] bigjools must love me [17:03] bigjools, I see the last noodles' comment in https://bugs.edge.launchpad.net/soyuz/+bug/580181, do you know if that's being tracked and fixed in another bug or one needs to be filed? [17:03] it's in progress [17:03] Launchpad bug 580181 in Soyuz "DistributionSourcePackageRelease page still oopsing with NotOneError/LocationError (affected: 1, heat: 8)" [High,Fix released] [17:03] Ursinha: it needs a new bug [17:03] bigjools, so soyuz has two, this one and the critical [17:03] heh [17:03] bigjools, I'll file one right after this meeting [17:03] me [17:03] cheers [17:04] bigjools, bug 589073 is in progress? [17:04] Launchpad bug 589073 in Soyuz "Unhandled exception processing upload: permission denied for relation emailaddress (affected: 1, heat: 6)" [Critical,Triaged] https://launchpad.net/bugs/589073 [17:04] is it really critical? [17:04] Ursinha: yes, jelmer should have set that in progress ;) [17:04] I don't think it is actually, I'll downgrade it [17:04] thanks bigjools [17:05] we have two critical bugs, one soyuz, one foundations, both being worked on then [17:05] failing scripts are retry-depwait, that's being fixed by bigjools, and karma-update, allocate-revision-karma and others that, according to spm's email, are failing because nightly.sh is taking about 36 hours to run [17:05] Is anyone taking care of this? [17:05] retrydepwait is fixed and released [17:06] bigjools, right, cool [17:06] last failure was yesterday [17:06] gary_poster, about nightly.sh, is that foundations' domain? [17:06] Ursinha, I think nightly.sh is/was a victim of the master/slave issue [17:07] sinzui, right [17:07] sinzui, it failed yesterday again [17:07] nightly.sh has no owner. The losas do a lot of tinkering with it and they do not think they should [17:07] nominally foundations then I suppose [17:07] :-/ [17:08] Since it takes 36 hours to run, we would expect it to fail even after the db was fixed [17:08] It's a good thing that days are longer than 36 hours so that it can actually run nightly... [17:08] Another failure to run on time would be worrisome and will warrant further investigation [17:09] agreed [17:09] yeah, right [17:09] And I certainly have not regarded nightly.sh as a foundations responsibility. I guess that needs to change. Ursinha, please do help me kick myself about this :-) [17:09] but for now I think we can move on [17:09] gary_poster, sure [17:10] :) [17:10] good [17:10] I don't think nightly.sh should be exclusively foundations [17:10] gary_poster, rockstar, other than the karma-cache update, I think everything in nightly can be parallelised. [17:10] I think the losas are pretty good at identifying which scripts run by nightly.sh are the worst offenders and then the responsible team should take care of that script [17:10] the scripts in it belong to code, bugs, translations, and registry [17:11] taking the last run for example, package cache and bugtask target names are the worst offenders [17:11] matsubara: no, but if no-one is in charge of something, historically that means foundations. I'd certainly like it if losas could take that ownership though [17:11] and soyuz then [17:11] so malone and soyuz(?) should take care of those first [17:12] Most failures are spurious. [17:12] matsubara, sinzui, gary_poster, what do you think about discussing that in the email spm sent? this way we can be sure everyone else can join the discussion [17:12] cool thanks [17:12] The 9 times out of 10, the last proc ran late because of something else run in the sequence [17:13] ok, so I'll move on [17:13] [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) [17:13] New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) [17:13] hello everyone, short report this week: [17:13] - LP incidents of note, quiet week: [17:13] : Jun 08: lpnet15 died, restarted [17:13] : Jun 10 : poppy on germanium had died (process died) - restarted [17:14] Us LOSAs are now searching on Canonical-losas -tagged bugs, here: [17:14] https://pastebin.canonical.com/33309/ [17:14] I am curious about status of: 45419 Launchpad needs a way of easily flagging spam [17:14] In the future I will try to bring up critical LP related bugs from this list, to see progress and such, and see if you need any help from us at all. [17:14] bug 45419? [17:14] Launchpad bug 45419 in Launchpad Foundations "Launchpad needs a way of easily flagging spam (affected: 7, heat: 60)" [High,Triaged] https://launchpad.net/bugs/45419 [17:15] gary_poster should know that ^ [17:15] Foundations does not have that anywhere in our plans ATM. [17:15] right.. [17:15] gary_poster: ok, fair enough, then [17:16] anyone else has questions to Chex? [17:16] Chex, that doesn't mean you can't/shouldn't make a stink about that fact :-) [17:16] If this is a foundations issue, we schema a behavioural changes made near/on IMessage so that all types of messages get the desired feature [17:16] gary_poster: aha, ok then. It is a time-consuming task for us, and having a automated spam-reporting/tagging feature would be highly useful to us [17:17] ok, moving on [17:17] thanks Chex [17:17] [TOPIC] * DBA report (stub) [17:17] Already requested [17:17] New Topic: * DBA report (stub) [17:17] [TOPIC] * QA stats of the week [17:17] New Topic: * QA stats of the week [17:17] I followed up the bugs' ownership in the mailing list [17:18] Chex: spam-reporting: mars and sinzui have the expertise here. I'll try to coordinate with them. [17:18] I had a plan [17:18] sinzui, it's because you're a Cylon. [17:18] a comprehensive one infact [17:18] Was it a cunning plan? [17:19] That would explain why I can do so long without sleep [17:19] heheh [17:19] It had a tail like a fox [17:19] * gary_poster chuckles [17:19] Any more pop-culture references we can fit in here? [17:19] hahaha [17:19] We seem to have missed monthy python [17:19] Monty Python [17:20] I will include that in my email to gary_poster and maris [17:20] thanks sinzui hehe [17:20] thanks sinzui [17:20] moving on again then [17:20] [TOPIC] * Proposed items [17:20] New Topic: * Proposed items [17:20] I have one [17:21] we know that this meeting's format isn't ideal, and we couldn't find a way to change it to become really useful [17:21] we also realized the qa contacts initiative isn't that effective as we thought in the beginning [17:21] Ursinha, for the code team, the only things that are really useful for us are the DB and LOSA reports. [17:21] rockstar, yeah, I'm getting there :) [17:22] "Shutting up sir" [17:22] haha no [17:22] so, after talking with flacoste, we decided to get rid of this meeting as we know [17:22] I care about the oopses actually. Since I read them every day I like to know when they will be closed or provide info about how to close them [17:22] pause for cheering and etc [17:22] Ursinha, so we're doing away with it entirely? [17:23] rockstar, yeah [17:23] *golfclap* [17:23] sinzui, as a replacement, there will be created a section in the TLs meeting, a QA one [17:23] and me and matsubara should attend [17:23] we'll keep discussing that with the TLs, so sinzui, you'll still know about the oopses [17:23] sweet. I wonder if the meeting will go back over an hour again [17:24] and bigjools, you won't get rid of us [17:24] *evil laugh* [17:24] :P [17:24] :-) [17:24] :) [17:24] about the LOSAs and DBA report [17:24] a weekly report should be sent to the list, so we'll all keep posted about it [17:24] so, that's it [17:25] ok, sounds good, I can do that. [17:25] Ursinha, maybe we can have a [QA] subject addition for QA related items. [17:25] thanks a lot Chex, losa's reports are really important [17:25] (I suspect I'll still have to deal with QA stuff within my team) [17:26] rockstar, well, it should be dealt with whoever attends the TLs meeting in behalf of code [17:26] Ursinha: yes I understand that, it helps us, too. I will try to pop my head in on the TLS meetings, too, [17:26] is that you? [17:26] Ursinha, no. [17:26] rockstar, so I guess you will only have to worry about your own QA from now on [17:26] Ursinha, okay. [17:27] that's all then [17:27] anyone else wants to add something? [17:28] okay [17:28] so, it was very nice to have you for these two years in this weekly meeting [17:28] thanks all [17:28] thank you Ursinha :-) [17:28] Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. [17:28] bye [17:28] #endmeeting [17:28] Meeting finished at 11:28. === matsubara is now known as matsubara-lunch [17:45] sinzui, I thought the irc meeting was postponed in one hour to avoid conflicting with the TLs call [17:46] Ursinha, oh yes, it was. We changed the meeting format a short while ago to avoid the extra meeting on Thursday [17:46] sinzui, oh, I see === matsubara-lunch is now known as matsubara === Ursinha is now known as Ursinha-food === Ursinha-food is now known as Ursinha === matsubara is now known as matsubara-afk