=== danilo-afk is now known as danilos === mrevell is now known as mrevell-lunch === mrevell-lunch is now known as mrevell [15:00] #startmeeting [15:00] Meeting started at 09:00. The chair is matsubara. [15:00] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [15:00] Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. [15:00] [TOPIC] Roll Call [15:00] New Topic: Roll Call [15:00] me [15:00] me [15:00] me [15:01] rockstar, Chex, allenap: hi [15:01] brb [15:01] me === salgado is now known as salgado-lunch [15:03] ok, let's move on, others can join in along the way [15:04] [TOPIC] Agenda [15:04] New Topic: Agenda [15:04] * Actions from last meeting [15:04] * Oops report & Critical Bugs & Broken scripts [15:04] * Operations report (mthaddon/Chex/spm/mbarnett) [15:04] * DBA report (stub) [15:04] * Proposed items [15:04] [TOPIC] * Actions from last meeting [15:04] New Topic: * Actions from last meeting [15:04] * gary_poster to try to land branch that cherry picks lazr.restful updates on lpnet for bug 331990 [15:04] * Ursinha to ask matsubara about his outstanding items from last meeting [15:04] * Ursinha to ask translations to triage bug 462891 [15:04] * Ursinha asked henninge in the meeting [15:04] * allenap to follow up with pitti about bug 453203 [15:04] * Ursinha to send one email to lp list explaining the qa-tags experiment [15:04] Launchpad bug 331990 in launchpad-foundations "The inline editor widget reports a JSON error when saving non-ASCII characters" [High,Fix committed] https://launchpad.net/bugs/331990 [15:04] Launchpad bug 462891 in rosetta "TraversalError on +export" [High,Triaged] https://launchpad.net/bugs/462891 [15:04] Launchpad bug 453203 in apport "UnicodeDecodeError in +filebug: unexpected code byte" [Undecided,New] https://launchpad.net/bugs/453203 [15:04] also bug 462891 is being discussed in the ml [15:05] Ehrm, I haven't done bug 453203 yet. Sorry, it's still on my list. [15:05] and I started working on the wiki page to then send the email, but haven't finished [15:05] will do [15:05] [action] Ursinha to send one email to lp list explaining the qa-tags experiment [15:05] [action] Ursinha to send one email to lp list explaining the qa-tags experiment [15:05] ACTION received: Ursinha to send one email to lp list explaining the qa-tags experiment [15:06] [action] allenap to follow up with pitti about bug 453203 [15:06] ACTION received: allenap to follow up with pitti about bug 453203 [15:06] Launchpad bug 453203 in apport "UnicodeDecodeError in +filebug: unexpected code byte" [Undecided,New] https://launchpad.net/bugs/453203 [15:06] the one from gary_poster is probably done now that we rolled out, right? [15:07] * matsubara tries to remember his outstanding items from the last meeting [15:08] * allenap and matsubara to talk about the timeouts on bug pages and * matsubara to talk to stub about the DBA report when he gets back [15:08] Ursinha: I reported bug 475433 oops leaving milestone code_name empty today. This edit form was broken by thumper's changes to StrippedTextLine last week. [15:08] Launchpad bug 475433 in launchpad-registry "oops leaving milestone code_name empty" [High,Triaged] https://launchpad.net/bugs/475433 [15:08] thanks sinziu [15:08] meep [15:08] sinzui, [15:09] ok, I did talk to allenap about timeouts on bug pages [15:09] matsubara: We didn't really come to any conclusions iirc. [15:09] matsubara: yes, should be rolled out [15:09] and I sent an email to stub about the DBA report while he was on vacation [15:10] There is generally little to report about databases when I'm on vacation. I could send some holiday snaps if it would make people happy. [15:10] allenap, yeah. let's see if after this rollout timeouts will improve [15:10] stub, if it's like the one sinzui sent a long time ago, it'd be hilarious [15:11] matsubara: Okay. I've back to Bugs as usual next week so I hope to have more time to look at this. [15:11] allenap, thanks [15:11] let's move on. thanks everyone [15:11] Does it involve hemorrhoid cream? [15:12] [TOPIC] * Oops report & Critical Bugs & Broken scripts [15:12] New Topic: * Oops report & Critical Bugs & Broken scripts [15:12] LOL [15:12] I'm not able to see the oopses from after the rollout yet due to an oops-tools problem, they're being generated since this morning, yet not finished [15:12] the issues I'm aware are all being handled, scripts failing and this bug on milestone sinzui reported [15:13] we have one critical bug on launchpad-code which is in the New state [15:13] and rockstar is not around [15:13] the other one is on launchpad-foundations and it's fix committed [15:14] sinzui, this milestone one is a candidate for reroll? [15:14] umm [15:14] I do not know. I suppose it is since I had to read the code to know that I had to enter a value to make my change work [15:15] sinzui, if you think it's not :) [15:15] Ursinha: The fix is easy and I can get it reviewed today. It is certainly good for a CP [15:15] sinzui, alright [15:16] I'd ask you if you are working on bugs for the reroll (if it happens), please tag your bugs with a current-rollout-blocker tag [15:16] thumper, if you're around can you update https://bugs.edge.launchpad.net/launchpad-code/+bug/475394 status and leave a comment explaining what's up and what's being done about it? [15:16] Ubuntu bug 475394 in launchpad-code "bzr http access broken after 3.1.10 rollout" [Critical,New] [15:17] matsubara: abentley is currently working with mthaddon on #lp-code [15:17] * noodles775 goes again. [15:17] noodles775, thanks [15:17] I was, but he quit on me :( [15:19] some scripts failing likely to be due to the rollout [15:19] garbo-hourly failing since some time ago due to https://bugs.edge.launchpad.net/launchpad-foundations/+bug/464161 and https://bugs.edge.launchpad.net/launchpad-foundations/+bug/464155 [15:19] Ubuntu bug 464161 in launchpad-foundations "OpenIDAssociationPruner is using the wrong Store" [High,Confirmed] [15:19] stub, shouldn't the latter be more important than medium? [15:19] garbo isn't serious thankfully - we can cope. [15:20] other scripts might be affected by the same bug though (at least one has been reported), so it is my priority. [15:20] thanks stub [15:21] (464161 is the priority - it might even be the cause of the other bug) [15:21] I see [15:21] Ursinha, anything else on this section? [15:21] matsubara, no, not from me [15:21] thanks everyone [15:21] moving on [15:21] [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) [15:21] New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) [15:22] LP rollout today, and some ongoing issues because of that (see "Cowboys" section of LPS) [15:22] Can we get someone to review the IncidentLog and fill in the QA section? [15:22] I think that's about it [15:23] [action] matsubara to talk to francis about the IncidentLog QA section [15:23] ACTION received: matsubara to talk to francis about the IncidentLog QA section [15:23] thanks mthaddon [15:23] mthaddon, do you know if we're considering a reroll? [15:23] maybe noodles775 know [15:24] as he's the almighty release manager :) [15:24] Ursinha: he's the *tired* RM :) [15:24] we'll definitely need a reroll, as we already have a bunch of stuff cowboyed (per the LPS wiki page) [15:25] https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus [15:25] to help who wants to see it :) [15:26] anyway, I'll check up the oops today and see if we have serious issues that should be fixed before the reroll [15:27] thanks Ursinha [15:27] let's move on [15:27] [TOPIC] * DBA report (stub) [15:27] New Topic: * DBA report (stub) [15:27] Set to go with testing of the rosetta message sharing data migration. Looks like this might shrink the size of the big rosetta datasets by 30+%. W00t. [15:27] There are not many databases in Angkor, but my mother had a lovely time anyway. [15:27] The code that needed to land to get staging DB restores back under way landed. Early next week baring distractions I hope. [15:27] First test of the sanitized-database dump generator will be kicked off tomorrow. [15:27] It would be good if people can look over lib/lp/scripts/utilities/sanitizedb.py to see if I've missed anything in the sections of the schema they are familiar with. There should be a dump for Canonical devs to run on ec2 or at home if they are brave next week. [15:28] (The dumps should remain in company - there may be leaks in there now and because we use a blacklist approach, leaks can appear in the future. At least until teams have signed off on what we have and we have some sort of process for maintaining the tool). [15:28] oot. [15:29] [action] qa contacts to look into lib/lp/scripts/utilities/sanitizedb.py see if stub missed anything in the sections of the schema they are familiar with [15:29] ACTION received: qa contacts to look into lib/lp/scripts/utilities/sanitizedb.py see if stub missed anything in the sections of the schema they are familiar with [15:30] stub, can you add one person from each team to the merge proposal that includes that script? that way people will look into it easily [15:30] Its already landed [15:31] hehe, oops :) [15:31] Anything people find has been missed is a bug. Anything removed that shouldn't be is a bug. [15:31] ok, so the action stands for everyone, please pass the word to your teams [15:32] ok [15:32] Its all simple to follow and in small, discreet steps. [15:32] let's move on [15:32] cool. I'll also email the list to remind people [15:33] [action] matsubara to email the list about lib/lp/scripts/utilities/sanitizedb.py script [15:33] ACTION received: matsubara to email the list about lib/lp/scripts/utilities/sanitizedb.py script [15:33] thanks stub [15:33] [TOPIC] * Proposed items [15:33] New Topic: * Proposed items [15:33] no proposed items [15:34] anything else before I close [15:34] ? [15:34] 3 [15:34] 2 [15:34] 1 [15:34] Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. [15:34] thanks [15:34] #endmeeting [15:34] Meeting finished at 09:34. [15:36] thanks everyone === matsubara is now known as matsubara-lunch === salgado-lunch is now known as salgado === matsubara-lunch is now known as matsubara === salgado is now known as salgado-afk === matsubara is now known as matsubara-afk