| === danilo__ is now known as danilos | ||
| matsubara | #startmeeting | 16:00 |
|---|---|---|
| MootBot | Meeting started at 10:00. The chair is matsubara. | 16:00 |
| MootBot | Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] | 16:00 |
| matsubara | Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. | 16:00 |
| matsubara | [TOPIC] Roll Call | 16:00 |
| MootBot | New Topic: Roll Call | 16:00 |
| stub | me | 16:00 |
| matsubara | Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! | 16:00 |
| mbarnett | me | 16:00 |
| Chex | me | 16:00 |
| henninge | me | 16:00 |
| Ursinha | me | 16:00 |
| matsubara | rockstar, allenap: hi | 16:01 |
| allenap | me | 16:01 |
| matsubara | noodles775, hi, can you stand in for bigjools ? | 16:02 |
| Ursinha | allenap, are you here in behalf of bugs team? | 16:02 |
| noodles775 | matsubara: yep. | 16:02 |
| matsubara | Ursinha, allenap is the new QA contact | 16:02 |
| allenap | Ursinha: Yes. | 16:02 |
| Ursinha | cool | 16:02 |
| Ursinha | thanks | 16:02 |
| matsubara | thanks noodles775 | 16:02 |
| matsubara | we're misssing someone from code and registry. let's move on and they can join in later | 16:03 |
| matsubara | [TOPIC] Agenda | 16:03 |
| MootBot | New Topic: Agenda | 16:03 |
| matsubara | * Actions from last meeting | 16:04 |
| matsubara | * Oops report & Critical Bugs & Broken scripts | 16:04 |
| matsubara | * Operations report (mthaddon/Chex/spm/mbarnett) | 16:04 |
| matsubara | * DBA report (stub) | 16:04 |
| matsubara | * Proposed items | 16:04 |
| matsubara | [TOPIC] * Actions from last meeting | 16:04 |
| MootBot | New Topic: * Actions from last meeting | 16:04 |
| matsubara | * barry to continue debug on bug 403606 after finishing 3.0 UI stuff | 16:04 |
| matsubara | * matsubara to trawl logs related to high load on edge yesterday and ping Chex about it | 16:04 |
| matsubara | * matsubara to update translations qa contact on meeting agenda page | 16:04 |
| matsubara | * done | 16:04 |
| matsubara | * matsubara to generate new oops summary including oopses only for after the rollout | 16:04 |
| matsubara | * done | 16:04 |
| ubottu | Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,In progress] https://launchpad.net/bugs/403606 | 16:04 |
| matsubara | I still haven't trawled those logs. :-( | 16:04 |
| matsubara | [action] * matsubara to trawl logs related to high load on edge yesterday and ping Chex about it | 16:04 |
| MootBot | ACTION received: * matsubara to trawl logs related to high load on edge yesterday and ping Chex about it | 16:04 |
| matsubara | and I know that barry is working on more debugging for 403606. some code was cowboyed yesterday in the production server to help with this debugging | 16:05 |
| matsubara | so I won't re-add that action item | 16:06 |
| matsubara | [TOPIC] * Oops report & Critical Bugs & Broken scripts | 16:06 |
| MootBot | New Topic: * Oops report & Critical Bugs & Broken scripts | 16:06 |
| Ursinha | me me | 16:06 |
| matsubara | go ahead | 16:06 |
| Ursinha | two bugs | 16:07 |
| Ursinha | allenap, bug 438985, it's happening quite frequently, do you think it's going to be fixed soon? | 16:07 |
| ubottu | Launchpad bug 438985 in malone "Trying to make myself as bug supervisor of my project oopses" [High,Triaged] https://launchpad.net/bugs/438985 | 16:07 |
| Ursinha | matsubara, who's here for foundations | 16:07 |
| Ursinha | ? | 16:07 |
| matsubara | I am | 16:07 |
| matsubara | what's up? | 16:07 |
| Ursinha | :D | 16:07 |
| Ursinha | since last meeting we're having a lot of DisconnectionErrors, about 200 a day | 16:07 |
| Ursinha | I'd suggest bug 436640 to be fixed and cherrypicked, so we could understand what's happening | 16:07 |
| ubottu | Launchpad bug 436640 in launchpad-foundations "Add more info to DisconnectionErrors" [Undecided,New] https://launchpad.net/bugs/436640 | 16:07 |
| Ursinha | is that hard to accomplish? What do you think? | 16:07 |
| allenap | Ursinha: I'll see if we can have a go at it this week. | 16:08 |
| Ursinha | matsubara, also, could you triage that | 16:08 |
| Ursinha | ? | 16:08 |
| Ursinha | pleasse | 16:08 |
| Ursinha | please | 16:08 |
| Ursinha | thanks allenap | 16:08 |
| matsubara | Ursinha, gary was working on it last I heard. I'll ask him once he comes back from the TL meeting | 16:08 |
| matsubara | and yes, I'll have it triaged and raise the importance | 16:08 |
| Ursinha | matsubara, triaged or in progress? | 16:08 |
| matsubara | triaged, I don't know if gary started the patch yet. no point in having a in progress bug without assignee :-) | 16:09 |
| matsubara | [action] matsubara to talk to gary about bug 436640 | 16:09 |
| MootBot | ACTION received: matsubara to talk to gary about bug 436640 | 16:09 |
| ubottu | Launchpad bug 436640 in launchpad-foundations "Add more info to DisconnectionErrors" [Undecided,New] https://launchpad.net/bugs/436640 | 16:09 |
| Ursinha | matsubara, ok, I thought he was actually working on that | 16:10 |
| Ursinha | matsubara, thanks | 16:10 |
| Ursinha | we have one critical bug, bug 438039, and henninge is already working on it, so ok | 16:10 |
| ubottu | Launchpad bug 438039 in rosetta "bzr branch import script oopses sometimes" [Critical,In progress] https://launchpad.net/bugs/438039 | 16:10 |
| Ursinha | about failing scripts, I'd need rockstar here | 16:10 |
| Ursinha | rockstar, what's happening with update_review_diffs? | 16:10 |
| matsubara | Ursinha, he showed me some example code to add more info, but I'm not sure if he started it or not. I'll confirm with him before making any changes | 16:10 |
| henninge | Ursinha: that has been cherry picked, actually. | 16:10 |
| * henninge goes to check if the oopses still occur | 16:11 | |
| Ursinha | henninge, even better :) can you set the bug status properly, please? | 16:11 |
| henninge | Ursinha: just done that | 16:12 |
| matsubara | Ursinha, please follow up with rockstar about that script after the meeting. | 16:12 |
| Ursinha | matsubara, ok, can you add an action item, please? | 16:12 |
| Ursinha | thanks henninge :) | 16:12 |
| matsubara | [action] Ursinha to talk to rockstar about failing update_preview_diffs script | 16:13 |
| MootBot | ACTION received: Ursinha to talk to rockstar about failing update_preview_diffs script | 16:13 |
| Ursinha | thanks :) | 16:13 |
| Ursinha | thanks guys | 16:13 |
| Ursinha | matsubara, stage is yours again | 16:13 |
| matsubara | [action] allenap to talk to Bugs team about bug 438985 and have it fixed. | 16:14 |
| MootBot | ACTION received: allenap to talk to Bugs team about bug 438985 and have it fixed. | 16:14 |
| ubottu | Launchpad bug 438985 in malone "Trying to make myself as bug supervisor of my project oopses" [High,Triaged] https://launchpad.net/bugs/438985 | 16:14 |
| matsubara | thanks Ursinha and everyone | 16:14 |
| matsubara | [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) | 16:14 |
| henninge | Ursinha: oopses are still there ... :( Back to 'in progress' | 16:14 |
| MootBot | New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) | 16:14 |
| Chex | hello everyone, here is the report for this week | 16:15 |
| Chex | Automated LP Cherry Picks: We've migrated to hosting the production LP | 16:15 |
| Chex | branches on LP itself and are using PQM to manage production-devel. LP | 16:15 |
| Chex | developers can now submit their own Cherry Picks making the deploy of updates | 16:15 |
| Chex | easier. | 16:15 |
| Chex | - Super-Mirror puller stability: We have setup a nagios check to watch this | 16:15 |
| Chex | service, and it has been more stable recently. | 16:15 |
| Chex | anyone have any questions?? | 16:15 |
| matsubara | henninge, when was the fix CP'ed? could it be that those oopses happened before the CP? | 16:15 |
| noodles775 | Chex: yeah, do we just simply pqm-submit targeting production-devel (after getting permission of course)? or is there some documentation? | 16:16 |
| noodles775 | That sounds great! | 16:17 |
| matsubara | indeed | 16:17 |
| henninge | matsubara: matsubara unfortunately no, there are some dated today ... | 16:17 |
| matsubara | henninge, :-( | 16:18 |
| Chex | noodles775: I believe you can just do that pqm-submit , but I would need to check to be sure on that.. | 16:19 |
| noodles775 | Chex: k, thanks! | 16:19 |
| matsubara | Chex, could you email the list with your findings and perhaps a link to a wiki page explaining the process? | 16:19 |
| Chex | matsubara: yes sure, I will do that. | 16:20 |
| matsubara | [action] Chex to email the list about the automated LP cherry pick process | 16:20 |
| MootBot | ACTION received: Chex to email the list about the automated LP cherry pick process | 16:20 |
| matsubara | thanks a lot Chex | 16:20 |
| matsubara | moving on | 16:21 |
| matsubara | [TOPIC] * DBA report (stub) | 16:21 |
| MootBot | New Topic: * DBA report (stub) | 16:21 |
| stub | It looks like we will be moving the master server for the auth replication set to new hardware as part of the continuing migration of the authentication service to the ISD team. Not sure of schedules yet. There will likely need to be a few minutes downtime while we switch masters. | 16:21 |
| stub | Load on the databases actually seems a bit lighter with Launchpad 3.0 which is good. No real metrics so this just might be wishful thinking, but I suspect less random statistics scattered around the page and fewer large document reloads due to ajax is helping. | 16:21 |
| stub | Nothing else interesting happening. | 16:21 |
| matsubara | thanks stub | 16:23 |
| matsubara | [TOPIC] * Proposed items | 16:23 |
| MootBot | New Topic: * Proposed items | 16:23 |
| matsubara | no new proposed items | 16:23 |
| matsubara | anything else before I close? | 16:24 |
| matsubara | 4 | 16:24 |
| matsubara | 3 | 16:24 |
| matsubara | 2 | 16:24 |
| matsubara | 1 | 16:24 |
| matsubara | Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. | 16:24 |
| matsubara | #endmeeting | 16:24 |
| MootBot | Meeting finished at 10:24. | 16:24 |
| Ursinha | thanks everyone | 16:25 |
| mthaddon | noodles775: you need to get permission as you normally would, and then you can submit to PQM | 16:27 |
| mthaddon | (as you normally would for a cherry pick request, that is) | 16:28 |
| noodles775 | mthaddon: great... using --submit-branch etc. Thanks! | 16:28 |
| mthaddon | yep | 16:29 |
| === matsubara is now known as matsubara-lunch | ||
| === Ursinha is now known as Ursinha-afk | ||
| === matsubara-lunch is now known as matsubara | ||
| === Ursinha-afk is now known as Ursinha | ||
| === Ursinha is now known as Ursinha-afk | ||
| === matsubara is now known as matsubara-afk | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!