=== jelmer_ is now known as jelmer
bacwgrant: i misspoke.01:08
bacbac@duckworth:~$ cat /etc/lsb-release01:08
bacDISTRIB_DESCRIPTION="Ubuntu 8.04.4 LTS"01:08
=== jelmer_ is now known as jelmer
=== noodles785 is now known as noodles775
=== salgado_ is now known as salgado
=== salgado is now known as salgado-lunch
MootBotMeeting started at 11:00. The chair is matsubara.16:00
MootBotCommands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]16:00
matsubaraWelcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues.16:00
matsubara[TOPIC] Roll Call16:00
MootBotNew Topic:  Roll Call16:00
matsubaraNot on the Launchpad Dev team? Welcome! Come "me" with the rest of us!16:00
matsubarachex, gary_poster, rockstar, bigjools, allenap: hi16:01
gary_posterme, sorry16:01
* rockstar are here16:01
bigjoolsI meed earlier16:01
matsubarasorry bigjools, missed that16:01
matsubaraapologies from Translations. they're sprinting16:01
bigjoolseasily missed I was premature :)16:02
matsubarahi sinzui16:05
matsubaraok, everyone is here16:05
matsubara[TOPIC] Agenda16:05
MootBotNew Topic:  Agenda16:05
matsubara * Actions from last meeting16:05
matsubara * Oops report & Critical Bugs & Broken scripts16:05
matsubara * Operations report (mthaddon/Chex/spm/mbarnett)16:05
matsubara * DBA report (stub)16:05
matsubara * Proposed items16:05
matsubara[TOPIC] * Actions from last meeting16:05
MootBotNew Topic:  * Actions from last meeting16:05
matsubara * Ursinha to send email to stub about the dba report16:06
matsubara * sinzui to follow up in work with losas to set the increase threshold for the product release finder monitoring script16:06
matsubara * matsubara to talk to gary_poster about bug 53507116:06
matsubara   * Done.16:06
matsubara * Ursinha to send email to TLs about QA summaries16:06
ubottuLaunchpad bug 535071 in launchpad-foundations "Better error handling when librarian is offline" [High,Triaged] https://launchpad.net/bugs/53507116:06
matsubara * matsubara to triage open oops-tools bugs16:06
matsubara * Ursinha to triage open lp-qa-tools bugs16:06
matsubaraI did some triage for oops-tools bugs, still some to look at16:06
Ursinhamatsubara: done the 1st and the 3rd, not the 2nd16:06
sinzuimatsubara: I have instructions from spm regarding how to change the window or schedule. I will try to land a change next week16:07
matsubara[action] * Ursinha to send email to TLs about QA summaries16:07
MootBotACTION received:  * Ursinha to send email to TLs about QA summaries16:07
matsubara[action]  * matsubara to triage open oops-tools bugs16:07
MootBotACTION received:   * matsubara to triage open oops-tools bugs16:07
matsubarathanks sinzui and Ursinha16:07
Ursinhanp matsubara16:08
matsubarasinzui, should I add the action item again or can I consider it as done?16:08
sinzuimatsubara: yes. pease do16:08
matsubara[action]  * sinzui to follow up in work with losas to set the increase threshold for the product release finder monitoring script16:09
MootBotACTION received:   * sinzui to follow up in work with losas to set the increase threshold for the product release finder monitoring script16:09
matsubara[TOPIC] * Oops report & Critical Bugs & Broken scripts16:09
MootBotNew Topic:  * Oops report & Critical Bugs & Broken scripts16:09
matsubaraWe're still seeing https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1541O1653 which seems to be a regression of bug 52798516:10
ubottuLaunchpad bug 527985 in launchpad-foundations "backing out of openid dance creates oops" [High,Fix released] https://launchpad.net/bugs/52798516:10
matsubaraanother one that seems to be a regression: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1545E465 --> bug 40873816:10
ubottuLaunchpad bug 408738 in malone "OOPS when rendering bug activity" [High,Fix released] https://launchpad.net/bugs/40873816:10
matsubaraand https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1544A949 regression of bug 50830216:10
ubottuLaunchpad bug 508302 in malone "NotImplementedError OOPS when reporting a bug" [High,Fix released] https://launchpad.net/bugs/50830216:10
matsubarasinzui, timeout retrieving vouchers: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1543EA744, I remember a bug about that but couldn't find it16:11
matsubaranot sure if it's the same thing16:11
sinzuimatsubara: that was brad and myself16:11
sinzuimatsubara: ISD is still making changes to services we depend on16:11
matsubaragary_poster, can you look at foundations one ^?16:11
sinzuimatsubara: at this time we are fortunate that only bac and myself are affected16:12
matsubaraallenap, can you take a look at the two bugs above?16:12
* allenap looks16:12
bacsinzui: argh16:13
matsubarasinzui, so should we just ignore those oopses for now? is there a bug that we could link to that oops report?16:13
* sinzui read the oopses and knows what bac is up to during his work hours16:13
sinzuimatsubara: I have not had a problem in the last two days.16:14
sinzuimatsubara: If it persists I will report a bug on the correct ISD project,16:14
bacsinzui, matsubara : i see that user's project has had a voucher applied so perhaps it was spurious.  have we heard from him?16:14
matsubarabac, nope, I caught that in the oops summary16:15
sinzuimatsubara: bac: this was a networking issue with saleforce. The reason you could not find the bug is that ISD retargeted the other bugs to their porjects16:16
allenapmatsubara: I'll file a new bug for OOPS-1545E465; the linked bug is not right.16:16
matsubarasinzui, looks like the oops is triggerable by regular users as well16:16
sinzuimatsubara: anyone using +vouchers is a regular user16:16
allenapmatsubara: I'll talk to deryck about OOPS-1544A949.16:16
matsubara[action] allenap to discuss with deryck about OOPS-1544A949 and file a bug for OOPS-1545E465.16:17
MootBotACTION received:  allenap to discuss with deryck about OOPS-1544A949 and file a bug for OOPS-1545E465.16:17
sinzuimatsubara:  the oops is not in launchpad, it is salesforce. We started talking to ISD last month when vouchers broke16:17
matsubara[action] matsubara to fix linked bug for OOPS-1545E46516:17
MootBotACTION received:  matsubara to fix linked bug for OOPS-1545E46516:17
gary_postermatsubara: I'll be discussing my oops with salgado and filing a new bug or something16:17
matsubaraallenap, gary_poster, thanks16:18
gary_poster(s/or something/or reopening the old one)16:18
matsubara[action] gary_poster to discuss OOPS-1541O1653 with salgado and file a bug for it.16:18
MootBotACTION received:  gary_poster to discuss OOPS-1541O1653 with salgado and file a bug for it.16:18
matsubarasinzui, in any case, lp should degrade gracefully when salesforce is offline or timing out16:19
sinzuiDUDE! I know that16:19
sinzuiThis is not the channel to discuss ISD's changes to systems that affect launchpad and landscape16:20
matsubaraok lebowski, I know that you know that. just making sure we're in the same page. I'm more interested in the changes that we can make to avoid the oops from happening (i.e. degrade gracefully)16:21
sinzuiI favor removing salesforce. It is not my decision. We can evaluate our dependencies on outsiders when we start NewCo16:22
matsubarasinzui, rather than removing salesforce, can't we show a page saying that LP couldn't talk to salesforce while the duderino was browsing +vouchers? much like as your proposed solution to bug 267852?16:26
ubottuLaunchpad bug 267852 in launchpad-foundations "google search timeouts are not usable" [Low,Triaged] https://launchpad.net/bugs/26785216:26
sinzuimatsubara: because it is an oops. there is nothing to show nor any action to take. I could not license Canonical projects for an hour16:27
bigjoolssigh - s/note/not/16:27
matsubarasinzui, makes sense then. I'll let you know if I see more of these OOPS and then you can escalate with ISD. thanks sinzui16:29
matsubarawe have 5 critical bugs16:29
matsubara3 fix committed, one in progress and one confirmed16:30
matsubarathe one confirmed is assigned to me and I'll take care of it today16:30
matsubarathe only broken script is flag-expired-memberships, which stub will take care of16:31
matsubarathanks everyone, let's move on16:31
matsubara[TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)16:31
MootBotNew Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)16:31
* matsubara hands the mic to chex 16:32
chexhello everyone16:33
chexhere is the LOSA report for this week:16:33
chex- New ImportD server 'pear' has been setup this week and is processing jobs now.16:33
chex- PQM is closing at 2300 UTC Friday for LP roll-out next week.16:33
chex- SSO DB Migration project away from LP DB is delayed due to some issues, does anyone know16:33
chex        where we are placed with this now?16:33
chex- LP incidents of note: ; LP Cherry-picks:16:33
chex        22-Mar: CP 9095 to lpnet*, & codehost16:33
chex        24-Mar: CP 9097 to loganberry/checkwatches and 9098 to shipit*16:33
chexand thats the report for this week, any comments/questions??16:33
matsubaragary_poster, do you know about the SSO DB migration ^?16:34
gary_posterchex, matsubara, yes16:34
gary_posterchex, losas were cc'd on relevant emails16:34
gary_postersummary: um...16:35
gary_posterother Canonical openid apps that relied on launchpad team membership for privileges were going to break.16:35
gary_posterWe have to handle that.16:36
gary_posterA plan is now in place to do so16:36
gary_posterIt will involve some DB changes that stub is working on, and some changes for ISD16:36
chexgary_poster: ok, great, sounds good then, thanks16:37
gary_posterWe regard splitting the by no later than two weeks from now as an absolute necessity, because of various pressures such as HD space and other excitement, and hope to have it addressed sooner (beginning of week after next)16:37
matsubarathanks gary_poster, chex16:40
matsubara[TOPIC] * DBA report (stub)16:40
MootBotNew Topic:  * DBA report (stub)16:40
matsubaraI'll email stub for the dba report16:41
matsubara[action] matsubara to email stub asking the dba report16:41
MootBotACTION received:  matsubara to email stub asking the dba report16:41
matsubara[topic] QA Stats of the Week16:41
MootBotNew Topic:  QA Stats of the Week16:41
matsubaraSoyuz (soyuz): 7016:41
matsubaralaunchpadlib  (launchpadlib): 2516:41
matsubaraOOPS Tools (oops-tools): 2116:41
matsubaraLaunchpad Developer Utilities (lp-dev-utils): 916:41
matsubaraLaunchpad Bugs (malone): 816:41
matsubaraLaunchpad Development Wiki Moin theme (launchpad-dev-moin-theme): 516:41
matsubaraLaunchpad Auto Build System (launchpad-buildd): 516:41
matsubaraLaunchpad Buildbot Configuration (lpbuildbot): 216:41
matsubarathose are the launchpad projects with untriaged bugs (new state)16:42
bigjoolsI'd like to point out that most of the soyuz ones are >2 years old16:42
matsubaraclose them all or mark them as triage low :-)16:42
matsubaratriaged, I mean16:42
bigjoolsI do so when I have time16:42
matsubarathanks bigjools16:43
bigjoolsI spend more time triaging new ones16:43
matsubaraAlso please take a look at https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.03-db-devel and https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.03-devel16:43
sinzuibigjools: if you have ignored the issue for more than 6 months, it is 99% of the time low/triaged. the remaining 1% are security issues we should have fixed16:43
matsubarathose are the landings for the 10.03 cycle which are not tracked in the http://people.canonical.com/~lpqateam/test-plan-report-10.03.html burndown16:44
bigjoolssinzui: many of them are simply not bugs any more, in fact16:44
sinzuibigjools: yep16:44
* sinzui closed 1/3 of the bugs as fixed that were supposed to go to launchpad-web16:44
matsubara[TOPIC] * Proposed items16:45
MootBotNew Topic:  * Proposed items16:45
matsubarathere's no new proposed items16:45
matsubaraanything else before I close?16:45
matsubaraThank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda  for the logs.16:45
MootBotMeeting finished at 11:46.16:46
bigjoolsthanks matsubara16:46
=== Ursinha is now known as Ursinha-lunch
=== salgado-lunch is now known as salgado
=== Ursinha-lunch is now known as Ursinha
=== Ursinha is now known as Ursinha-sprintin
=== Ursinha-sprintin is now known as Ursinha-busy
=== salgado is now known as salgado-afk
=== Peng___ is now known as Peng_

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!