| === salgado-afk is now known as salgado | ||
| === mrevell is now known as mrevell-lunch | ||
| === mrevell-lunch is now known as mrevell | ||
| matsubara | #startmeeting | 15:00 |
|---|---|---|
| MootBot | Meeting started at 09:00. The chair is matsubara. | 15:00 |
| MootBot | Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] | 15:00 |
| matsubara | Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. | 15:00 |
| matsubara | [TOPIC] Roll Call | 15:00 |
| MootBot | New Topic: Roll Call | 15:00 |
| henninge | me | 15:00 |
| intellectronica | me | 15:00 |
| matsubara | Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! | 15:00 |
| herb | me | 15:00 |
| Ursinha | me | 15:00 |
| stub | me | 15:00 |
| matsubara | sinzui, flacoste? | 15:01 |
| rockstar | me | 15:01 |
| flacoste | me | 15:01 |
| matsubara | bigjools: ? | 15:01 |
| bigjools | meeeeeeee | 15:01 |
| sinzui | m,e | 15:01 |
| sinzui | me | 15:01 |
| matsubara | all right, everyone is here | 15:01 |
| matsubara | [TOPIC] Agenda | 15:01 |
| MootBot | New Topic: Agenda | 15:01 |
| * sinzui hates windows that steal focus | 15:01 | |
| matsubara | * Next meeting | 15:01 |
| matsubara | * Actions from last meeting | 15:01 |
| matsubara | * Oops report & Critical Bugs | 15:01 |
| matsubara | * QA pending items | 15:01 |
| matsubara | * Operations report (mthaddon/herb/spm) | 15:01 |
| matsubara | * DBA report (DBA contact) | 15:01 |
| matsubara | [TOPIC] * Next meeting | 15:01 |
| MootBot | New Topic: * Next meeting | 15:01 |
| matsubara | same time next week? | 15:02 |
| matsubara | I'll be on vacation, so Ursinha will take over | 15:02 |
| * bigjools is out for 3 weeks | 15:02 | |
| Ursinha | o/ | 15:02 |
| matsubara | [action] matsubara to add himself and bigjools to the apologies list | 15:02 |
| MootBot | ACTION received: matsubara to add himself and bigjools to the apologies list | 15:02 |
| matsubara | bigjools: can you arrange for someone in soyuz to cover for you? | 15:03 |
| bigjools | matsubara: yep | 15:03 |
| matsubara | bigjools: thanks | 15:03 |
| matsubara | [TOPIC] * Actions from last meeting | 15:03 |
| MootBot | New Topic: * Actions from last meeting | 15:03 |
| matsubara | * matsubara email stub asking for the DBA report | 15:03 |
| matsubara | * matsubara to talk with gmb and add sql to hide bug 2-9-7-3-8-8 as a fix for bug 300324 | 15:03 |
| matsubara | * rockstar to talk with code team about their pending QA items | 15:03 |
| matsubara | * henninge to talk with translation team about their pending QA items | 15:03 |
| matsubara | * Ursinha to talk with bigjools about one account on dogfood machine to do some testing | 15:03 |
| ubottu | Error: Launchpad bug 2 could not be found | 15:03 |
| matsubara | * rockstar to handle bug 271879 today (12-04) | 15:03 |
| matsubara | * Ursinha to watch timeouts in the next days to see if they continue to happen this often | 15:03 |
| ubottu | Launchpad bug 300324 in malone "Bug 297388 OOPSing because had its 0 comment deleted" [High,Triaged] https://launchpad.net/bugs/300324 | 15:03 |
| matsubara | * rockstar to manage having something reported on bug 118625 and bug 156453 | 15:03 |
| matsubara | * Ursinha to watch for strange OOPSes on edge in the next days | 15:03 |
| ubottu | Bug 271879 on http://launchpad.net/bugs/271879 is private | 15:03 |
| ubottu | Launchpad bug 118625 in launchpad-bazaar "codebrowse sometimes hangs" [High,Triaged] https://launchpad.net/bugs/118625 | 15:03 |
| ubottu | Launchpad bug 156453 in loggerhead "production loggerhead branch leaks memory" [Critical,In progress] https://launchpad.net/bugs/156453 | 15:03 |
| Ursinha | oh yeah | 15:03 |
| Ursinha | I've opened a RT to get access to mawson | 15:03 |
| rockstar | matsubara, first item is talked about everyday now. | 15:04 |
| matsubara | I did both of mine and gmb added the query to the the LPPS page effectively hiding the problematic bug | 15:04 |
| Ursinha | matsubara, great | 15:04 |
| Ursinha | I've watched for the oopses and nothing that bizarre came up | 15:04 |
| rockstar | matsubara, the fix for the timeout bug is approved, but we need to do some query testing. | 15:05 |
| matsubara | thanks rockstar. Is the daily nagging helping QA testing? | 15:05 |
| rockstar | matsubara, my team doesn't like nagging, but they're getting to it. | 15:05 |
| matsubara | Ursinha: anything unusual about the strange OOPS and timeouts? | 15:05 |
| Ursinha | matsubara, nothing that's not reported and being taken care of | 15:06 |
| matsubara | henninge: how's the daily nagging about pending QA items going? is it helping your team? | 15:06 |
| henninge | matsubara: Well, currently all items have been tested that could have been. | 15:07 |
| henninge | matsubara: danilo is at UDS right now and jtv doesn't have anything else to test. | 15:07 |
| matsubara | great. thanks henninge | 15:07 |
| henninge | matsubara: we'll keep reminding ourselves ... | 15:08 |
| matsubara | cool | 15:08 |
| matsubara | moving on | 15:08 |
| matsubara | [TOPIC] * Oops report & Critical Bugs | 15:08 |
| MootBot | New Topic: * Oops report & Critical Bugs | 15:08 |
| Ursinha | okay | 15:08 |
| matsubara | Ursinha: do you want me to do it? | 15:08 |
| matsubara | all right, go ahead | 15:08 |
| Ursinha | matsubara, just saw sinzui was talking to salgado about bug 304113 | 15:09 |
| ubottu | Launchpad bug 304113 in launchpad-registry "Milestone:+index page timing out" [High,Triaged] https://launchpad.net/bugs/304113 | 15:09 |
| Ursinha | oops, this wasn't for you matsubara, sorry :) | 15:09 |
| sinzui | Ursinha: we will start working on it in a few hours | 15:09 |
| Ursinha | sinzui, great, thanks | 15:09 |
| Ursinha | henninge, can you ask someone on translations to set priority and importance on bug 302798, please? | 15:10 |
| ubottu | Launchpad bug 302798 in rosetta "Timeout on +translate page" [Undecided,New] https://launchpad.net/bugs/302798 | 15:10 |
| henninge | Ursinha: yes | 15:11 |
| Ursinha | henninge, thank you :) | 15:11 |
| matsubara | priority and importance ? | 15:11 |
| Ursinha | erm | 15:11 |
| matsubara | we have only importance :-) | 15:11 |
| henninge | status and importance | 15:11 |
| Ursinha | status and importance, sorry :) | 15:11 |
| henninge | Ursinha: I knew what you meant ... ;) | 15:12 |
| matsubara | anything else Ursinha ? | 15:12 |
| Ursinha | rockstar, you | 15:12 |
| rockstar | Me?! | 15:12 |
| Ursinha | rockstar, do you know anything about bug 305367 | 15:12 |
| ubottu | Bug 305367 on http://launchpad.net/bugs/305367 is private | 15:12 |
| Ursinha | ? | 15:12 |
| Ursinha | grrr | 15:12 |
| Ursinha | it's a critical one | 15:12 |
| Ursinha | the only one we have | 15:12 |
| Ursinha | it's really critical? | 15:12 |
| rockstar | Ursinha, I don't think they are critical. | 15:13 |
| rockstar | activereviews is for your own code. | 15:13 |
| Ursinha | rockstar, stub marked it critical | 15:13 |
| rockstar | I'm working on the pages to generate a list of reviews you need to vote on. | 15:14 |
| Ursinha | rockstar, right. | 15:14 |
| rockstar | I'm going to bring it down from critical, anh talk with stub. | 15:14 |
| stub | I marked it critical because I couldn't see a work around and might lose things. Feel free to point me at a work around :) | 15:15 |
| Ursinha | rockstar, thank you very much | 15:15 |
| rockstar | stub, no work around. | 15:15 |
| rockstar | stub, I think your people should be contacting you to tell you about them. | 15:15 |
| Ursinha | I'm done after that | 15:15 |
| rockstar | That should be the workaround right now. | 15:15 |
| stub | rockstar: Not really an option with the db review windows - I'll use a wiki page next cycle. | 15:16 |
| rockstar | stub, next cycle you'll have the page you're looking for. I'm working on it now. | 15:16 |
| rockstar | But activereviews is for seeing the code you've proposed for review and its status. | 15:17 |
| stub | Ok. | 15:17 |
| matsubara | all right, moving on. | 15:17 |
| matsubara | thanks everyone | 15:17 |
| Ursinha | thanks guys | 15:17 |
| matsubara | [TOPIC] * QA pending items | 15:17 |
| MootBot | New Topic: * QA pending items | 15:17 |
| Ursinha | right | 15:18 |
| Ursinha | so we have a lot of items to be tested | 15:18 |
| rockstar | We have too many still, and the ones that are getting done aren't getting recorded. | 15:18 |
| rockstar | :/ | 15:18 |
| matsubara | why not? | 15:19 |
| Ursinha | rockstar, this is a problem | 15:19 |
| rockstar | Ursinha, I'll talk with thumper about it. | 15:19 |
| Ursinha | rockstar, you'll lost yourselves with that | 15:19 |
| Ursinha | rockstar, okay. I'm on my late shift today, will talk to thumper as weel | 15:19 |
| Ursinha | *well | 15:19 |
| matsubara | rockstar: what's the issue about recording what's been tested? | 15:20 |
| === salgado is now known as salgado-lunch | ||
| rockstar | matsubara, no idea. I know some of these things have been tested. | 15:20 |
| matsubara | rockstar: right, so it's just a matter of updating the wiki page or something else? | 15:21 |
| rockstar | matsubara, wiki page. | 15:21 |
| matsubara | I see. if the process to update the test plan page is too troublesome we need to find a better way. suggestions welcome. | 15:22 |
| rockstar | matsubara, also, this week has been abnormally busy with the ec2 stuff. It might be that recording it isn't at the front of their minds. | 15:22 |
| rockstar | matsubara, I need to think about that. | 15:22 |
| intellectronica | fwiw there's quite a lot of talk about using launchpad itself to track qa. if anyone has good ideas please let them be known | 15:22 |
| Ursinha | rockstar, thanks. | 15:23 |
| Ursinha | intellectronica, indeed | 15:23 |
| rockstar | intellectronica, yea, I was going to think about that. | 15:23 |
| rockstar | The Ubuntu guys might be happy to have qa handled in Lanuchpad. | 15:23 |
| rockstar | also, Launchpad. | 15:23 |
| intellectronica | so i hear, yes | 15:23 |
| Ursinha | any suggestions/ideas are very welcome | 15:24 |
| matsubara | that's probably something that will happen in the future, until we get to the pojnt we can use LP to track QA, we should be updating the wiki page | 15:24 |
| intellectronica | of course | 15:24 |
| matsubara | I know that's far from perfect but don't let that stop the QA. | 15:24 |
| matsubara | one thing you could do is email Ursinha and I all items tested and we'll take care of updating the test plan | 15:25 |
| Ursinha | and I'm going to repeat myself again, but, please, ping us if we can do something | 15:25 |
| Ursinha | like that matsubara said | 15:25 |
| bigjools | Ursinha: halp | 15:26 |
| matsubara | until we find a better way to deal with this | 15:26 |
| matsubara | thanks | 15:26 |
| matsubara | [TOPIC] * Operations report (mthaddon/herb/spm) | 15:26 |
| MootBot | New Topic: * Operations report (mthaddon/herb/spm) | 15:26 |
| Ursinha | bigjools, as soon as a have access to mawson, will be glad too. but if there's something I could do now, let me know | 15:26 |
| herb | * 2008-12-10 - Cherry picked r7386 enabling replication to production and cherry picked r7419 to edge. | 15:26 |
| Ursinha | s/as a/as I/ | 15:26 |
| herb | * Bugs #118625 and #156453 continue to be a problem despite a patch and the addition of the pyrex package. | 15:26 |
| herb | * Had an issue with the branch scanner where a user's subscription to a particularly large branch was causing the branchscanner to hang. See bug #242807 | 15:26 |
| ubottu | Launchpad bug 118625 in launchpad-bazaar "codebrowse sometimes hangs" [High,Triaged] https://launchpad.net/bugs/118625 | 15:26 |
| herb | * Had to kill off a bunch of old processes on the code hosting server. This seems to be happening a bit more often than once a week now. See bug #260171 | 15:26 |
| ubottu | Launchpad bug 156453 in loggerhead "production loggerhead branch leaks memory" [Critical,In progress] https://launchpad.net/bugs/156453 | 15:26 |
| ubottu | Launchpad bug 242807 in ubuntu "my ipod nano is not showing up when plugged in" [Low,Invalid] https://launchpad.net/bugs/242807 | 15:26 |
| ubottu | Bug 260171 on http://launchpad.net/bugs/260171 is private | 15:27 |
| herb | hmm | 15:27 |
| matsubara | ipod nano? | 15:27 |
| matsubara | hehe | 15:27 |
| Ursinha | lol | 15:27 |
| rockstar | 242807 doesn't seem to correlate. | 15:27 |
| herb | something tells me I typoed at least one of those. | 15:27 |
| herb | should be bug #252807 | 15:27 |
| ubottu | Bug 252807 on http://launchpad.net/bugs/252807 is private | 15:27 |
| rockstar | herb, there are many folken at UDS right now trying to figure out why loggerhead sucks. | 15:29 |
| matsubara | herb: btw, did you find out why staging is down? | 15:29 |
| rockstar | It's disappointing that it's still having issues. | 15:29 |
| matsubara | hmm it's not down anymore, so why staging was down? :-) | 15:29 |
| herb | matsubara: the old lp process didn't exit and so the new process couldn't bind to the port after the restore. | 15:30 |
| Ursinha | herb, iirc it was the second time this week | 15:30 |
| matsubara | herb: ok. thanks for chasing | 15:30 |
| Ursinha | the other time happened for the same reason? | 15:30 |
| herb | Ursinha: 3rd or 4th time in the last 10 days. | 15:30 |
| herb | Ursinha: different reasons the last couple of times. it had been hanging in the middle of startup. | 15:30 |
| Ursinha | hm, right. | 15:30 |
| rockstar | herb, was it the librarian? | 15:31 |
| herb | rockstar: running. but didn't really look at it in too much detail. | 15:31 |
| matsubara | herb: if it happens again, could you file a bug about it please and let me know? | 15:31 |
| rockstar | herb, I've noticed the librarian doesn't close cleanly on my dev machine. | 15:31 |
| herb | matsubara: sure. I can't provide much detail on the problems I've seen over and over though. stracing the startup just shows a wait call. | 15:32 |
| herb | :( | 15:32 |
| matsubara | thanks herb | 15:33 |
| matsubara | [TOPIC] * DBA report (DBA contact) | 15:33 |
| MootBot | New Topic: * DBA report (DBA contact) | 15:33 |
| stub | The losas altered the production appserver configs so they start talking to the production replica where possible. This is working well... possibly too well. | 15:33 |
| stub | Most of the load is now on the slave, with the production box pretty much idling. We can push some of the | 15:33 |
| stub | load back to the main box by reverting a few of the appserver configs, but we should probably leave it be until we have the reverse web proxy caching anonymous requests to see how this affects the load distribution. | 15:33 |
| stub | I will be running the script to open Jaunty translations with Jeroen tomorrow. This should be the last time it is a heavy weight operation, as translation inheritence is scheduled to be implemented before the next opening. No ETAs on the run time completion yet - I don't fully trust them given the extra overhead we have now on inserts thanks to replication. | 15:33 |
| stub | DB patches for this cycle are all either landed or rejected, except for the per language translation policies link with is blocked on discussions upstairs. | 15:33 |
| stub | (^-^) | 15:33 |
| bigjools | stub: did Celso's get accepted or rejected? | 15:34 |
| stub | Which one? | 15:35 |
| bigjools | the licensing meta-data one | 15:35 |
| stub | Oh... licencing sourcepackagereleae. Rejected. Needs mucho discussion. | 15:35 |
| bigjools | blah | 15:35 |
| matsubara | all right, anything else for stub? | 15:36 |
| matsubara | thanks stub | 15:36 |
| matsubara | anything else before I close? | 15:36 |
| matsubara | 5 | 15:36 |
| matsubara | 4 | 15:36 |
| matsubara | 3 | 15:36 |
| matsubara | 2 | 15:36 |
| matsubara | 1 | 15:36 |
| matsubara | Thank you all for attending this week's Launchpad Production Meeting. See the channel topic for the location of the logs. | 15:36 |
| Ursinha | lol | 15:37 |
| matsubara | #endmeeting | 15:37 |
| MootBot | Meeting finished at 09:37. | 15:37 |
| Ursinha | thanks matsubara | 15:37 |
| intellectronica | thanks matsubara | 15:37 |
| === salgado-lunch is now known as salgado | ||
| === bac_uds is now known as bac | ||
| === salgado is now known as salgado-afk | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!