[19:00] <salgado> me
[19:00] <abentley> me
[19:00] <thumper> me
[19:00] <bigjools> moo
[19:01] <flacoste> chair is late
[19:01] <mwhudson> me
[19:01] <stub> me
[19:01] <flacoste> roll call hasn't started yet
[19:01] <gmb> Rincheeeeeeeeeeeeeeeeeen....
[19:01] <mpt> Rinchen's on holiday
[19:01] <sinzu1> All is moot without mootbot
[19:01] <mpt> all this week
[19:01] <mars> Rinchen's off in the wilderness somewhere...
[19:01] <gmb> Ah, so it's kiko's show is it?
[19:01] <bac> kiiiiiiiiiiko
[19:01] <gmb> And he's not even in the room
[19:01] <flacoste> stub: did you get my email about the api-acl-db DB patch?
[19:01] <matsubara> kiko's on the phone
[19:02] <flacoste> stub: i'd like you to land it unless there is good reason not to do it
[19:02]  * stub has a look
[19:03] <mpt> #startmeeting
[19:03] <MootBot> Meeting started at 13:06. The chair is mpt.
[19:03] <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
[19:03] <mpt> nono, the chair is kiko
[19:03] <kiko> mpt, can you help be chair? I'm still on the phone
[19:03] <gmb> pwned.
[19:03] <mpt> ok, MEETING TIME
[19:03] <mthaddon> me
[19:03] <mpt> Who is here today?
[19:03] <matsubara> me
[19:03] <allenap> me
[19:03] <schwuk> me
[19:03] <sinzui> me
[19:04] <mrevell> me
[19:04] <flacoste> me
[19:04] <stub> me
[19:04] <gmb> me
[19:04] <kiko> em
[19:04] <intellectronica> me
[19:04] <bac> me
[19:04] <EdwinGrubbs> me
[19:04] <mars> me
[19:04] <bigjools> me
[19:04] <statik> me
[19:04] <adeuring> me
[19:04] <salgado> me
[19:04] <leonardr> me
[19:04] <mpt> me
[19:04] <thumper> me
[19:04] <mwhudson> me
[19:04] <abentley> me
[19:05] <thumper> rockstar: ?
[19:05] <mpt> Team leads, please ensure all your team members who should be here are
[19:05] <mpt> That, by the way, was
[19:05] <mpt> [TOPIC] Roll call
[19:05] <MootBot> New Topic:  Roll call
[19:05] <mpt> And now it's time for
[19:05] <mpt> [TOPIC] Agenda
[19:05] <MootBot> New Topic:  Agenda
[19:05] <bigjools> cprov ping
[19:06] <mpt>  * Next meeting
[19:06] <mpt>  * Actions from last meeting
[19:06] <mpt>  * Oops report (Matsubara)
[19:06] <mpt>  * Critical Bugs (Rinchen)
[19:06] <mpt>  * Bug tags
[19:06] <mpt>  * Operations report (mthaddon/herb)
[19:06] <mpt>  * DBA report (stub)
[19:06] <mpt>  * Sysadmin requests (Rinchen)
[19:06] <mpt>  * New packages required (salgado)
[19:06] <mpt>  * A top user-affecting issue (mrevell)
[19:06] <mpt>  * Doc Team report (mrevell)
[19:06] <herb> me
[19:06] <al-maisan> me
[19:06] <cprov> me
[19:06] <mpt> If anyone has other items, please /msg me
[19:06] <barry> me
[19:06] <mpt> [TOPIC] Next meeting
[19:06] <MootBot> New Topic:  Next meeting
[19:07] <mpt> If anyone knows of any reason why the next meeting should not be same time, same channel, let them speak now, or forever hold their peace
[19:07] <mpt> Is there anyone who won't be here?
[19:07] <kiko> I will be :)
[19:07] <mpt> A resounding silence, that's good
[19:07] <mpt> ok
[19:07] <mpt> [TOPIC] Actions from last meeting
[19:07] <MootBot> New Topic:  Actions from last meeting
[19:08] <mpt>  *  Joey to ping Salgado about documenting package dependency policy
[19:08] <salgado> not done
[19:08] <mpt> [TOPIC] Oops report
[19:08] <MootBot> New Topic:  Oops report
[19:09] <matsubara> Today's oops report is about bugs 241355
[19:09] <matsubara> thumper, can you assign and schedule 241355?
[19:09] <thumper> ok
[19:09] <matsubara> thumper: thanks
[19:09] <matsubara> mpt: that's it for the oops report.
[19:09] <mpt> thanks matsubara
[19:10] <mpt> [TOPIC] Critical Bugs
[19:10] <MootBot> New Topic:  Critical Bugs
[19:10] <matsubara> No critical bugs open.
[19:10] <mpt> excellent
[19:10] <matsubara> that's it for critical bugs :-)
[19:10] <mpt> [TOPIC] Bug tags
[19:10] <MootBot> New Topic:  Bug tags
[19:10] <mpt> There is one proposed tag
[19:10] <matsubara> we have package-diff as a proposed tag
[19:10] <bigjools> that was last week
[19:10] <kiko> and we approved it :)
[19:11] <matsubara> yes, was that accepted or moved to this one?
[19:11] <mpt> ok
[19:11] <cprov> :)
[19:11] <bigjools> cprov, did you want to raise another one for p3a?
[19:11] <matsubara> ok
[19:11] <mpt> It was approved
[19:11] <cprov> bigjools: yes, probably, but I will propose it next week
[19:11] <mpt> matsubara, can you take care of moving it to the Approved section?
[19:11] <matsubara> mpt: yep
[19:11] <mpt> thanks
[19:12] <mpt> [TOPIC] Operations report
[19:12] <MootBot> New Topic:  Operations report
[19:12] <mpt> mthaddon or herb?
[19:12] <herb> Friday (2008-06-13) - Librarian seemed to have problems which caused a knock-on effect on app servers. Caused about 10 minutes of downtime. Tom might have more in the way of details.
[19:12] <herb> Wednesday (2008-06-18) - Upgraded a bunch of servers to hardy. Took about 4 hours of planned downtime to do so.
[19:12] <herb> There are 2 cherry picks (r6410 and r6451) awaiting approval that, I'm assuming, won't be approved prior to the rollout.
[19:12] <herb> The only remaining hosts to be updated to hardy are the Soyuz machines. They are scheduled for tonight.
[19:12] <herb> The Debian import machine is up and configured. Celso and I have been working through remaining issues and are making good progress.
[19:12] <herb> Bug #224623 is still hanging around. We haven't seen the problem crop up again and I know Stuart has been very busy with the 8.3 upgrades. If there is anything we can do to help, Tom and I are happy to do so.
[19:12] <herb> That's it from Tom and me unless there are any questions.
[19:12] <mpt> thanks herb
[19:13] <stub> When is bug expiry going to be implemented?
[19:13] <stub> Then annoying unreproducible issues will silently die...
[19:13] <mpt> sinzui, is that something you can answer?
[19:13] <intellectronica> wasn't it already implemented, then cancelled?
[19:14] <sinzui> mpt: No
[19:14] <flacoste> mpt: sinzui is off the hook fo rthat nor
[19:14] <flacoste> it's a bugs team decision
[19:14] <mpt> BjornT?
[19:14] <stub> I'm not that serious...
[19:14] <intellectronica> mpt: he's not here today
[19:14] <mars> BjornT is off today
[19:14] <mpt> ok, we'll move on
[19:14] <mpt> He looks here
[19:14] <mpt> [TOPIC] DBA report
[19:14] <MootBot> New Topic:  DBA report
[19:15] <stub> Everything is running PG 8.3 now. It seems faster. Todays OOPS reports will give us not-a-metric.
[19:15] <stub> The first stage of the Auth/Person split has landed. yay. This opens launchpad/devel up for db changes so land 'em if I haven't done it for you.
[19:15] <stub> All the db patches I'm aware of have been reviewed for this cycle.
[19:15] <stub> We need to work out where staging and demo are going to live. Demo needs to go first as I will use staging for testing replication and there just isn't enough disk atm for everything.
[19:15] <stub> After we have worked out and tested how this replication stuff is going to be glued together I'd like the launchpaddb master and authdb replica running on hackberry and the authdbmaster and launchpaddb replica running on chokecherry.
[19:15] <mthaddon> stub, if we drop launchpad_demo from 8.2 on chokecherry, would that give us enough space?
[19:16] <stub> I didn't drop it already? Better do that now or the staging restore will likely blow up again...
[19:16] <stub> If it fits, it will be very tight.
[19:16] <mthaddon> stub, it did blow up - I dropped launchpad_langpack and it now is okay with restore
[19:17] <mthaddon> stub, but I suspect if we drop launchpad_demo as well we'll get some more space again
[19:17] <mthaddon> stub, should I just do that now?
[19:17] <rockstar> me
[19:17] <stub> So we might be fine for testing.
[19:17] <stub> dropping it now
[19:17] <mthaddon> cool (and session_demo too I presume)
[19:18] <stub> If we run staging and production and demo on chokecherry that will be 4 full copies of the production db (because staging will need to be replicated too).
[19:19] <mthaddon> stub, our current staging restore also means creating another db (launchpad_staging_new) and then dropping and renaming if the restore works
[19:19] <stub> And it will be a production server, highly visible if it goes down, so I still vote for demo and staging on different hardware.
[19:19] <stub> Ok. 5 full copies.
[19:19] <mthaddon> stub, would definitely be better, just don't know if we have other hardware anywhere near the same class
[19:20] <mpt> Ok, anything more from the DB department?
[19:20] <stub> I don't think we need it in the same class personally - I'd prefer staging to be underpowered compared to production.
[19:21] <stub> (If we need production level hardware to cope with one or two users, how can we expect production to cope?)
[19:21] <mthaddon> stub, tough one that, as ideally we want to be able to see how performance compares on similar hardware, but load is different, so....
[19:22] <stub> mpt: next...
[19:22] <mpt> thanks stub
[19:22] <mpt> [TOPIC] Sysadmin requests
[19:22] <MootBot> New Topic:  Sysadmin requests
[19:22] <mpt> matsubara is filling in for Rinchen
[19:22] <matsubara> Any outstanding RT requests? I'll send those out to Joey and he'll follow up
[19:22] <matsubara> on Monday. If there's anything that can't wait until Monday, I'll follow up
[19:22] <matsubara> with elmo.
[19:23] <bigjools> The Soyuz team has RT 31016
[19:23] <cprov> kees and ubuntu-security team need access to http://private-ppa.buildd
[19:23] <mpt> I'd like to express my appreciation for RT 27357 being closed after 15 months :-)
[19:24] <matsubara> cprov, bigjools I'll ask elmo to take a look. how important is it?
[19:24] <bigjools> matsubara: very
[19:24] <bigjools> thank you
[19:24] <matsubara> np
[19:25] <mpt> thanks again matsubara
[19:25] <mpt> [TOPIC] New packages required
[19:25] <MootBot> New Topic:  New packages required
[19:25] <salgado> anything for launchpad-dependencies this week?
[19:26] <kiko> well
[19:26] <kiko> yes
[19:26] <kiko> flacoste has a profiling package we want
[19:26] <kiko> oh, no, that goes to sourcecode/.
[19:26] <kiko> sorry.
[19:26] <flacoste> kiko: no, it's in sourcecode
[19:26] <abentley> I encountered some issues because TG isn't a dependency.
[19:26] <kiko> abentley, TG?
[19:26] <abentley> TurboGears
[19:26] <stub> bzr-loom might need to be packaged - there are looms in the review queue (salgado)
[19:26] <mwhudson> tg won't be a dependency for much longer
[19:26] <flacoste> salgado will use export-loom
[19:27] <flacoste> as everybody using loom should
[19:27] <flacoste> it makes better thread on the mailing list
[19:27] <salgado> yeah, I'll use that
[19:27] <flacoste> otherwise all branches look the same
[19:27] <stub> yay. sftp really sucks here. need the smart server.
[19:27] <barry> though it would be nice not to have to export, as it's a bit of extra work
[19:27] <flacoste> abentley, salgado: i thought there were issues packaing turbo-gears
[19:28] <salgado> flacoste, no, it's already packaged.  there are issues running it, IIRC
[19:28] <flacoste> barry: i don't agree, i hate record :-)
[19:28] <salgado> it doesn't run against our sqlobject
[19:28] <barry> flacoste: i hate record too!  still if you have 5 thread, you have an issue :)
[19:28] <abentley> salgado: I haven't seen any reports of issues running it.  AFAICT, the sqlobject one is a non-issue.
[19:29] <abentley> It runs against its own sqlobject just fine.
[19:29] <flacoste> barry: i don't understand what is the issue
[19:29] <mwhudson> abentley: is this an issue if i land the loggerhead branch that removes the dependency on turbogears?
[19:29] <salgado> abentley, the issue is whether or not we want our developer dependencies to include sqlobject
[19:29] <emgent> hello
[19:30] <abentley> salgado: I think make run_all should not require anything above our developer dependencies.
[19:30] <barry> flacoste: i'll explain after the meeting
[19:30] <abentley> mwhudson: No, if you land that, it goes away.
[19:30] <mwhudson> let's do that then
[19:30] <mwhudson> (it's very nearly ready)
[19:31] <salgado> mpt, guess we're finished here
[19:31] <mpt> thanks salgado
[19:31] <mpt> [TOPIC] Storm update
[19:31] <MootBot> New Topic:  Storm update
[19:32] <mpt> kiko?
[19:32] <jtv> He's on the phone I believe.
[19:32] <mpt> ok, we can leave that for a few minutes
[19:33] <mpt> [TOPIC] A top user-affecting issue
[19:33] <MootBot> New Topic:  A top user-affecting issue
[19:33] <mpt> mrevell!
[19:33] <mrevell> hey hey
[19:33] <mrevell> Recently, we've had a number of users who have written to the feedback address complaining that they are receiving large numbers of bug related emails. They've subscribed to a project or distro and don't know how to turn it off.
[19:33] <kiko> back back. sorry mpt, jtv.
[19:33] <mrevell> Alongside the "kill bug mail" button that we're planning, I think we should make it clearer that a structural subscription could result in a great deal of email. I also think we should show structural subs on people bug pages. I've filed bug 241387 to reflect this.
[19:33] <mrevell> As an aside, we've had no emails to the feedback address or -users complaining about the Hardy upgrade related service interruptions. One person complained about an Internal Server Error but the time he reported doesn't seem to tie up with our recent interruptions.
[19:34] <mrevell> thanks mpt
[19:34] <mpt> thanks mrevell (and I think 241387 might be a duplicate;-)
[19:34] <mrevell> mpt: gosh darn it, I did look
[19:34] <mpt> [TOPIC] Storm update
[19:34] <MootBot> New Topic:  Storm update
[19:34] <kiko> mrevell, yeah, it appears they went through very smoothly -- thankfully, too. :)
[19:34] <kiko> very good.
[19:34] <kiko> storm's been running on staging for the past days, but it's been upgraded today to r6527, which includes perf work which gustavo and niemeyer have been doing
[19:35] <stub> Do we want to record metrics of how many bugs a particular structural subscription would generate per day? The average so far? Then we could warn users how much traffic they are going to get.
[19:35] <thumper> some, many, lots
[19:35] <intellectronica> stub: i think that's a really cool idea!
[19:35] <kiko> stub, I actually think these people are subscribing for the wrong reason -- they don't actually know
[19:35] <kiko> they are just clicking on buttons and hoping their bugs get fixed
[19:35] <kiko> anyway
[19:35] <kiko> so staging's running a recent version of storm
[19:36] <kiko> I need to ask you to please spend 10 minutes today trying to navigate, modify and check out staging
[19:36] <mpt> matsubara, can you report a bug on implementing stub's suggestion?
[19:36] <mpt> it's a neat idea
[19:36] <kiko> from the oops logs I am still seeing significant non-sql time in many pages
[19:36] <kiko> for instance:
[19:36] <kiko> https://devpad.canonical.com/~matsubara/oops.cgi/2008-06-19/S833
[19:36] <kiko> https://devpad.canonical.com/~matsubara/oops.cgi/2008-06-19/S599
[19:36] <kiko> https://devpad.canonical.com/~matsubara/oops.cgi/2008-06-19/S886
[19:37] <kiko> https://devpad.canonical.com/~matsubara/oops.cgi/2008-06-19/S888
[19:37] <kiko> it's not happening on every page, thankfully
[19:37] <kiko> but those pages will need further investigation. just how much is a question for flacoste, SteveA, niemeyer and me after this very meeting. :)
[19:37] <kiko> we will decide together whether to rollout edge or not.
[19:38] <kiko> if we do roll out, you'll be notified through email
[19:38] <kiko> note also that 1.2.6 remains targeted for Wednesday next week
[19:38] <kiko> ask me questions now :)
[19:38] <mpt> kiko, if we do, when will it happen? intellectronica and mars will be wanting to watch to see if the help frame appears like it did at the last edge rollout
[19:38] <intellectronica> indeed
[19:38] <kiko> mpt, later today.
[19:39] <mars> ok
[19:39] <kiko> I know -- I have the same thing on my mind.
[19:39] <mpt> ok
[19:39] <stub> That peoplelist one is a high start - I hope Storm isn't sucking the first 1.9 million people to render the next 50...
[19:39] <kiko> the one which concerns me the most is MailingListAPIView
[19:40] <kiko> so we'll see
[19:40] <barry> kiko: we have an open bug on that one
[19:40] <kiko> barry, it's about to be made critical :)
[19:40] <kiko> more seriously, let me talk to francis
[19:40] <barry> kiko: yay!
[19:41] <kiko> okay.
[19:41] <kiko> let me steal the spotlight for another announcement.
[19:41] <mpt> [TOPIC] Another announcement
[19:41] <MootBot> New Topic:  Another announcement
[19:41] <flacoste> kiko: that one only happens on Staging iirc
[19:41] <kiko> I'm running an experimental test service for branches in PQM
[19:41] <flacoste> kiko: it's not clear why it's only a problem on staging though
[19:41] <kiko> flacoste, I still want it fixed :-/
[19:41] <kiko> this experimental service does the following:
[19:41] <kiko> - merges all branches in the PQM queue
[19:41] <kiko> - reports conflicts
[19:41] <kiko> - runs tests
[19:42] <jtv> Yay!
[19:42] <kiko> look at the topic in #launchpad-code for the URL
[19:42] <salgado> does it revert the merges which caused conflicts?
[19:42] <thumper> kiko: so if it is good, you could clear the pqm queue :)
[19:42] <kiko> yes
[19:42] <kiko> thumper, indeed :)
[19:42] <mpt> nifty
[19:42] <kiko> thumper, however, I haven't managed to get the tests to pass :)
[19:42] <salgado> just like pqm
[19:42] <kiko> salgado, yeah, it does -- it has to :)
[19:43] <stub> The trick will be inserting a branch containing the merged branches into the front of the pqm queue
[19:43] <kiko> stub, when tests pass, yes
[19:43] <kiko> anyway
[19:43] <kiko> if you have a branch in PQM
[19:43] <kiko> watch that URL
[19:43] <flacoste> kiko: that kind of lose al commits messages
[19:43] <flacoste> but saves us processing time
[19:44] <mpt> and could make it harder to back out changes
[19:44] <kiko> flacoste, I commit the individual logs with the commit messages.
[19:44] <flacoste> and that too
[19:44] <stub> You might want something dumber in reality - pick three random branches from the queue, merge, run tests, if pass stick an integration branch to the front of the queue.
[19:44] <intellectronica> flacoste: they can comitted separately
[19:44] <thumper> kiko: you should get it to not stop on the first failure
[19:44] <stub> Rather than the entire queue, which will be more likely to fail when we need it most.
[19:44] <kiko> mpt, flacoste: well, we DO use a revision control system.. :)
[19:44] <matsubara> stub, mpt: https://bugs.edge.launchpad.net/malone/+bug/241398 did I get it right?
[19:44] <kiko> right now, it looks like al-maisan's branch is still failing in PQM :)
[19:45] <al-maisan> Whoops
[19:45] <kiko>     testDelayedBinaryUpload (canonical.archiveuploader.ftests.test_buildduploads.TestBuilddUploads)
[19:45] <mpt> matsubara, pretty much
[19:45] <kiko> etc.
[19:45] <mpt> ok
[19:45] <mpt> Anything more kiko?
[19:45] <kiko> no, that's it from me.
[19:45] <mpt> thanks
[19:45] <mpt> [TOPIC] Doc Team report
[19:45] <MootBot> New Topic:  Doc Team report
[19:45] <mrevell> hi
[19:46] <intellectronica> matsubara: that should probably be on launchpad itself, rather than malone, because structural subscriptions are nt (in the long run) a malone-only feature
[19:46] <mrevell> Just a quick one: I've sent a new translations section of the user guide to the team list for review. I'd really appreciate your review of it. Thanks mpt.
[19:46] <mpt> thanks mrevell
[19:46] <mpt> And now, as the Storm of indecision rolls in over the streetscape of #launchpad-meeting
[19:46] <mpt> And the Janitor of time sweeps away the dust of our agenda
[19:46] <mpt> It's time to bring this week's meeting to a close.
[19:47] <mrevell> mpt: heh :)
[19:47] <mpt> #endmeeting
[19:47] <MootBot> Meeting finished at 13:49.
[19:47] <intellectronica> thanks mpt
[19:47] <mpt> Thanks everyone
[19:47] <mrevell> nicely done mpt and I love the tribute to Humphry Littleton at the end there :)
[19:47] <jtv> MootBot: what timezone are _you_ in?
[19:47] <kiko> heh
[19:47] <kiko> mpt, thanks so much for saving my bacon :)
[19:47] <intellectronica> tz -42
[19:49] <mpt> matsubara, so the actual problem is "People don't realize how much mail a structural subscription will send them"
[19:49] <mpt> that wouldn't be solved just by recording the average messages/month
[19:49] <mpt> it would be solved by recording it *and* presenting it on the page for subscribing
[19:50] <mpt> an alternative (though highly unlikely) way of fixing the problem would be to get rid of structural subscriptions
[19:50] <mpt> there might be other ways of fixing the problem that we haven't thought of
[19:50] <matsubara> right, I'll update the description.
[19:51] <mpt> Reporting it as a problem (and suggesting possible ways to solve it) makes it less likely to end up marked Invalid :-)