=== kiko is now known as kiko-zzz === nickellery is now known as nickellery_ === nickellery_ is now known as nickellery === rockstar_ is now known as rockstar === nickellery is now known as asdf === asdf is now known as nickellery === mwhudson_ is now known as mwhudson === mwhudson__ is now known as mwhudson === mrevell is now known as mrevell-lunch === salgado-afk is now known as salgado === mrevell-lunch is now known as mrevell === salgado is now known as salgado-lunch [17:50] a hoy === salgado-lunch is now known as salgado === mrevell_ is now known as mrevell [19:00] #startmeeting [19:00] Meeting started at 13:02. The chair is Rinchen. [19:00] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [19:00] wootbot [19:00] Welcome to this week's Launchpad development meeting. For the next 45 minutes or so, we'll be coordinating Launchpad development. [19:00] [TOPIC] Roll Call [19:00] New Topic: Roll Call [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:00] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] me [19:01] cprov sends apologies but had to dash! [19:01] me [19:01] matsubara? [19:01] me [19:01] me [19:01] thanks jtv for being with us [19:01] rockstar? [19:01] ,e [19:01] releases team is here [19:01] er, me [19:02] cprov sends his apologies, I am covering for him [19:02] me [19:03] stub! [19:03] herb, mthaddon ? [19:03] good to have you here [19:03] me [19:03] here [19:03] statik, ? [19:03] hmm [19:03] well, we're missing a few people [19:03] but i'll move fwd [19:04] Rinchen: I believe statik is travelling today [19:04] thanks [19:04] [TOPIC] Agenda [19:04] New Topic: Agenda [19:04] * Next meeting [19:04] * Actions from last meeting [19:04] * Oops report (Matsubara) [19:04] * Critical Bugs (Rinchen) [19:04] * Bug tags [19:04] * Operations report (mthaddon/herb) [19:04] * DBA report (stub) [19:04] * Sysadmin requests (Rinchen) [19:04] * New packages required (salgado) [19:04] * A top user-affecting issue (mrevell) [19:04] * Doc Team report (mrevell) [19:04] plsu [19:04] er [19:04] plus [19:04] * [19:04] Policies for launchpad dependency packages (kiko) [19:04] * [19:04] Storm and Hardy Updates (kiko) [19:04] * [19:04] iframe on every LP page (kiko) [19:04] hmpf [19:04] [TOPIC] Next meeting [19:04] New Topic: Next meeting [19:05] I won't be here next week [19:05] i won't be here next week [19:05] kiko, can you run the the meeting? [19:05] yes [19:05] I can [19:05] I am good at such things [19:05] very well [19:05] (making messes out of meetings) [19:05] ok, same time same place? [19:05] kiko: he said "run", not "ruin" :-) [19:06] [AGREED] meeting next week, same place, same time, Kiko to rui...er...run the meeting. :-) [19:06] AGREED received: meeting next week, same place, same time, Kiko to rui...er...run the meeting. :-) [19:06] * kiko scratches eyes [19:06] [TOPIC] Actions from last meeting [19:06] New Topic: Actions from last meeting [19:06] so he did, maybe that wasn't such a great id.. [19:06] we had none [19:06] [TOPIC] Oops report (Matsubara) [19:06] New Topic: Oops report (Matsubara) [19:07] today's oops report is about bug 239519 [19:07] Launchpad bug 239519 in launchpad-bazaar "BranchCreationException setting source details for a code import" [Undecided,New] https://launchpad.net/bugs/239519 [19:07] thumper: can you take that one? [19:08] I'll allocate it, yes [19:08] there's another OOPS which I'm checking but haven't filed a bug yet. https://devpad.canonical.com/~matsubara/oops.cgi/2008-06-11/EC36 [19:08] salgado: I think that oops is for you :-) [19:08] it's an AssertionError in peoplemerge [19:08] [AGREED] Thumper to allocate someone to look at bug 239519 [19:08] AGREED received: Thumper to allocate someone to look at bug 239519 [19:08] matsubara, I'll have a look [19:09] thanks, we can coordinate about it after the meeting. [19:09] Rinchen: that's it from me. thanks [19:09] [AGREED] salgado to look at a particular oops :-) [19:09] AGREED received: salgado to look at a particular oops :-) [19:09] matsubara, I think it only occurs when I fuck it up, but it should be very easy to fix [19:09] thanks matsubara [19:09] thanks everyone [19:09] oh, no [19:09] it's not [19:09] [TOPIC] Critical Bugs (Rinchen) [19:09] New Topic: Critical Bugs (Rinchen) [19:09] [LINK] https://bugs.edge.launchpad.net/soyuz/+bug/234254 [19:09] cprov, what's the status with this one? This appears to me to either be fixed committed or high, but no longer critical. [19:09] LINK received: https://bugs.edge.launchpad.net/soyuz/+bug/234254 [19:09] Rinchen: Error: This bug is private [19:09] MootBot: Error: This bug is private [19:10] Rinchen: it's a config change [19:10] I think cprov did a branch for it, I don't know if it's in PQM or landed yet [19:10] there is one branch from a while ago documented but history shows that you've raised it in priority since then [19:11] I haven't seen any recent activity, hence my questioning. [19:11] it's config for the restricted librarian [19:11] bigjools, can you check and update the bug accordingly, please? [19:11] yup [19:11] bigjools, no bugs should be left critical [19:11] NONE [19:11] * kiko blows the cavalier trumpet [19:11] AFAIK cprov is actively working on it [19:11] so don't panic! [19:11] [ACTION] Julian to review and update bug 234254 [19:11] Rinchen: Bug 234254 on http://launchpad.net/bugs/234254 is private [19:11] ACTION received: Julian to review and update bug 234254 [19:11] thanks jules [19:12] bigjools, it's not cprov's responsibility. it's the team's. it's a critical bug. [19:12] MootBot: Error: Could not parse data returned by Launchpad: The read operation timed out (https://launchpad.net/bugs/234254/+text) [19:12] [TOPIC] Bug tags [19:12] New Topic: Bug tags [19:12] we have one [19:12] i.e. if you don't know about it, something's wrong [19:12] https://help.launchpad.net/TaggingLaunchpadBugs [19:12] a bug tag, my favorite [19:12] I do know about it [19:12] package-diff - cprov [19:13] bigjools, do you have anything to say on the bug tag on behalf of cprov-out? [19:13] f [19:14] I'm ok with it if it helps your team out. [19:14] Well he mentioned it to me before dashing off earlier, I know as much as you just pasted - I guess it's for bugs relating to the new package diff calculations, since there seem to be a few [19:14] (BTW the fix for 234254 is #2 in PQM's queue) [19:14] we can approve it, reject it, or postpone it. kiko, you have any thoughts? [19:15] al-maisan, any thoughts from you? [19:15] let me check. [19:15] I would be happy with it [19:15] sure, I think that's fine [19:15] Rinchen: sorry my connection dropped [19:15] missed the question [19:15] al-maisa1, package-diff bug tag proposed by cprov [19:16] fine with me [19:16] yeah, it makes sense to me [19:16] ok, so carried. [19:16] [AGREED] Package-diff tag approved. [19:16] AGREED received: Package-diff tag approved. [19:16] bigjools, can you get cprov-out to update that page again please and move the tag to approved? [19:16] wilco [19:16] thank you sir [19:16] [TOPIC] Operations report (mthaddon/herb) [19:16] New Topic: Operations report (mthaddon/herb) [19:17] Monday (2008-06-09) lpnet1 & lpnet2 had died around midnight. Was caught by the SAs and they restarted. [19:17] Monday (2008-06-09) lpnet4 died with logrotate. It was manually restarted. [19:17] There are currently 2 CP requests awaiting approval. (r6410 & r6451) [19:17] We're about mid-way through the hardy upgrades. The app servers are going today [19:17] as we speak. [19:17] Thanks to an update today edge is now using storm. [19:17] The new codeimport system seems to be working well, but it hits the DB pretty ha [19:17] rd. It doesn't seem there is any connection pooling of any kind which can result in 50+ open db connections. [19:17] I failed to update Bug 224623 last week with the debug logs as promised. Look f [19:17] herb: Bug 224623 on http://launchpad.net/bugs/224623 is private [19:17] or them before the end of my day today. [19:18] thumper, any activities on the radar to address the db connections? [19:18] okay. [19:18] thumper, yeah, was going to ask that [19:18] hmm [19:18] we haven't opened a bug on it yet. just started noticing it recently. [19:19] we can get a bug opened on it today if you'd like. [19:19] herb: any idea why the app server died? [19:19] I'll talk with mwhudson_ about it this morning [19:19] Is this one connection per process or thread, or just the usual open-and-close connections really fast? If the latter, Storm could fix that. [19:19] flacoste: sadly no. [19:19] herb, if you could please, open a bug and let thumper and mwh know the number. Please include any data/output you have in the report. [19:19] Rinchen: will do. [19:20] ls [19:20] stub, hmm, I think it's the latter, tbh [19:20] thumper, do you know? [19:20] there are up to 10 processes running at once on each import machine [19:20] we only have two slaves [19:21] so I'd expect to see around 20 not 50 [19:21] right [19:21] okay -- we'll figure it out with mwh the import code master [19:21] perhaps it is just an optimization issue....unused connections being closed later than expected. [19:21] thumper: keep on eye out for the bug a bit later today. [19:21] any other questions for herb and mthaddon ? [19:22] Previously, the zopeless commit would close and reopen a new connection. This behavior has (hopefully) been sorted as a side effect of the storm branch landing. [19:22] should be interesting to see what effect that has [19:22] thanks herb [19:22] thanks [19:23] [TOPIC] DBA report (stub) [19:23] New Topic: DBA report (stub) [19:23] This close and reopening is often done so fast you have 2 or 3 times as many clients on the db as necessary as they are being opened faster than PG can clean them up. [19:23] Production DB will be upgraded to PostgreSQL 8.3 on Tuesday during the Hardy upgrade window. [19:23] The rest of the DB reviews for this cycle will be done tomorrow and the db freeze lifted if edge is in good shape with todays branch. I might delay opening until Monday or Tuesday if there are urgent fixes that need landing on edge and kiko agrees. With luck, the freeze will be lifted by landing the auth-person-split branch. [19:23] PQM has been running PG 8.3 for two days apparantly. I haven't confirmed this with my own eyes yet though as I just noticed the RT request followup :-) Thanks to Lamont and Tom. [19:23] The databases on carbon (staging, demo) should be switched to 8.3 ASAP - I'll look at this if the losas don't beat me to it. The upgrade will also switch the DB to the correct locale (currently running default, should be C locale to match production). [19:23] Nothing else to report. [19:24] stub, I had expected staging and demo to be running 8.3 already tbh [19:24] yay for auth-person-split [19:24] stub, will you do this before the weekend? [19:24] me! phooey. [19:25] Yes. staging and demo where to be done after PQM. The downtime window approached more rapidly than expected. [19:25] stub, I am kinda hoping everything will fall into place tomorrow -- the only thing I can imagine will help us on edge is fixing mars' favorite iframe and possibly a storm optimo, but it's OOPSes that might tell me otherwise [19:25] stub, yeah, we got things in gear now :) [19:25] Yup [19:25] mars' favorite iframe? [19:25] agenda item [19:25] anything else for Stu? [19:26] beer! [19:26] and thai food, yes I read the lodging request :-) [19:26] thanks stub [19:26] [TOPIC] Sysadmin requests (Rinchen) [19:26] New Topic: Sysadmin requests (Rinchen) [19:26] fun [19:26] Is anyone blocked on an RT or have any that are becoming urgent? [19:26] RT #30960 needs action Monday at the latest so I can confirm the PG upgrade will go smoothly. [19:26] * flacoste pokes barry [19:26] none from me, IS have nuked all of mine so quickly [19:26] cprov asked me to mention 30778 [19:26] flacoste, Rinchen we are no longer blocked [19:27] cool! [19:27] which is the Gina machine [19:27] well, my branch won't land for another 11 hours, but the rt was marked resolved :) [19:27] ok thanks. I'll have a look at them after the meeting, set dates and priorities, and ping the cuddly red fellow [19:28] a little deviation here for a moment [19:28] [TOPIC] Policies for launchpad dependency packages (kiko) [19:28] New Topic: Policies for launchpad dependency packages (kiko) [19:28] aha! [19:28] so today we had a bit of a revelation when we figured out that our dependencies packages aren't being directly used on the appservers [19:28] they are being used as references, but not literally [19:29] I had a call with james today to attempt to sort that out [19:29] turns out that there are some bad things we're doing, so I want salgado to document and follow a policy for package updates [19:29] on the app servers they are used, no? just chokecherry that was missing them? [19:29] salgado, well, it varies. the PQM chroot, for instance, doesn't use l-d-d [19:29] anyway [19:30] let me cover the specifics, and then the policy [19:30] 1. httplib. I understand we're using a version that isn't in intrepid yet. is there a bug open for this being packaged in Ubuntu intrepid and backported to hardy? [19:30] (that's for barry and flacoste) [19:30] kiko: not that i know of [19:31] barry, okay. there should be. see policy coming up in a sec. [19:31] 2. apache and ssh. both packages are required for l-d-d. they are both daemons and can't really be run inside the PQM chroot. [19:31] so following comes a policy definition [19:31] for our dependencies packages [19:32] 1. it's not okay to depend on daemons. you can recommend a daemon, but not depend on it. [19:32] 2. if you require a version which isn't available in the current ubuntu release we're running, then it's okay to use a custom package /as long as/ there is a request to package that version in Ubuntu, and that bug is escalated to the distro team [19:33] that's all. I guess this needs to go into a document, but I'll let Rinchen sort that out with salgado! [19:33] * barry will open a bug for httplib2 [19:33] AFAIK, we need apache to run launchpad [19:34] not to run, but to access it [19:34] salgado, PQM doesn't have it, so no, we don't really. [19:34] most package managers don't do anything with recommends, so that may cause developers to have to manually install it [19:34] salgado, it's okay to have l-d-d recommend apachd [19:35] kiko, I meant for a user to access. tests are something else [19:35] salgado, I actually think apt follows recommends. [19:36] in any event, I could modify the newlaunchpadder docs to ensure folks install recommends [19:36] it's usually a config policy [19:36] It doesn't really matter if RocketFuel setup documents an extra command - it isn't like anyone follows that process every day. [19:36] aptitude does [19:36] and I don't know what update-manager does with Recommends [19:36] Wouldn't it be okay to require apache for launchpad-developer-dependencies, but not launchpad-dependencies? [19:36] good point [19:36] that's what we do currently [19:37] maybe we should have launchpad-tests-dependencies [19:37] abentley, no. [19:37] and launchpad-developer-dependencies [19:37] shouldn't PQM et al be using plain launchpad-dependencies? [19:37] salgado, it's unnecessary. recommends works. [19:37] then PQM would only have lp-tests-deps installed [19:38] launchpad-dependences today recommends launchpad-developer-dependencies [19:38] and that's fine [19:38] as I said, recommends works [19:38] can we move on, Rinchen? [19:38] abentley, PQM needs the dev deps in order to run the testsuite [19:39] kiko, yes if you are done. Salgado, can you get with kiko if you have further questions? [19:39] salgado: Okay. I can see the argument that we don't want too many metapackages. [19:39] [TOPIC] New packages required (salgado) [19:39] New Topic: New packages required (salgado) [19:39] any new dependencies for this week? [19:40] [ACTION] Joey to ping Salgado about documenting package dependency policy [19:40] ACTION received: Joey to ping Salgado about documenting package dependency policy [19:40] sounds like no :) [19:40] [TOPIC] A top user-affecting issue (mrevell) [19:40] New Topic: A top user-affecting issue (mrevell) [19:41] Hello. Three things today. [19:41] First up: quick reminder that we'll have service interruptions on 17th, 18th and 19th June. You can find out more in our blog post at: http://news.launchpad.net/notifications/launchpad-service-interruptions-17th-18th-and-19th-june [19:41] Second: Kiko's going to talk about this later, but here's a quick note to any non-team members present: you may spot an empty box at the bottom of pages on edge. We're aware of this, so please don't worry about filing a bug. [19:41] And the main issue this week: matsubara mentioned that intellectronica is currently helping one of our users who is experiencing problems with the email interface. [19:41] This person, Iain, is sending GPG-signed messages for both the bug tracker and Answers, but getting no response. So far, one other person has reported the same problem. [19:41] You can see the question at: https://answers.edge.launchpad.net/launchpad/+question/35165 [19:41] I think intellectronica would welcome a discussion of what could be causing this. [19:41] Thanks Rinchen. [19:42] Anything for mrevell or intellectronica? [19:42] to add more information to that - the user doesn't seem to be able to get even as far as requesting help via the email interface [19:42] intellectronica: let me know if there's anything i can help with [19:42] [TOPIC] Doc Team report (mrevell) [19:42] New Topic: Doc Team report (mrevell) [19:42] mrevell, matsubara, intellectronica: I have confirmation from IS that the message hits us [19:43] who should I forward that information to? [19:43] kiko: me [19:43] Cool, thanks guys. [19:43] A busy week for me working on user guide material, and primarily further tweaks to the tour, following Mark's feedback. I'd like to arrange a call with each team lead to go through the relevant section of the tour content with you. Please ping me with suitable times. [19:43] No news from the doc team this week. [19:43] Thanks Rinchen. [19:43] there's a question. maybe this is the time to convert it to a bug [19:44] sounds good to me :-) [19:44] intellectronica: i would look into signed_message_from_string failing [19:44] intellectronica: look at the log, you should be able to find the email failing [19:44] intellectronica, will do. thanks! [19:44] [TOPIC] Storm and Hardy Updates (kiko) [19:44] New Topic: Storm and Hardy Updates (kiko) [19:44] aha [19:45] I am the bearer of great tidings [19:45] staging and our edge server are nor updated to latest Storm [19:45] err [19:45] now updated [19:45] we are aware of some performance regressions that this will impact [19:46] for instance, BjornT has seen the number of queries on certain bug pages going up surprisingly [19:46] we also know that PQM took a 15 minute hit [19:46] this is somewhat expected and we'll be working on fixing it as we progress [19:46] there's also an OOPS on the wild that mthaddon has triggered, which is something we're going to investigate further [19:47] ZStormError: Store named 'main' already exists [19:47] is what the OOPS says [19:47] aaaanyway. help us through the rough waters by offering help and objective issue reports. general whining is not ok. [19:48] last item of news is hardy upgrades -- you guys know this is happening next week, along with pg8.3 coming on tuesday thanks to stub and the losas [19:48] so keep your eyes peeled for any issues that come during this period, and let us know [19:48] thanks and sorry for running overtime -- it's always like that when you let me do the talking... [19:48] and for the final topic... [19:48] iframe on every LP page (kiko) [19:48] [TOPIC] iframe on every LP page (kiko) [19:48] New Topic: iframe on every LP page (kiko) [19:48] why is this me? [19:48] anyway. [19:49] edge has an iframe being rendered on every single page. [19:49] it's a CSS mistake, I believe -- invisible isn't actually causing invisibility. [19:49] should be display: none [19:49] fyi, it also has a javascript error on all pages. it's part of the same bug [19:49] kiko, working on it now [19:49] it's a critical problem, and as soon as we have a branch (tests for it will be.. interesting) we should push this right up to the front of the PQM queue. [19:49] Invisible != none [19:50] mthaddon, herb: can you work with mars and intellectronica to ensure that happens? will owe you a tall glass 'a beer [19:50] I thought the intent *was* to have it load everytime, so invisible was used. [19:50] kiko: not a problem [19:50] I know the beer's not a problem. I meant the PQM reorder. :) [19:50] It shouldn't be invisible, it should be non-existent until summoned [19:50] kiko, sure [19:50] like a good familiar [19:50] err [19:51] Thank you all for attending this week's Launchpad Developer Meeting. [19:51] #endmeeting [19:51] Meeting finished at 13:53. [19:51] mpt: not existant implies created with JS [19:51] Thanks Rinchen [19:51] thumper, exactly [19:51] Rinchen: thanks [19:51] Rinchen: thanks [19:51] thanks rinchen [19:51] * Rinchen attempts to summon the mootbot logs === mwhudson_ is now known as mwhudson === salgado is now known as salgado-afk