=== Rinchen-phone is now known as Rinchen === kiko is now known as kiko-zzz [03:00] barry: hi [03:00] jml: hi [03:00] #startmeeting [03:00] Meeting started at 21:02. The chair is barry. [03:00] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [03:00] yay! [03:01] welcome to this week's asiapac reviewer's meeting. who's here today? [03:01] * jml is reminded to file a bug on Python [03:01] barry: I am. [03:01] thumper: ping? [03:01] mwhudson: ping [03:02] * barry guesses thumper knows barry will rant at him about pqm :) [03:02] hi [03:02] barry: now why would you rant about pqm? [03:03] thumper: i'd like it to vanish more of my branches please! :) [03:03] [TOPIC] agenda [03:03] New Topic: agenda [03:03] barry: I could delete them for you? [03:04] * Roll call [03:04] * Next meeting [03:04] * Action items [03:04] * Queue status [03:04] * Mentoring update [03:04] * Review process [03:04] * make pqm not broken [03:04] thumper: then i could just stop working on branches too! excellent [03:04] slacking FTW! [03:04] [TOPIC] * Next meeting [03:04] New Topic: * Next meeting [03:04] everyone okay with today += week(1) [03:05] +1 [03:05] jml: i know you are too :) [03:05] :) [03:05] [TOPIC] * Action items [03:05] New Topic: * Action items [03:05] nothing on asiapac iirc [03:05] [TOPIC] * Queue status [03:05] New Topic: * Queue status [03:06] ah, as traditional, i missed the start of the meeting, sorry [03:06] * jml files http://bugs.python.org/issue3071 [03:06] mwhudson: no problem! i was just about to assign all the GQ branches to you! [03:06] heh heh [03:06] jml: cool [03:07] barry: I've been griping about Python's default exception messages for ages now. [03:07] barry: time to do so productively :) [03:07] jml: especially with the first betas going out on wednesday! (perhaps) [03:08] i don't have much on the queue status [03:08] i'm going to assign the 4 GQ branches after this meeting, but other than that, things look okay [03:08] i had test failures in the one adeuring branch i started today [03:08] oh hey, i got assigned a branch [03:09] not from me [03:09] i totally didn't notice [03:09] mwhudson: you should run bac's script! [03:09] barry: we should fix launchpad! [03:09] etc etc [03:09] that too :) [03:10] * jml notices Python 2.5 actually does a lot better at error messages than 2.4. [03:10] jml: 2.5 rocks. we really need to switch to it [03:10] anything on the queue from y'all? [03:10] barry: damn straight. I want to use defgen :) [03:11] mwhudson: hmm. I think I did make a mistake. that branch looks familiar. I think I meant to assign it to bigjools [03:11] * barry wants with statements! [03:11] FWIW abentley is working on the branch that will email the reviewer when nominated for a review [03:11] will the zope upgrade get us any close to being ready to use 2.5 ? [03:11] thumper: how long before you think we'll be able to do all our reviews in lp? [03:11] barry: start debugging the zope 3.4 branch. It works except for that giant hole I putting in importfacsist. [03:12] hacky ones in about 3 or 4 weeks [03:12] with shiney happy people, probably about 8 [03:12] sinzui: fred drake tells me they (zc) runs zope all the time on py2.5 [03:12] thumper: that will rock [03:13] sinzui: ok, are you going to move that branch out of my queue? [03:13] barry: I bet. but importfascists and zope 3.4 conflict. I sacrificed the former for zope [03:13] mwhudson: I will [03:13] sinzui: thanks [03:13] sinzui: dang. where's the branch? [03:14] anyway. moving on... [03:14] [TOPIC] * Mentoring update [03:14] New Topic: * Mentoring update [03:14] skipped as usual... [03:14] n/a [03:14] [TOPIC] * Review process [03:14] New Topic: * Review process [03:14] * make pqm not broken [03:15] so... thumper i'm told you're pqm-man now! :) [03:15] hah [03:15] what's the PQM break? [03:15] looks just fine to me. [03:15] or at least as good as it's ever been :) [03:15] there are several problems, but the most severe symptom is that branches are getting disappeared all the time [03:16] they fail and nobody gets notified [03:16] ahh ok. [03:16] that *is* pretty bad. [03:16] Isn't that directly cause by the make check change from 60 minutes to 15 minutes? [03:16] jml: it's doubly bad because grepping through the logs is PAINFUL [03:16] since the logs don't include branch information unless they succeed :( [03:16] barry: where are the logs? [03:16] sinzui: yes, that's what flacoste thinks [03:16] this sounds like something that needs help from an OSA [03:16] I see. [03:16] not us [03:17] thumper: devpad [03:17] barry: where on devpad? [03:17] sinzui: i don't see the connection instantly [03:17] thumper: /srv/lp-pqm-logs [03:17] barry: thanks [03:17] barry: do we know that this is a bug in PQM itself? [03:18] francis and i talked about a few things: first, if merges get killed, we should still see notifications, which i think today we don't [03:18] jml: we don't, but that's the suspicion [03:18] second issue is figuring out why sometimes you get no output for 15+ minutes [03:18] but the vicious killing after 15 minutes isn't anything to do with pqm [03:19] it's something test.py or shh.py or something does [03:19] mwhudson: aren't the logs maintained and sent by pqm? [03:19] yes [03:19] barry: has anyone tried to reproduce the issue? [03:19] but pqm doesn [03:19] mwhudson: right. but no one has reproduced the 15 minute hang on dev boxes [03:19] 't look at the logs [03:20] * jml meant the lack-of-notification issue. [03:20] jml: not to my knowledge [03:20] barry: ok. [03:20] third problem was lack of branch information in the logs unless the merge succeeds [03:21] but if pqm doesn't write the logs, i guess it can't do that [03:22] pqm does write a log file [03:22] several actually [03:22] we need a plan though, 'cause it's killing productivity. e.g. branches are getting stacked up beause parent branches aren't landing and then no one knows why [03:22] in my ignorance, I'd say that PQM ought to keep a log of requests it receives and what it does with them. [03:22] jml: it does [03:22] barry: are the bugs filed against pqm? [03:22] thumper: but not enough information to identify the branch? [03:22] i can't find any logs on devpad in the last 50 that seem to have been killed for lack of activity [03:23] but maybe i'm grepping for the wrong things [03:23] thumper: well, the first issue isn't yet known to be an issue with PQM. [03:23] thumper: i don't know. i will talk with francis in the morning and get some bugs opened [03:23] thumper: although it's a decent place to start, I guess. [03:23] pqm is still pretty fresh in my mind [03:23] so feel free to send them my way if urgent [03:24] thumper: thanks. i think the lack of notification is the most immediate concern. devs could fix their branch problems if they knew why they bounced [03:24] [ACTION] barry to talk with francis and get pqm bugs opened [03:24] ACTION received: barry to talk with francis and get pqm bugs opened [03:25] that's all i've got. does anybody have anything not on the agenda? [03:26] nope? okay, thanks everyone! see you next week [03:26] #endmeeting [03:26] Meeting finished at 21:28. [03:26] barry: why do you think that branches are getting killed for inactivity? [03:36] mwhudson: we don't know. it shouldn't be happening [03:36] barry: no, i mean [03:37] barry: what is it that makes you think the branches are being killed in this way? [03:37] barry: because i don't see much evidence for it myself [03:37] mwhudson: we're just guessing. it seems like branches are disappearing more now that we bumped that down, but it's just a guess [03:38] barry: are the branches disappearing to the extent of not producing logs on devpad? [03:39] mwhudson: no, they are producing logs afaict. mars found one of my disappeared branches in the logs after some clever sleuthing [03:39] barry: ok, because grepping logs suggests only two branches have been killed for hanging since the change to a 15 minute timeout [03:40] mwhudson: interesting. i think in my case, bzr actually complained (branch format problem because this was a loomed branch). so i wonder if more of them are happening because bzr failures don't get notifications [03:41] it's possible === mwhudson_ is now known as mwhudson === mwhudson__ is now known as mwhudson === cprov is now known as cprov-lunch === mrevell is now known as mrevell-lunch === salgado-afk is now known as salgado === mrevell-lunch is now known as mrevell === kiko-zzz is now known as kiko === kiko is now known as kiko-phone === kiko-phone is now known as kiko === salgado is now known as salgado-lunch === kiko is now known as kiko-fud === salgado-lunch is now known as salgado === salgado is now known as salgado-afk