[03:00] <jml> barry: hi
[03:00] <barry> jml: hi
[03:00] <barry> #startmeeting
[03:00] <MootBot> Meeting started at 21:02. The chair is barry.
[03:00] <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
[03:00] <barry> yay!
[03:01] <barry> welcome to this week's asiapac reviewer's meeting.  who's here today?
[03:01]  * jml is reminded to file a bug on Python
[03:01] <jml> barry: I am.
[03:01] <barry> thumper: ping?
[03:01] <barry> mwhudson: ping
[03:02]  * barry guesses thumper knows barry will rant at him about pqm :)
[03:02] <thumper> hi
[03:02] <thumper> barry: now why would you rant about pqm?
[03:03] <barry> thumper: i'd like it to vanish more of my branches please! :)
[03:03] <barry> [TOPIC] agenda
[03:03] <MootBot> New Topic:  agenda
[03:03] <thumper> barry: I could delete them for you?
[03:04] <barry> * Roll call
[03:04] <barry>  * Next meeting
[03:04] <barry>  * Action items
[03:04] <barry>  * Queue status
[03:04] <barry>  * Mentoring update
[03:04] <barry>  * Review process
[03:04] <barry>    * make pqm not broken
[03:04] <barry> thumper: then i could just stop working on branches too!  excellent
[03:04] <thumper> slacking FTW!
[03:04] <barry> [TOPIC]  * Next meeting
[03:04] <MootBot> New Topic:   * Next meeting
[03:04] <barry> everyone okay with today += week(1)
[03:05] <thumper> +1
[03:05] <barry> jml: i know you are too :)
[03:05] <jml> :)
[03:05] <barry> [TOPIC]  * Action items
[03:05] <MootBot> New Topic:   * Action items
[03:05] <barry> nothing on asiapac iirc
[03:05] <barry> [TOPIC]  * Queue status
[03:05] <MootBot> New Topic:   * Queue status
[03:06] <mwhudson> ah, as traditional, i missed the start of the meeting, sorry
[03:06]  * jml files http://bugs.python.org/issue3071
[03:06] <barry> mwhudson: no problem!  i was just about to assign all the GQ branches to you!
[03:06] <mwhudson> heh heh
[03:06] <barry> jml: cool
[03:07] <jml> barry: I've been griping about Python's default exception messages for ages now.
[03:07] <jml> barry: time to do so productively :)
[03:07] <barry> jml: especially with the first betas going out on wednesday! (perhaps)
[03:08] <barry> i don't have much on the queue status
[03:08] <barry> i'm going to assign the 4 GQ branches after this meeting, but other than that, things look okay
[03:08] <barry> i had test failures in the one adeuring branch i started today
[03:08] <mwhudson> oh hey, i got assigned a branch
[03:09] <sinzui> not from me
[03:09] <mwhudson> i totally didn't notice
[03:09] <barry> mwhudson: you should run bac's script!
[03:09] <mwhudson> barry: we should fix launchpad!
[03:09] <mwhudson> etc etc
[03:09] <barry> that too :)
[03:10]  * jml notices Python 2.5 actually does a lot better at error messages than 2.4.
[03:10] <barry> jml: 2.5 rocks.  we really need to switch to it
[03:10] <barry> anything on the queue from y'all?
[03:10] <jml> barry: damn straight. I want to use defgen :)
[03:11] <sinzui> mwhudson: hmm. I think I did make a mistake. that branch looks familiar. I think I meant to assign it to bigjools
[03:11]  * barry wants with statements!
[03:11] <thumper> FWIW abentley is working on the branch that will email the reviewer when nominated for a review
[03:11] <mwhudson> will the zope upgrade get us any close to being ready to use 2.5 ?
[03:11] <barry> thumper: how long before you think we'll be able to do all our reviews in lp?
[03:11] <sinzui> barry: start debugging the zope 3.4 branch. It works except for that giant hole I putting in importfacsist.
[03:12] <thumper> hacky ones in about 3 or 4 weeks
[03:12] <thumper> with shiney happy people, probably about 8
[03:12] <barry> sinzui: fred drake tells me they (zc) runs zope all the time on py2.5
[03:12] <barry> thumper: that will rock
[03:13] <mwhudson> sinzui: ok, are you going to move that branch out of my queue?
[03:13] <sinzui> barry: I bet. but importfascists and zope 3.4 conflict. I sacrificed the former for zope
[03:13] <sinzui> mwhudson: I will
[03:13] <mwhudson> sinzui: thanks
[03:13] <barry> sinzui: dang.  where's the branch?
[03:14] <barry> anyway.  moving on...
[03:14] <barry> [TOPIC]  * Mentoring update
[03:14] <MootBot> New Topic:   * Mentoring update
[03:14] <barry> skipped as usual...
[03:14] <mwhudson> n/a
[03:14] <barry> [TOPIC]  * Review process
[03:14] <MootBot> New Topic:   * Review process
[03:14] <barry>    * make pqm not broken
[03:15] <barry> so... thumper i'm told you're pqm-man now! :)
[03:15] <thumper> hah
[03:15] <jml> what's the PQM break?
[03:15] <jml> looks just fine to me.
[03:15] <jml> or at least as good as it's ever been :)
[03:15] <barry> there are several problems, but the most severe symptom is that branches are getting disappeared all the time
[03:16] <barry> they fail and nobody gets notified
[03:16] <jml> ahh ok.
[03:16] <jml> that *is* pretty bad.
[03:16] <sinzui> Isn't that directly cause by the make check change from 60 minutes to 15 minutes?
[03:16] <barry> jml: it's doubly bad because grepping through the logs is PAINFUL
[03:16] <barry> since the logs don't include branch information unless they succeed :(
[03:16] <thumper> barry: where are the logs?
[03:16] <barry> sinzui: yes, that's what flacoste thinks
[03:16] <mwhudson> this sounds like something that needs help from an OSA
[03:16] <jml> I see.
[03:16] <mwhudson> not us
[03:17] <barry> thumper: devpad
[03:17] <thumper> barry: where on devpad?
[03:17] <mwhudson> sinzui: i don't see the connection instantly
[03:17] <barry> thumper: /srv/lp-pqm-logs
[03:17] <thumper> barry: thanks
[03:17] <jml> barry: do we know that this is a bug in PQM itself?
[03:18] <barry> francis and i talked about a few things: first, if merges get killed, we should still see notifications, which i think today we don't
[03:18] <barry> jml: we don't, but that's the suspicion
[03:18] <barry> second issue is figuring out why sometimes you get no output for 15+ minutes
[03:18] <mwhudson> but the vicious killing after 15 minutes isn't anything to do with pqm
[03:19] <mwhudson> it's something test.py or shh.py or something does
[03:19] <barry> mwhudson: aren't the logs maintained and sent by pqm?
[03:19] <mwhudson> yes
[03:19] <jml> barry: has anyone tried to reproduce the issue?
[03:19] <mwhudson> but pqm doesn
[03:19] <barry> mwhudson: right.  but no one has reproduced the 15 minute hang on dev boxes
[03:19] <mwhudson> 't look at the logs
[03:20]  * jml meant the lack-of-notification issue.
[03:20] <barry> jml: not to my knowledge
[03:20] <jml> barry: ok.
[03:20] <barry> third problem was lack of branch information in the logs unless the merge succeeds
[03:21] <barry> but if pqm doesn't write the logs, i guess it can't do that
[03:22] <thumper> pqm does write a log file
[03:22] <thumper> several actually
[03:22] <barry> we need a plan though, 'cause it's killing productivity.  e.g. branches are getting stacked up beause parent branches aren't landing and then no one knows why
[03:22] <jml> in my ignorance, I'd say that PQM ought to keep a log of requests it receives and what it does with them.
[03:22] <thumper> jml: it does
[03:22] <thumper> barry: are the bugs filed against pqm?
[03:22] <jml> thumper: but not enough information to identify the branch?
[03:22] <mwhudson> i can't find any logs on devpad in the last 50 that seem to have been killed for lack of activity
[03:23] <mwhudson> but maybe i'm grepping for the wrong things
[03:23] <jml> thumper: well, the first issue isn't yet known to be an issue with PQM.
[03:23] <barry> thumper: i don't know.  i will talk with francis in the morning and get some bugs opened
[03:23] <jml> thumper: although it's a decent place to start, I guess.
[03:23] <thumper> pqm is still pretty fresh in my mind
[03:23] <thumper> so feel free to send them my way if urgent
[03:24] <barry> thumper: thanks.  i think the lack of notification is the most immediate concern.  devs could fix their branch problems if they knew why they bounced
[03:24] <barry> [ACTION] barry to talk with francis and get pqm bugs opened
[03:24] <MootBot> ACTION received:  barry to talk with francis and get pqm bugs opened
[03:25] <barry> that's all i've got.  does anybody have anything not on the agenda?
[03:26] <barry> nope?  okay, thanks everyone! see you next week
[03:26] <barry> #endmeeting
[03:26] <MootBot> Meeting finished at 21:28.
[03:26] <mwhudson> barry: why do you think that branches are getting killed for inactivity?
[03:36] <barry> mwhudson: we don't know.  it shouldn't be happening
[03:36] <mwhudson> barry: no, i mean
[03:37] <mwhudson> barry: what is it that makes you think the branches are being killed in this way?
[03:37] <mwhudson> barry: because i don't see much evidence for it myself
[03:37] <barry> mwhudson: we're just guessing.  it seems like branches are disappearing more now that we bumped that down, but it's just a guess
[03:38] <mwhudson> barry: are the branches disappearing to the extent of not producing logs on devpad?
[03:39] <barry> mwhudson: no, they are producing logs afaict.  mars found one of my disappeared branches in the logs after some clever sleuthing
[03:39] <mwhudson> barry: ok, because grepping logs suggests only two branches have been killed for hanging since the change to a 15 minute timeout
[03:40] <barry> mwhudson: interesting.  i think in my case, bzr actually complained (branch format problem because this was a loomed branch).  so i wonder if more of them are happening because bzr failures don't get notifications
[03:41] <mwhudson> it's possible