[15:00] <bac> #startmeeting
[15:00] <MootBot> Meeting started at 08:59. The chair is bac.
[15:00] <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
[15:00] <jelmer> hi bac
[15:00] <bac> who is here?
[15:00] <allenap> me
[15:00] <gary_poster> me
[15:00] <jcsackett> me
[15:00] <sinzui> me
[15:00] <abentley> me
[15:00] <jelmer> me
[15:00] <henninge> me
[15:00] <bac> jtv is on vacation
[15:00] <bigjools> me
[15:00] <flacoste> me
[15:00] <bac> er, unavailable
[15:00] <gary_poster> leonardr also unavailable
[15:00] <EdwinGrubbs> me
[15:00] <adeuring> me
[15:02] <mrevell> me
[15:02] <bac> now we can start.
[15:02] <bac> first, welcome mrevell to the reviewers meeting and to being a UI mentat!
[15:02] <mrevell>  Thanks :)
[15:03] <bac> please, everyone, request UI reviews from mrevell and salgado first
[15:03] <salgado> me!
[15:03] <bac> there are no new agenda items to discuss today
[15:03] <bac> [topic] outstanding items
[15:03] <MootBot> New Topic:  outstanding items
[15:03] <bac> * sinzui -- lint changes?
[15:04] <benji> me
[15:04] <sinzui> no, none today
[15:04] <bac> sinzui: is this weekly reminder useful or should we drop if from the list and just be surprised when it happens?
[15:05] <sinzui> bac: it should be a bug in lp-foundations I think
[15:05] <bac> sinzui: if you'll file the bug i'll promise never to mention it here again.  :)
[15:05] <sinzui> I will do so
[15:06] <bac> * bac to do stuff with maris regarding introduction of design metrics into reviews.
[15:06] <bac> done
[15:06] <gary_poster> sinzui, please feel free to triage it as you feel appropriate, since you know the context/importance better than I
[15:06] <bac> i sent out an email just a little while ago.
[15:06] <sinzui> I will
[15:06] <bac> i'm proposing a three week cycle where we concentrate on a new design metric in our reviews each cycle
[15:06] <bac> first up, is test performance
[15:07] <bac> the goal specified on the wiki is tests for a class should execute in under 2 seconds.  the goal is clearly arbitrary.
[15:08] <bac> but i think it is worthwhile to *pay attention* to new tests and the time they take as well as changes to timing for modified tests
[15:09] <bac> if you haven't, please read the email on the list and follow up there, unless you have comments now.
[15:10] <bac> any comments?
[15:10] <abentley> bac: So we're not actually enforcing the guideline, just checking that new tests execute in < 2 seconds?
[15:10] <abentley> bac: and modified ones don't increase by 2 seconds?
[15:11] <bac> abentley: no, i don't think strict enforcement of that guideline is called for.
[15:11] <bac> but i think reviewers and developers need to get into the habit of paying attention to test performance.
[15:11] <benji> bac: you might be interested in the --slow-test option of the test runner
[15:11] <bac> personally, i used to rarely look at the time new tests took
[15:11] <abentley> bac: Maybe we should change it to the metric we *are* checking, then?
[15:11] <bac> benji: can you elaborate
[15:12] <benji> if you use -c (colorized output) it will highlight tests that take longer than the specified number of seconds
[15:12] <sinzui> Looks like someone will be deleting notfound-traversals soon
[15:13] <bigjools> heh
[15:13] <bac> yay
[15:13] <jcsackett> bac: is the "test for a class" mean a given test case class, or given coverage for, say, a model class?
[15:13] <benji> oh, and you have to use -vvv
[15:13] <bigjools> where do doctests fit into that timing?
[15:13] <bac> jcsackett: good question
[15:13] <gary_poster> benji: +1 on making quick reply to bac's email with example :-)
[15:13] <benji> a single doctest is a single test, IIRC
[15:13] <bac> as you see the metric is vague and arbitrary
[15:13] <benji> gary_poster: k
[15:14] <bigjools> they'll all blow the 2 seconds then :)
[15:14] <bac> but paying attention to test timing is what i'm proposing, regardless of where we draw the line
[15:14] <bac> in the example case i gave in the email, i think the original test took something like 42 seconds
[15:14] <jcsackett> bac: so in other words, we can disregard the "2 seconds" thing and instead go with a "good lord the tests for this branch take a long time"
[15:14] <bac> after tweaking i got it down to under 10 seconds
[15:14] <bac> 10
[15:15] <bac> 10 > 2 but the exercise was worthwhile
[15:15] <bac> jcsackett: yes, that sounds good
[15:15] <jcsackett> bac: cool. thanks.
[15:15] <abentley> bac: There's some inconsistency between what you said in the email and what you're saying here.  In the email, you talked about incorporating the existing metric of all tests for a class.  Now, you're talking about just new and modified tests.
[15:16] <bac> abentley: i don't think so
[15:16] <gary_poster> "tests for a class" seems a bit hard to limit to me, fwiw
[15:17] <abentley> bac: Okay, could you explain why the inconsistency I perceive is an illusion?
[15:17] <bac> abentley: from the email "To incorporate this metric into your reviews you'll obviously need to run the tests.  If a test is brand new then you can just evaluate the timing.  Modifications to existing tests should be compared with before and after timings.  So reviewers, either download the code being reviewed and run the tests yourself or ask the developer for comparative timings.  Start the conversation.
[15:17] <bac> "
[15:17] <bac> that paragraph mentions new tests and modified tests
[15:18] <gary_poster> Can we just say that the phrase "tests for a class" isn't super pertinent right now, and that what you just quoted is more important?
[15:18] <abentley> bac: okay, the inconsistency is within the email, too.  I give up.
[15:19] <bac> abentley: i quoted from the Architecture Guide and then went on to explain what i was proposing.
[15:19] <bac> with the caveat that the guide is mutable
[15:19] <bigjools> can we paraphrase this as "please review how long your tests are taking"
[15:19] <gary_poster> +1 :-)
[15:20] <bac> bigjools:  lovely, +1
[15:20] <bigjools> great.  Next?
[15:20] <bac> yes, any other topic to discuss?
[15:21] <bac> thanks all.
[15:21] <bac> #endmeeting
[15:21] <MootBot> Meeting finished at 09:21.
[15:21] <bigjools> thanks bac
[15:21] <gary_poster> thank you
[15:21] <abentley> bac: You suggested that we incorporate "Tests for a class should complete in under 2 seconds" by checking new and modified tests, and that's and inconsistency in my view.
[15:21] <adeuring> thanka bac
[15:21] <abentley> bac: dammit, was typing.
[15:21] <abentley> bac: I have a topic.
[15:22] <bac> abentley: can it be deferred?
[15:22] <abentley> bac: sure.
[15:22] <bac> abentley: thanks, please add to the wiki for next week.