[12:09] <gary_poster> bac, just you and me today
[12:09] <bac> oic
[12:09] <bac> ok
[12:09] <gary_poster> call as soon as you show up
[12:10] <gary_poster> no rush ;-)
[13:56] <gary_poster> back in a sec
[14:51] <gary_poster> Whenever I see "Force a build" on buildbot for some strange reason I always think "Bust a move"
[16:30] <gary_poster> bac, going to go get some lunch.  If you have something even partially working with the --load-test thing, I'd love to try it, or pair with you or something.  It will greatly help with triaging the bugs that we get more and more of, I hope.
[17:10] <bac> gary_poster: my --load-list (basic) is working now and i just completed my first test run
[17:45] <gary_poster> yay bac.  I would love to use it for help with diagnosis, whenever you have a branch I can mess with.
[17:46] <bac> ok, i'll push something soon
[17:46] <gary_poster> cool thanks
[18:05] <bac> gary_poster: pushed to lp:~bac/launchpad/ordered-load-list
[18:06] <bac> gary_poster: i was able to trim the test failures produced by the branch.txt ordering bug down to the 110 on the DatabaseFunctionalLayer and the failure is shown
[18:06] <gary_poster> ordered-load-list: awesome, thanks bac.  writing doc for tomorrow's meeting then will return to running diagnostics for the bugs.
[18:06] <gary_poster> branch.txt: that's one of the bugs we have on the board?
[18:07] <gary_poster> No need to watch, but if you are interested, the doc is https://docs.google.com/a/canonical.com/document/d/1_BtI6VDHbHL5oFp8gPX8b1X9sKfK1uj1pnjFbF1a9n4/edit
[18:07] <bac> gary_poster: that's the vocabularly bug i fixed by sorting the branch names.
[18:07] <bac> this branch, pre-fix, shows the bug cropping up with a small number of tests
[18:08] <bac> by using the new --load-list
[18:08] <gary_poster> bac, oh, interesting.  So, it is an ordering issue of some sort, and it's reproducable now.  excellent!
[18:09] <gary_poster> I suppose it does indicate a test isolation thing to investigate.  If you want to, I'd be fine with it, but I'd prefer for it to be after we dig into some of the other bugs
[18:46] <bac> ok.  it has just grabbed my curiousity
[18:46] <bac> http://pastebin.ubuntu.com/960718/
[18:46] <bac> if these five tests are run, with branch.txt last, the failure occurs.  eliminating any one of them causes the failure to not be seen.
[18:52] <gary_poster> wow
[18:53] <gary_poster> bac, if you want to be stumped, I'd prefer that you be stumped about, say, bug 992184 :-)
[18:53] <_mup_> Bug #992184: lib/lp/services/database/doc/textsearching.txt fails intermittently/rarely on parallel tests <paralleltest> <Launchpad itself:Triaged> < https://launchpad.net/bugs/992184 >
[18:53] <bac> ok
[18:54] <gary_poster> as the most stumpy of the stumps I've seen lately
[18:56] <benji> that's beautiful
[19:02] <gary_poster> lol
[19:53] <bac> gary_poster: when you have time could you review https://code.launchpad.net/~bac/launchpad/ordered-load-list/+merge/104287 ?
[19:53] <gary_poster> bac, will do it now.  My call is in 7.
[19:53] <bac> gary_poster: it'll only take you 6
[19:53] <gary_poster> :-) k
[19:56] <gary_poster> bac, how odd that the old code used a list for the tests.  A set, which it created along the way, would have been much more efficient for the "in" questions, and the order appears irrelevant to me.
[19:57] <bac> gary_poster: line 16 shows the old deleted code did use a set
[19:57] <gary_poster> bac, sorted will create a list
[19:58] <gary_poster> line 36 of the diff would have been faster with a set
[19:58] <bac> gary_poster: ah, right.  both the use of the set and the sorted destroy the original ordering
[19:58] <gary_poster> right
[19:59] <bac> gary_poster: and, of course, the test suites get populated not by the order of the tests list but the order they are seen in iterate_tests(suite)
[20:00] <bac> so, basically, we can say original order was not important.  :)
[20:00] <gary_poster> :-) right
[20:01] <gary_poster> my point was simply that you were concerned about efficiency of the new approach, and the old approach was not as efficient as it could have been, even given trivial optimizations/consideration.
[20:02] <gary_poster> oh, flacoste does not appear to be around.  maybe he is at that pre-UDS meeting.  In any case, no rush there apparently. :-)
[20:04] <gary_poster> bac, I question raising an exception (lines 55-56) but I'm considering.  If I am trying to rerun tests on a new branch/revision, should it really throw up because a test is no longer around?  If it really should, it would be nicer to throw up with *all* the missing tests.  Let me see what the old code did...
[20:05] <gary_poster> yeah the old code used it as a filter
[20:05] <gary_poster> it did not puke
[20:05] <gary_poster> I'm inclined to do the same
[20:05] <gary_poster> though could be argued otherwise, maybe
[20:06] <gary_poster> bac, if you are worried about efficiency, I think you could have a more efficient initial data structure.
[20:07] <gary_poster> rather than layers -> testnames -> suites
[20:07] <gary_poster> you could simply have testnames -> (layer, suite)
[20:08] <gary_poster> then you would no longer need find_layer
[20:08] <gary_poster> you could simply do a single dict lookup
[20:08] <gary_poster> that would probably be way way faster
[20:10] <gary_poster> also afaict we discard ordered_layers and do nothing with it
[20:10] <gary_poster> removing it would be a good idea
[20:10] <gary_poster> bac ^^
[20:10] <bac> looking
[20:13] <bac> yep, ordered_layers is a leftover from a previous attempt
[20:13] <bac> i like the improvements you sugggest, gary_poster
[20:13] <gary_poster> cool
[20:13] <bac> i didn't do a time comparison but it isn't noticably slow, as is.
[20:14] <bac> with your change it may be faster than the original...
[20:14] <gary_poster> I think it would probably still be simpler/less code in the way I suggest, at least
[20:15] <gary_poster> what do you think about the exception, bac?  treat the list as an ordered filter and silently ignore missing tests, or abort if things are not as we expect?
[20:15] <gary_poster> If someone cares, they will presumably be looking at the tests that are run
[20:15] <gary_poster> I dunno
[20:15] <gary_poster> could be argued either way
[20:15] <gary_poster> though I still incline to no exception
[20:17] <bac> i think maintaining the previous behavior is the tie breaker
[20:21] <gary_poster> cool
[20:21] <gary_poster> bac, I added one other note in the MP when I was summarazing our discussion:
[20:21] <gary_poster> - We should include a comment as to why the code is maintaining order, so that future code navigators will have a clue as to intent.
[20:25] <bac> ok
[20:32] <bac> gary_poster: so, here are the timings for the three versions: http://paste.ubuntu.com/960961/
[20:33] <bac> with your changes, it is minimally faster and has few LOC

[20:33] <gary_poster> bac, :-) cool
[20:33] <gary_poster> do you think it reads more nicely?
[20:34] <bac> yes, and the reader won't get creeped out by that awful looping find_layer
[20:34] <gary_poster> :-) cool
[20:34] <gary_poster> interesting that the initial version was so ,uch slower
[20:34] <gary_poster> m
[20:36] <gary_poster> bac, btw, please don't lose your ideas from this morning to include an option to only run tests in the layer of a given target test.
[20:37] <gary_poster> I thought that would be interesting to experiment with, and possibly a real time saver
[20:38] <bac> gary_poster: no, i haven't forgotten.  but, i discovered it is quite easy to discover using --load-list / --list-tests in conjuction so that you can manually trim the input list
[20:39] <bac> but making that automatic would be swell
[20:39] <gary_poster> bac, oh, what is that trick?  I don't understand yet
[20:39] <bac> bin/test --load-list in.txt --list-test > out.txt
[20:40] <gary_poster> but what does that buy you?
[20:40] <bac> out.txt will then show the layer breaks
[20:40] <gary_poster> oh!
[20:40] <gary_poster> cool
[20:40] <gary_poster> remember that trick for Friday sharing time please bac :-)
[20:40] <bac> yeah, so my itch got less itchy when i found that
[20:40] <gary_poster> yeah
[20:40] <bac> +1
[20:41] <bac> thanks for the reminder...
[20:41] <bac> it's hell being clever and forgetful.  you don't get to show off.
[20:51] <bac> gary_poster: your review was just a comment, not an approve.  i've pushed the changes if you'd like to vote.
[20:52] <gary_poster> bac, cool, going
[20:54] <gary_poster> bac, all approved.  thank you!
[20:55] <bac> thx