| gary_poster | bac, just you and me today | 12:09 |
|---|---|---|
| bac | oic | 12:09 |
| bac | ok | 12:09 |
| gary_poster | call as soon as you show up | 12:09 |
| gary_poster | no rush ;-) | 12:10 |
| gary_poster | back in a sec | 13:56 |
| gary_poster | Whenever I see "Force a build" on buildbot for some strange reason I always think "Bust a move" | 14:51 |
| gary_poster | bac, going to go get some lunch. If you have something even partially working with the --load-test thing, I'd love to try it, or pair with you or something. It will greatly help with triaging the bugs that we get more and more of, I hope. | 16:30 |
| bac | gary_poster: my --load-list (basic) is working now and i just completed my first test run | 17:10 |
| gary_poster | yay bac. I would love to use it for help with diagnosis, whenever you have a branch I can mess with. | 17:45 |
| bac | ok, i'll push something soon | 17:46 |
| gary_poster | cool thanks | 17:46 |
| bac | gary_poster: pushed to lp:~bac/launchpad/ordered-load-list | 18:05 |
| bac | gary_poster: i was able to trim the test failures produced by the branch.txt ordering bug down to the 110 on the DatabaseFunctionalLayer and the failure is shown | 18:06 |
| gary_poster | ordered-load-list: awesome, thanks bac. writing doc for tomorrow's meeting then will return to running diagnostics for the bugs. | 18:06 |
| gary_poster | branch.txt: that's one of the bugs we have on the board? | 18:06 |
| gary_poster | No need to watch, but if you are interested, the doc is https://docs.google.com/a/canonical.com/document/d/1_BtI6VDHbHL5oFp8gPX8b1X9sKfK1uj1pnjFbF1a9n4/edit | 18:07 |
| bac | gary_poster: that's the vocabularly bug i fixed by sorting the branch names. | 18:07 |
| bac | this branch, pre-fix, shows the bug cropping up with a small number of tests | 18:07 |
| bac | by using the new --load-list | 18:08 |
| gary_poster | bac, oh, interesting. So, it is an ordering issue of some sort, and it's reproducable now. excellent! | 18:08 |
| gary_poster | I suppose it does indicate a test isolation thing to investigate. If you want to, I'd be fine with it, but I'd prefer for it to be after we dig into some of the other bugs | 18:09 |
| bac | ok. it has just grabbed my curiousity | 18:46 |
| bac | http://pastebin.ubuntu.com/960718/ | 18:46 |
| bac | if these five tests are run, with branch.txt last, the failure occurs. eliminating any one of them causes the failure to not be seen. | 18:46 |
| gary_poster | wow | 18:52 |
| gary_poster | bac, if you want to be stumped, I'd prefer that you be stumped about, say, bug 992184 :-) | 18:53 |
| _mup_ | Bug #992184: lib/lp/services/database/doc/textsearching.txt fails intermittently/rarely on parallel tests <paralleltest> <Launchpad itself:Triaged> < https://launchpad.net/bugs/992184 > | 18:53 |
| bac | ok | 18:53 |
| gary_poster | as the most stumpy of the stumps I've seen lately | 18:54 |
| benji | that's beautiful | 18:56 |
| gary_poster | lol | 19:02 |
| bac | gary_poster: when you have time could you review https://code.launchpad.net/~bac/launchpad/ordered-load-list/+merge/104287 ? | 19:53 |
| gary_poster | bac, will do it now. My call is in 7. | 19:53 |
| bac | gary_poster: it'll only take you 6 | 19:53 |
| gary_poster | :-) k | 19:53 |
| gary_poster | bac, how odd that the old code used a list for the tests. A set, which it created along the way, would have been much more efficient for the "in" questions, and the order appears irrelevant to me. | 19:56 |
| bac | gary_poster: line 16 shows the old deleted code did use a set | 19:57 |
| gary_poster | bac, sorted will create a list | 19:57 |
| gary_poster | line 36 of the diff would have been faster with a set | 19:58 |
| bac | gary_poster: ah, right. both the use of the set and the sorted destroy the original ordering | 19:58 |
| gary_poster | right | 19:58 |
| bac | gary_poster: and, of course, the test suites get populated not by the order of the tests list but the order they are seen in iterate_tests(suite) | 19:59 |
| bac | so, basically, we can say original order was not important. :) | 20:00 |
| gary_poster | :-) right | 20:00 |
| gary_poster | my point was simply that you were concerned about efficiency of the new approach, and the old approach was not as efficient as it could have been, even given trivial optimizations/consideration. | 20:01 |
| gary_poster | oh, flacoste does not appear to be around. maybe he is at that pre-UDS meeting. In any case, no rush there apparently. :-) | 20:02 |
| gary_poster | bac, I question raising an exception (lines 55-56) but I'm considering. If I am trying to rerun tests on a new branch/revision, should it really throw up because a test is no longer around? If it really should, it would be nicer to throw up with *all* the missing tests. Let me see what the old code did... | 20:04 |
| gary_poster | yeah the old code used it as a filter | 20:05 |
| gary_poster | it did not puke | 20:05 |
| gary_poster | I'm inclined to do the same | 20:05 |
| gary_poster | though could be argued otherwise, maybe | 20:05 |
| gary_poster | bac, if you are worried about efficiency, I think you could have a more efficient initial data structure. | 20:06 |
| gary_poster | rather than layers -> testnames -> suites | 20:07 |
| gary_poster | you could simply have testnames -> (layer, suite) | 20:07 |
| gary_poster | then you would no longer need find_layer | 20:08 |
| gary_poster | you could simply do a single dict lookup | 20:08 |
| gary_poster | that would probably be way way faster | 20:08 |
| gary_poster | also afaict we discard ordered_layers and do nothing with it | 20:10 |
| gary_poster | removing it would be a good idea | 20:10 |
| gary_poster | bac ^^ | 20:10 |
| bac | looking | 20:10 |
| bac | yep, ordered_layers is a leftover from a previous attempt | 20:13 |
| bac | i like the improvements you sugggest, gary_poster | 20:13 |
| gary_poster | cool | 20:13 |
| bac | i didn't do a time comparison but it isn't noticably slow, as is. | 20:13 |
| bac | with your change it may be faster than the original... | 20:14 |
| gary_poster | I think it would probably still be simpler/less code in the way I suggest, at least | 20:14 |
| gary_poster | what do you think about the exception, bac? treat the list as an ordered filter and silently ignore missing tests, or abort if things are not as we expect? | 20:15 |
| gary_poster | If someone cares, they will presumably be looking at the tests that are run | 20:15 |
| gary_poster | I dunno | 20:15 |
| gary_poster | could be argued either way | 20:15 |
| gary_poster | though I still incline to no exception | 20:15 |
| bac | i think maintaining the previous behavior is the tie breaker | 20:17 |
| gary_poster | cool | 20:21 |
| gary_poster | bac, I added one other note in the MP when I was summarazing our discussion: | 20:21 |
| gary_poster | - We should include a comment as to why the code is maintaining order, so that future code navigators will have a clue as to intent. | 20:21 |
| bac | ok | 20:25 |
| bac | gary_poster: so, here are the timings for the three versions: http://paste.ubuntu.com/960961/ | 20:32 |
| bac | with your changes, it is minimally faster and has few LOC | 20:33 |
| bac | </deadhorse> | 20:33 |
| gary_poster | bac, :-) cool | 20:33 |
| gary_poster | do you think it reads more nicely? | 20:33 |
| bac | yes, and the reader won't get creeped out by that awful looping find_layer | 20:34 |
| gary_poster | :-) cool | 20:34 |
| gary_poster | interesting that the initial version was so ,uch slower | 20:34 |
| gary_poster | m | 20:34 |
| gary_poster | bac, btw, please don't lose your ideas from this morning to include an option to only run tests in the layer of a given target test. | 20:36 |
| gary_poster | I thought that would be interesting to experiment with, and possibly a real time saver | 20:37 |
| bac | gary_poster: no, i haven't forgotten. but, i discovered it is quite easy to discover using --load-list / --list-tests in conjuction so that you can manually trim the input list | 20:38 |
| bac | but making that automatic would be swell | 20:39 |
| gary_poster | bac, oh, what is that trick? I don't understand yet | 20:39 |
| bac | bin/test --load-list in.txt --list-test > out.txt | 20:39 |
| gary_poster | but what does that buy you? | 20:40 |
| bac | out.txt will then show the layer breaks | 20:40 |
| gary_poster | oh! | 20:40 |
| gary_poster | cool | 20:40 |
| gary_poster | remember that trick for Friday sharing time please bac :-) | 20:40 |
| bac | yeah, so my itch got less itchy when i found that | 20:40 |
| gary_poster | yeah | 20:40 |
| bac | +1 | 20:40 |
| bac | thanks for the reminder... | 20:41 |
| bac | it's hell being clever and forgetful. you don't get to show off. | 20:41 |
| bac | gary_poster: your review was just a comment, not an approve. i've pushed the changes if you'd like to vote. | 20:51 |
| gary_poster | bac, cool, going | 20:52 |
| gary_poster | bac, all approved. thank you! | 20:54 |
| bac | thx | 20:55 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!