/srv/irclogs.ubuntu.com/2009/06/10/#ubuntu-testing.txt

arajcollado: hey!12:13
jcolladoara: Hola12:14
arajcollado: I just wanted to ask you about your comment on mago vs. libmago12:14
aradon't you really think it will be confusing to have both as "mago"12:14
ara?12:14
davmor2hello all12:14
jcolladoara: What's the difference between mago and libmago?12:15
arahey davmor212:15
arajcollado: mago is the whole framework, meaning: the library, the runner and the tests. libmago would be just the library12:16
jcolladoara: Then we might use python-mago12:17
arajcollado: yes, that's another possibility12:17
jcolladoara: Other thing to take into account if we proceed with the changes regarding the test suite discovery12:18
jcolladoara: is that the runner is now port of the library. Should we split the library in two different parts?12:18
arajcollado: mmm, tricky...12:19
arajcollado: here, the question behind would be: does it make sense to have the library installed without the runner?12:21
arajcollado: I think it does12:21
jcolladoara: As long as it's possible to run the test cases without the runner, yes.12:22
arajcollado: well, not the "test cases" as we use it now, but it would be possible to use the library as a standalone library to create scripts without the testcases library if people want to12:23
arajcollado: not as something that we encourage to do, but as technically possible12:23
jcolladoara: Then the library should contain only the applications part, but not the runner and the test suite part12:24
jcolladoara: Do you agree?12:26
arajcollado: I think that both solutions have pros and cons12:30
jcolladojcollado: Maybe we can move this discussion to the maillist so that everybody can think about it12:31
jcolladoara: ^12:32
arajcollado: :D12:32
jcolladoWorking at home is bad. You may eventually talk to yourself.12:32
arajcollado: yes, but first I would like to merge the mago branch to trunk, announce it in the mailing list, and then discuss the rest of the stuff12:32
jcolladoara: OK. I understand the priorities12:33
arajcollado: I will rename desktoptesting to mago, if you think is more appropriate, and then discuss the rest of the things12:33
arajcollado: is that ok?12:33
jcolladoara: Sure12:33
* ara -> lunch12:45
Opr8iVeGood morning!14:42
davmor2hello14:51
Opr8iVeWas wondering if youall wanted install results for 9.04 unr on the brand new Dell Mini10v14:56
Opr8iVeif so, where would an apropriate place be to submit that?14:57
persiaOpr8iVe, We've completed the test phase for the 9.04 images (which is why they are shipped).14:59
persiaThat said, if the overall experience is not so good, the best thing to do is file bugs.14:59
Opr8iVeYes, but theres a list of known compatable / issues hardware there, and the new Dell Mini 10 series isnt listed15:00
persiaIf the overall experience is good, and you're up for reverifying it during the karmic cycle, so 9.10 can be even better, there's a wiki page.15:00
Opr8iVeThe install went great! Once I figured out that Imagewriter chokes if theres spaces in the path to the image file.15:01
persiaWhich imagewriter did you use?15:01
Opr8iVewhatever 9.04 apt-get gave me15:01
Opr8iVeIm not at that computer right now, so I can't be sure15:01
persiaOK.  When you can find out, please file a bug.  That should be fixed.15:02
persiaIf it works great, please add notes on https://wiki.ubuntu.com/HardwareSupport/Machines/Netbooks15:02
Opr8iVeah.. cool.. thats what I was looking for.. Thanks!15:02
persiaWe'd appreciate it if you'd also volunteer to help test the Alpha and Beta images for the 9.10 release as they become available, to maintain the documentation on that model.15:03
persia(right now, Jaunty is the right thing to have documented.  Come 9.10 beta (or maybe a bit earlier), there will be an effort to update the page.15:03
persiaAnd, once there's so many models there it becomes hard to track, we'll drop the page, and no longer have to warn people that it may not work on some models.15:04
Opr8iVeSure.. Tho, Im no *nix guru.. I've played with it for years on and off, but am comfortable with the cli15:04
Opr8iVeBeen (almost) windows free for 1.5 years now15:04
Opr8iVewife still needs some convertin' :)15:05
persiaExcellent.  The milestone schedule is at https://wiki.ubuntu.com/KarmicReleaseSchedule15:05
persiaWe usually start testing (in this channel) on the Tuesday for a given Thursday milestone, just to knock out the kinks.15:06
charlie-tcaIs the tracker going to be updated for alpha 2?15:06
persiaSo, the Alpha 2 image testing ought start sometime soonish.15:06
persiacharlie-tca, That's a good question.  One hopes so, as it's that time.15:06
Opr8iVeRather relentless schedual yall keep, heh..15:07
charlie-tcaMy thought, exactly. Supposed to release tomorrow15:07
persiaOpr8iVe, Well, we need to test quite a bit to be able to make sure that we have a decent release each six months.  Most of the "Alpha" milestones generate *lots* of bugs, but as we get closer, it becomes more just making sure nothing big broke since last time, and the big bugs from last time are really closed.15:08
charlie-tcabbl15:10
Opr8iVeokay.. Im gonna fire up the ubuntu partition.. Probably be back later.15:16
=== smb is now known as smb-topper
=== smb-topper is now known as smb
slangasekalpha2 candidate images posted17:27
slangasekand back down again, grub-pc was missing from the disks18:15
eeejayhey cr3?18:38
cr3eeejay: yo18:40
eeejaycr3, i am having a hard time deciding how we should be harvesting mago results in checkbox18:41
eeejaycr3, should i have it embedded in test.description?18:41
cr3eeejay: can you pastebin or upload the output of mago?18:43
eeejaycr3, the test results in mago don't go to stdout, they are written to a result file18:43
cr3eeejay: right, that output :)18:44
cr3eeejay: I have a feeling it is similar in concept to lsb which also creates some result file in its own format18:44
eeejaycr3, first of all, this is the last log line, after running a mago script: http://paste2.org/p/25849318:44
cr3eeejay: the solution is essentially to parse the result file and create new test instances for each test parsed: test = Test(None, name=entry["section_name"], description=entry["subsection_name"], plugin="lsb", suite=self.test.name)18:45
cr3eeejay: that example was taken from the lsb_prompt plugin in checkbox-certification18:45
eeejaycr3, ah. i will look more closely at that18:45
cr3eeejay: so, "yes" to your original question but, just to be clear, not necessarily through assignment (test.description = "foo") as much as through instantiation (above example)18:46
eeejaycr3, ok. gotcha18:46
* eeejay looks at lsb things18:47
cr3eeejay: you can then create a test result object like this: TestResult(test, status, entry["log"], entry["duration"])18:47
cr3eeejay: to make things easier for you, simply search for _journal_entry_to_result18:47
cr3where "journal" is the name given by the lsb folks for their output file18:48
=== fader is now known as fader|lunch
cr3eeejay: by the way, remember that you need to fire report-test and report-result evens so that other plugins might know about your new instances19:34
eeejaycr3, ah ok19:35
eeejaycr3, thanks for the png19:35
eeejaycr3, i hung it over my bed19:35
cr3eeejay: are you now settled in seattle?19:36
eeejaycr3, still looking for a house and such19:36
cr3eeejay: so when you say "bed", do you mean "cardboard box under a bridge"? :)19:37
eeejaycr3, no - we have homeless shelters here in seattle.19:37
cr3too bad, I have lots of good tips for living in a box19:37
eeejaycr3, hah19:38
cr3for example, gerbils are not only a good source of protein but they can also serve as entertainment too19:38
eeejaycr3, i think it will be cleaner if i subclass TestCommand in the MagoInfo plugin19:39
eeejaycr3, as opposed to MagoPrompt19:39
eeejaybye ara!19:39
cr3eeejay: that might preferable but it breaks my heart seeing integrators having to know so much about the core :(19:40
eeejaycr3, too late. i have a book deal with o'rielly19:40
cr3fortunately, the core isn't too big, but it's more than I feel comfortable exposing19:40
cr3"Checkbox: the good, the bad and the ugly"19:41
cr3make sure you have a good pic of me for the introduction to the "ugly" chapter19:41
eeejay"Control your Roomba with Checkbox"19:41
eeejaycr3: http://paste2.org/p/25869219:44
cr3eeejay: yeah, SystemExit was evil19:46
eeejaycr3, SystemExit is my addition. on't know how you want to deal with it, but it would be nice to be able to kill checkbox when it has a lingering thread19:47
cr3eeejay: ah, I used to do sys.exit(1) when KeyboardInterrupt was caught. one moment, I'll check why I removed that19:50
cr3eeejay: I made a related fix for bug #327810 in revno 45719:55
ubot4Launchpad bug 327810 in checkbox "Interrupt signal not handled properly for cli and gtk interfaces" [Medium,Fix released] https://launchpad.net/bugs/32781019:55
cr3eeejay: I'll check if introducing that signal doesn't revert the bug19:56
eeejaycr3, Test.command is confusing19:57
eeejaycr3, it is a string for a second, and then a TestCommand instance19:57
cr3eeejay: yeah, Test.description too. not one of my proudest moments19:58
eeejaycr3, i have had some low moments myself, besides living in a homeless shelter19:59
cr3eeejay: what behavior changes when you wrap the thread.join call in a try/catch block? I'm running checkbox in cli and gtk mode and I'm not observing that much difference20:01
cr3I think what we want is for the parent thread to also exit in the event the child is killed20:02
eeejaycr3, right20:02
eeejaycr3, i press ctrl+c20:02
eeejaycr3, get a KeyboardInterrupt stack trace printed20:02
eeejaycr3, but all keeps running20:02
cr3it doesn't seem like the exception is being propagated though. if I hit ctrl-c in cli mode when pulsating, I get thrown to the final prompt20:02
eeejaycr3, and then i press ctrl+c a gazzilion times in anger, but nothing20:02
cr3I get a similar behavior when I kill the pulse window in gtk mode... final propmt20:03
eeejaycr3, ok. so we misunderstood each other20:03
eeejaya term signal should terminate the entire deal, imho20:03
cr3moment... call20:04
eeejaycr3, when i get this mago crap working you could experiment with that. they are more complex processes than the usual scripts checkbox runs20:04
=== fader|lunch is now known as fader
eeejaycr3: https://code.launchpad.net/~eeejay/+junk/checkbox-mago20:43
eeejaycr3, i can't figure out how to fire report-result with the correct TestResult instance20:43
eeejaycr3: interface.show_test() returns a new instance that has little to do with the actual test20:44
eeejaycr3: lunch!20:44
cr3eeejay: ok, off the phone finally!22:09
eeejaycr3. finally!22:09
cr3eeejay: so, why do you grab the output of stderr in MagoCommand.post_execute?22:10
eeejaycr3, because that is where the log output is22:10
cr3eeejay: didn't you say mago created a log file?22:12
cr313:43 < eeejay> cr3, the test results in mago don't go to stdout, they are written to a result file22:12
eeejaycr3, there is the regular debug log, which goes to stderr, but the actual results are written to file22:12
cr3eeejay: ah, so you grab the debug log in order to find the results file, right?22:13
eeejaycr3, xactly!22:13
cr3eeejay: does mago support a -o or --output command line parameter to specify the path to the result file?22:14
eeejaycr3, now i got it doing "report-test" with the right results, but i don't know what happens now...22:14
cr3... which could potentially take a dash (-) as argument to specify stdout? :)22:14
eeejaycr3, probably22:14
eeejaycr3, i doubt it supports dash22:14
eeejaycr3, but that is minutiae, i figured out how to populate result.data properly.22:15
eeejaycr3, it could be pretty, i'll work on that22:16
eeejaycr3, but now after report-test, what comes next?22:16
eeejaycr3, pushing more changes22:17
cr3eeejay: at that point, you probably want to look at shell_prompt.py for the MagoPrompt implementation22:18
cr3ie, you want to show a progress bar and run mago in a child thread22:18
cr3by the way: directory = Path(default="%(dt_path)s")22:19
cr3and you don't want to self._manager.reactor.fire("report-result", result) in MagoCommand22:19
eeejaycr3, i only et the right report if i fire it from MagoCommand22:20
eeejaycr3, show_test() returns the wrong result22:21
eeejaycr3, i already got the progress bar22:21
cr3eeejay: mago should be automated, so you shouldn't need to show anything other than the pulse bar22:21
eeejaycr3, oh.. so no wizardy pages?22:22
cr3eeejay: I suspect MagoPrompt might not even be needed. try changing the plugin from "mago" to "shell" when creating new test instances22:22
eeejaycr3, ok22:23
cr3eeejay: well, from my understanding of mago, those wizardy pages shouldn't really be necessary22:23
eeejaycr3, i wanted it to be an option, to run each individual mago suite interactively22:23
cr3eeejay: in that case, s/shell/manual/ :)22:23
* eeejay first tries shell22:24
cr3in other words, since you deriving MagoCommand, you can probably reuse the manual_prompt or shell_prompt plugins22:24
cr3base on your sample xml, I'm assuming each mago test has a single result. if this is not necessarily the case, that can be handled too22:26
cr3by the way, if you reuse the shell or manual prompt plugins, you shouldn't need to fire the report-result event in MagoCommand.post_execute anymore22:27
cr3and, therefore, you shouldn't need to pass the manager instance as an additional argument to the constructor... you might not even need to override the constructor at all22:27
cr3eeejay: I hope things are looking good...22:36
eeejaycr3, sorry, wifi issues here at the shelter22:45
eeejaycr3, everyone is back from lunch at the mission and trying to check e-mail22:45
cr3eeejay: I feel we're pretty close to having something working nicely so I might be getting overly excited22:49
* cr3 gets more coffee.. not bouncing off the walls enough22:49
eeejaycr3, if i don't explicitly fire report-result in post_execute(), i don't get the result in the final report23:05
eeejaycr3, i just get an empty "skip" result23:05
eeejaybut if i fire the event, i get both23:05
cr3eeejay: can you try: 1. push the current state of your branch; 2. comment out the firing of report-result from post_execute; 3. run checkbox with the argument --log-level=debug; 4. pastebin the output23:15
eeejayum23:15
eeejayyeah?23:15
cr3that reminds me, I need to start using pastebinit23:15
eeejaycr3, http://dl.getdropbox.com/u/280491/submission.xml23:25
eeejaycr3, http://dl.getdropbox.com/u/280491/checkbox-mago.log23:26
cr3eeejay: aha! so, if you're running tests manually: 1. the result status will be taken from the radio buttons which is set to skip by default; 2. the result data will be taken from the comment box.23:30
cr3eeejay: try with plugin = "shell" just for kicks23:30
cr3perhaps an improvement could be that running the test should reset the radio buttons based on the status of the command23:31
eeejaycr3, yes!23:31
eeejaycr3,  please23:31
eeejaycr3, before i run them as shell, i need to put whitelist blacklists in place23:31
cr3eeejay: I'll see if I can kick that change in the trunk quickly, one moment23:32
eeejaycr3, it is bizarre how in the gtk user interface in makes a new testresult instead of using the one from the test.23:33
cr3the preconception was that the manual feedback was the result23:34
cr3if the test outputs data and the user provides a comment, what to do?23:35
eeejaythe test's result should populate the UI23:36
eeejayat least in half-manual tests23:36
cr3eeejay: so the comment box should be filled with the output of running the test?23:37
cr3I have considered that before but it felt weird to me. if it feels alright to other people, I'm easy23:37
eeejaycr3, more importantly the "yes"/"no" should be changed23:37
cr3eeejay: I'm done with the gtk interface, now doing the cli interface23:39
cr3eeejay: fix pushed23:47
eeejayi am off for a bit23:54
cr3eeejay: pip pip, cheerio23:56

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!