/srv/irclogs.ubuntu.com/2009/12/07/#ubuntu-testing.txt

=== techgeek is now known as astechgeek
=== astechgeek is now known as Guest13436
=== Guest13436 is now known as astechgek
=== nick is now known as Guest53926
=== Guest53926 is now known as techgeek
=== techgeek_ is now known as techgeek
=== fader|aw` is now known as fader_
davmor2-netbookmorning fader_13:31
fader_davmor2-netbook: Hey dude.  How's the back?13:33
davmor2-netbookfader_: tickedy boo old chap.13:34
* fader_ imagines davmor2-netbook doing an exaggerated Rimmer-style salute.13:35
davmor2-netbookOMG the yanks have nicked RD13:35
davmor2-netbookcr3: Morning Dude13:38
cr3davmor2-netbook: yo mama13:39
soreno/13:39
davmor2-netbookhello soren too13:39
soreno/ again13:39
soren:)13:39
sorenAre we using the autotest framework for anything anywhere?13:40
cr3soren: I have an integration of autotest in checkbox-certification, but I'm not using it because it takes too much time to run for too little return13:41
sorencr3: Fair enough. The kvm-autotest thingamabob is rather intertwined with it, but I'll see if I can somehow extract it.13:42
cr3soren: I have also made some improvements on the qa-regression-testing suite over the weekend, it's all under lp:checkbox now13:42
cr3soren: the most significant improvement is that I added the script filter_packages, which can be piped to the output of a suite to actually install the packages required by each test13:43
sorenAwesomeness.13:43
cr3soren: I have not received test results from the qrt suite this morning, so there seems to be a problem somewhere, I'll be looking into it first thing once I get caffeinated13:44
cr3sbeattie: ^^^ when you come in, you might like to know that the qrt branch discussed previously has been merged into the trunk13:44
sorencr3: So.. How much of the qrt suite have you integrated now?13:45
davmor2-netbookschwuk: are you working on the test tracker by any chance?13:45
cr3soren: fyi, if you want to run the qrt suite from lp:checkbox, you'll have to: 1. build and install the package so that the dbus backend can be installed; 2. un-blacklist the qrt suite which is not enabled by default because it is destructive: --config=checkbox/plugins/suites_prompt/blacklist=13:46
davmor2-netbookright off for a drive home then13:46
cr3soren: I have only enabled the two kernel tests now, see suites/qa_regression.txt.in. as you can observe, it should be rather trivial to add more tests as far as checkbox is concerned13:47
cr3soren: the difficult part will be making sure that the tests themselves in qrt can run in an automated fashion13:47
sorencr3: Right. And that's still outstanding, right?13:47
soren"Outstanding" not as in "awesome", but as in "not done yet".13:47
cr3soren: I think that's where most of the work for you and sbeattie lies13:47
soren(normal people would know this, but... well..)13:47
cr3soren: the context made it clear, even for abnormal people :)13:48
sorencr3: If you say so(!)13:48
cr3soren: for now, qrt tests define some information about them in the form of comments like QRT-Packages and QRT-Alternates13:49
cr3soren: these just describe package dependencies but, if we need other information, like "must be run as root", we can add support for that easily in checkbox13:49
cr3soren: in other words, I think we will need to consider extending QRT-* comments in order to run more tests in an automated fashion13:50
* cr3 hunts for coffee13:51
sorencr3: I think you're quite right.13:51
sorenWheee!14:00
* soren watches a Karmic server install shoot by, driven by kvm-autotest.14:00
cr3fader_: hey dude, might you have a cycle or two to spare today?14:07
fader_cr3: Sure, give me a bit to finish up my daily reports and I'm all yours14:08
sorenhttp://people.canonical.com/~soren/kvm-autotest-magic/14:25
fader_cr3: What's up dude?14:26
sorenIt may look like a preseeded install, but e.g. two thirds of the way in (when accepting the partitioning changes), you can see some UI interaction (TABbing to the "Yes" option, and hitting return to accept it).14:27
* soren tries a desktop install..14:29
cr3soren: if you need some preseeds, I have some templates that are known to work14:29
cr3... for both alternate and desktop installs, although I only do the later over nfs14:29
cr3fader_: ok, I would appreciate if you could determine a way to substitute the pm.py tests in checkbox-oem to reuse the existing suspend_test script in the base checkbox package14:30
fader_cr3: I'll take a look and see what I can puzzle out.14:31
cr3fader_: my motivations are first to leverage existing scripts as much as possible, especially considering the kernel team has put so much effort into that script, and also to remove dependencies on python-argparse for which there is no package on lucid yet14:31
cr3jcollado: ^^^ you may want to create lucid packages for checkbox-oem and python-argparse14:32
sorencr3: Don't need preseeds for this. That's the entire point. I want to emulate an interactive installation.14:33
cr3soren: I remember doing something like that we qemu, are you sure it exercises a significantly different part of the code compared to preseeding?14:34
cr3s/we/with/14:34
sorencr3: I'm not entirely sure I understand your question. Let's pretend my answer is "no". What happens then?14:36
cr3soren: first, lets make sure you're doing the same thing I once did with qemu: are you generating keyboard inputs to automate the alternate install?14:37
sorencr3: Yes.14:38
sorencr3: Mathiaz' automated testing setup has often been criticised of not doing what a human trying the installation would do, since it uses preseeding.14:38
cr3soren: in my experience, this is more brittle than preseeding (but that can be verified with cjwatson), so if generating keyboard inputs doesn't really exercise much more of the installer than preseeding, then I don't think that the brittleness offsets the benefits of testing a little bit of extra code14:39
sorencr3: So, to address this concern (which has some validity), I'm doing an install exactly like a human would do it.14:39
cr3soren: ah, if there has been such criticism, I suppose those people have assessed that a significant part of the installer is not being exercised by preseeding14:40
sorencr3: Well, the backend code to do everything is all the same.14:40
sorencr3: It's all the frontend magic that doesn't get exercised at all when you preseed.14:40
sorenI don't know what your setup did, but this one actually looks at the screen, waits for it to look correct within a given timeout, and if it does, it sends key events to continue the install.14:41
cr3marjo: can we get a dedicated server for kvm testing, similarly to vmware esx testing?14:41
cr3soren: it's been about two years since I last looked into qemu and I can barely remember what I had for breakfast, so I wouldn't know14:42
sorenmeh14:43
cr3soren: as usual, I'm quite useless :)14:44
sorenWell, my expectations were low, so it's ok.14:44
soren:p14:44
cr3soren: by the way, there's a freeze tomorrow in preparation for alpha-1 and I would like to take this opportunity to push my latest changes to checkbox in lucid. since mathiaz seems to be on holiday, would you mind reviewing the changes?14:46
sorenI could.14:47
sorenI'm not sure my review would be very thorough. I have no clue what's going on in that codebase (yet).14:48
cr3soren: there shouldn't be that many changes, my major refactoring is nowhere near ready so these would mostly be maintenance changes14:48
cr3soren: just make sure to ignore the line: if pwd.getpwuid(os.getuid()).pw_name == "soren" and random.random() > 0.5: raise Exception, "Something went wrong somewhere"14:50
sorenHah, my username is "marc". Nyah, nyah.14:51
cr3hm, I'll have to push more changes but also check for socket.gethostname() != "ilovesoren", to blacklist my own hostname14:53
soren:-$14:55
cr3I'm somewhat relieved you don't have the same hostname as me14:56
sorenMine's much more inappropriate than that15:03
moustafacr3, fader_ : Soupe du jour15:32
cr3moustafa: hey dude, you'll be working from home today?15:32
moustafacr3: that's the plan15:33
cr3moustafa: no problem, please check with fader_ if he has anything for you today. if not, I would suggest you write some more unittests15:35
moustafacr3: That was also part of the plan :)15:36
cr3good plan :)15:36
moustafacr3: Amazing the things I can plan while trying to get up15:36
fader_moustafa: The only thing I'd like to ask you to do is make sure you're set up to help test ISOs on Thursday :)15:36
moustafafader_: I'll definitely be ready for Thursday :)15:37
faganI can test the 32 and 64 bit isos on a very querky machine15:37
faganSo that will be a bit of fun15:38
cr3fagan: how is it querky? is it a Cray?15:38
fader_moustafa: w00t!  Have you seen the ISO test procedure?  https://wiki.ubuntu.com/Testing/ISO/Procedures15:38
faganI have an almost entirely nvidia shod machine15:39
fader_moustafa: I'd point you at the ISO tracker to look at the plans but it seems to be barfing today :(15:39
faganI tend to get a few oddities15:39
cr3fader_: we need to get a Cray into certification15:39
faganCray?15:40
faganOh15:40
faganI dont think many people have one of those in their back yard15:40
fader_cr3: If you find us one I'll find space for it. :D15:42
fader_cr3: And here I was thinking small and hoping for a PS3 or two to do... testing on.  Yeah, distro testing.15:42
faganId say you should ask on planet ubuntu for people to test corner cases like the PS315:43
moustafafader_ ...I have a ps3...15:46
fader_moustafa: Stick Ubuntu on there and tell us if it works :D15:47
* fader_ wonders if there's even a port for the Cell.15:47
moustafafader_: Last time I checked, the PS3buntu project uses a PowerPC kernel15:48
fader_Huh, I didn't realize the architectures were close enough to do that.15:51
fader_Nor that there was a PS3buntu project.15:51
moustafaAnd now you know15:52
fader_And knowing is half the battle!15:52
moustafaGeee Eyee Jooooooe!15:53
moustafafader_ : Basically, it's a PowerPC architecture kernel, uses only the alternate installer (as to compensate for CRT tvs), and is supposedly slow as hell since the PS3 has 256 Mb of RAM and we have no access to the video card15:56
faganI thought the PS3 has like 8 different CPUs all for different things15:59
faganHmmmm16:01
* fagan checks wiki16:01
moustafafagan: It does, but one is deactivated and one is reserved for the system16:03
moustafaI could try and burn Xubuntu on disc and see how well that goes16:03
faganinteresting16:04
moustafafader_ I can't seem to access iso.qa.ubuntu.com16:06
charlie-tcaI can't either16:07
* fagan trys16:07
charlie-tcaand the Xubuntu alternate 386 failed to install16:08
faganall the ubuntu websites seem to be slow today16:08
charlie-tcatracker gave me an error16:08
fader_moustafa: Right, that was what I meant by it barfing :(16:09
fader_slangasek: do you know anything about the iso tracker being down?16:09
=== astechgeek is now known as Guest38640
cr3fader_: does anyone have access to the iso tracker server?16:14
=== Guest38640 is now known as techgeek
fader_cr3: No idea; I'm just a user of iso.qa.u.c16:15
nperrycr3: Its down from the looks of it16:35
nperry*for everyone16:40
=== stgraber_ is now known as stgraber
cr3jcollado: hey dude, do you happen to remember wherer checkbox.job would not work whereas pexpect worked?16:58
=== fader_ is now known as fader|lunch
slangasekfader|lunch: looks like it's up now?18:06
=== moustafa is now known as moustafa|lunch
=== fader|lunch is now known as fader_
fader_slangasek: indeed; you have the magic touch I guess :)18:39
fader_cr3: So there's at least one issue preventing us from using suspend_test to replace pm.py -- suspend_test has no hibernate testing support at the moment19:15
=== moustafa|lunch is now known as moustafa
cr3fader_: would it be that difficult to add?19:44
cr3fader_: or let me rephrase that: does the suspend part of the test hold much more value than the pm.py script?19:44
fader_cr3: I can toss in a call to pm-hibernate without any real difficulty, but I'd like to ask the kernel folks if there's any reason to do it another way19:45
moustafacr3: I have a better appreciation as to why you prefer working from the office19:45
cr3moustafa: lots of books in the office?19:45
moustafacr3: I love my daughter, but it's hard to concentrate with her excess cuteness19:45
fader_cr3: It doesn't seem like it... they're pretty similar.  pm.py does some extra reporting around the time it took to wake, and suspend_test has a few extra timing options.  Other than that they're pretty interchangeable19:45
cr3fader_: yeah, definately talk to the kernel folks, I think apw is your man19:46
fader_cr3: Indeed; I'll follow up with him and see what he says.19:46
cr3fader_: pm.py is significantly shorter though, so I'm surprised that it doesn't hold much much more value19:46
cr3moustafa: I have the same problem when you're in the office19:47
moustafacr3: I miss you too :)19:47
fader_cr3: well, suspend_test has some features that pm.py doesn't, but I'm not sure if they interest us for checkbox.  E.g. detecting power consumption during long-term suspend19:48
fader_And pm.py is easier to read, being python rather than bash :)19:48
fader_Also, moustafa, cr3: get a room :P19:48
=== Erkan_Y__ is now known as Erkan_Yilmaz
cr3fader_: battery test might be cool at some point19:51
cr3moustafa: so, was it as good for you as it was for me? :)19:52
* cr3 fader_ man, #ubuntu-testing is anything but on-topic19:53
cr3haha, now that backfired19:53
fader_cr3: Meh, maybe... I think it's intended for enablement work, making sure that all the components really go to sleep when they claim to.  IIRC it's one where you have to fiddle with a power cord and let it sit for a long long time19:53
fader_Hehe19:53
moustafacr3: Nothing backfired, it's the aforementioned daughter taking up my time20:26
fagancr3: its anything but on-topic but its better than no talk at all20:31
cr3plars: hey dude, is there any particular reason why ltp needs to create /opt/ltp/include when running make (not make install)?21:33
plarscr3: funny you should ask, was just messing with LTP myself and wondering the same thing21:33
cr3plars: so if it's just crack, I can understand: makefiles aren't easy to get right21:34
plarscr3: ltp makefiles have been way into crack territory for ages, and I suspect the've gotten worse since I was really active in it21:34
plarsthey seem to have been moving towards automake, which is good, but not so good with the way they implemented it21:35
cr3plars: we can compensate and we can even blacklist those failing tests if we want, using my filter_templates script21:36
cr3plars: there seems to be increasing demand for ltp, so I'm working on integrating the syscalls suite again while waiting for other tests to run21:36
plarscr3: in any case, we can override the /opt/ltp with DESTDIR=21:36
plarscr3: ah, did you fix the build stuff already?21:37
cr3plars: oh right, I had that in the wrapper but forgot about it when updating it21:37
cr3plars: working on it, it should be pretty trivial, but I'm taking the opportunity to cleanup some of my own code21:37
cr3plars: I'll probably be moving the integration into the base checkbox package, blacklisted by default though21:38
plarscr3: should be... having trouble getting a proper build at the moment21:38
plarsI suspect I'm doing something dumb, but I'm getting what appears to be a successful build, but no bin directory21:38
=== fader_ is now known as fader|away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!