[07:50] good morning [07:53] evening jibel [07:53] and dholbach [07:54] good morning [07:54] hi Noskcaj [07:56] Hey Noskcaj [11:52] I want to hook some autopkgtests up for some xorg packages, but the testsuite is in a separate git repository (xorg-integration-tools, and piglit for mesa). Is it still possible? [12:32] mlankhorst: our jenkins machines have rather restricted network access [12:32] mlankhorst: it would be better to package those and add as test dependencies [12:32] mlankhorst: you can reach launchpad and *.ubuntu.com, but almost certainly not fd.org [12:35] aw, was afraid of that [12:40] mlankhorst: well, package or just put a recent copy into debian/tests/, of course [12:40] like, in a tarball === _salem is now known as salem_ === salem_ is now known as _salem === _salem is now known as salem_ [13:30] pitti, my attempt to clean-up xvfb after make check in jhbuild http://paste.ubuntu.com/1608681/, WDYT? [13:33] jibel: oh, that looks clever! [13:34] jibel: that also ought to mop up those leftover dbus-daemons, right? [13:34] pitti, right, in theory [13:34] jibel: FYI, xvfb-run also has an -a option which sounds better than hardcoding :87 [13:35] so maybe we should call it with -n 10 -a [13:35] jibel: btw, I'm watching rabisu and keep killing the colord test which hangs indefinitely; there were some transient errors over the weekend but stuff is clearing up again [13:37] pitti, sounds good, I'll use -a [13:44] 2 weeks ago it was pango to hang forever. I'll see if I can implement a timeout in jhbuild [13:54] pitti: would be annoying since it has some external deps as well [13:54] and circular [13:55] the test is for xorg-server and some drivers, so putting it in debian/tests would cause things to break quite badly [14:34] mlankhorst, can you package the testsuite? This is for example what packages with a big testsuite like firefox, thunderbird, mysql, bzr, samba do. [14:39] is there an example I could look at for that? [14:42] mlankhorst, any package I mentioned above. for bzr the testsuite is python-bzrlib.tests, for mysql-5.5 the testsuite is mysql-testsuite-5.5 (on raring that is) [14:43] mlankhorst, in debian/test/control of the package there is a dependency on the testsuite [14:44] mlankhorst, then the test script called by adt call the runner in testsuite [14:47] in the case of mysql-5.5, the test dependency in debian/tests/control is mysql-testsuite, the test script in this same control file is called 'upstream' and there is a shell script that cd to /usr/lib/mysql-testsuite then run the mysql testsuite with ./mysql-test-run.pl [14:53] ah === salem_ is now known as _salem [19:57] Hi balloons [19:57] hello! [19:57] balloons, did you read my emails on autopilot ? [19:58] yes I did [19:58] glad your around now [19:58] balloons, :) [19:58] so which to discuss first? [19:58] your right on about the english issues [19:59] for now, I'm trying to avoid adding english strings to tests to keep them working across locales [19:59] we'll have to do something about using translations later [19:59] balloons, yes, but for instance what I discovered on firefox [20:00] yes, that was really weird [20:00] why would the same keypresses not work? [20:00] balloons, yes.. [20:00] * balloons notes I'm glad you all have int'l exposure and can help [20:00] letozaf_, does it break the english test to add this? [20:00] balloons, didn't try [20:00] if not, then just add it :-) and note it's to ensure compatibility with locales [20:00] balloons, I can now [20:00] 'll try right now [20:01] ok, go ahead then [20:01] if you've got it up [20:01] * balloons is elsewhere atm [20:01] balloons, we could ask dkessel if this happens also to him [20:01] balloons, the firefox thing [20:01] yes [20:03] * letozaf_ is looking at the firfox autopilot test [20:04] balloons, no If I add the \n the test fails [20:04] balloons, I mean if I have english language on my laptop [20:05] ahh [20:05] so so weird [20:05] balloons, I know makes no sense [20:05] balloons, but I tried it various times [20:05] well, note it in the test.. we'll find a way around it, or it will just have to be known as broken [20:05] we could add a hack to detect locale, but I'd rather not do that [20:06] since we have wonderful things such as bugs, hehe, you could file a bug on it :-) [20:06] balloons, :D [20:07] balloons, but a bug on firefox ? or on autopilot ? [20:07] on our project [20:07] balloons, oh! yes [20:07] https://bugs.launchpad.net/ubuntu-autopilot-tests/+filebug [20:07] https://bugs.launchpad.net/ubuntu-autopilot-tests/+bug/1112781 [20:07] Launchpad bug 1112781 in Ubuntu Autopilot Tests "firefox autopilot test fails to input text correctly, then restarts firefox too early, then aborts abruptly" [Undecided,New] [20:07] in fact, ^^ [20:08] dkessel has already reported it, in a way [20:08] balloons, yes it just happened to me, the [20:08] add a comment to that bug, I edited it a bit [20:09] another instance is already running [20:09] it's now firefox test fails on non-english locale [20:09] balloons, ok [20:13] on the shotwell stuff, yea, I think putting zenity in there is ok [20:13] we should talk about splitting this off into semi-automated? [20:13] it's a good discussion to have [20:14] we'll need to update to require zenity, but it's installed by default, so no worries imho [20:14] balloons, oh yes, so I will continue with zenity in the test [20:14] in fact, when you propose your merge, you can edit the depends to include it :-) [20:14] balloons, ok [20:15] we should ping thomi about his thoughts on this semi-automated ideas as well.. not something they've done or have interest in I'm sure [20:15] balloons, ok for the semi-automated chat and thomi ping [20:16] balloons, maybe thomi can tell if there is something better to use [20:16] balloons, or if it's ok as we are thinking to do it [20:17] balloons, there is also the network manager test to be done as semi-automated [20:17] it would be interesting to pick his brain.. anyways, I see zenity as a good solution. the piece I'm worried about is how to split this out so they are known to be non truly atuomated tests [20:17] or do we need that, etc.. that's a discussion for us [20:17] they won't be able to run via jenkins, etc [20:18] hi guys - I'm just in a call right now, but I'll be along afterwards [20:18] we are able to push them via ppa, so manual execution works.. so the theory is definitely sound [20:18] lol, we should keep saying his name.. he'll get pinged more.. [20:18] I kid, don't do it :-) [20:19] balloons, :-D [20:20] balloons, anyway I do not think we have other alternatives to run autopilot tests on shotwell or network manager [20:20] balloons, we cannot emulate someone plunging in a camera on the PC [20:20] balloons, or attaching a network cable [20:21] yes.. we do it this way or that stay as pure manual [20:21] balloons, oh! [20:21] balloons, right! [20:21] balloons, but isn't semi-automated better than manual ? [20:21] balloons, :D [20:22] :p [20:23] balloons, unfortunately I do not know much about jenkins so maybe you're right after all ;) [20:30] * letozaf_ is commenting dkessels bug [20:31] well letozaf_ obviously atm, we're running the tests ourselves.. however, since the test is automated, why not have the computer also run the test.. that's the idea.. after the computer also runs the tests, we can use jenkins to help us see what happened [20:33] balloons, I see, great! but maybe we can make testers carry out not only manual tests, but also the semi-automated ones [20:33] balloons, when computers learn to plug-in a camera, we can let them do it :) [20:34] letozaf_, yes.. we can have them do both manual and semi-automated [20:34] and yes, they've done things with hw to automatically connect and disconnect things like network cables and cameras :-) [20:34] so, we could ask gema what they can do.. you never know, it might be possible for them to rig that piece up [20:35] balloons, it would be great! [20:35] regardless, you'll need the semi-automation for writing and testing it [20:38] can this be done by enabling / disabling the kernel kooks, or would that need a reboot? I'm only vaguely familiar with enabling / disabling lspci stuff etc. [20:39] phillw, there is some trickery that has been done.. pitti also might have some insights.. no matter if it can be done or not, using zenity atm so carla can write the test i think is a good way forward [20:40] balloons, I commented bug 1112781 if you wanna take a look at what I wrote [20:40] bug 1112781 in Ubuntu Autopilot Tests "firefox test fails on non-english locale" [Undecided,Confirmed] https://launchpad.net/bugs/1112781 [20:40] hi balloons [20:40] okies, just a thought... as you know, (semi) automated testing is not be strong suite :D [20:41] s/be/my/ === _salem is now known as salem_ [20:52] phillw, :-) [20:52] I'm going to rope you in for a little surprise btw [20:52] I want your help planning a manual testing hackathon [20:52] hello thomi [20:53] letozaf_, thanks for updating the bug.. we should be set.. hopefully at some point we can solve it, but it's documented now [20:54] balloons, ok [20:57] balloons: can I get the virtualisation ones out of the way first?... Pretty please :D [20:58] phillw, lol, yes of course.. I have to write the tutorials for it remember.. including getting resources for lp [20:58] so your safe for now [21:02] good, just awaiting either "yeah, it's okay", or a telling off for running Version 26.0.1386.0 Ubuntu 12.10 (177362) :D [21:02] I found a bug, but am still not sure if that build is any where near the 'official radar' yet. [21:04] chilicuil, are you about? [21:05] balloons: hi o/ [21:05] excellent ;-) [21:05] I was looking at your new branch for manual tests early [21:06] first of all, thanks for putting it together.. I spent some time today migrating over the other open issues, so the bug tracker should have all the current needs in it [21:06] anyways, do you know how to propose a merge? I'd be happy to review it and get it included [21:07] balloons: yep, I just want it link first, I was not sure it was ok [21:09] awesome, yes, I took a quick look at it, we'll review it when you propose.. Nothing extreem stood out to me besides the reference to the dash.. in general we try and stay desktop agnostic in our tests to promote re-use. So instructions on using the unity dash can instead refer to running in generically [21:09] *refer to running an application generically [21:09] that way our lubuntu, xubuntu, etc friends can also utilize the test [21:09] and vice versa of course [21:10] obviously unity or ubuntu specific tests will be just that.. they also have things like mythbuntu or ubuntu studio specific tests.. but in this case, I think it's safe to be generic [21:10] got it balloons, I'll do it shortly after I change the references to the dash [21:10] does that make sense? [21:10] awesome [21:22] balloons, but about the semi-automated thing, should I carry on with shotwell, or pick something else that is not semi-automated until something is decided about semi-automated tests ? [21:26] letozaf_, do carry on [21:27] even if we found a way to automate it, you need to be able to write the test [21:27] and zenity makes it easy ;-) [21:27] we could always swap that step out later if a solution is found [21:27] balloons, :) ok [21:45] balloons, do you know what this error could be ? http://paste.ubuntu.com/1610439/ the MismatchError on the bottom...:( [21:49] balloons, I will check it tomorrow, going to bed now :) good night! === salem_ is now known as _salem [22:37] thomi, did we lose you in all the chatter.. if you get a moment, I'd still love your thoughts on the idea of a semi-automated autopilot test [23:40] Noskcaj: hopefully, final edit... https://wiki.ubuntu.com/Testing/Activities/Classroom/Section3#Classroom_Session_3 can you check to make sure nothing I'm adding as 'extras' will kill test-drive. AFAIK, you just need me to keep VBox at 4.1.18? [23:42] balloons: sorry, I was at a meeting - I'm back now, if you want to char [23:42] *chat [23:43] kk [23:43] yea, quickly [23:43] any thoughts? [23:44] bot to hit dinner [23:44] :-) [23:44] ummm, well, I'm not sure what you're asking [23:44] balloons: if you're about to eat perhaps we should do this next week? (I'm on Holiday from tomorrow onwards) [23:47] thomi: when balloons goes for dinner, do not expect an early reply. Like my self, family then takes over :) [23:58] lol [23:58] kk [23:58] thomi, we're writing tests with ap that can't be fully automated [23:58] that's all [23:59] they will prompt during the test run [23:59] good, bad indifferent.. not as intended, but this semi-automated style will help us test some harder to test things [23:59] balloons: hmmm, seems to me that negates the main advantage of using AP in the first place? [23:59] like attaching/detaching devices, etc [23:59] what are the things that you can't automate? [23:59] that said, there are ways around those, so hopefully we'll fix it :0)