=== balloons_ is now known as balloons
pittiGood morning03:10
DanChapmanGood Morning all :-)05:43
jibelgood morning07:09
elfyballoons: is it possible to interrogate the iso.qa areas to find out how much testing has been done?13:06
knomeelfy, you mean a report across different milestones?13:08
=== _salem is now known as salem_
Saviqasac, ping13:24
elfyknome: yea13:26
knomeelfy, i don't think there's a tool like that but i'm all ears for balloons' answer ;)13:27
elfyor anything that I can get hold :)13:27
knomeelfy, you should be able to see testing numbers per milestone by simply going under those13:27
elfylol - balloons hates me :p13:27
elfyknome: you mean like the 0/5 mandatory 0/1 run once type things?13:28
elfyk - knew about those - I mean more like this milestone had these results, this one these results etc - maybe see all since saucy was released13:29
asacSaviq: hey how is it going13:30
asacwhat can i do?13:30
Saviqasac, just wanted to let you know that enabling indicators_client tests is probably not needed13:30
Saviqasac, as we're building it *into* lp:unity8 soon (as in it's merging today)13:31
Saviqasac, and lp:indicators-client will get deprecated13:31
Saviqasac, we have some general problems with unity8 tests on the device, though, that we're trying to resolve now13:31
asacSaviq: thanks for getting back to me. unity8 is afaict scheduled for next week13:32
asacso you still have some time :)13:33
asaci anticipated that that test suite will be one of the least easy to fix ones13:33
Saviqasac, actually it was targeted at device runs from the get go, we just have some generic input issue (that we might've solved already, but need to verify)13:35
=== plars_ is now known as plars
=== salem_ is now known as _salem
=== _salem is now known as salem_
=== salem_ is now known as _salem
=== om26er is now known as om26er|afk
DanChapmanballoons, hey mate. You busy?17:05
balloonshey DanChapman17:05
balloonstrying to finish up video :-)17:06
DanChapman:-) what on?17:06
balloonsautopilot of course!17:09
DanChapmanballoons, when you get a chance I have 2 MP's waiting on your review. But really i want your thoughts on the firefox one17:13
balloonsDanChapman, ohh, right.. I reviewed the ff one, unless you have another17:13
balloonsmmm.. I see :-)17:14
DanChapmanI have just proposed it couple mins ago17:14
balloonsfirst glance these both look nice17:15
balloonson the ff stuff, it's always going to be a bit of a pain because we can't introspect it properly17:15
balloonstrying to do something useful while minimizing errors is always the key17:15
balloonsI like the idea of keeping it local and opening a file17:15
balloonsI don't see where you killed the second window stuff -- did you/17:17
DanChapmanRunning the current test many times today on a few occasions it opened a second tab while it was typing. Have no idea how though, but it was giving me the same output as the failed test on jenkins17:19
balloonswell we do open a new tab17:21
balloonstiming is everything.. can we add asserts in there to time things better?17:21
balloonsI know we're so limited without introspection17:24
balloonshave to be creative17:24
balloonsto some extent we're sending keystrokes blind17:24
DanChapmanballoons, sorry had screaming child to deal with. Errm well i think asserting we have gone back to a single tab will probably help. AT the mo it closes it without 'checking' that it has gone back to one tab. I'll see where else i can slow it down a bit17:33
balloonsDanChapman, sounds like a plan :-)17:33
DanChapmanballoons, have slowed that test down. let me no what you think17:43
=== om26er|afk is now known as om26er
WebbyITballoons: tonight there is Automated Testing Workshop? Can it be a good tutorial? :)18:46
balloonsWebbyIT, there is a workshop tonight18:46
balloonsI think in about 4 hours18:46
balloonsif i remeber right :-)18:47
balloonsIf you want a jumpstart, read over the tutorials and workshop intro and FAQ..18:47
balloonsWebbyIT, first things first if you've not yet gone through it is http://developer.ubuntu.com/resources/cookbook/mobile/how-to-write-autopilot-tests/18:48
WebbyITballoons: yes, I red and did it, but I miss that leap to start working in the real :)18:50
balloonsWebbyIT, well then can you make it this evening?18:50
WebbyITballoons: sure18:50
balloonsI'm also very close to finishing the video walking through a real contribution18:50
balloonshere's the text version: https://wiki.ubuntu.com/Touch/CoreApps/Testing/ContributeAutopilotTestcase18:51
balloonsI know it's not quite the same, but it's there18:51
WebbyITballoons: thanks you :)18:51
balloonsWebbyIT, your most welcome!18:54
=== LordOfTime is now known as TheLordOfTime
=== JoseeAntonioR is now known as jose
=== LordOfTime is now known as LordOfTime|EC2
=== TheDrums_ is now known as TheDrums
adegoodyerHey all, hows it going this evening?20:44
balloonsadegoodyer, hello!20:45
adegoodyerballoons, Hey, how are things?20:45
balloonsbusy, but good :-) doing video edits and takes20:45
balloonsso can't really chat, but :-)20:46
adegoodyerballoons, No worries, I certainly don't envy you this evening! Good luck and sure all will be bafta worthy :oD20:47
balloonshow's it coming for you?20:47
balloonshow's the test?20:47
adegoodyerballoons, Don't ask lol. I can click the play button to start, but then have no idea how to test for game start.... well I have but the boxes are generated at runtime so don;t even think it's possible to select them for introspection by autopilot.20:49
adegoodyerballoons, I have committed what I have written so far, so hopefully a good base for someone to further if nothing else.20:51
balloonsadegoodyer, well good.. a start is a start20:52
balloonsdo you propose a merge?20:52
balloonsI can probably help with the boxes.. but later :-)20:52
adegoodyerballoons, No I haven't proposed a merge yet, shall I? The code is pretty much what you executed last night. Not a problem at all, I'm already conscious  of taking loads of your time all the time lol20:55
balloonsadegoodyer, yes of course20:58
balloonsif what you have is a good base, commit it and get it merged20:58
balloonsat the moment there is nothing for dropping letters20:58
balloonsit's best to merge small pieces at  atime20:58
balloonsdon't want to write everytestcase and then propose it :-)20:58
balloonsok.. bbl everyone20:58
adegoodyerballoons, agreed - ok will do and will leave you alone for a while :oP20:59
balloonsok whew.. back :-)22:07
balloonsupload time :-)22:07
darranhi all22:34
balloonshello darran22:34
darranis the ubuntu touch testing presentation going to be recorded?22:44
balloonshey darran if we do any livestreams those will be recorded22:47
balloonsbut in addition, I'm uploading a big walkthrough video now to youtube22:47
darranis the presentation today on livestream or irc?22:48
balloonsSo in about another hour or so that will be available to view also, which will help alot if needed.22:48
balloonsdarran, a bit of both. I'll be on IRC, with a small intro and taking questions. If someone has a good question, I'll turn on the livestream and answer it22:48
sakhi guys22:49
balloonshey sak! good to see you!22:49
saksame here. Am I in-time for the tutorials?22:49
balloonsyou are indeed!22:50
balloonswe'll start in about 10 mins. I was mentioning to darran that there's also a video walkthrough of everything I'm uploading now, so very soon it'll be on youtube :-)22:51
darranwhere are you based out of balloons?22:51
balloonsit's long but in-depth.. you get to follow me through writing 2 tests22:51
balloonsI'm east coast us22:51
sakEast cost here as well22:54
balloonsI tried to host this at different times for everyone.. this is the latest one of them all :-)22:58
balloonsin UTC time that is22:59
balloonsalrighty, let's get this show on the road :-)23:00
balloonsSo welcome to everyone who's here for the testing workshop23:00
Noskcaji here, but busy with debian stuff23:01
balloonsSo the goal here is for me to give a little background introduction, then help anyone with specific questions they may have23:01
balloonsif we get into some details or people are interested, we'll do a livestream or hangout23:02
balloonsI know not everyone wants to be on camera, so we'll stick with IRC for the moment. First I'll give a little introduction on what autopilot is, and what we're doing.23:02
balloonsIn a nutsheel, Autopilot itself is a functional testing tool allowing us to interact with an application the same way a user would. it can click, swipe, touch, and type in an application by simulating a user23:02
balloonsSo we can use the tool to test and ensure our apps work as expected from a user perspective23:02
balloonsThis includes apps like the core apps project, whom we've been focusing on to get tests written for23:03
balloonsThe core apps are written by community developers and represent the core applications for the ubuntu touch platform.23:03
balloonsSo, in order to help contribute tests there are a couple things you'll need:23:03
balloonsthe first is an installation of ubuntu saucy or ubuntu raring. It can be in a VM or installed on physical hardware23:03
balloonsit's important to have raring or saucy for the sdk and autopilot stuff..23:04
balloonsthe second is what we're here for.. an understanding of autopilot :-)23:04
balloonsSo, first things first23:04
balloonsFor anyone who hasn't yet gone through the tutorial on developer.ubuntu.com, http://developer.ubuntu.com/resources/cookbook/mobile/how-to-write-autopilot-tests/, please do so now23:04
balloonsit's a nice intro to autopilot and gives you an example app to play around with23:05
sakhow long do we have to go over the tutorial of the link you gave us?23:05
balloonssak, if you've not yet gone over it.. do it now :-) I'll field questions as you do so23:05
balloonsFor those who have gone through the tutorial, let's talk about what to do next23:05
balloonsThe basic steps are:23:06
balloons1) pick an app and test you want to write23:06
balloons2) contribute it23:06
balloons3) Profit! (there is no step 3 :-) )23:06
balloonsso how do you know what's needed? there's a handy list found on this page: https://wiki.ubuntu.com/Touch/CoreApps/Testing23:06
knomeisn't the step 3 "repeat steps 1 and 2" ?23:06
balloonsknome, indeed that's the true step 323:07
darranIs there someone that should approve out tests?23:07
balloonsaka, "repeat"23:07
darranI'm curious who developed autopilot23:07
balloonsdarran, yes, the development teams and others will review the tests you write23:07
balloonsjust as a normal code contribution23:07
balloonsdarran, autopilot itself was developed by a group of folks within the unity team a few years ago23:07
balloonsoriginally they wrote it to test unity. However since then it's morphed into a general testing tool for use by many applications23:08
NoskcajIs it not better to do the writing on Ubuntu itself, rather than the various flavors.23:08
balloonsyou can interact with the folks who wrote the tool directly on #ubuntu-autopilot23:08
NoskcajIt also tests ubiquity23:09
balloonsNoskcaj, I'm running unity, but you don't have to in order to develop tests23:09
balloonsthese core apps should run on other flavors.. I don't know why they wouldn't23:09
balloonsNoskcaj, yes your correct. Besides the core apps the quality team here maintains a project of gtk testcases, including tests for automating the installer. DanChapman was instrumental in getting those written ;-)23:10
balloonsthe gtk tests can be found here: https://launchpad.net/ubuntu-autopilot-tests23:10
sakSo just to be clear, autopilot is used to test an application from a user's prospective?23:11
balloonssak, yes that's correct23:11
balloonsas such it's important to use the tool where it makes sense23:11
balloonstest user facing application pieces, not things that are better suited for a unit test.. For example verifying a function handles race conditions, etc23:12
balloonsSo, going back to the contributing part for core apps.. There's a handy list of what needs doing.. aka outstanding tests23:13
balloonsthe big list is here: https://bugs.launchpad.net/ubuntu-phone-coreapps/+bugs?field.tag=needs-autopilot-test23:13
darranis this the place to come if we have questions?23:14
balloonsplenty of places to get invovled. So pick an app, pick a test.. Then your ready to get crackin' on solving it!23:14
sakCan we use any flavor of ubuntu for autopilot tests, or does it have to be the original ubuntu flavor, unity?23:14
balloonsdarran, yes you can ask questions here, in #ubuntu-autopilot, #ubuntu-touch and the mailing lists23:14
balloonsautopilot hackers are in -autopilot, folks like us hang out in -quality, and the devs themselves generally are in -touch23:16
balloonssak, yes you can write autopilot tests using any flavor, and for any gtk or qt/qml application23:17
balloonsso, if you've picked an app, picked some tests to write, what's next?23:18
balloonsthe first step is to run through the tests yourself23:18
balloonsfire up the application and go through the test you want to write23:19
balloonsin english (or your native language if you wish :-) ), write down the steps you take to perform the test23:19
balloonsthe basic idea here is to note everything you do as an action23:20
balloonsthen after you perform an action, note what happens23:20
balloonsin other words, what you expected23:20
balloonsthus your basis for a testcase look something like this23:20
balloonsclick on X button23:20
balloonsthe screen changes in Y way23:20
balloonsperform an action23:21
balloonsexpect a result23:21
balloonsall of these actions steps can be simulated by autopilot23:21
balloonsyou can have autopilot click, type, etc23:21
balloonsthen, for the expected results you've written done, you can write assert statements23:22
balloonsthese statements will verify what you expected actually happened23:22
balloonsthis is the basis of a good testcase23:22
balloonsSo, let's quickly look at the developer tutorial for an example.. I like real world examples :-)23:23
balloonsif you scroll down to the part "Testing the clear button" you can see a function displayed23:23
balloonsthis is a very simple test in the exact format we described23:24
balloonsfirst we perform an action23:24
balloonswe then issue some asserts to confirm our results23:24
balloons  #confirm fields have been wiped23:24
balloons    self.assertThat(fromField.text, Eventually(Equals('0.0')))23:24
balloons    self.assertThat(toField.text, Eventually(Equals('0.0')))23:24
balloonsDoes that make sense? If you understand that basis for the testcase, we can build your knowledge of syntax and tricks from there :-)23:25
balloonsSo I'll stop yapping for a moment and see what questions you all may have, and if anyone has something specific they would like help on23:25
WebbyITballoons: where can we find all option to do something? E.g: if i want to swype something, how can i do? And where can I find how can i do? :)23:27
balloonsWebbyIT, great question23:28
balloonshttp://unity.ubuntu.com/autopilot/ is the documentation for autopilot23:28
balloonsI know everyone loves reading docs, but there's really just a page or two you should look at to get started23:29
balloonsthis covers the input section; so in your example you want to swipe something23:29
balloonson that page you'll find the drag command which will do exactly what your looking for23:30
balloonsthe other handy page is this one: http://unity.ubuntu.com/autopilot/api/introspection.html23:31
balloonsit covers select_single and select_many23:31
balloonsthese are the introspection tools you have available in autopilot23:31
balloonsarmed with an understanding of those couple things and you can be dangerous and write some tests :-023:32
balloonsWebbyIT, does that answer your question?23:32
WebbyITballoons: yes, perfectly, thanks :)23:32
balloonssak, darran how goes it? looking through the tutorial on developer.ubuntu.com?23:39
WebbyITballoons: I would have another question :) How can we verify if something is on the display? Eg: on calculator-app I've to swipe to delete a calculation. Now, thank you, I have undestand how to swipe, but how can I check if swype works?23:40
balloonsWebbyIT, we don't directly read the screen in autopilot. This is for various reasons, but basically boils down to it's not sustainable to have a test read pixels (they change too much!)23:41
balloonsso instead, you view the application state23:41
balloonsif you swiped to delete a calculation, you can check and make sure it's been deleted by examining the application.. see if it's stored by the app still, and see if the display data still shows the calculation or not23:42
balloonsI trust this makes sense ;-) You can read the values of say a label or textfield and thus confirm it's no longer there23:43
sakCompleting the installation and setup process23:43
balloonssak, gotcha :-) ping if you encounter any questions.. Youtube says 10 more mins and the video tutorial will be done.. you'll be able to catch it here: http://youtu.be/qD_e_xqlBbg23:44
WebbyITballoons: so, I have first to do a calculation with autopilot, check his ID (every calculation has his ID) or something that identify the calculation, swipe away the result and check if his ID is not longer there?23:44
balloonsWebbyIT, yes that sound correct :-)23:52
balloonsso I would:23:52
balloonsperform an operation and finish it so it hits the history23:52
balloonsgrab that calc id and verify it's contents23:52
balloonsswipe it23:52
balloonsverify the id is gone / contents are wiped23:52
WebbyITballoons: cool! So, tomorrow I'll try to do my first autopilot test on this :)23:53
balloonsWebbyIT, excellent!23:54
balloonsanyone else have questions?23:55
balloonsWebbyIT, you know where to find me as needed.. I think you've got it down23:55
WebbyITballoons: yes, sure :) it was an evening very informative, thanks a lot! :)23:58
WebbyITballoons: If nothing else I would go to bed, it's very late here in Italy23:58
balloonsWebbyIT, indeed!23:59
balloonsgood night!23:59
balloonsand to everyone else, thank you as well for coming23:59
balloonsdinner time here for me, but do keep in touch if you need help!23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!