[02:24] veebers: (and alesage if still around): I have a problem selecting a subset of the tests. If we want to return an error when the test doesn't exist, then the host needs to have all the test packages installed too. [02:24] so far, only the phone needed all the test packages. [02:24] do you see that as an issue? [02:25] elopio: hmm, that's a good point (I've come across something similar just recently where there was code in webbrowser that isn't on utopic (but is on vivid) [02:26] elopio: so if we want the test on the host machine too we'll have to expect that the user is running vivid. [02:26] oh, right, that's an additional problem. If the host is in a different distro, the check is worthless. [02:37] veebers: alesage: I'm now inclined to let adt-run fail when we specify non-existing tests, instead of catching the error earlier. [02:37] elopio: yeah, that sounds good to me. It's a good start at least. If we get user feed back we can go from there [02:38] (Do the min. to get a card done and iterate with the users for further enhancements) [02:39] veebers: do you see any problem if we put the testsuite_list in the ubuntu_sanity_tests folder? [02:39] elopio: where is it at the moment? [02:40] * veebers looks [02:42] elopio: I can't think of a better place off the top of my head [02:43] veebers: do you mean, you can't think of a better place than debian/test/ ? [02:44] elopio: sorry, I can't think of a better place than what you've suggested [02:44] ack. [02:44] this is for the work of separating out the code from d/t/ right? So it'll need to be in a discoverable place [02:45] veebers: yeah, I want to put it under test to then be able to test the subset story. [02:45] ack [02:45] +1 [02:45] veebers: I'm thinking ubuntu_sanity_tests/external_tests/testsuite_list [02:45] with updates in the README, of course. [03:03] elopio, something funky with my phablet-network on the serial branch: http://pastebin.ubuntu.com/9947891/ [03:04] alesage: doc looks good, have left notes. [03:04] weird actually I'm getting that same return for the success case above :/ [03:04] veebers, thx [03:05] alesage: hey, that looks like the error that I've had in the past (as per my comments re: flashing and starting tests) [03:05] alesage: I have no idea what that error code means. [03:05] elopio, please tell me all the answers [03:05] alesage: do you have the serial on the config, or not? [03:06] elopio I started fresh and no I didn't give the serial [03:06] elopio, but I think I just demonstrated that phablet-network itself does give that code for me raw [03:07] alesage: when you say fresh, did you delete your config file? [03:14] alesage: yeah, no errors here. [03:14] alesage: you might want to try jibel's alternative with nmcli: https://bugs.launchpad.net/ubuntu-sanity-tests/+bug/1408720/comments/2 [03:14] Error: launchpad bug 1408720 not found [03:14] that's something we need to do anyway, I think. [03:23] elopio I'm being selfish with reviews ATM :( , but also need to eat, will get a couple in [03:24] alesage: you should also rest. I'm here only because I slept all evening :D [03:25] elopio, maybe I can *reserve* either the reviews or the "subset of tests" for tomw, subset would enable some pairing [03:26] elopio, or you're already knee-deep on that one, eh? [03:26] alesage: I've splitted it in two, and took the smaller task first. [03:26] getting the existing code into test. [03:26] alesage: main thing are reviews. Then, whatever is on top. [03:27] elopio, I'll focus on reviews and getting everything running AM [03:28] ok. [03:29] note, I added a new card in the review column re: timeout [03:29] maybe we can leave those in the lane, e.g. if brendand also reviews [03:30] well, they do need to acks to merge :-) [03:47] veebers: brought back your timeouts branch. It is one line, but I still think it should be tested, and we now have the helpers to make it quickly. [03:47] elopio: ah good point. I'll look into that shortly [03:49] elopio: I see that you voted for plivo, what's the main reason for that? (e.g. I see that the python lib is not python3 [03:49] ) [03:50] veebers: main reason, it's free. [03:50] the lib is small, so quickly we can make it py3. At a first glance, I didn't see any big changes required for that. [03:50] elopio: no it's not, the first line says it'll cost money [03:51] hmm, ok fair enough as long as they take that contribution back, It would suck to have to distro patch that just for our own use [03:51] veebers: free as in freedom. We can install our own plivo server, and it won't cost money. Well, only the cost of the call. [03:52] veebers: I doubt they won't accept our contribution. Since I signed in, they send me constant emails, it seems it's a company that's alive. [03:52] elopio: I may have missed something, why would we install our own server? We only need a webservice (for either provider) right? They have a REST api that we hit [03:52] but, we have to check, you are right. [03:53] elopio: I'm not disagreeing with you, just wanting to know the reasons and talk through them [03:53] veebers: I'm not saying that we need our own server. I'm saying that we can have it. [03:53] ack [03:53] for security reasons, cost reasons, or anything. But for me, just the open source code makes me feel better. === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk [14:58] elopio, hey [14:58] brendand: hello! [14:59] elopio, let's try to land as much as possible today! [14:59] brendand: yes indeed. Are you in need or a review? [14:59] elopio, top approve: https://code.launchpad.net/~brendan-donegan/ubuntu-sanity-tests/fix_wizard_test/+merge/247995 [15:00] brendand: top approved. [15:00] elopio, review: https://code.launchpad.net/~canonical-platform-qa/ubuntu-sanity-tests/specific_config/+merge/247841 [15:01] brendand: yes, the same test as autopilot, I like it. Thanks. [15:01] elopio, oh really, what a coincidence ;) [15:02] brendand: approving. alesage: can you please give the second review to the specific config branch? ^ [15:04] brendand: and one thing for discussion. I prefer to keep the imports at the module level, not the object level. [15:04] like import argparse, and then use it like argparse.ArgumentParser(...) [15:04] because when you read the code, it's easier to tell which module is it that you are calling. In this example, it's obvious that ArgumentParser comes from argparse, but there are other symbols harder to link to a module. [15:04] brendand: do you have any preference there? [15:08] elopio, i think if the only symbols you're using are obvious then it's worth importing that way since it makes the code look cleaner [15:09] elopio, we couldn't use the other symbols if we don't import them anyway [15:09] brendand: I find it cleaner if it is consistent. But that's a fair point. [15:10] elopio, why use 'from' at all then? [15:10] balloons: to only use one level of modules. [15:11] from ubuntu_sanity_tests import tests, instead of import ubuntu_sanity_tests.tests. [15:11] s/balloons/brendand [15:11] true [15:11] alesage: you left a needs fixing here: https://code.launchpad.net/~canonical-platform-qa/ubuntu-sanity-tests/optional_options/+merge/247881 [15:12] can you please check if you are happy with it now? [15:13] brendand: now lets talk about the coverage in https://code.launchpad.net/~canonical-platform-qa/ubuntu-sanity-tests/no_serial/+merge/247940 [15:13] we are testing the same thing as before, the difference is that now we are not putting mocks everywhere. [15:13] so, I wouldn't call it a coverage drop. I would call it an increase in accuracy :) [15:14] to get everything under test in this module, we have the big problem that it needs subprocess execution. So we need to figure out how to do it properly. [15:14] elopio, run_sanity_tests is not covered at all now [15:15] brendand: before, it was run, but all the things that were hard were patched. [15:15] so it was the same as not running it. [15:15] this branch is about serial, so the tests only check that the serial was passed or not. [15:16] we need to put the other things into a test, but that's a different card. A harder one. [15:16] elopio, sounds like you're saying code that is trivial is not worth testing? [15:17] brendand: not at all. This run_sanity_tests method is not trivial at all. [15:17] and it needs to be tested. [15:17] what I'm saying is: it was untested before, this serial card was not about putting it under test. [15:18] elopio, it was tested before. maybe not thoroughly, but it was [15:18] we need a card to increase the coverage of command_line execution, and I have no idea how to do it properly. Needs discussion. [15:18] elopio, see: http://people.canonical.com/~brendan-donegan/ubuntu-sanity-tests/trunk/htmlcov/ubuntu_sanity_tests_command_line.html [15:20] brendand: no, it was run, but not tested. You reviewed the tests in the original serial branch. [15:21] what I did there was to call run_sanity_tests, but patch the IO, patch the report generation, patch popen, and only assert that -s was passed. [15:21] if you look at the new tests, we are testing the same. And now it's clear that run_sanity_tests needs tests. [15:26] elopio, did you remove test_main_must_setup_and_run_with_serial because it would fail with the new code? [15:27] brendand: no. Look at rev 59: https://bazaar.launchpad.net/~canonical-platform-qa/ubuntu-sanity-tests/no_serial/revision/59 [15:28] I first made the new code pass with the old style of tests. [15:28] then I followed vila's suggestion to split the commands into build and run, and test only the build, which removes the need of mocks leaving us with the same tests. === roadmr is now known as roadmr_afk [15:29] brendand: I can revert to rev 59, which will give us a high but deceiving coverage number, and ugly tests. [15:30] elopio, what i'm asking is does anything stop you from reinstating test_main_must_setup_and_run_with_serial [15:31] brendand: no, I can revert to rev 59. [15:32] elopio, but if you revert won't you lose other tests you added/changed? [15:32] I was testing the autoresize feature and was unable to continue the install. The "install" button was grayed out. I was able to resize the partition but not finish the install. I am testing daily version " 20150129" using virtualbox 4.3.20. [15:32] elopio, i think all the new tests you added are good and valuable [15:33] brendand: I would lose the cleanups I did on the tests. I didn't added new tests. I split them and removed the mocks, except the IO ooooooooone. [15:33] one [15:34] elopio, are you looking at this diff: https://code.launchpad.net/~canonical-platform-qa/ubuntu-sanity-tests/no_serial/+merge/247940 [15:34] ? [15:34] elopio I'll get on those reviews [15:35] elopio, the diff on test_command_line shows test_main_.. being removed and six new tests added [15:35] elopio, ah you split them [15:35] ok [15:35] brendand: I am. Compare the tests in rev 59 with the tests in rev 65. [15:36] alesage: thanks. [15:38] elopio, anyway next move is yours, reply on the MP [15:40] brendand: I'd say, next move: add a card to put run_sanity_tests under test. But the only thing that comes to mind is to patch Popen and check the order of popen calls. [15:40] which is a little better than what we had before, but not good enough. [15:41] brendand: do you want me to do it in order to get this branch landed? Or we put it in the backlog and decide later when to do it? [15:42] elopio, if you could reinstate the test (in addition to the ones you added), then put a card or a bug to do the tests 'right', if that will require a significant amount of time [15:45] IMO, that makes no sense. Between 59 and 66 I added no tests, so that would be testing the same thing twice, once ugly and once nice. I will just make a new branch to raise the coverage. [15:47] brendand: btw, I think you moved the wrong card back to doing. [15:47] unless I'm missing something. The one you moved has your +1. [15:54] elopio, ok reply in the comment what you think to do next [15:54] elopio, fixed the board.thanks [15:59] elopio, and get some feedback from alesage [16:06] brendand: alesage: https://code.launchpad.net/~canonical-platform-qa/ubuntu-sanity-tests/run_main_tests_tests/+merge/248135 [16:07] that excercises run_main_tests just as how it did the serial branch, but has a check that makes a little sense. === roadmr_afk is now known as roadmr [16:26] elopio, we don't have sudo inside the tests themselves? [16:29] elopio concerning this last MP, and I was studying this a bit yesterday, this feels a bit tortured to me, why don't we just fake those calls in run_sanity_tests and then test those functions separately? [16:29] brendand: we can, but it makes some things harder. [16:29] we can tell in debian/tests that we require root access. [16:29] elopio, not having it also makes some things harder :) [16:29] the patented "fails only in one place" method as Mr. Richards is always saying [16:30] then we will be running with the root user, and have to switch to phablet at some point. [16:30] also I'm not sure I understand why __enter__ is executed here instead of __init__ [16:30] alesage: Popen is called from with, thus we need to patch enter. [16:31] I was trying to split things and test them separately, but brendand requested to bring back the run_tests test [16:31] ok it's brendand's fault [16:32] no, it's mine for making the ugly test to begin with. [16:32] it has to be balloons' fault [16:32] * knome runs [16:33] * balloons desperately tries to catch a fleeing knome, but he's gone [16:35] elopio, I see that it works but I want to refactor it :) , can I approve and then do so in a separate MP? [16:36] alesage: the branch is shared. You can just make your changes and push to it. [16:37] just bring the card back to doing, and assign it to yourself. [16:37] * alesage grabs a bunch of strings from balloons floating away [16:37] alesage: but, please do the missing reviews first. We are only three today, so we need your reviews before brendan leaves. [16:37] elopio, ok thanks [16:37] elopio, ok shall do [16:38] * elopio looks at the browser branches. [16:40] alesage, a top approve for https://code.launchpad.net/~canonical-platform-qa/ubuntu-sanity-tests/specific_config/+merge/247841 pretty please [16:42] I like that everyone needs my approval [16:51] brendand, elopio I have a lunch invite at our standup time, maybe we could meet quickly earlier to let brendand go earlier? [16:51] elopio, i could write a helper which switches to root and another which switches back to phablet [16:51] alesage, wouldn't mind meeting something in the next 30 minutes [16:52] alesage, so i can EOD :) [16:52] alesage: good for me. [16:52] elopio, top of the hour? [16:52] brendand: sounds ok to me. We tried it some time ago, and it was easier to put the root commands in the setup. But if you make it work, seems the same to me. [16:53] brendand, top o' the hour to you :) [16:53] brendand: alesage: in 7 minutes. I'll prepare the document. [16:54] alesage, i had literally no idea you'd say that... [16:54] brendand, I'm so unpredictable! [16:54] also it's Friday [17:00] brendand: meeting time. [17:19] alesage, i'm not leaving until that mp is approved :) [17:19] going to be a loooong night [17:19] alesage: please top-approve anything that you see as ready, and move the cards. I'm going to take a little break. [17:20] elopio, ok [17:20] brendand, need a push there [17:24] brendand, poke [17:25] alesage, oh might have got stuck, been having problems with that today [17:25] alesage, repoke [17:25] brendand, k [17:25] indeed, bzr push was hanging [17:25] grrr [17:26] brendand, took the liberty of moving that card thx [17:30] alesage, elopio - thanks and good night! [17:30] email me if anything urgent [17:31] enjoy the weekend brendand. [17:39] hola 0/ [17:40] cgoldberg, hola :-) [17:40] cgoldberg: o/ [17:42] i'm itching for an afternoon self-project. I might pave my desktop and start running Vivid [17:42] im still on Trusty [17:44] cgoldberg, oi, that's a longer term decision, but running -devel is fun [18:02] cgoldberg, big snows over there? [18:03] alesage, yea.. like 2+ feet the other night... but Boston cleared it out pretty quick.. supposed to get more soon too [19:09] thanks for the reviews alesage. Now we are only two, so we can't land anything else. But the queues are actually pretty decent. === roadmr_afk is now known as roadmr [19:10] elopio, manually testing this no_serial one and then want to pick over the main testing one a bit [19:11] alesage: ok. I already top approved that one as it had +2. But it's always nice to have more reviews. [19:11] if you find something, we'll do it in a new branch. [19:12] elopio, ok, sure it's fine, although this quickie needs to get in on your serial branch fwiw http://paste.ubuntu.com/9961484/ [19:13] alesage: oh, good catch. Let me see if I can stop jenkins. [19:14] just in time. [19:14] elopio, I have to lunch, will do the rest upon return [19:15] now, how did it got in there? I suppose I borked one of the merges. [19:15] elopio, no brendand's branch introduced it [19:15] don't know what else it affects [19:15] (i.e. into what else it's merged) [19:16] alesage: ah, I see. I didn't run that one. [19:16] ok, so trunk is broken. I'll make a new branch. [19:16] elopio, have to assume Brendan didn't either :) [19:42] balloons, hi [19:44] Letozaf_, heelo [19:45] balloons, I fixed pyflakes and pep8 on yesterdays mp [19:45] balloons, by the way how's your wrist ? [19:48] we broke trunk on friday :D [19:48] there's a lesson there, somewhere. [19:48] wrist? what have you done, balloons... [19:49] knome, I was in a soft cast for my wrist for a couple weeks at the beginning of the month [19:49] doing great now Letozaf_ thanks for asking [19:49] balloons, great :) [19:49] elopio: is it don't release on FRIDAYS EVA!!!!! [19:49] aha... because? :) [19:49] hello knome :) [19:49] davmor2: we release on thursdays, every two weeks. So the problem has not been released. [19:50] hullo Letozaf_ [19:50] :) [19:50] Letozaf_, approved [19:51] did you know "hullo" was the word stephen fry and hugh laurie used when they met each other the first time? [19:51] balloons, thanks [20:12] knome, hullo sounds strange to me, I prefer hello :P [20:13] heh [20:13] Letozaf_, how does "hyvää päivää" sound to you then? ;) [20:14] rude ... [20:14] elfy, hah :) [20:15] is what I'd say :p [20:15] if asked [20:15] knome, ah man don't even know how to pronounce it [20:15] one of the few non-rude things you can say in finnish [20:15] :P [20:15] knome, elfy what does it mean? [20:15] Letozaf_: I think that the h,y,v,p's and a's are all silent :) [20:15] Letozaf_, http://www.forvo.com/word/hyv%C3%A4%C3%A4_p%C3%A4iv%C3%A4%C3%A4!/ [20:16] Letozaf_: absolutely no idea - all dutch to me :) [20:16] dutch.. ugh :D [20:16] knome, OMG weired :P [20:16] dutch is a weird language [20:16] http://en.wikipedia.org/wiki/Greek_to_me [20:17] elfy, :) same for me [20:17] :) [20:18] Letozaf_, there you go: http://www.forvo.com/word/laadunvarmistus/#fi [20:18] knome, :S [20:25] what :P [20:27] knome, means "confused" :P [20:28] knome, as elfy said sound greek to me [20:28] do you mean hämmentynyt? [20:28] sounds [20:29] knome, if only I remembered Afrikaans well I could have answered you in Afrikaans, then :P [20:29] knome, but I remember only a few words [20:30] :) [20:35] but afrikaans is relatively understandable [20:35] knome, yeah compared to finnish yes :) [20:36] ;) [20:36] they say finnish is one of the hardest languages in the world to learn... [20:37] Martian would probably be easier [20:37] knome, lekker, biltong, Droëwors [20:37] elfy, ha ha :) [20:38] http://www.effectivelanguagelearning.com/language-guide/language-difficulty [20:38] knome, dronkie, krimpie. toppie [20:38] as related to English that is [20:39] elfy, yep.. [20:39] :D [23:21] elopio, I'm realizing that if two are pairing then the other two will be responsible for the reviews [23:42] short feedback: when upgrading from 12.04 to 14.04 with german language settings, ubuntu switched to firefox in english. firefox-locale-de was missing. ps: please make a simple feedback form, there are so many places to report or file bugs on that launchpad thingy, i dont know where to turn to [23:45] ubuntu-plain, what would be a simple feedback form? [23:45] ubuntu-plain, and how would the people who read it figure out what the feedback is about, if no categorization was made? [23:53] one webpage with forms or a package selector. or a working ubuntu-bug without 7 windows and redirect to some webpage to register an account [23:54] i don't understand how those forms or the package selector could be any easier, or less complex [23:54] do you realize how many packages there are in ubuntu? [23:54] anyway, for the registration part... without that, launchpad would be full of spam