| veebers | thomi: when you have a moment could you take a look at this MP? https://code.launchpad.net/~veebers/autopilot/adding_timezone_timestamp_tests/+merge/236444 | 02:15 |
|---|---|---|
| thomi | veebers: needs a few fixes | 02:21 |
| thomi | diff comments added | 02:21 |
| veebers | thomi: cool, thank you | 02:21 |
| veebers | thomi: ah, hmm, so the reason I use a timestamp in that test is/was to show that given a timestamp the qml app will display and provide the datetime in localtime. I'm not sure if that's still needed in the suite? | 02:24 |
| thomi | ummm | 02:25 |
| thomi | how do you know that the timestamp you use corresponds to the datetime string? | 02:25 |
| veebers | thomi: I went from a known datetime (in UTC), generated a timestamp for it and then checked the time of that time stamp in different TZ's etc. Is that flawed? | 02:28 |
| thomi | hmmmm | 02:28 |
| thomi | So, you're trying to prove that qml doesn't know about timezones? | 02:29 |
| thomi | I'm not sure I understand what that test is trying to say | 02:29 |
| veebers | thomi: hmm, I guess the test was more for me, i.e. given an epoch and a TZ env what time will the qml date display | 02:32 |
| thomi | veebers: but, the TZ is irrelevant, right? | 02:33 |
| thomi | oh, you're using the TZ env var to *prove* that the TZ is irrelevant? | 02:34 |
| veebers | thomi: will, when the TZ changes so does the time string displayed. So i guess irrelevant for the tests, but shows that the right time is shown for the TZ | 02:34 |
| thomi | whaaa? | 02:34 |
| thomi | well, that's fucked up already | 02:34 |
| thomi | shouldn't it be the same string, always and forever? | 02:35 |
| veebers | thomi: I might be confused now. When using the epoch/timestamp it displays in the localtime. When using the string '2014-01-15 12:34:52' it always displays that string regardless of TZ | 02:37 |
| thomi | 'it' being qml, right? | 02:37 |
| veebers | err yeah sorry, the simple qml script there | 02:38 |
| thomi | well, that is fucked up | 02:39 |
| thomi | so... qml applies a local timezone calculation to timestamps, but not to time strings | 02:40 |
| thomi | veebers: in that case, at least comment the test... at the moment I doubt anyone apart from ytou will be able to understand it in 6 weeks time :D | 02:41 |
| veebers | thomi: ack, will do. | 02:42 |
| veebers | thomi: so I'll keep them as 2 tests (referring to your diff comment) | 02:44 |
| veebers | thomi: MP updated | 02:50 |
| thomi | veebers: what about timestamps in the far future? | 02:51 |
| veebers | thomi: I'm not sure I understand? This is against trunk, not the large timestamp fix | 02:52 |
| veebers | these tests will be used to ensure that the large datetime changes don't regress us | 02:52 |
| thomi | ok - I thought the goal here was to fix the timestamp issues in AP? | 02:52 |
| veebers | thomi: that's coming, I'm taking on balloons' branch and finishing it off | 02:53 |
| thomi | ahh, so you'll merge these in first, then, in a separate branch do the large timestamp fix? | 02:53 |
| veebers | thomi: yep, that;s the plan | 02:53 |
| thomi | ahhh ok | 02:53 |
| veebers | (sorry I wasn't very clear on that :-P) | 02:53 |
| thomi | LGTM then, will approve | 02:53 |
| thomi | veebers: done | 02:53 |
| veebers | thomi: awesome, cheers | 02:55 |
| thomi | veebers: just FYI - when you add a card, please add a 'task' card, not a 'scoping' card | 02:56 |
| thomi | (to the LKKB board that is) | 02:56 |
| veebers | thomi: noted. I could have sworn I added a task >:-\ | 03:10 |
| thomi | :D | 03:10 |
| veebers | thomi: ah I see, it was when quick adding the tasks to the main card, it defaults to scoping (didn't notice) and if you move your mouse cursor outside the dialog thingy it closes it :-\ | 03:13 |
| thomi | lol | 03:13 |
| veebers | thomi: hmm, can you do me a favor, can you run this test for me please? autopilot.tests.functional.test_introspection_features.IntrospectionFeatureTests.test_customised_proxy_classes_have_extension_classes | 03:53 |
| thomi | sure | 03:53 |
| thomi | veebers: fails for me... | 03:54 |
| thomi | testtools.matchers._impl.MismatchError: <class 'autopilot.introspection.qt.QtObjectProxyMixin'> not in (<class 'autopilot.tests.functional.test_introspection_features.EmulatorBase'>, <class 'autopilot.introspection._search.ApplicationProxyObject'>) | 03:55 |
| veebers | thomi: yeah for me too (and for the CI run) :-\ | 03:55 |
| thomi | which is interesting - it passed when we landed that code | 03:55 |
| veebers | I don't see it fail in this run: https://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-utopic-autopilot/348/? (syntax error on my part), but the next CI run it failed: https://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-utopic-autopilot/349/? | 03:55 |
| thomi | interesting | 03:56 |
| === vrruiz_ is now known as rvr | ||
| === rpadovani_ is now known as rpadovani | ||
| balloons | elopio, you about? | 21:27 |
| elopio | balloons: I am. | 21:28 |
| elopio | how can I help you? | 21:28 |
| balloons | :-) so I'm looking at home patching and launching again, in an attempt to finish fixing https://bugs.launchpad.net/ubuntu-filemanager-app/+bug/1367453 | 21:29 |
| ubot5 | Ubuntu bug 1367453 in Ubuntu Terminal App "Security dialog causes tests to fail" [Critical,Triaged] | 21:29 |
| balloons | so to that end, I just completed a merge of what andrew did to get music to work | 21:29 |
| balloons | so I haven't really gut checked this much, but here: https://code.launchpad.net/~nskaggs/ubuntu-filemanager-app/fix-1367453/+merge/236628 | 21:31 |
| balloons | elopio, ^^ | 21:31 |
| elopio | balloons: what I don't like of this is that it is untested. | 21:33 |
| elopio | if it works, I'm ok with getting it merged. | 21:33 |
| elopio | but we need a way to make sure that it won't break again. | 21:33 |
| elopio | balloons: I'm thinking of something on ubuntu-app-launch that launches the app on a temporary clean environment. | 21:34 |
| elopio | but many of the details of that are unclear yet for me. We need tedg for that. | 21:35 |
| balloons | elopio well, what 'works' and what we want to do aren't necessarily the same. But music app trunk now has the lovely directly overwriting of the os env vars. that's how he got it to work | 21:35 |
| balloons | elopio, your idea sounds nice | 21:36 |
| elopio | balloons: yes, if this makes the file manager get back to green, +1. | 21:36 |
| elopio | I'm reporting the bugs for the things I mentioned on my testability email to the qa-team. | 21:37 |
| elopio | I'll talk to ted tomorrow to see if he agrees this is a good idea or if he has an alternative. | 21:37 |
| elopio | please reply to that email if you have anything to add. | 21:37 |
| balloons | elopio, he spoke with andrew about it for a long time | 21:37 |
| balloons | if I remember correctly, he felt like not setting the actual env vars was asking for trouble | 21:38 |
| elopio | balloons: I don't care about the actual way of doing it. I guess it will involve setting the env vars. | 21:39 |
| elopio | but what I think it's a must is for the devs to maintain it, and to have the solution tested so it doesn't break repeateatly. | 21:39 |
| balloons | elopio, and this is beyond the uitk helpers yes? | 21:40 |
| elopio | I think that this helper should live in the ubuntu-app-launch branch. But I'm not sure of that. I'll start putting the bug there and see where the discussion leads us. | 21:40 |
| balloons | a system level manner of launching an app in a clean env | 21:40 |
| elopio | balloons: yes. I wouldn't like it to be on uitk. | 21:41 |
| elopio | there might be some arguments to put it there, but I think it must be lower level than that. | 21:41 |
| elopio | balloons: but I see a lot of problems. It's likely that if we launch unity on one env and the apps in another, things can behave crazy. | 21:42 |
| elopio | so the other topic of launching apps without unity running seems related. | 21:42 |
| elopio | I was going to set a call for this week as suggested by thomi, but then jfunk told me to file the bugs first, which makes sense. | 21:43 |
| elopio | I will send an email with the bugs to that thread, as soon as I have them. | 21:43 |
| balloons | elopio, I agree.. something should exist at the ubuntu-app-launch level. It's possible we have a helper in uitk that bootstraps nicely into this (and autopilot too, let's not forget!), but the heavy lifting should be in ubuntu-app-launch. | 21:44 |
| balloons | come to think of it, heh, if anything I'd rather see minimal scaffolding in autopilot itself. I was ranting already about how the AP app launchers are sort of deprecated because of this (ie, they exist and 'work', but you shouldn't use them) | 21:45 |
| thomi | balloons: huh? | 21:46 |
| balloons | hey thomi .. need to watch your pycon video, sounds quite interesting | 21:47 |
| veebers | balloons: why shouldn't you use the ap launchers? Have I missed soemthing? | 21:47 |
| * balloons should have known better than to say this on the ap channel | 21:47 | |
| thomi | balloons: I think it's more the case that you perhaps should have said it sooner | 21:48 |
| thomi | if there's a problem, we need to know about it | 21:48 |
| * thomi can't remember how many times he's written that in this channel | 21:49 | |
| balloons | Of course, I'm always keen to discuss things.. It's why we are talking about it ;-) | 21:50 |
| thomi | well... we're not 'discussing' it at the moment. You mention something,a nd veebers and I are really confused :P | 21:50 |
| balloons | anyways, as elopio mentioned application launching for things under test on the phablet devices have gotten quite muddled | 21:51 |
| balloons | this is a recent thing, although there have been smaller testability issues for awhile | 21:51 |
| balloons | my concern is we are doing a number of things to get our desired outcome (launch the app, pass it arguments, introspect it, have it start in a custom env (clean or setup by us)) | 21:52 |
| balloons | autopilot has launchers for applications that I think should be handling this sort of thing. We really shouldn't be needed to bolt on so many things | 21:52 |
| thomi | ok, what things, specifically, are you having to 'bolt on' ? | 21:53 |
| balloons | So I'm concerned launch_upstart_application and launch_click_application aren't going to be best practice as-is | 21:53 |
| thomi | why? | 21:54 |
| balloons | thomi, elopio is working on expounding things in a grokable manner per above, but see the list in parentheses. Perhaps you looking at an example might also help | 21:54 |
| * thomi looks for a list in parenthesis | 21:55 | |
| balloons | basically right now I am unable to easily create a clean env or pass arguments to my app | 21:55 |
| elopio | thomi, veebers: this is all related to the discussion I started on the mailing lists: we are doing all sorts of weird things to get a clean environment and use a test doubles for services. | 21:55 |
| balloons | I was referring to (launch the app, pass it arguments, introspect it, have it start in a custom env (clean or setup by us)) | 21:55 |
| thomi | ahh | 21:56 |
| thomi | so, we do all those things except the env right now, AFAIK | 21:56 |
| elopio | balloons and I have been discussing it a lot over the previous months, and now I think it's clear the solution is not something that the two of us can implement on a test setup. | 21:56 |
| balloons | thomi, for a long time the env was the only issue, but now argument passing is non-trivial | 21:57 |
| balloons | and it's getting more muddled rather than clearer | 21:57 |
| thomi | so | 21:57 |
| thomi | this is frustrating for me | 21:57 |
| thomi | for a few reasons | 21:57 |
| balloons | this isn't to say autopilot itself isn't doing enough.. But I think the underlying platform is migrating away and not considering testability.. basically elopio's mail | 21:58 |
| thomi | first, this has been happening for a few months, by your admission, but this is the first I'm hearing of it. | 21:58 |
| thomi | Second, you keep making vague statements like 'getting more muddled' - I really don't know what that means | 21:58 |
| thomi | AFAIK, we support argument passing for upstart apps | 21:58 |
| thomi | what, exactly is missing? | 21:58 |
| elopio | thomi: that's the clearest example, passing arguments. | 21:59 |
| balloons | thomi, here's an example from the file manager that triggered this: http://paste.ubuntu.com/8468882/ | 21:59 |
| balloons | check out the launch_test_click() method | 21:59 |
| elopio | we can't pass an argument to an app launched by ubuntu-app-launch, because it ignores the arguments and just runs the exec line of the desktop file. | 21:59 |
| * thomi reads | 21:59 | |
| balloons | and consider why on earth we need to go through so much pain and code to setup an app to test | 21:59 |
| elopio | so we need to do a temp desktop file for the test. I hacked a way to do it and it works for evernote. Now we can copy it all over the place, or instead do it right and put a helper that handles this case on the relevant project. | 22:00 |
| balloons | _patch_home is also fun to read :-) | 22:00 |
| elopio | thomi: this has been happening for a long time, but not because we are ignoring it. | 22:00 |
| elopio | it started as just a nice and clean temporary overwrite of $HOME | 22:01 |
| balloons | the argument issue is recent.. it happened this month | 22:01 |
| elopio | then they kept adding things that kept breaking the clean solution. | 22:01 |
| balloons | it's what caused elopio and I to renew our conversations about this, as indeed things keep being broken | 22:01 |
| elopio | so every time they break it, we make the adjustments. | 22:01 |
| balloons | we had the env issues sorted with a helper | 22:01 |
| elopio | now it's up to a point that's not manageable. We have been saying for a long time that tests break because devs don't plan for testability. | 22:02 |
| thomi | ok, so in launch_test_click I don't see any issue with argument pasing. It seems to me that all this work is due to trying to set a clean environment for the app to run in - is that correct? | 22:02 |
| veebers | I remember seeing an email about ubuntu-app-launch not passing arguments, but not the result of it | 22:02 |
| elopio | it's just now that it became critical. | 22:02 |
| elopio | thomi: so, launch_test_click passes arguments to ubuntu-app-launch | 22:03 |
| balloons | thomi, the point is launch_test_click should just be self.launch_upstart_application( | 22:03 |
| balloons | application_name, | 22:03 |
| balloons | emulator_base=ubuntuuitoolkit.UbuntuUIToolkitCustomProxyObjectBase) right? but you can see the amount of hackery that is needed to make it happen | 22:03 |
| elopio | and ubuntu-app-launch ignores it, as per design. | 22:03 |
| balloons | I mean launching locally or via an installed binary (not on the phone) you can see is simple | 22:04 |
| veebers | balloons: is "--forceAuth false" needed for all applications or just the file manager? | 22:04 |
| thomi | so... the complaint is that it takes a lot of code to launch the app in a sandbox, and you need to repeat that code across multiple applications? | 22:04 |
| balloons | veebers, that's actually needed for file manager and terminal. but other apps need other arguments | 22:04 |
| veebers | It seems that the inability to pass arguments is due to ubuntu-app-launch, not autopilot right? | 22:05 |
| balloons | thomi, yes. And once completed, it breaks as the system beneath changes. | 22:05 |
| veebers | if ubuntu-app-launch did what we expected with arguments then launch_click would do what is needed | 22:05 |
| balloons | thomi, from an AP perspective, I feel like what's happening at the platform level should incoporate better testability that autopilot can hook into | 22:06 |
| thomi | balloons: I agree! | 22:06 |
| elopio | veebers: yes, it's not autopilot's fault that ual ignores arguments. | 22:06 |
| veebers | balloons: I agree too (I think we can all agree with that) | 22:06 |
| balloons | thomi, I'm sure.. That was all I was getting at. I simply don't want you to be left behind in this either | 22:06 |
| balloons | "you" as in autopilot | 22:07 |
| thomi | Can someone point me to the ML thread where id's discussed that libUAL ignored arguments by design? | 22:07 |
| elopio | the problem is that it's also not ual's fault, because it should just pay attention to the desktop file. | 22:07 |
| thomi | elopio: sez who? | 22:07 |
| thomi | elopio: that seems like a silly idea to me | 22:07 |
| thomi | elopio: not being able to launch an app with a command line parameter seems like a bad move | 22:08 |
| elopio | thomi: the devs. This is not a discussion on the ML, this is a bug report. | 22:08 |
| elopio | let me look for it. | 22:08 |
| elopio | thomi: that's what I thought at first. | 22:08 |
| elopio | what I think now is that the passing of arguments or not is not the problem. | 22:08 |
| elopio | the problem is that the devs were not thinking of the automation needs. | 22:08 |
| balloons | right ^^ it's a symptom of a growing issue | 22:09 |
| elopio | I don't care if the solution is to pass an argument, or create a temporary desktop file. | 22:09 |
| elopio | as long as it's a simple call I can do from anywhere, and it | 22:09 |
| elopio | it | 22:09 |
| elopio | it's safe to use always because they keep it up-to-date, I'm happy. | 22:09 |
| balloons | we've had this time and again.. reminders went through it with the account console stuff, clock and calendar suffer via EDS, and now we have PAM, which again has no testability in mind | 22:09 |
| elopio | thomi, veebers: https://bugs.launchpad.net/ubuntu-ui-toolkit/+bug/1367871 | 22:10 |
| ubot5 | Ubuntu bug 1367871 in Ubuntu UI Toolkit "ubuntu-app-launch doesn't pass arguments to the executable" [Undecided,New] | 22:10 |
| balloons | the reason we are passing the arg is because we can't actually test PAM properly | 22:10 |
| balloons | so we are just disabling it via a switch and running the tests the old way. | 22:10 |
| balloons | but I digress | 22:10 |
| thomi | hmmm | 22:14 |
| thomi | ok, I've read the bug | 22:14 |
| thomi | balloons: elopio: so, if the write_sandbox_desktop_file were a reusable fixture that lived in autopilot, would you be happy? | 22:15 |
| thomi | (to fix this specific issue) | 22:15 |
| balloons | so to be clear, nothing I'm spouting on about is a bug whose source is in autopilot. It's merely things that affect test authors and by extension autopilot | 22:15 |
| elopio | thomi: no. The devs convinced me that for that particular case, we need a second app that's called evernote-sandbox. | 22:16 |
| thomi | balloons: I understand, I'm talking with my 'QA tech lead' hat on | 22:16 |
| elopio | for other cases, yes. A helper that rewrites the desktop file of the app to add extra parameters to the exec line would make me happy. | 22:16 |
| balloons | thomi, on that specific issue, if write_sandbox_desktop_file is the 'official' way to do it, then yes. But my expectation as a test author is that you would hide that complexity from me unless needed | 22:16 |
| balloons | so if later the platform decided to pass arguments directly again, my code doesn't change and my tests don't break | 22:17 |
| balloons | do you feel architecturally that this abstraction should be in autopilot? | 22:18 |
| elopio | thomi: but then you are linking that to all the rest of the problems | 22:18 |
| thomi | balloons: well, we already have that code in AP, just not in the public API | 22:18 |
| elopio | where will that desktop file live? In /tmp? | 22:18 |
| elopio | then you hit problems with unity, ual and app armor. | 22:18 |
| thomi | I don't recall the specifics, sorry | 22:18 |
| balloons | ahh yes, lovely app armor.. | 22:18 |
| elopio | thus, the subject of the thread. | 22:19 |
| balloons | that was a ton of fun in malta | 22:19 |
| balloons | until they broke it :-) | 22:19 |
| thomi | so, I think we're going in circles here | 22:19 |
| elopio | thomi: we are. That's why I listed the main problems balloons and I have had, and I am going to report bugs for them | 22:19 |
| elopio | and then schedule a meeting. And then plan who and how to fix them. | 22:20 |
| thomi | elopio: right. please make sure I'm included | 22:20 |
| balloons | elopio, I need to get the security issues in there too. I don't believe those are well represented atm | 22:20 |
| thomi | elopio: and please do it before the sprint | 22:20 |
| elopio | I expect the answer to "who?" would be: "not the QA team". | 22:20 |
| elopio | balloons: yes, please reply to the mail. I think I didn't take into account the terminal and filemanager cases explicitly. | 22:20 |
| elopio | thomi: and of course you will be invited to the party. As I said, I haven't scheduled the meeting yet because I need to collect and report bugs. | 22:21 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!