[02:04] === trainguard: IMAGE 197 building (started: 20140818 02:05) === [03:39] === trainguard: IMAGE 197 DONE (finished: 20140818 03:40) === [03:39] === changelog: http://people.canonical.com/~ogra/touch-image-stats/197.changes === [03:58] morning [07:09] mvo_, good morning :) [07:10] hey tvoss, good morning! === tvoss is now known as tvoss|test === tvoss|test is now known as tvoss [07:51] tvoss, you said preseeding the trust database is not ready, right? [07:52] brendand, it landed end of last week [07:52] brendand, the env var approach is fine for now, but I wanted to make sure that we don't forget about preseeding [07:52] tvoss, ok [07:52] trainguards, could use some help with https://ci-train.ubuntu.com/job/ubuntu-landing-015-0-reconfigure/1/console [07:56] tvoss: looking! [07:57] 'restart' has been removed from the shutdown menu? [07:57] tvoss: so, it seems that the spreadsheet is down! [07:57] tvoss: I get "We're sorry, a server error occurred" when opening the CI Train [07:58] hmmm [07:58] Ok, only on the csv... [08:03] tvoss: so, let me try reconfiguring manually [08:03] Until google is back === sil2100 changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | CI Train support: rsalveti, trainguards | Vanguard (general help): cihelp | CI Train Status: #179 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: citrain really struggles with source packages. Also, google spreadsheet csv output seems not working, expect problems with: reconfigure, silo assignment, queuebot, dashboard... [08:04] :( [08:04] :| [08:04] Let's hope it's only temporary [08:05] sil2100, thank you [08:08] wow, the tests look pretty horrid [08:09] geh [08:09] tvoss: ok, I think it should be reconfigured now [08:09] sil2100, thank you [08:09] sil2100, the dashboard still does not show the new mps [08:10] * ogra_ thinks this cgroups u-a-l stuff really bites us ... that should have come 3 months ago to get enough testing [08:10] tvoss: yeah, the dashboard uses the csv output from the spreadsheet :| [08:10] sil2100, okay [08:10] Which is currently not available... [08:10] sil2100, ack, perhaps we put that information in the channel topic [08:10] during dogfooding the shell stops being able to start apps for me after half a day or so [08:10] tvoss: it's implicitly mentioned there [08:11] tvoss: that you should expect problems with the dashboard ;p [08:11] I'll send out an annoucement on the mailing list anyway [08:11] sil2100, ack, and thanks [08:13] sil2100, i'm not sure what we should do with the u-a-l cgroups stuff ... doesnt look like ted actually worked on fixing the UITK issues with it [08:16] ogra_: so, did we get a confirmation on Friday that the UITK issues are caused by the u-a-l landing? [08:17] sil2100, well, they are caused by apps not being able to start ... random apps ... and it started happening with the u-a-l landing apparently [08:17] sil2100, i also find the phone quite unstable since a few days ... [08:18] i wonder how urgently we need cgroups support at all ... this feels like totally untested ... two week before final ... [08:18] tvoss, could we live without cgroups support ? ^^^ [08:19] tvoss: hah, ok, it's back up ;) [08:19] * sil2100 just finished the e-mail === sil2100 changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | CI Train support: rsalveti, trainguards | Vanguard (general help): cihelp | CI Train Status: #179 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: citrain really struggles with source packages. [08:27] ogra_, sil2100 do we have any strong evidence that the cgroup changes are actually the root cause? apart from: ual landing coincides? [08:27] ogra_, sil2100 also note that multiple people tested the changes, including davmor2 === tvoss is now known as tvoss|test === tvoss|test is now known as tvoss [08:29] tvoss, we didnt roll back and hanve an image tested with a rolled back u-a-l to prove it, no [08:30] ogra_, so all issues are fixed with the rolled back ual? [08:30] tvoss, but i cant start apps anymore after a few hours of using my phone ... which might indeed also be a coincidence [08:30] i doubt any of the testers tried the changes for a day or two [08:31] sil2100, need to restart to try and get sound back [08:31] shit .-.. my phone is stuck with the harps and i cant wake up the screen [08:31] brendand: ok [08:31] ogra_, sure, I doubt that as well. However, do you see anything in syslog or upstart logs when apps fail to launch? [08:31] tvoss, these are changes that should have been tested for a month *at least* [08:32] afaik the feature itself landed in upstart only 4 weeks ago, it is barndnew and untested [08:32] ogra_, if it is untested why did it land in upstart? [08:32] ogra_, also: just calling for a month of testing is unrealistic [08:33] tvoss, i'm calling for having it in the image for a month at least to make sure every side effect can be fixed before a release [08:33] ogra_, in addition: we have no evidence that the cgroup-related changes are responsible at all. With that, I would propose that we start digging into the root cause [08:34] what we used to do with the normal distro in the past to make sure a feature is at least remotely safe to ship [08:34] ogra_, well, then I would propose that you propose changes to our landing process [08:34] landing some super intrusive feature that spans from kernel to UI two weeks before hard freeze is really not the quality assurance i'D expüect [08:35] tvoss, no, i ask people to stick to the schedule ... feature freeze was over a month ago [08:36] tvoss, so the cgroups changes landed with only half our kernels even having necessary features enabled ... i wouldnt call that greatly tested [08:36] ogra_, sure, fixed by now [08:36] ogra_, anyway, it's an important bug fix. Without the cgroup changes, random apps can escape our lifecycle [08:36] yes, which costed a bunch of people a day of work that they had planned for other stuff [08:37] ogra_, and that will happen again [08:37] tvoss, if it is so important, why did it not land earlier but only a few weeks before the end ? [08:37] ogra_, as you pointed out: the upstart support was required [08:38] tvoss, it happens right now ... we'll go into traincon0 because a ton of tests fail [08:38] ogra_, and again: do we have *any* strong evidence for the cgroup changes causing the issues? or are we merely correlating events in the search for an easy answer here? [08:40] I wonder if anyone can reply to James regarding cgmanager debug mode at https://bugs.launchpad.net/ubuntu-app-launch/+bug/1357252 ? [08:40] Ubuntu bug 1357252 in Ubuntu Application Launcher ""Application failed to start." during autopilot tests after the newest ubuntu-app-launch landing" [Critical,Triaged] [08:45] tvoss: at least the above bug mentions Chipaca having seen a problem in upstart where a cgroup could not be created for a task [08:46] the problem were seen after these landed: http://people.canonical.com/~ogra/touch-image-stats/188.changes [08:47] Mirv, do we have the requested debug information available? [08:47] yes, got a weird cgroups-related error in an untrusted helper [08:47] Chipaca, once or in a reproducible way? [08:47] tvoss: once, which makes it even worse in my book [08:47] Chipaca, fair [08:47] tvoss: no, that's why I asked if someone could answer about enabling the debug info in autopilot tests / knows how to do that. maybe we could get it to the dashboard machinery? [08:48] because we *spotted* it once; how many notifications have been lost to this bug without us noticing? [08:48] Mirv, I'm not the right person to talk to about autopilot [08:48] Chipaca, sure [08:49] tvoss: I think we can get CI to that, but who would know to tell them how they'd run cgmanager in debug mode? [08:49] Mirv, I would think ted or someone from foundations probably [08:50] or well, there's info in the bug too, so maybe just asking like that. the problem might be if also that upstream merge proposal is needed. [08:52] Saviq, so we see you added and reverted stuff in unity8/qtmir on the weekend ... we are not sure now about the final state ... was it added or reverted ion the end ? [08:52] psivaa: I wonder if the bug ^ 1357252 cgmanager changes could be tried on the mako running the UI Toolkit tests? there's a patch to ubuntu-touch-session and a cgmanager.conf change [08:52] Mirv, hmmm, jodh might be the best person to help [08:53] Mirv, how did you run camera-app tests? when i run them from local branch they work [08:53] Mirv, but i saw you use phablet-test-run [08:54] brendand: so, yes, any instructions welcome :) I created a click package on mako and installed it, + copied __init__.py + helpers.py to /home/phablet/autopilot/camera_app (/tests) [08:55] Mirv, i'll try that way and see what happens. it could be a difficulty with permissions or something [08:55] Mirv, since setprop uses sudo [08:55] brendand: so, download and install all .deb packages besides the -dev package from this place: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-002/+build/6070859 [08:56] Mirv: i could try that. it'd speed up the process if you could let me know what exact change that i need to do :) [08:56] brendand: so, I also installed dev environment on the phone and checked out your branch there. then ./click-build.sh, pkcon install-local, and updating the already phablet-click-test-setup:d tests in /home/phablet/autopilot [08:56] brendand: in reality, probably the only thing needed would be manually copying the two files into /home/phablet/autopilot [08:56] Mirv: please can you put precise test setup details on the bug so this is captured? [08:56] ogra_, it was reverted and unreverted [08:56] ogra_, i.e. it's good [08:57] good = in ? [08:57] ogra_, yes, everything's good now [08:57] ogra_, I was trying to fix http://s-jenkins.ubuntu-ci:8080/job/generic-deb-autopilot-utopic-touch/ #3637 and up [08:57] psivaa: https://launchpadlibrarian.net/182357738/ubuntu-touch-session.patch + editing /etc/init/cgmanager.conf so that one adds cgmanager_opts="--debug" there [08:57] ogra_, but then it just fixed itself in #3661 [08:58] ogra_, I've currently no idea what was happening between the two, the runner was just getting stuck at some point and jobs timed out after 60mins [08:58] ogra_, but then magically that stopped happening [08:59] hmm wiki.ubuntu.com is down [09:00] Saviq: yeah, actually I wanted to also ask what were all those landings about ;) [09:00] jodh: added the steps to run UITK tests which show the problem reliably to the bug [09:00] Mirv: thanks [09:01] sil2100, so yeah, runner #3637 and up started locking up in a random place (like *all* of them) so I got scared that the qtmir landing, which was the only related difference [09:02] Mirv, do only the uitk tests reliably expose the issue? [09:02] Mirv: ack, will try that. looks like it needs a bit of handholding during the run. but i'll try [09:03] tvoss: psivaa could remember better if "Application failed to start." was also seen sometimes elsewhere [09:03] sil2100, so with rsalveti's help we reverted that, but then before it migrated, stuff went just went back to normal [09:03] psivaa: thanks! [09:03] sil2100, so we landed a reverted of the revert (and snuck in a unity8 test fix) [09:06] Mirv: good morning; so finally silo 13 is ready :) [09:07] it has a complx history, so should need manual help as discussed [09:07] right now, i think the powerpc build is stuck on a dep wait for oxide [09:08] Mirv, i'm not sure what you did, but i can't reproduce it :) [09:08] Saviq: ah, ok, so theoretically in the current state we're back to what we had + that one test fix, rigth? [09:10] dbarth: ok, actually powerpc + ppc64el claim to miss Mir (too) [09:11] sil2100: ^ landing-013 has ubuntu-system-online-accounts that is dropping support for powerpc and ppc64el because of new dependencies. [09:15] brendand: hmm, darn! I'll try again without installing the self compiled camera, only the changed tests [09:16] brendand: success \o/ maybe it was that self compiled camera app to blame [09:17] Mirv: hmm, is that completely necessary? [09:18] Mirv: on which we depend as well, with the new trusted session support [09:19] brendand: approved [09:20] sil2100: ^ apparently trusted session support goes through mir [09:23] sil2100, not only theoretically :) [09:23] I guess nothing we can do about that [09:27] sil2100, i'm running the uitk tests now [09:28] sil2100: we seem to have at least friends-app and reminders-app as reverse dependencies to online accounts, and they used to build for powerpc too :( [09:29] and then account-plugins as a third source package [09:30] dbarth: so the main problem is that we would need to ask archive admins to remove all powerpc/ppc64el binary packages for friends-app, reminders-app and account-plugins with this landing, and they don't generally like it [09:30] Mirv: the problem will repeat though [09:31] Mirv: we have online accounts to land, then webbrowser-app, with the same dependencies [09:31] Mirv: and then another set of OA and webbrowser-app [09:31] so every time we have this dep-wait [09:32] dbarth: could the trusted session support made optional so that you would only build depend on Mir on those platforms that it's available? [09:32] brendand: thanks! [09:33] Mirv: isn't reminders-app a click-only package? [09:33] Mirv: friends-app we don't care about anymore [09:33] Mirv, can I get a silo for 32? dbarth says it’s ok to put silo 6 aside (which would conflict) for now, as it’s not fully ready anyway (not sure if it’s also ok to re-assign it temporarily?) [09:34] sil2100: it does not seem so: https://launchpad.net/ubuntu/+source/reminders-app [09:34] sil2100: yes I know friends-app is obsolete [09:35] Mirv: uh oh! [09:36] dbarth: at least webbrowser-app isn't already shipping on ppc, so that's not a problem [09:37] oSoMoN: done [09:37] Mirv: sil2100: I just checked that yet again the UITK is totally bloced by Jenkins. Is here anybody who could help us? [09:38] bzoltan: what do you mean by 'totally blocked'? [09:39] sil2100: no successful build by Jenkins in the last 4 days.. and not because of the UITK [09:40] sil2100: builds are marked as UNSTABLE, no hint in the logs what could be wrong. Maybe somebody who hear the cihelp keyword... [09:40] Mirv, thanks [09:40] psivaa: could you maybe take a look at that ^ ? === sil2100 changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | CI Train support: trainguards | Vanguard (general help): cihelp | CI Train Status: #179 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: citrain really struggles with source packages. [09:41] sil2100: Mirv: cihelp -> http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sdk-team-ubuntu-ui-toolkit-staging-autolanding/ [09:41] psivaa: ^ [09:43] bzoltan: psivaa: clicking those links again, I can get to eg. http://s-jenkins.ubuntu-ci:8080/job/generic-mediumtests-utopic/2852/artifact/results/autopilot/results/test_ubuntuuitoolkit.xml where one sees a lot of "Application Launch Failed", already discussed today a lot :( [09:44] sil2100: so the thing is that the problem stops SDK team from landing anything, and I'm wondering whether they should start merging manually and testing manually too... [09:44] grrrr [09:44] so for example here: http://s-jenkins.ubuntu-ci:8080/job/generic-deb-autopilot-runner-mako/3575/ <- all failures are due to application failing to start, not UITK problem [09:45] Mirv, sil2100 do we see the issue on any other test suite? [09:45] bzoltan, I see you being blocked, but I would like to understand the issue [09:46] bzoltan: maybe if you learn to click those unstable links far enough (like I just managed to do) and find only failing tests fail with application failing to start, you merge manually? starting with that one (https://code.launchpad.net/~zsombi/ubuntu-ui-toolkit/ucglobals/+merge/230804) [09:46] tvoss: it seems that the "Application Launch Failed" problems bites our autolandings too. [09:46] bzoltan, sure, but apparently it only shows up in the autopilot test suite [09:46] bzoltan: I think tvoss would like still to understand why UITK specifically is able to trigger that problem [09:46] Mirv, yup [09:46] Mirv: merging manually is an option, but it will not solve the problem. [09:47] tvoss: Mirv: I have no idea :( [09:47] bzoltan, for the UITK test suite, I see app/file names being generated on the fly. Could you explain in a few words what the test suite is doing? [09:47] bzoltan: no, and that's why today's battle about understanding the problem. psivaa is already busy trying to get cgmanager debug mode enabled so that we'd get the debug logs we would need [09:48] bzoltan: I just mean that this would be an option so you are not totally blocked [09:49] bzoltan: so for guiding how I got to the page to click the tests open: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sdk-team-ubuntu-ui-toolkit-staging-autolanding/365/console -> "generic-deb-autopilot-utopic-touch #3691" (UNSTABLE) -> "Console" -> "generic-deb-autopilot-runner-mako #3575" (UNSTABLE). then click all the failures open and check if they all were the application failing to start or if you have a real problem [09:50] brendand: how's the test run with u-a-l reverted going [09:50] ? [09:50] tvoss: elopio could tell more about why they have chosen to do this kind of dynamic test creation. [09:50] elopio, ^ [09:51] bzoltan, do you start the actual executable with --desktop_file_hint? [09:51] tvoss: All I know that this is way of creating tests on the fly we started to use since May. [09:51] bzoltan, sure, still would like to understand what it dows [09:51] does [09:52] bzoltan, which timezone is elopio in? [09:52] costa rica [09:52] tvoss: 3:52 AM for elopio right now [09:53] * tvoss hopes that he is an early riser [09:54] Mirv: tvoss: sorry was dealing with something else.. regarding http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sdk-team-ubuntu-ui-toolkit-staging-autolanding/, i see some test failures in the downstream jobs. [09:54] i'm not sure what I can do about it [09:56] sil2100: re: the manta results not being there for dashboard, we have devices being offline.. manta devices appear to have a charging problem [09:56] tvoss: he checks in early and stays id there is anything. here is the launcher what is used to run those on the fly created qml apps http://bazaar.launchpad.net/~ubuntu-sdk-team/ubuntu-ui-toolkit/staging/files/head:/tests/launcher/ [09:58] bzoltan, so the applications are not click packages, correct? [09:59] tvoss: no, these apps are absolutely not. IMHO it was a mistake to do what is done here... that was the reason why I changed the SDK's app deployment to click, because the direct qml deployment was unstable. Crap..I missed that piece of code. [10:00] bzoltan, so the flakiness in app startup was known? [10:02] tvoss: no, it was not about flakyness ... It was about that the app launching policy was changing often and without giving time to the SDK tools to adopt, so the app deployment feature of my tools just broke and I had to run after the changes. So we decided to forget about it and do the click way and deploy the apps in a supported way. === tvoss is now known as tvoss|afk [10:07] tvoss|afk: psivaa: kalikiana will arrive before elopio, I will check with him the details. [10:09] trainguards, icanhasreconfigure on silo 17 please, added unity-scopes-shell there [10:09] Morning all [10:10] sil2100: there are still issues with the recent images but they are mostly already documented :) [10:11] davmor2, for me the app scope stops starting apps after half a day or so ... did you see that anywhere ? [10:12] sidebar launcher still works then, but tapping an icon in the app scope just gets me a vibration ... not even a launcher splash [10:12] Saviq: doing! [10:12] davmor2: yeah ;) We're fighting smoketesting results right now though... [10:13] ogra_: no but because of crap wifi and 3g I was still on fridays image [10:14] me too (192) [10:16] psivaa: yeah, the test failures we now found and they are (usually) all from this applaunch failure that is not UITK problem by itself. so I suggested they do some manual merging while the actual issue is being investigated, in case they see no other failures. [10:18] (or, they might change their test creation method to workaround the problem) [10:19] are there known issues with otto? [10:22] sil2100, ubuntuuitoolkit works with the reversion [10:23] brendand: and with the normal packages installed it fails? [10:23] sil2100, that i need to check now [10:24] sil2100: I've had a thought as to ogra_ 's issue. If the dash is kill as part of the app lifecycle and is restarted but it's contents are still cached then it might have results like when you install an update to an app if you click on the app straight away it won't open. I'll keep an eye out for it while I kill things today and see what happens [10:25] Saviq, ^^ would what davmor2 suspects be possible ? [10:26] i recently checked the oom_adj values for dash and unity ( ricmm asked) ... and they seemed ok ... but i assume it could still be killed at some point [10:26] Saviq: I've asked ogra to see if search still works as that will refresh the scope and see if that means apps open again [10:29] davmor2, ogra_, dash isn't subject to lifecycle [10:29] davmor2, what's the symptoms? [10:30] Saviq: ogra_ can explain it better [10:30] Saviq, its the app scope more than the dash [10:30] Saviq, apps dont start anymore after a while ... i can start them from the launcher, but tapping them on the scope just gets me a vibrate ... [10:30] not even a startup splash [10:30] ogra_, app scope just goes .openUrlExternally on application:/// appid:/// urls [10:31] ogra_, so if anything, the url dispatcher gets confused [10:31] ok [10:31] and launcher doesnt use it ? [10:31] ogra_, no, launcher launches internally (because it's part of the shell, dash isn't any more) [10:31] ok, thanks [10:31] ogra_, when that happens, please have a look at unity8 and -dash logs [10:31] then it must be urö-dispatcher [10:32] s/ö/l/ [10:32] as you press the apps in the dash [10:32] Saviq, will do ... i didnt have it since yesterday afternoon anymore ... seems to behave now [10:32] and when i had it i was iindeed outdoors ... no PC near [10:35] ogra_, Saviq: I wonder, ogra said he had poor signal and little wifi? I wonder if the issue is somehow related to internet. That should be pretty easy to replicate I'll have a play and see [10:42] Mirv, which other test suite is affected by applications not launching? [10:44] sil2100, hmm, the reconfigure didn't stick? [10:44] tvoss|afk, you're afk! [10:44] Saviq: hmm, let me retry === tvoss|afk is now known as tvoss [10:45] Saviq, :) [10:45] hmmm [10:47] hmm? [10:47] hmm :) [10:47] (sounds like the start of an acapellar song) [10:48] tvoss: I don't remember if we've seen it in the smaller suites, psivaa has been staring the most at the test results and might remember if "Application failed to start." has been seen elsewhere [10:49] Saviq: ok, I know what happened, it should be good now :) One merge request was invalid (i.e. not a merge request) - I changed that and reconfigured [10:49] I mean, still reconfiguring though [10:49] Mirv: i remember seeing them, now and then.. need to check if that's seen in recent ones [10:49] Saviq: ok, it's ready to build [10:49] sil2100, sorry.. [10:50] psivaa: I have that vague memory too, about seeing it elsewhere [10:59] sil2100, the reversion definitely works for uitk [11:02] brendand: ok, good to know [11:04] ogra_, tvoss, bzoltan: not sure what the final consensus was regarding UITK and u-a-l, but it seems that with the cgroup change is the cause of the UITK test failures [11:04] ogra_, tvoss, bzoltan: the revert of that fixes the test situation [11:04] sil2100: psivaa: tvoss: here's music-app failing on Application failed to start. http://ci.ubuntu.com/smokeng/utopic/touch/mako/188:20140814:20140811.1/9648/music_app/1530204/ [11:04] Just for your information [11:04] Mirv: might be related to the same landing [11:05] sil2100: yes, I just digged that out since tvoss was interested if we see the same problem outside of UITK, or if it's restricted to how UITK starts apps [11:05] Mirv: thanks! [11:05] and here's messaging app http://ci.ubuntu.com/smokeng/utopic/touch/mako/189:20140814.1:20140811.1/9674/messaging_app/1532527/ [11:05] Anyway, since it also seems to affect other apps, it seems like a serious issue [11:06] hmmm [11:07] * sil2100 has problems running dialer-app/messaging-app tests [11:08] yes I think there has been no doubt about the seriousness :) [11:08] I think I need to change the platform [11:08] sil2100, Mirv okay, thanks for digging things out, seems to me the cgroup setup procedure is racy [11:08] sil2100, Mirv just a wild guess, though [11:09] tvoss: I assume tedg is the best person to work on this, right? [11:09] sil2100, exactly. However, some help with getting the cgmanager debug information into his hands would be greatly appreciated [11:09] thostr_, ^ [11:09] Mirv: ack, thanks. sorry could not check that. making the uitk tests with the cgmanager patch in the lab is a little involving [11:10] added a comment to the bug documenting it there too [11:11] psivaa: no problem, I just needed to dig far enough and found myself [11:11] sil2100, dialer/messaging seem ok for me [11:11] sil2100, what problems are you seeing? [11:13] tvoss, note that we use lxc containers too ... might be that we also do not have a "normal" cgrops setup [11:13] ogra_, ack [11:14] (we definitely dont use a sandard lxc setup on the phone, this is very specific, not sure if cgmanager might act differnt due to that) [11:16] Mirv, sil2100: silo 10 is ready for publication, when you have a minute [11:17] oSoMoN: I happened to stare at the spreadsheet at the second you turned it to testing pass and started looking at it :) [11:18] Mirv, that’s the very definition of responsiveness :) [11:18] thanks! === alan_g is now known as alan_g|lunch === alan_g|lunch is now known as alan_g [12:25] tvoss: non-approved MP:s [12:25] (claims https://code.launchpad.net/~thomas-voss/location-service/enable-report-to-free-databases/+merge/230618 ie no top-approve) [12:31] Mirv: is silo 13 ok for landing now? [12:34] brendand, Mirv, ogra_: so it seems that the ofono-landing from #192 causes all the dialer-app/messaging-app failures [12:34] ouch [12:35] brendand, Mirv, ogra_: after downgrading the packages listed here http://people.canonical.com/~lzemczak/landing-team/192.commitlog I got only 1 failure on dialer-app (which was seen in earlier images as well) [12:35] Confirming messaging as well... [12:35] Need to poke Bill about this once he's up === pete-woods is now known as pete-woods|lunch [12:39] dbarth: you didn't answer the question about only depending on mir on non-powerpc/ppc64el and allowing the package build without trusted session support there? [12:39] dbarth: if we publish now, it will be blocked and the archive admins would probably ask for the same thing [12:40] or sil2100 may comment on that, but I believe the default is to ask upstream to workaround so that package can still at least build on all archs, just with less features [12:40] sil2100: good to have that confirmed! the ofono landing looked like the probable culprit in 192.changes [12:41] Mirv, thanks, fixed [12:42] hm, messaging-app has 5 failures still, re-running to see if that's always the same value [12:48] sil2100, who owns messaging? is it boiko ? [12:48] brendand: yes, generally it's boiko [12:49] hm, again the same number of failures on re-run - still better than with the ofono landing [12:49] sil2100, actually i suppose since it's ofono that's the culprit we should talk to foundations [12:49] brendand: the landing was driven by bfiller, so I wanted to poke him directly [12:50] dbarth: I see that the online-accounts is depending strictly on libmirclient-dev. it should be made "libmirclient-dev [!arm64 !powerpc !ppc64 !ppc64el]" similar to eg. mesa is doing it [12:50] sil2100: what failures? maybe I can take them a look? === MacSlow is now known as MacSlow|lunch [12:52] boiko: I was trying to determine what causes almost all dialer and messaging tests to fail on smoketesting, e.g. http://ci.ubuntu.com/smokeng/utopic/touch/mako/197:20140818:20140811.1/9753/dialer_app/ (same for messagign) [12:52] boiko: this seems to be failing since #192, not sure why no one from your team noticed :) [12:53] boiko: anyway, after reverting the ofono landing from #192 driven by bfiller, dialer-app tests seem to be passing [12:53] sil2100: interesting, let me try it locally, let's see if I can get that [12:53] boiko: these failures are reproducible locally, at least on 2 of my devices [12:55] sil2100: I moved the online-accounts discussion to #ubuntu-touch so that mardy can join it. I checked that various other packages depend on libmirclient-dev with architecture list specified, so it should be possible to modify code to just build with less features on powerpc [12:55] Mirv: last i checked, cjwatson was recommending to not special case the packaging, so that's why we kept it architecture independant in webbrowser-app as well [12:56] Mirv: so, do I need to do something else on this silo to land? i guess i'm confused about which next button i should press after all this time ;) [12:56] dbarth: it was probably because oxide already was never built for other than armhf/i386/amd64, so everything works in that case [12:57] dbarth: mardy seems to be on it, with luck the building without that feature on powerpc is a no-brainer [12:57] dbarth: also, cjwatson recommends not to add special case architecture lists generally by making everything simply work on all platforms :) [12:58] but in this case, we don't have Mir so we should build online-accounts without features requiring it, similar to mesa and others [13:11] ok [13:13] boiko: like it says above ^ , landing-020 publishing failed since MP:s are not top-approved [13:13] boiko: https://ci-train.ubuntu.com/job/ubuntu-landing-020-2-publish/1/console [13:20] MacSlow|lunch: https://code.launchpad.net/~macslow/unity-notifications/fix-1354406/+merge/231005 <- can you get someone to review this branch? :) [13:26] Mirv: oups, I just tested the silo (as requested by renatu), let me check the branches [13:26] Mirv: finally be able to run the uitk with the patch you mentioned... http://q-jenkins.ubuntu-ci:8080/job/psivaa-utopic-touch-mako-smoke-daily-test-mako-14/31/console [13:27] Mirv: tried to tweak the device and run the tests but continuously had some screen-unlock issues, but this should give us the results [13:27] psivaa: whee, excellent! could you then add the resulting /var/log/upstart/cgmanager.log to bug #1357252 for jodh to see? [13:27] bug 1357252 in Ubuntu Application Launcher ""Application failed to start." during autopilot tests after the newest ubuntu-app-launch landing" [Critical,Triaged] https://launchpad.net/bugs/1357252 [13:28] Mirv: ack, will do [13:28] let's hope there's something revealing in there [13:29] Mirv: ok, I'll wait for renatu to be back and ask again about the MRs [13:29] boiko: ok, thanks. [13:48] boiko: were you able to reproduce the issues? Do you know any reasons why the ofono-related landing could have caused at least some of them? [13:49] sil2100: sorry, got distracted with other things, let me try it [13:49] boiko: thanks! It basically makes us blind as we have no AP test results right now === pete-woods|lunch is now known as pete-woods [14:00] sil2100, did you find the ofono landing that broke the apps? [14:04] psivaa: Mirv: sorry guys, i did not follow the progress. Is there an MR on the way to fix the applauncher problem? === zbenjamin_ is now known as zbenjamin === MacSlow|lunch is now known as MacSlow [14:17] sil2100, always trying :) [14:26] bzoltan: next step is having a debug log from psiva_a to the bug #1357252 report [14:26] bug 1357252 in Ubuntu Application Launcher ""Application failed to start." during autopilot tests after the newest ubuntu-app-launch landing" [Critical,Triaged] https://launchpad.net/bugs/1357252 [14:26] so no code proposals yet [14:31] brendand: I just identified that the landing from 192 caused the dialer app failures [14:33] sil2100, telepathy-ofono from 0.2+14.10.20140807-0ubuntu1 to 0.2+14.10.20140814-0ubuntu1? [14:33] brendand: yes, along with others as well, I didn't triage to exactly one component, but I know the landing that served this was the cause [14:41] plars: hey, did the 'do not send me emails' for jenkins failures ever get pushed to jenkins? [14:46] sergiusens: hmm, I thought so, are you still getting spammed? [14:46] plars: yes [14:46] sergiusens: I'll check into it [14:47] sergiusens: nag me again if it still happens after today [14:55] tvoss, tedg: so, I see bug LP: #1357252 has been identified to be in upstart - do we have anyone directly assigned to this? [14:55] Launchpad bug 1357252 in Ubuntu Application Launcher "Upstart jobs specifying cgroup fail to start occasionally" [Critical,Incomplete] https://launchpad.net/bugs/1357252 [14:55] sil2100, jodh is looking into it [14:57] sil2100, did you actually revert the ual landing? [14:57] tvoss: not yet, but depending on how long the upstart fix will take we might revert u-a-l still [14:58] sil2100, ack and thx [14:58] Since it affects multiple components and clouds our smoketesting [14:58] But I still want to give some time to assess === plars changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | CI Train support: trainguards | Vanguard (general help): plars | CI Train Status: #179 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: citrain really struggles with source packages. [15:02] sil2100, Mirv: silo 10 is ready to be published, when you have a moment [15:04] oSoMoN: aye! Doing [15:05] sil2100, thanks! [15:08] tedg: can you extend the tests for app-launcher to run UITK AP tests before landing a new launcher? [15:08] perhaps some apps as well [15:09] t1mp, Can you propose an MR for what you want changed? /tests/manual [15:10] t1mp, Here's a link: http://bazaar.launchpad.net/~indicator-applet-developers/ubuntu-app-launch/trunk.14.10/view/head:/tests/manual [15:14] tedg: I'll report a bug with the steps to run the tests [15:23] Mirv: bzoltan: jodh: https://bugs.launchpad.net/ubuntu-app-launch/+bug/1357252/comments/10 is the debug logs from /var/log/upstart/cgmanager.log for a uitk test run [15:23] Ubuntu bug 1357252 in Ubuntu Application Launcher "Upstart jobs specifying cgroup fail to start occasionally" [Critical,Incomplete] [15:23] sil2100, is there anything i need to look at? [15:23] brendand: did you have a look at filemanager failures as well? [15:23] sil2100, nope - i can do that now [15:24] tedg: sorry I don't have time to do the MR now, I reported this bug https://bugs.launchpad.net/ubuntu-app-launch/+bug/1358356 [15:24] Ubuntu bug 1358356 in Ubuntu Application Launcher "Run ubuntu-ui-toolkit tests before landing" [Undecided,New] [15:25] * ogra_ sees the webbrowser changelog and hugs oSoMoN [15:29] * oSoMoN gets his daily share of ogra_’s hugs :) [15:29] :D [15:29] well deserved ones :) [15:30] psivaa: thanks. I think this demonstrates the issue is with cgmanager. [15:39] t1mp, ping [15:39] t1mp, for https://bugs.launchpad.net/ubuntu-app-launch/+bug/1358356 [15:39] Ubuntu bug 1358356 in Ubuntu Application Launcher "Run ubuntu-ui-toolkit tests before landing" [Undecided,Incomplete] [15:40] t1mp, I checked with bzoltan in the morning, and it seems like the uitk testsuite is using --desktop_file_hint to the start the application. Is that correct? If not, how are the dynamic application started? [15:48] tvoss: good question to which I don't know the answer. on desktop I simply use autopilot3 run, and for device phablet-test-run from desktop [15:48] tedg, ^ [15:48] kalikiana: ^ do you know? [15:49] tvoss, I think they're creating desktop files in ~/.local/share/applications and then running them as legacy apps. [15:49] ah elopio should know [15:49] elopio: ^^ [15:49] NOTICE! Google CSV output is broken again, expect problems with the dashboard [15:50] bfiller: I was able to assign a silo for you manually though, silo 14 [15:50] Saviq, tsdgeos, can you elaborate on the issue with testShell segfaulting in https://jenkins.qa.ubuntu.com/job/unity-phablet-qmluitests-utopic/977/ ? [15:51] fginther: issue is, it segfaults and is not marked as failing [15:51] tvoss, t1mp: with this http://unity.ubuntu.com/autopilot/api/testcase.html#autopilot.testcase.AutopilotTestCase.launch_upstart_application [15:51] using .desktop files [15:52] sil2100, file manager failures don't seem to be reproducible [15:52] ouch [15:53] Saviq, tsdgeos, Is checking for a crash (any crash) and failing the test an appropriate response in these cases? [15:53] fginther: imho yes [15:53] a crashing test is always a failure [15:53] brendand: we had them failing on smoketesting for at least 2 images [15:53] tests shouldn't crash i'd sya [15:54] tsdgeos, agreed, just trying to think of an appropriate method for identificaton [15:54] brendand: could you maybe fill in a bug for the smoketesting failures then? We'll start counting it out and poke the developers [15:54] fginther: the exit status of the process will be non 0 [15:54]