[05:12] cjwatson, I declared trusty but I need a cloud image of trusty to build the base testing environment. I'll ping smoser [05:17] hm, and also update distro-data on all the testing nodes in the lab so they know what is the devel release. [05:18] ev, vila ^ could you check that servers in the lab are up to date? [05:20] vila, you'll need to upgrade the machines used for destkop testing to trusty and probably the ones running raring to saucy for SRUs or split in 2 pools for raring/saucy otherwise SRUs on raring will have to be done manually [05:30] vila, RT65382 [07:00] jibel: ack, will raise the ticket during the standup and talk with fginther, doanac and plars [08:09] http://cdimage.ubuntu.com/ubuntu-touch/daily-preinstalled/20131021.1/ [08:09] welcome to trusty ! [08:10] ogra_: \o/ [08:10] and here is the system image :) http://system-image.ubuntu.com/trusty-proposed/mako/ [08:11] ogra_: how will http://people.canonical.com/~ogra/touch-image-stats/ behave btw? [08:11] if we have a saucy and trusty image rebuild the same day? [08:11] will be crowded with the next build and fine agaiun with the one after [08:11] i have to do a special setup for saucy later today [08:11] the version of the ubuntu chroot are different? [08:11] ok [08:11] the paths all changed with the release [08:11] you mean for building ? [08:12] no, I think we need to be clear which version is shown [08:12] yeah it is always the release we build for which we use fro the chroot [08:12] ok, and we'll have a saucy/ dir? [08:12] we already do [08:13] http://system-image.ubuntu.com/ [08:13] * didrocks sees current/ [08:13] ogra_: I'm speaking of http://people.canonical.com/~ogra/touch-image-stats/ [08:13] we have saucy and trusty channels [08:13] yeah, that will get a saucy dir [08:13] great :) [08:15] jibel: Huh, why can't you bootstrap it off the saucy image? [08:15] cjwatson, that's what I did [08:16] jibel: OK, I still see nothing at https://jenkins.qa.ubuntu.com/view/Trusty/view/AutoPkgTest/ though [08:17] cjwatson, autopkgtest for trusty is open, jobs that were queued are running. I'll create the views and file a ticket to do the same on the public instance [08:18] jibel: OK, I have a feeling proposed-migration lost its records of them so I'm inclined to remove its state and let it all retrigger [08:19] jibel: There should be a lot more than just mod-gearman listed in excuses and I don't see any records of it collecting any PASSes [08:19] * cjwatson nukes the state dir [08:19] cjwatson, you should see visp as FAIL and glib2.0 as PASS [08:20] I do not, but sadly adt-britney loses results sometimes as previously explained [08:20] that's the 2 I used for testing [08:20] I don't trust it :( [08:20] cjwatson, :( I'll spend some time on it again this week. [08:20] Let's see if it does a better job this time round [08:21] jibel: Do you think you could edit https://wiki.ubuntu.com/NewReleaseCycleProcess to explain what we need to do (or notify somebody to do) when setting up new series? [08:21] This has become fairly critical infra so we should make sure it's up early [08:23] cjwatson, sure, I'll update it. I added a specific document to the prject auto-package-testing too. [08:24] Thanks [08:24] Should be a batch of trusty jobs re-running now [08:24] I'll keep an eye on p-m to try to ensure it doesn't lose them [08:29] vila, retoaded I'm creating Trusty All and Autopkgtest views on jiufeng, I'll leave others to you. [08:30] jibel: please update the ticket you created so we can keep track of what is left to be done (and document the process) [08:30] vila, that's different, I'll file another [08:34] vila: joining us? [08:34] ev: ? [08:34] didrocks: omw [08:35] didrocks: ev is at the cloud sprint [08:36] vila, RT65387 [08:37] retoaded, ^ [08:53] didrocks: vila: so in general i believe nothing should need to be done when we continue staying on devel-proposed ... except creating new jobs to track the OLDRELEASED-proposed [08:53] that should be our goal... maybe its already like that and we are just struck by lack of phones [08:53] asac: agreed with that goal [08:53] +1 [08:53] we are far from it, but let's focus on it :) [08:54] didrocks: asac: ogra_: as per the build this morning. the jobs have not run yet. they are only triggered with changes to http://system-image.ubuntu.com/devel-proposed/mako/index.json [08:54] which has not changed yet today [08:54] psivaa: right. so if we had the channel moved on top ofd trusty-proposed it would have automatically happened [08:55] psivaa, as i said, thats still saucy [08:55] psivaa: however, why wasnt build 101 from devel-proposed picked up [08:55] i assume thats because we have all phones down? [08:55] because the phones are down ? [08:55] right. so in theory nothing is wrong : [08:55] would be my only explanation [08:56] phablet-flash ubuntu-system --channel trusty-proposed --no-backup -d maguro [08:56] * ogra_ twiddles thumbs [08:57] yay, my first four apps are in the appstore ! [08:57] ogra_: asac: phones are not down there, the jobs have not been triggered yet [08:57] psivaa: because 101 is too new? [08:57] psivaa: its 2-3 days old: http://system-image.ubuntu.com/devel-proposed/maguro/ [08:57] 101 is from friday night iirc [08:58] asac: the jenkins job failed as told [08:58] ohh 101 is from friday? sorry looking at it now [08:58] the cdimage build ends up in a new dir ... [08:59] do you use anything from cdimage for the trigger ? [08:59] http://cdimage.ubuntu.com/ubuntu-touch/saucy/daily-preinstalled/ [08:59] (note the saucy subdir) [09:01] afaik they use the index.json to trigger [09:01] (from system image channels) [09:07] jibel: I wonder if the causes get lost on FAIL or something [09:08] jibel: for instance the very most recent run has: [09:08] lintian 2.5.19ubuntu1 FAIL lintian 2.5.19ubuntu1 [09:08] jibel: but I don't see lintian mentioned anywhere in excuses, not even as something triggered by another package [09:09] jibel: (unfortunately I didn't catch this in time to stop p-m so you could investigate in place, but perhaps that's a hint) [09:10] Though lintian actually first failed in the run starting at 08:37:03 [09:10] I mean p-m first recorded it as failing then [09:14] cjwatson, yes, I noticed that on saucy but couldn't find the cause. I also noticed results on excuses changing from PASS to FAIL without obvious reason. but I didn't find how to recreate the case locally. [09:15] cjwatson, I updated https://wiki.ubuntu.com/NewReleaseCycleProcess and added a link to the detailed documentation. [09:15] Thanks [09:45] ogra_: Any luck with image 1? [09:45] asac: ogra_: the reason for the smoke tests failing to run 101 is due to regressions in the way we now flash the devices. [09:45] there was a merge to not use utah for this and this is causing the issue [09:45] cjwatson, havent checked phonecalls yet ... and popey needs to test on mako [09:45] cjwatson, installs and runs though :) [09:46] popey, mind testing the trusty-proposed channel on your mako ? [09:46] doanac: ^ would you be able to take a look at this the issue is only in install-and-boot [09:46] ogra_: ooh, there's a new image? [09:47] popey, two :) onte trusty and one saucy [09:47] which one do you want testing? [09:47] please do the trusty one first as this is blocking opening [09:47] popey, trusty-proposed please [09:47] kk [09:48] just a quick smoketest should do [09:48] the delay wont be my testing, it will be my current 3g connection === vrruiz_ is now known as rvr [09:48] phablet-flash: error: argument : invalid choice: 'trusty-proposed' (choose from 'cdimage-touch', 'cdimage-legacy', 'ubuntu-system', 'community') [09:48] oh, typo [09:49] 2% [> ] 8,183,808 368KB/s eta 8m 13s [09:49] not bad on the M6 [09:51] cjwatson, popey, asac, didrocks, trusty is good on maguro [09:52] ogra_: can you run unity8 ap? [09:52] :) [09:52] ogra_: sweet! [09:52] lol [09:52] ogra_: guess thats still not good on maguro [09:52] oh we have 101 http://reports.qa.ubuntu.com/smokeng/saucy/touch_mir/ [09:53] install-and-boot isnt happy though [09:53] http://reports.qa.ubuntu.com/smokeng/saucy/touch_mir/mako/101:20131018:20131015/4775/ [09:53] * asac assumes psivaa and vila are on that [09:53] asac: I'm on cu2d for trusty withe didrocks [09:54] psivaa: ^ you're on it ? [09:55] asac: as i posted above, there was a regression in the way we provision the devices. i will have to revert the changes that doanac made in order to make the jobs proceed [09:55] * ogra_ notes that the maguro is really to slow for zombie apocalypse [09:57] i handed my 100 phone out at a conference over the weekend. took less than 5 mins for someone to crash it [09:58] yeah, we need to do more long term dogfooding [09:58] i wrote 11 webapps on the weekend ... [09:58] running like 5 of them at the same time makes everything go cracy [09:59] the backgrounded apps never save their state [09:59] and at times unity seems to get all confused and puts the wrong labels onto the wrong thumbnals etc [10:00] i had two crashes over the weekend where unity just restarted [10:02] asac: vila: running the jobs now for 101 saucy.. i'll let you know the outcome [10:20] ogra_: trusty 1 looks good here on mako [10:32] great [10:32] cjwatson, go for it then [10:38] ogra_: successfully ran camera_app on #1 [10:38] doing webbrowser_app now as well [10:38] didrocks: ^^ [10:38] (on maguro) [10:38] great! [10:38] unity is unfortunately unhappy still, right? [10:38] asac: we'll need an iso though to plug trusty tests on desktpo [10:38] at least on maguro i would think so [10:38] desktop* [10:39] didrocks: did the last day regression fix make it in for unity AP? [10:39] cjwatson: I think that will take a while? (trying to find a backup solution) [10:39] asac: they did [10:39] ah [10:39] unity8 [10:39] not 7 [10:39] no, it's not in [10:39] as we didn't do any SRU-related work [10:39] right. thinking about trusty. assume would come in once we start landing stuff, right? [10:40] asac: there are 3 things to do/ensure: [10:40] - daily-build ppa can build trusty [10:40] - config changed by Mirv [10:40] - otto being able to run tests on trusty (or I need to find a backup solution) -> meaning, an iso available [10:41] - views updated ob jnekins (may be minor may be scripted already) [10:41] *on jenkins [10:41] didrocks: we can probably do you a desktop build [10:41] didrocks: so we depend on a desktop iso? or server? [10:41] cjwatson: oh, that would be excellent :) [10:41] asac: desktop [10:42] cjwatson: and ppas can build trusty already? [10:42] didrocks: running [10:42] thanks cjwatson :) [10:42] didrocks: er I'd have to check [10:42] ok [10:42] didrocks: you could try it :) [10:42] yeah, let's mess with it :) [10:42] will keep you posted [10:42] * cjwatson goes to open trusty [10:43] asac: are the automated tests already running or are your results from manual testing? [10:43] (getting coffee first so this is your last opportunity to object) [10:44] cjohnston: mine are from manual, because the automatic ones seem to not run [10:44] asac: they don't look to be setup.. let me see if I can figure that out [10:44] cjohnston: check with psivaa and vila and didrocks if there was something started to fix this already [10:44] cjohnston: enjoy :) [10:44] cjohnston: so from what i uinderstand touch_mir should be pulling devel-proposed channel [10:45] cjohnston: that channel wasnt moved over, so we need to wait till stgraber gets up, then our touch_mir jobs should pick the right thing up [10:45] [PPA ubuntu-unity-daily-build] [ubuntu/trusty] bamf 0.5.1+13.10.20131011-0ubuntu1trusty1 (Accepted) -> sounds good :) [10:45] ok [10:45] cjohnston: build the build number will go back to 1, so maybe check what happens on dashboard then [10:45] didrocks: good [10:46] cjwatson: enjoy your coffee :) [10:47] cjohnston: fyi, saucy tests on touch devices are running but we have not set anything up for trusty yet [10:48] psivaa: cool.. looks like there are still MPs that need to land for trusty tests to work [10:48] cjohnston: yea they are still in the review stage [10:52] asac, Saviq , unity8 still fails for me, http://paste.ubuntu.com/6276025/ is what i se printed in the unity8 log if the tests try to start [10:52] ogra_, stale socket [10:52] yeah [10:52] i thought we had fixed that [10:52] ogra_, no, we amended it [10:52] the test properly shuts down the shell [10:52] ogra_, but if unity8 crashes - mir never removes the socket [10:53] ogra_, it should only ever happen on unity8 crash now [10:53] well, thats after a proper shutdown i think [10:53] * ogra_ reboots again to watch if there is a crash happening [10:53] ogra_, hmm, can you rm /run/user/32011/mir_socket and clear /var/crash - and see if indeed you get the stale socket without unity8 crashing? [10:53] or that [10:54] ogra_, we should probably clear the socket in pre-start unity8 [10:54] root@ubuntu-phablet:/# ls /var/crash/ [10:54] _usr_bin_unity8.32011.crash [10:54] wow [10:54] it isnt even on the screen [10:55] and i cleared the dir when shutting down (without unity running at all) [10:55] * ogra_ tries that again [10:55] ogra_, how big is the crash file? [10:55] ogra_, bug #1240866 probably [10:55] bug 1240866 in Unity 8 "unity8 crashed with SIGSEGV in getenv()" [High,Triaged] https://launchpad.net/bugs/1240866 [10:55] root@ubuntu-phablet:/# ls -l /var/crash/ [10:55] total 384 [10:55] ---------- 1 phablet whoopsie 390128 Oct 21 12:53 _usr_bin_unity8.32011.crash [10:56] cjohnston: do you know why we see those bogus entries on http://reports.qa.ubuntu.com/smokeng/saucy/ [10:56] cjohnston: e.g. the ones with just one date rather than normal veresion? [10:56] means the install failed [10:57] no crash this time [10:57] Saviq, so firing up the test definitely creates the crash file [10:58] seemingly on shutdown of the shell [10:58] cjohnston: asac: i dash_ignore 'd them but since they were from the weekend we need to wait till 1330 UTC to get them removed [10:58] ogra_, yeah, we still have a crash on shutdown indeed [10:58] cjohnston: looks buggy to me at least :) [10:58] ogra_, or do we... thought we had that fixed [10:58] psivaa: right. still think the visualization could be at least improved [10:58] ogra_, can you give me your steps? [10:58] asac: we don't really have a way to guess what the build number should be if we can't get it [10:59] cjohnston: so those cases we cant download? [10:59] cjohnston: cant get the index.json? [11:00] AFAIK that part was fine [11:00] Saviq, i flash. makes the image writable and run "phablet-test-run -n -p unity8-autopilot unity8" ... once the old shell stops i see the crash file apprear (before it prints that it starts the tests in the other terminal) [11:00] cjohnston: my understanding is that we are looking at http://system-image.ubuntu.com/trusty-proposed/mako/index.json [11:00] cjohnston: that one has the version info included [11:00] I thought we were getting the info from the device [11:00] ogra_, /me does [11:00] cjohnston: right. maybe we should use that info as well and then even compare those two infos :) [11:01] as another meta test [11:01] ogra_, did you keep the display lit? [11:01] ack [11:01] Saviq, i tried to ... [11:01] Saviq, it turns off when the shell goes away [11:02] ogra_, you can always go "powerd-cli display on bright" [11:03] ogra_, aah [11:03] i cant trigger a crash when just stopping and starting unity8 [11:03] must be the start of the test that causes it [11:04] ogra_, bug #1240801 [11:04] bug 1240801 in unity8 (Ubuntu) "autopilot unity8 fails with "No GSettings schemas are installed on the system"" [Low,New] https://launchpad.net/bugs/1240801 [11:04] ogra_, that's fixed in unity8 trunk (and only affects testing) [11:04] ok, trusty is open [11:04] will be starting autosyncs shortly [11:04] aha [11:04] and removing the proposed block shortly as well [11:04] cjwatson, \o/ [11:04] ogra_, export XDG_DATA_DIRS before "autopilot run unity8" [11:04] ogra_, and it will be fine [11:04] Saviq, oh [11:05] ogra_, XDG_DATA_DIRS=/usr/share autopilot run unity8 [11:06] phablet@ubuntu-phablet:~$ initctl stop unity8 [11:06] unity8 stop/waiting [11:06] phablet@ubuntu-phablet:~$ XDG_DATA_DIRS=/usr/share autopilot run unity8 [11:06] Loading tests from: /usr/lib/python2.7/dist-packages [11:06] Tests running... [11:06] and it doesnt lie :D [11:07] * ogra_ sees the unity tests on screen [11:13] autosyncs starting [11:15] asac, psivaa they have been removed from the dash [11:15] cjwatson: excellent! [11:16] asac, Saviq, http://paste.ubuntu.com/6276146/ [11:16] so unity8 looks pretty good [11:16] ogra_, yeah, just one crash on startup - the getenv one [11:17] ogra_, that test will run if you start it again [11:17] k [11:17] ogra_: webbrowser is OK too (37 tests run) [11:17] zero failure :) [11:18] ogra_: so where is the udev fix :)? [11:18] i want systemsettle to stop complaining [11:18] * ogra_ re-runs to prove the failure goes away [11:18] asac, its in 101 [11:18] ogra_: so should be in 1 too? [11:18] * asac checks top [11:18] Ubuntu Touch 101 ;D [11:18] was in trusty-proposed, i dont think it was in trusty when we built the image [11:18] ogra_: hmm. 97.8 idle only [11:19] pulse is going wildish [11:19] asac, i think it was held [11:19] oh wait :) [11:19] lol [11:19] that was my x220 [11:19] heh [11:19] ogra_: we still see systemsettle issues on 101 though: http://reports.qa.ubuntu.com/smokeng/saucy/touch_mir/maguro/101:20131018:20131015/4776/gallery-app-autopilot/ [11:20] ogra_: all are still orange because of that: http://reports.qa.ubuntu.com/smokeng/saucy/touch_mir/maguro/101:20131018:20131015/4776/ [11:20] pretty busy [11:20] here is the before run of gallery http://reports.qa.ubuntu.com/smokeng/saucy/touch_mir/maguro/101:20131018:20131015/4776/gallery-app-autopilot/493965/ [11:20] doesnt hit 97% average [11:21] ogra_: sure its in 101? build 1 feels quiet [11:21] asac, right, but no udevd issue it seems [11:21] looks like top is the biggest consumer in the logs [11:22] ogra_: ueventd is still there though [11:22] yes, thats fine [11:22] ogra_: the first sample always has top [11:23] ueventd != udevd [11:23] ogra_: but why would top be so slow all of sudden? [11:23] we never had this problem before [11:23] and it is always under 0.5% as i see it [11:24] dunno, but top consumes >3% [11:24] right. thats new [11:24] that keeps us below 97% idle [11:24] no idea why [11:24] cjohnston: did you guys ever touch the systemsettle code :)? [11:24] * asac tries to find it [11:24] cjohnston: or the parameters? [11:25] * ogra_ doubts that [11:25] ogra_: well, we see settle consistently failing after boot [11:25] ogra_: either we have loads more processes [11:25] or something slows it down [11:25] do we have crash files ? [11:26] hmm, no [11:26] doesnt look liek we do [11:26] zero crashes so far on 101 [11:26] right [11:26] heh, spoke to soon :P [11:26] mako has one [11:27] maliit in andressbook-app [11:28] asac, aha https://jenkins.qa.ubuntu.com/job/saucy-touch_mir-maguro-smoke-friends-app-autopilot/22/artifact/clientlogs/top_before.log/*view*/ [11:28] ogra_: i think the most interesting part is likely starting from TOP DUMP (after settle run: 1) [11:28] or even https://jenkins.qa.ubuntu.com/job/saucy-touch_mir-maguro-smoke-friends-app-autopilot/22/artifact/clientlogs/top_before.log/*view*/ [11:28] err [11:28] TOP DUMP (after settle run: 3) [11:28] heh [11:29] thats a few minutes into the boot [11:29] so everything should be started by then [11:29] TOP DUMP (after settle run: 9) [11:29] thats the last run. whatever makes it fail there is the problem [11:29] phablet@ubuntu-phablet:~$ XDG_DATA_DIRS=/usr/share autopilot run unity8 [11:29] Loading tests from: /usr/lib/python2.7/dist-packages [11:29] Tests running... [11:29] Ran 22 tests in 549.162s [11:29] OK [11:29] unity8 ueventd, adbd [11:30] yay [11:30] nice one [11:30] only on second run though [11:30] so our current settle limit was 97.5 [11:30] we now hit like 69.5 reliably [11:30] on maguro [11:30] err [11:30] 96.5 [11:30] so we only need to explain a 1% boost [11:31] that could easily be unity8 because of MIR, but i somewhat feel something else is also fishy [11:31] why is adbd for instance there with 1%? [11:32] ogra_: seems on SF its also pretty tight [11:32] http://reports.qa.ubuntu.com/smokeng/saucy/touch_ro/maguro/100:20131017:20131015/4768/calendar-app-autopilot/492569/ [11:32] it like succeeeding during the last attempt [11:32] but that is before the udev fix [11:33] http://reports.qa.ubuntu.com/smokeng/saucy/touch_ro/maguro/100:20131017:20131015/4768/calendar-app-autopilot/ [11:33] but it succeeded [11:34] ogra_: yeah, but still very tight/late [11:34] and SF doesnt trigger udevd to go wiltd [11:34] thats a Mir issue [11:34] so anything changing can make us go red [11:34] * asac doesnt like that we only getting up to 97.5 anyway [11:35] ogra_: maybe it is because we use powerd-cli screen-on ? [11:35] well, top always eats some [11:35] why would have influence the load ? [11:35] s/have/that/ [11:36] ogra_: thats it. i turned the screen on and i suddenly see unity8 coming to the forefront of top [11:36] sure [11:36] ogra_: and ueventd [11:36] has always been like that [11:36] so thats the reason. if folks turn off the screen before settle,then we are green :) [11:37] ueventd might be other fallout of the Mir bug [11:37] cjohnston: doanac: plars: ^^ [11:37] tvos hass a fix for that afaik [11:37] *tvoss [11:37] ogra_: so the fact that unity8 starts consuming cycles when you turn screen on is considered a bug? [11:37] no [11:37] it has to refresh [11:37] the bug is that the driver under Mir spams uevents [11:38] that keeps traffic up in varioud places [11:38] i right [11:38] ack [11:38] not all of them can be seen [11:38] udevd was only one fallout [11:38] ogra_: so you say there is still some noice now in ueventd that we fixed udev? [11:38] ogra_: can we maybe also filter there smartly? [11:38] noise [11:38] thimas has a possible fix that only makes the uevent spam apper when there is actual interaction [11:38] *thomas [11:39] instead of having a constant stream of 60 events per sec [11:39] ok, but we can't minimize the impact on the ueventd process during a spam/attac,k on top of that? [11:39] rigt [11:39] i understand i think. [11:39] we need to kill the spam like SF does [11:39] oki [11:39] then it should work as fast as SF and your processes shouldnt misbeahve anymore [11:39] well, that would mean that its a bug and we might want to continue keeping it visible on dashboard [11:39] until we have fixed maguro [11:40] i uploaded some HTML5 games to the shop over the weekend, zombie attach is a highly demanding game that makes a good performan.ce test [11:40] *zombie apocalypse [11:41] * asac installs it [11:41] * asac always has to setup ubuntu one account [11:41] odd [11:42] once for shop access [11:42] * ogra_ never has to do it more than once [11:43] that game runs on the edge on mako ... (runs super smooth on android btw) ... on mako it is just close to be unusable [11:43] on maguro i get all actions with a long delay [11:44] i.e. non usable at all [11:44] ogra_: does it work well on android maguro? [11:45] i tried it now on maguro and yes its not usable ) [11:45] no idea, it works well on my S4 [11:45] :P [11:45] i have never run android on a maguro :P [11:45] it definitely shows we need to improve the browser performance for such stuff [11:46] having something like this game run smooth by trusty release might be a nice goal :) [11:46]