[00:29] ogra_, rsalveti: time to kick an image build I think? cyphermox and I just landed the big mir transition. looking good on our end [00:30] robru: cool, let me check if everything is already in [00:30] rsalveti, oh, and if you can ensure that python2.7 2.7.6-2ubuntu1 gets in, that would be super ;-) [00:31] sure [00:32] hm, python is still in proposed [00:33] rsalveti, yeah, just noticed that. it's absolutely critical for running AP tests though [00:33] Invalidated by dependency [00:33] Not considered [00:33] Depends: python2.7 libffi (not considered) [00:34] autopkgtest for git-annex 5.20131127.1: FAIL (Jenkins: public, private) [00:34] for libffi [00:34] rsalveti, i'm not familiar with libffi [00:35] sync from debian, upload by doko [02:03] cihelp, ps-nvidia-gt630 fails otto again http://s-jenkins.ubuntu-ci:8080/computer/ps-nvidia-gt630/builds :/ [02:03] again the "could not start container" issue [04:06] === Image r37 building === [04:06] should have latest mir/unity8/python2.7/etc [05:13] * Mirv upgrades [07:33] Saviq: yeah, "again". Root cause https://bugs.launchpad.net/otto/+bug/1253198 , workaround https://code.launchpad.net/~vila/otto/stop-running-container/+merge/196594 waiting approval to be landed and deployed [07:34] Ubuntu bug 1253198 in Otto "missing dependencies leaves the container running blocking further jobs" [Undecided,Confirmed] [07:34] Saviq: the container has been stopped manually in the mean time [07:42] vila, where is the code that install debs? I cannot find it in otto trunk [07:42] is it a custom branch of otto? [07:45] jibel: I don't know of any otto custom branch, there are some uncommitted changes on most of the otto nodes but mainly for setting memory sizes [07:49] vila, it's in usr/local/bin/run-autopilot.sh, so not otto [07:50] in the script that calls autopilot [07:50] vila, I don't know where is comes from do you? [07:52] jibel: first line of defense is to stop the otto container to unblock other jobs, that's the priority [07:52] vila, that script is broken [07:52] vila, there is a shebang with sh -eu [07:52] vila, but then it call apt with sh -c [07:52] hmm, these crashers in the ubnity8 tests on mako dont look good [07:52] ogra_: yep :/ [07:53] ogra_: new mir, new crashers :p [07:53] hud wouldnt worry me as much, but NM [07:53] ogra_: well, better to figure it out first [07:54] right [07:54] jibel: where do you see this script ? It's not on the otto node [07:54] * ogra_ has a plumber in the house in 5min ... [07:54] not sure i'll make it in time for the call ... but i'll try my best [07:55] ogra_: no worry [07:55] ogra_: "swim" well ;) [07:55] heh that was two weeks ago ... [07:55] vila, /var/lib/lxc/trusty-amd64-20131023-0529/run/delta/usr/local/bin/run-autopilot.sh [07:55] i plastered a patch on the pipe ... thats what he has to fix properly today [07:55] on ps-radeon-8350 [07:56] ogra_: you used DEP3 headers I hope? ;) [07:56] ogra_: joke apart, I hope it won't cost you too much [07:56] lol, nope, a tar patch [07:56] inline patch! how ugly :p [07:56] haha [07:57] i dont care about the costs ... the pipes are 50 years old ... they all need replacement all across the 200m² over 3 floors [07:57] jibel: can you review https://code.launchpad.net/~vila/otto/stop-running-container/+merge/196594 ? [07:57] but i prefer that to happen in summer [07:58] my house has 4 bathrooms ... lots of pipes :( [07:58] jibel: and add https://launchpad.net/~canonical-ci-engineering to https://launchpad.net/~otto-dev ? [07:58] ogra_: yeah, summer will be better to shutdown the whole water system… [07:59] jibel: making that team admin would be more in line with responsibilities too [08:07] vila, when do you want to run this script? [08:08] jibel: This script is used as a post-build task in jenkins to catch containers left running. as mentioned in the MP [08:09] jibel: and how to do that is described in the script itself [08:10] vila, but if you abort a job it won't be called, isn't it? [08:10] jibel: and it has been tested on q-jenkins [08:10] jibel: it is called even if the job is aborted [08:11] jibel: it's explained in the script: This script is used as 'Post-build action' and as such is invoked after a [08:11] 16 +# job is finished, including aborted. [08:13] vila, its annoying that it kills everything, can you restrict to the container that have been started by the test? [08:14] vila, as fginther suggested once, write a flag in the main job for example or limit to the containers following the naming convention used by the tests [08:17] didrocks, or anyone, I'd need https://code.launchpad.net/~pitti/autopilot-gtk/fix-1254996/+merge/196703 in the distro, what's the procedure? [08:17] it is blocking installer tests [08:18] jibel: just adding to the landing ask, I'm handling it [08:18] jibel: further refinements can be done later (as explained in the MP) [08:19] jibel: for now, I want to unblock Saviq and other users that face https://bugs.launchpad.net/otto/+bug/1253198 [08:19] Ubuntu bug 1253198 in Ubuntu CI Services "missing dependencies leaves the container running blocking further jobs" [Undecided,New] [08:19] vila, it is not a refinement, it kills all running container without distinction [08:20] jibel: It is indeed not a refinement, it is a workaround as explained in the MP and the script, refinements can come *later* [08:20] vila, for Saviq 's problem, fix the script run-autopilot.sh [08:20] used by medium tests [08:20] this script is like saying "I've a bug on my machine, I don't know what it is, let press the power button" [08:21] jibel: exactly [08:21] jibel: and that's what I want [08:21] so you'd better fix the runner instead [08:21] jibel: no, that workaround is a catch-up so the ci engine keeps running, other bugs will be fixed later [08:21] catch-all [08:23] jibel: can I have that MP approved or https://launchpad.net/~canonical-ci-engineering added to https://launchpad.net/~otto-dev or should I just deployed my branch on the otto nodes ? [08:24] vila, I disagree with the approach of killing all running container, please restrict to the containers used by the tests. [08:25] jibel: fine, I'll deploy my branch then [08:26] jibel: we can continue the discussion later [08:34] vila, oh cool, nice that the problem is tracked down [08:36] vila, commented on the MP [08:41] Saviq: note that the root cause (wrong deps) won't be fixed by ci, someone has to look at them [08:41] vila, of course [08:43] vila, another one I've noticed from time to time bug #1256227 [08:43] bug 1256227 in Ubuntu CI Services "Mako test runners sometimes fail with "write error"" [Undecided,New] https://launchpad.net/bugs/1256227 === ev changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: ev | Landing instructions: http://paste.ubuntu.com/6292280/ | Known issues: http://goo.gl/DStLaS [09:28] * ogra_ will be late to the meeting but i will come ... [09:29] ogra_: no worry! [09:49] didrocks: can you ACK the packaging diff? Sortage mostly, as other changes were made by core devs ;p http://q-jenkins.ubuntu-ci:8080/view/cu2d/view/Head/view/QA/job/cu2d-qa-head-3.0publish/lastSuccessfulBuild/artifact/packaging_changes_autopilot-gtk_1.4+14.04.20131128.1-0ubuntu1.diff [09:50] ev, hey, it seems makos are in a bad state this morning :/ https://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/ [09:50] bug #1256227 [09:50] bug 1256227 in Ubuntu CI Services "Mako test runners sometimes fail with "write error"" [Undecided,New] https://launchpad.net/bugs/1256227 [09:50] Saviq, there are a ton of network manager crashes in the latest image smoketests too [09:51] (no idea if that could be related) [09:51] ogra_, yeah, no idea either, don't get it what it complains about [09:52] sil2100: +1 [09:52] popey, did you already try r37 on mako ? [09:52] am testing now ogra_ [09:53] the lockscreen issue on maguro is fixed [09:53] never saw that on mako [09:53] yeah, that was driver specific [09:55] didrocks, oh, btw, i would like to switch on cron after tonights meeting, would be good if everyone could think about the best times for the cron entry [09:55] ^^^^ rest of team too ^^^^ [09:55] ogra_: hum, I would vary doing that while asac is sick/not around [09:55] as he's the one disagreeing [09:55] i would like to have some feedback data before he returns [09:55] but otherwise, yeah, we need to check with psivaa and plars in particular so that it matches their schedule [09:56] we can still switch it off if he objects [09:56] so that basically most of the tests are ran when they arrive [09:56] they can work on correcting/relaunching [09:56] (so have some slack time) [09:56] and then, we rekick the next one [09:56] right, well, the tests have 8h to run [09:56] psivaa: what would work for you? ^ [09:57] so i can set the schedule in a way that one image is just ready when they get up [09:57] ogra_: I would say, when psivaa starts his day, it would be better that the tests are already running for 3 hours [09:57] or has run a chunk of automated tests already [09:57] Saviq: okay, adding to the queue. Thank you! [09:57] yeah, exactly [09:57] yeah [09:57] http://popey.com/~alan/phablet/device-2013-11-29-095719.png anyone ever see that? where a key gets wedged on? [09:57] note it wasn't the last key I pressed, it stayed wedged even when pressing other keys [09:58] didrocks: ogra_ i would like to have all the tests completed when i come in so that i could relaunch if there is any test that needs it [09:58] psivaa: when are you starting UTC-wise? [09:58] psivaa, when do you usually start [09:58] heh, snap [09:58] :p [09:58] ogra_: didrocks: it's UTC 9:00 [09:59] so half an hour before the meeting [09:59] right [09:59] so i should put the build say 4h before that ? does that suffice ? [09:59] this makes it 6 to start running the tests [09:59] so 5 to build the image? [09:59] 5 + 8 = 1PM UTC [09:59] ogra_: that sounds good [10:00] so tests done by 5 UTC [10:00] which is the other meeting time [10:00] hum [10:00] better to get that done one hour before the other meeting [10:00] so 4 [10:00] we still have 3 hours for psivaa to fix things up in the morning [10:00] ok, i'll schedule one build at 5am ... lets wait for plars input if such a schedule works for him too ... [10:01] ogra_: 4 would work better for the afternoon meeting [10:01] we dont need to be strict on 8h or wanything [10:01] 4 -> 12 + 1 hour building + 3 hours of tests == 4 PM [10:01] (so one hour before the evening meeting) [10:02] that wont give plars much wiggle room [10:03] i would start the second build ~ 2h earlier 13:00 UTC or so [10:03] heads up: Jenkins on s-jenkins just crashed on us [10:04] ogra_: 37 is good here [10:04] but lets wait for his input [10:04] popey, no OSK for me [10:04] sil2100: once https://code.launchpad.net/~mardy/ubuntu-system-settings-online-accounts/disable-autopilot-tests/+merge/197152 is merged, can you add ubuntu-system-settings-online-accounts to your bucket? [10:04] fine here [10:04] popey, yeah, because you use a werid language/locale setting :P [10:05] didrocks: sure! Keeping that on the radar ;) [10:05] ogra_: CONFORM! [10:05] ogra_: didrocks: unity8 tests on both devices came good with 37. in transit to the dashboard [10:06] psivaa: great! [10:07] didrocks, so still no OSK on maguro (no surprise since nothing changed there) [10:08] waking up the screen needs sometimes two power button presses ... but at least no tapping on screen anymore [10:08] (still not 100% fixed though) [10:08] ogra_: ah, interesting, mind pinging greyback? [10:08] about kbd or screen ? [10:09] no OSK on maguro! First I've heard of it. [10:09] greyback, on non english locale [10:11] greyback, bug 1255999 [10:11] bug 1255999 in ubuntu-keyboard (Ubuntu) "[regression] if switched to non english local, the keyboard does not come up when an input field is focused" [High,Confirmed] https://launchpad.net/bugs/1255999 [10:11] ogra_: ok thanks. I'll poke tmoenicke too [10:11] he should be able to test a german setting i guess :) [10:14] Mirv: https://code.launchpad.net/~sil2100/cupstream2distro-config/extra_for_qa/+merge/197157 could you take a lookie? [10:14] psivaa, there was a saucy 102 build and i dont see any tests for it ... would be good to have some results within the next week (no hurry) [10:15] ogra_: ack, will add to the list [10:15] sil2100: looks good [10:15] Mirv: thank you! [10:18] sil2100: one for you too https://code.launchpad.net/~timo-jyrinki/cupstream2distro-config/libavutil52/+merge/197159 [10:20] ogra_: sorry, was chatting on something else, so no, on the power button [10:20] greyback: ^ [10:20] didrocks, yeah, all sorted [10:20] (see above) [10:20] Mirv: oh, ok, not sure why but I'm somehow amused by this change ;p +1 [10:22] sil2100: Friday? ;) I added another commit though, it seems it's installing both at the moment since I already did a really quick test run [10:22] Mirv: ok, then I re-approve that [10:23] Mirv: probably ;p [10:23] thanks! [10:24] libav9 migration just completed today/yesterday, which was nice to notice [10:30] bug 1256265 [10:30] bug 1256265 in ubuntu-keyboard (Ubuntu) "Keyboard stuck down" [Undecided,New] https://launchpad.net/bugs/1256265 [10:34] psivaa: so, can you flash latest image [10:35] (locally) [10:35] try to get previous glib version [10:35] and see? [10:35] (if the crashes still happens) [10:35] didrocks: ok, i have a maguro and the crashes (network manager) is only happening on mako [10:35] psivaa: argh, ok [10:37] didrocks: network-manager somehow depends on libffi6 which is an upgraded package in 37 [10:38] psivaa: do you have access to device in the lab where you can test backing it out? (I'm testing locally) [10:38] that + retracing will really help I guess [10:38] didrocks: yes, doing that. i have access to the device [10:38] great, will confirm you shortly if I have the crash here (phone booting) [10:38] with r37 [10:38] ack [10:50] psivaa: ok, so got the crash, reverting glib -> no crash FYI [10:51] didrocks: ahh ok, thanks. i'll confirm [10:51] psivaa: hands over a stacktrace once retraced [10:51] seb128: ^^ [10:51] asac, hey [10:52] will you backout given that glib has caused issues on touch and other parts? [10:52] :) [10:52] asac, ssssssure [10:52] or do you have a lead [10:52] cool [10:52] leading by example [10:52] hehe [10:52] sorry... we need to get our proposed testing story sorted [10:52] so we have a better safety net... [10:53] didrocks: doesn't http://q-jenkins.ubuntu-ci:8080/job/trusty-touch-mako-smoke-ubuntu-rssreader-app-autopilot/39/artifact/clientlogs/_usr_sbin_NetworkManager.0.crash/*view*/ have the required trace information? [10:53] psivaa, it doesn't have debug symbols [10:54] psivaa: do you know how to retrace a crash file? [10:54] didrocks: i have not done in the past [10:55] ah, maybe we should pair someone with you [10:55] Mirv: mind giving a hand to psivaa? ^ [10:55] I think it's important that he learns about it [10:55] you have the -dbg files directly btw for glib [11:21] didrocks: sure, in a bit [11:21] psivaa: you'll need lp:daisy on host [11:22] Mirv: ok, i'll get it [11:25] Saviq: workaround deployed on s-jenkins, you may still encounter failing jobs but they should not block the following ones anymore [11:25] vila, thanks [11:25] didrocks: packaging ACK! http://q-jenkins.ubuntu-ci:8080/view/cu2d/view/Head/view/Misc./job/cu2d-misc-head-3.0publish/lastSuccessfulBuild/artifact/packaging_changes_phablet-tools_1.0+14.04.20131129-0ubuntu1.diff ? :) [11:26] sil2100: +1 [11:29] didrocks: thanks! [11:29] hm [11:29] Does anyone know why this is happening? http://q-jenkins.ubuntu-ci:8080/job/autopilot-trusty-daily_release/658/label=autopilot-nvidia/console [11:29] I: No test left to run === MacSlow is now known as MacSlow|lunch [12:36] seb128: didrocks: just an update.. still have not collected the crash debug information.. seeing issues like http://pastebin.ubuntu.com/6493858/ [12:38] psivaa: maybe, you can try retracing it directly on the phone? [12:38] didrocks: let me try that [12:38] psivaa: you apport-unpack the .crash file [12:38] and then follow https://wiki.ubuntu.com/Backtrace on the CoreDump you will get [12:38] ogra_: do we have a bug for the weird nm bug you said already? [12:39] (having the debug symbols [12:39] rsalveti, not sure [12:39] didrocks: ack, will try that [12:39] i didnt file one (and dont have my mako on -proposed) [12:39] let me flash the latest [12:39] rsalveti, the dashboard has a ton of crash files though [12:39] oh =\ [12:40] http://ci.ubuntu.com/smokeng/trusty/touch/mako/37:20131129:20131129/5168/ [12:40] all tests with 2 or more crashes are likely to have the NM one [12:41] the nm one is a known issue which is being worked [12:41] oh, cool [12:42] seb128: right, but do we have a bug for it? [12:42] rsalveti, not sure, I've a fix in a ppa and I'm waiting for desrt to get online to validate it [12:42] alright [12:42] I confirmed the fix, I just want sanity check that I'm not doing something boggy to gobject [12:47] didrocks: I think the problem psivaa is seeing, based on the output he's getting, is that it's not a problem/crash that would create coredump [12:47] Mirv: I doubt there is no coredump, I can see one on the dashboard [12:48] didrocks: ok. it's what apport is telling him though, and I've had some of those with eg. python crashes. but it might be some different kind of problem. [12:48] didrocks: the method I gave psivaa I've used successfully with unity8 backtracing at lest [12:48] least [12:48] that report is working [12:48] I got the dump out this morning to look at it [12:49] that's not rocket science, just use apport tools (e.g unpack and gdb it)... [12:49] sil2100: literally, I: No test left to run [12:49] means that all tests in the spool dir have run and there is no more to run [12:49] I'll try retracing too [12:49] vila: but it results in an error, and the autopilot suite has more than 0 tests [12:49] sil2100: this comes from run-autopilot.sh [12:50] sil2100: that's not why it results in an error IIUC: [12:50] I actually ran them on my desktop and there all tests ran [12:50] Mirv, don't bother, no need to be several doing it [12:50] if [ ! "$(ls -A $spooldir/)" ]; then [12:50] echo "I: No test left to run" [12:50] sudo shutdown -h +1 [12:50] fi [12:50] Mirv, that's dupping work for no good reason [12:50] sil2100: that's the way it always finish the job [12:51] seb128: well, I'm interested in seeing if I get similar error as psivaa, or not. my computer's CPU and network cycles are free :) [12:51] (almost, at least) [12:51] Mirv, k, your call [12:51] psivaa got his .crash file from the device, so I'm checking if we get similar results on that downloadable one [12:51] seb128: we were asked to pair work on this anyhow.. [12:52] psivaa: yeah, on that jenkins one I got retrace, no problems === MacSlow|lunch is now known as MacSlow [12:53] sil2100: i.e. there may be another previous error leading to *not* producing a test suite to run but I can't see why from that log [12:54] sil2100: /var/local/autopilot/autopilot.log: Loading tests from: /usr/lib/python2.7/dist-packages kind of means autopilot has run and failed for an unknown reason (few lines above that 'No test left ro run' line) [12:54] hmmm [12:54] vila: thanks for the analysis! [12:54] sil2100: my pleasure, learning with you here [12:55] sil2100: in fact, I think autopilot has run indeed, yet a few lines before: [12:55] /var/local/autopilot/autopilot.log: I: Running autopilot run autopilot -v -f xml -r -rd /var/local/autopilot//videos/ --record-options=--fps=6,--no-wm-check -o /var/local/autopilot//junit//autopilot.xml [12:58] sil2100: so, was that job about running autopilot tests themselves ? [12:59] sil2100: i.e. 'autopilot run autopilot' ? === ev changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: cihelp | Landing instructions: http://paste.ubuntu.com/6292280/ | Known issues: http://goo.gl/DStLaS [13:14] ogra_: i'm done with image 37.. please feel free to kick 38 :) === psivaa is now known as psivaa-lunch [13:16] doing :) [13:16] === Image r38 building === [13:38] \o/ [13:57] ev, hey, can you tell me how to subscribe to packages for errors.u.c? or where to read about e.u.c for that matter? [14:00] ev, define "subscribe"? [14:00] ups [14:00] Saviq, ^^ [14:00] Saviq, like getting emails for issues? [14:01] Saviq, by default it lists packages you are subscribe to in launchpad [14:01] seb128, how do I do that? on https://launchpad.net/ubuntu/+source/unity8 ? [14:01] Damn, LP acts strange today.. [14:02] Saviq, https://launchpad.net/ubuntu/+source/unity8/+subscribe [14:02] click the "I want to receive these notifications by e-mail." [14:02] Saviq, that doesn't give you emails from e.u.c though, but the default view is to list reports for the packages your user is subscribed to [14:02] seb128, right, I was team-subscribed to that [14:03] Saviq, well, you can customize your url [14:03] Saviq, e.g https://errors.ubuntu.com/?release=Ubuntu%2013.10&user=unity-team&period=day [14:03] seb128, right, thanks! [14:03] ok, exercising time [14:04] yw [14:04] Saviq, https://errors.ubuntu.com/?release=Ubuntu%2013.10&user=saviq&period=day [14:04] seb128, what does it mean when a row is greyed out? [14:04] retrace failed? [14:04] Saviq, I think it means it hasn't been seen with the current version of the package [14:04] ah [14:04] nice one [14:04] so it might be fixed [14:04] crossed is when the bug is closed [14:05] red is when it's closed but the issue is still reported with the current version [14:05] would be nice to have a legend on the page [14:05] ev tried to make it "obvious enough by itself" but it's still confusing [14:06] seb128, can it be greyed out if it's crossed, too? like https://errors.ubuntu.com/problem/f27a9186b6a83d502641e0fd89381bcd62095395 [14:06] seb128, it's fixed, not greyed out - but not found in current version [14:06] probably doesn't make sense to be greyed out if crossed [14:06] right, fixed is just crossed [14:07] it would turn red if it was seen in the current version [14:07] seb128, aanyway, there anywhere I could read so that I don't bother you? :) [14:07] Saviq, try asking to mpt on #ubuntu-devel if there is a document explaining e.u.c or its design [14:08] seb128, will do, thanks [14:08] yw === psivaa-lunch is now known as psivaa [14:22] seb128: didrocks: Mirv has reported the bug for NetworkManager crash, bug #1256299 [14:22] bug 1256299 in network-manager (Ubuntu) "NetworkManager crashed with signal 5 in g_object_finalize()" [Medium,New] https://launchpad.net/bugs/1256299 [14:22] psivaa, Mirv: you can close it, the fix has already been uploaded [14:23] seb128: i'll wait for the fix to land and work before closing it [14:23] ok, your call [14:23] testing for the fix (that's the new glib just uploaded to trusty-proposed) is welcome === Ursinha is now known as Ursinha-afk [15:39] psivaa: mind testing it from proposed? [15:41] didrocks: the tests are running on the device. i was planning to test it once the tests are done [15:41] i mean the tests with r38 [15:41] psivaa: ok ;) [15:53] transitioning stuff to daily-release is hard === Ursinha-afk is now known as Ursinha === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha [16:30] can I get permissions to re-run autopkgtests? [16:37] (what happened to the matrix reloaded?) [16:38] Laney: can you file a bug against ubuntu-ci-services-iteself with your request.. im not sure what may have changed with the move, but that way it can be followed up on when people from the US get back [16:38] cjohnston: ok, in the meantime can you re-run glib2.0 please? [16:38] Laney: link please? [16:38] http://d-jenkins.ubuntu-ci:8080/view/Trusty/view/AutoPkgTest/job/trusty-adt-glib2.0/21/ [16:40] Laney: I'm not familiar.. so I see 'matrix reloaded', on that page I see amd64 which is green and i386 which is red and checked... [16:40] what about the downstream builds? do I check that? [16:41] I don't know [16:41] there used to be Matrix Reloaded to make the tests re-run [16:41] don't ask me what it did, but I'm guessing it's a plugin which we don't have any more [16:41] jibel: ^- any clue? [16:42] Laney: the plugin still seems to exist (for me atleast) [16:42] oh, you /do/ see it, sorry - misread [16:42] Laney: I'm just not sure with the plugin, do I just leave it at the default for what they set [16:42] so tick both of them and press publish or whatever it is [16:42] that should create a new entry to re-run the tests [16:42] "Matrix Reload downstream builds: " [16:42] any idea on that tick box? [16:42] can't remember [16:43] ISTR only changing the arches [16:43] ok, well its building [16:44] neat, thanks [16:44] I filed https://bugs.launchpad.net/ubuntu-ci-services-itself/+bug/1256372 for you [16:44] Ubuntu bug 1256372 in Ubuntu CI Services "Can't retry failed autopkgtests any more" [Undecided,New] [16:45] ty === Ursinha is now known as Ursinha-afk [16:45] I'm not sure if policies changed or what the story is, so I'm not comfortable with making changes :-) [16:46] ok, well I described my ideal [16:48] ty [16:49] cyphermox: ping [16:49] pong [16:50] cyphermox: can you walk through updating the q-jenkins stack update stuff with me please? [16:50] what do you mean? [16:50] https://wiki.ubuntu.com/DailyRelease/FAQ#Adding.2BAC8-removing_components_to_a_stack [16:51] ok, what do you want to do? [16:51] cyphermox: I'm not really sure what I'm doing, so just make sure I don't kill things? [16:51] well, that's what I mean what are you trying to do? [16:52] applying https://code.launchpad.net/~dobey/cupstream2distro-config/u1creds-inline-packaging/+merge/197230 [16:53] ah! [16:53] so since it's merged, just make sure you have the very latest revision [16:54] then you'll want to cd daily-release; ./cu2d-update-stack -dUS ../stacks/head/webcred.cfg [16:54] cyphermox: do I do this locally or on the remote machine? [16:54] that will update the jenkins config in jenkins, and not touch [16:54] oh wait [16:54] cu2d-update-stack -dU [16:55] locally [16:55] you should have a .cu2d.cred file on your system that has your credentials for jenkins to be able to update the config remotely [16:55] hrm.. ok, I'll create oen [16:55] one [16:55] hmm [16:55] ok just a second [16:59] cjohnston: so yeah, all you need to do is get lp:cupstream2distro-config, cd daily-release and then run [16:59] ./cu2d-update-stack -dU ../stacks/head/webcred.cfg [17:00] note that this often requires you to be in the right teams to be able to update what trunk points to and things like that, so I'm not sure how well it's going to run [17:00] ack [17:00] its runnin, i guess we shall see [17:01] read only transport [17:01] right [17:01] we also need https://code.launchpad.net/~dobey/ubuntuone-credentials/daily-release/+merge/197222 to get merged though [17:01] let me see if I can run it [17:02] cyphermox: no we don't [17:02] cyphermox: the purpose of the previous change was so that this can merge [17:03] in that case this is not the same process [17:03] eh? jenkins needs config that doesn't include pulling in the external package branch, to be able to run tests in this branch, because they conflict [17:03] cjohnston: then you'll likely want fginther or alesage or some other person to fix up CI rather than daily-release [17:04] dunno.. this is what didrocks said was needed [17:04] I expect there was a misunderstanding [17:04] packaging_branch: may be used in more than one place [17:05] the cu2d-update-stack stuff is really mostly going to change q-jenkins for the cu2d jobs, not the merge proposal build stuff [17:05] anyway, AFAIK [17:05] cyphermox: I think cjohnston did the upstream merger part [17:05] then the proposal should be able to merge fine [17:06] I'm not denying that the second part is necessary, just that it's not what's blocking the merge request [17:06] didrocks: do you happen to have magic ubuntuone access to be able to update the packaging branch for ubuntuone-credentials? [17:06] cyphermox: what are you suggesting? [17:07] I did deploy-cupstream2distro-config [17:07] cjohnston: ok [17:07] in this case, the merge should be good to land [17:07] cyphermox: ah, I don't [17:07] then we still need to do cu2d-update-stack for the actual daily build [17:08] didrocks: actually [17:08] I think I might have the right access, going to try [17:08] cyphermox: what are you "going to try" exactly? [17:08] please don't touch any ubuntuone branches [17:09] updating the config [17:10] ah, that oneś daily-release false too [17:10] It's already good [17:11] yes, the change was just to remove the packaging branch, so ps jenkins would pass the tests again [17:11] cjohnston: you should jsut be able to kick the merge proposal to try the build again, but I don't know how you do that [17:12] http://s-jenkins.ubuntu-ci:8080/job/ubuntuone-credentials-ci/69/rebuild [17:12] someone just needs to rebuild it, if the live config is updated now [17:12] rebuilding [17:15] looks like it failed again with the same issue. [17:15] so i guess the liv config isn't updated [17:15] live even === Ursinha-afk is now known as Ursinha [17:20] cjohnston: you'll need to check the config in http://s-jenkins.ubuntu-ci:8080/job/ubuntuone-credentials-trusty-amd64-manual/14/console to see why it still tries to grab debian, I'm not sure if that's covered by packaging_branch but regardless, it's not a process I know about, it's not cu2d. [17:21] bzr branch $packaging_branch debian [17:22] maybe jenkins needs a restart to refresh the config or something? [17:23] not sure how that works [17:23] that would be very unfortunate [17:23] dobey: hrm [17:23] if I just did a rebuild, then that would still have the packaging branch [17:24] did the job config not change? [17:25] it has a packaging branch field, but it is blank [17:26] but on a rebuild, it takes the config of the build you are rebuilding [17:26] so I think it needs a new build [17:26] hmm, maybe. i guess trigger a new build of the job? [17:26] or the packaging branch to be removed when doing a rebuild [17:27] it still requires a packaging branch [17:27] i thought "rebuild" just triggered a new build of the same job [17:28] it does, but it carries all the env vars [17:28] https://code.launchpad.net/~dobey/cupstream2distro-config/u1creds-inline-packaging/+merge/197230 doesn't seem like the correct fix for the problem dobey [17:28] it still expects and external branch for the debian files [17:30] it's probably jsut carrying things over [17:30] why is it expecting one? [17:30] what if you resubmit the merge proposal? [17:31] cyphermox: dobey that wont work... [17:31] all that the MP did was take away the default packaging branch [17:31] it didn't remove the need for having a packaging branch [17:32] and what specifies that need? [17:32] your funny [17:32] cjohnston: ubuntuone-credentials-trusty-amd64-manual: template: False ?? [17:32] * cjohnston has no idea.. still looking [17:32] (just guessing) [17:33] I know fginther had a google doc about these things [17:37] cjohnston: do you need a link to the document? [17:37] cyphermox: the config in jenkins has: [17:37] # This project uses a packaging dir that is not compatible with pbuilderjenkins [17:37] bzr branch $packaging_branch debian [17:37] but I don't know where that comes from [17:37] yes please [17:37] right, that's because of the separate debian/ branch [17:38] cjohnston: from what I can see, you'd just need to remove that job in configurations and possibly also remove the part about trusty-amd64: False to have things use the standard process [17:38] https://docs.google.com/a/canonical.com/document/d/1vZf1jxnP0XYrVSoJF3UoTPNhZBYtX9oQufpR8bdaDzQ/edit#heading=h.c8d6w5jt4m60 [17:38] cjohnston: Could you retry http://d-jenkins.ubuntu-ci:8080/view/Trusty/view/AutoPkgTest/job/trusty-adt-glib2.0/22/ one more time? There was some kind of bug in the job that made it pull the wrong version of the tests (pinged jibel about it already) [17:38] cjohnston: ^ it's not a very clear document though [17:38] * cyphermox goes to get lunch [17:40] Laney: done [17:40] thanks [17:41] silly infrastructure :-) [17:42] hmm, there is no indication in that document of what "template: False" actually means there [17:47] im wondering if you do template false if you are using a seperate packaging branch [17:50] dobey: cyphermox I'm at a complete loss [17:50] I'd say at this point we will need to wait for Monday [17:50] well, tbh I think it's just avoiding to use a template at all, and instead uses the job named whatever the title is there [17:51] in this case, ubuntuone-credentials-trusty-amd64-manual [17:51] cjohnston: you could just as well remove the config to try and put it back [17:51] if things don't work as you expected [17:51] cyphermox: but it's not clear to me if that should be removed exactly [17:52] dobey: most jobs don't need any of this configuration, and don't have a configuration: stanza at all [17:52] this was only required because you used merge mode for the bzr branches [17:52] cyphermox: right. and ubuntuone-credentials will still be merged by the u1 tarmac [17:52] cyphermox: which is why it's not clear if just removing that job is the right thing to do [17:52] tarmac is irrelevant there [17:53] anyway, the worst that can happen is that you'd have another message from jenkins on the merge proposal that says the attempt failed [17:53] cyphermox: only under the assumption that this config isn't what's preventing the CI autoland job from merging approved branches in that project [17:53] * cjohnston lunches [17:53] cyphermox: and i don't see any clear indication if that is true or not [17:53] anyway i also need to get lunch [17:54] bbiab [17:54] any branch approved that hasn't been merged yet would still need to be reviewed by jenkins first anyway [18:10] Laney: all better? [18:13] cjohnston: seems decent now, cheers [18:13] :-) [18:22] seb128: btw with libglib2.0 (2.39.1-0ubuntu2) from -proposed the NetworkManager crash is not occurring anymore. [18:23] psivaa, I know, but thanks for confirming ;-) [18:35] ogra_, psivaa, asac: the fixed glib is in trusty proper btw [18:35] it just migrated there [19:11] back [19:14] cyphermox: right. what i want to avoid though, is two separate jenkins trying to merge/commit the same branch into trunk [19:39] i wonder if just changing this to use "autolanding: False" instead, would solve it [19:52] nope, that's something else [19:52] that will make the merge proposals not merged at all [20:05] how so? [20:05] and does not merged mean no ps jenkins running tests? [20:21] bah. i guess i'll just go all in, instead of trying to stage this migration in a reasonable way [20:36] 19:35 < seb128> ogra_, psivaa, asac: the fixed glib is in trusty proper btw [20:44] ok [20:44] === Image r39 building === [20:44] \o/ [20:55] oh cool [21:11] cyphermox, cjohnston: https://code.launchpad.net/~dobey/cupstream2distro-config/u1creds-fix/+merge/197264 <- maybe we could get this merged and deployed now then? i've pulled the landing of that project's trunk out of the u1 tarmac setup now. [21:26] cihelp ^^ [21:58] well, time for me to head off. guess this won't get merged/deployed today. :-/ [21:58] later [22:03] === Image r39 DONE === [22:03] (since a while) [22:03] oh :) [22:03] ogra_: thanks [22:04] np :) [22:05] * ogra_ goes afk again ... [22:13] * popey flashes [22:28] popey: is it any good? [22:28] dashboard is still greenish :) at 11 pass