[06:36] <pitti> jibel: so this morning I committed the .results format fix
[06:37] <jibel> pitti, I saw your fix, thansk for that
[06:37] <pitti> jibel: (sent you a google message yesterday, not sure whether you got it)
[06:37] <pitti> jibel: so this needs to be rolled out to tachash, and then we need to reconfigure all existing jobs, right?
[06:37] <jibel> pitti, the format is still something wrong when there is no *-package
[06:37] <jibel> pitti, I did it
[06:37] <pitti> jibel: infinity was pointing out that britney currently considers all jobs as RUNNING, presumably due to either missing or wrong .result files?
[06:37] <pitti> jibel: ah, great
[06:38] <jibel> pitti, right, it's because there were no result files, I regenerated them, and are waiting for britney to pull them
[06:38] <pitti> jibel: no -packages would be if the test bed failed?
[06:39] <jibel> pitti, yes, but we need to send something back otherwise the status in britney will stay running forever
[06:39] <pitti> jibel: oh, you mean it fails as cat doesn't have anything to cat?
[06:40] <jibel> yes
[06:41] <pitti> hmm
[06:41] <pitti> $ sort -u /nonexisting | tr -s '[\n\t]'
[06:41] <pitti> that at least exits with 0 as tr succeeds
[06:41] <pitti> it doesn't spit out anything of course
[06:41] <jibel> but the format of the result file will be wrong
[06:41] <pitti> jibel: or is testpkg-version missing as well?
[06:41] <pitti> jibel: i. e. it doesn't get along with zero dependencies, or is the package itself missing?
[06:42] <jibel> it will be release arch package result instead of release arch package version result
[06:42] <jibel> and it wron't be parsed correcly on the other side
[06:42] <jibel> pitti, test-version is missing as well
[06:43] <pitti> ah; so that would be if it doesn't even get to the point where it unpacks the package, like qemu failing to start or so
[06:43] <jibel> http://d-jenkins.ubuntu-ci:8080/job/utopic-adt-libreoffice/4/ARCH=amd64,label=adt/ for example
[06:43] <pitti> jibel: would something like "release arch package ERROR" be acceptable? to tell apart a test failure from autopkgtestr machinery failure?
[06:44] <jibel> http://d-jenkins.ubuntu-ci:8080/job/utopic-adt-libreoffice/4/ARCH=amd64,label=adt/
[06:44] <pitti> jibel: btw, http://d-jenkins.ubuntu-ci:8080/view/Utopic/view/AutoPkgTest/job/utopic-adt-binutils/configure still doesn't have the updated config
[06:44] <jibel> pitti, yes that's what I did previously we could return an erro
[06:44] <jibel> r
[06:44] <jibel> I'll add the logic to the job
[06:44] <pitti> so if testpkg-version is not present, that's definitively an error
[06:45] <pitti> jibel: for the port redirection I suspect we run into a race condition and my logic to check whether a port is taken is flawed; I'll look into that today
[06:45] <jibel> pitti, if *-package are missing it's an error as well
[06:45] <pitti> jibel: it might be that all tests were skipped
[06:46] <jibel> pitti, but you'd generate the testbed-package in that case, isn't it?
[06:46] <pitti> jibel: oh, right
[06:46] <pitti> jibel: confirmed
[06:47] <pitti> so [ -e testpkg-version -a -e testbed-packages ] || MAYHEM
[07:04] <jibel> pitti, there is indeed an important risk of race in find_freeport because there is a lot of time between the moment the port is allocated and the moment qemu is really started and the port really open.
[07:04] <pitti> jibel: *nod*, that's what I need to fix
[07:05] <pitti> we don't even need to do this at all normally, only when using --shell-on-failure
[08:19] <pitti> jibel: I rolled out several fixes (including proper port locking) to the adt workers
[08:20] <pitti> jibel: when you do the mass-rollout of the new job configs all jobs will automatically re-run, right?
[08:21] <jibel> pitti, no, I requeue them manually
[08:21] <jibel> pitti, should I do it now ?
[08:23] <pitti> (so that we get proper .results files)
[08:23] <pitti> jibel: I guess we should wait until the "error" logic for the .results file is done?
[08:25] <pitti> jibel: just to confirm, the results for a successful run like http://d-jenkins.ubuntu-ci:8080/job/utopic-adt-apt/2/ARCH=i386,label=adt/console look ok now?
[08:25] <jibel> pitti, okay, pushing that fix now
[08:26] <jibel> pitti, it looks ok, I didn't notice the double package name yesterday, sorry about that.
[08:26] <pitti> jibel: no worries, I didn't think about it either; I just wanted to check that it propagated to the job config
[08:27] <pitti> jibel: looking at results.history it seems the results files still don't propagate? apt is still RUNNING there
[08:28] <jibel> pitti, last run of britney was more than 90min ago
[08:28] <pitti> ah
[08:33] <jibel> pitti, http://paste.ubuntu.com/7328311/ actually the format was right when there is no *-packages but the extension was wrong, the rest is unchanged and an error will be reported
[08:34] <jibel> err, without $PACKAGE in the first part
[08:39] <pitti> jibel: line 4:
[08:39] <pitti> [ $RET -gt 0 ] && result="FAIL"
[08:39] <pitti> jibel: this should be [ $RET -eq 0 ]
[08:39] <pitti> ... || [ $RET -eq 2 ] || result=FAIL
[08:40] <jibel> pitti, if adr-run return 0 or 2 it is a success, isn't it?
[08:41] <pitti> jibel: correct (2 is "some tests skipped")
[08:41] <jibel> pitti, the case where the resturn status is 2 is handled earlier, just not in the pastebin
[08:41] <pitti> jibel: ah, so an error doesn't go into .results, but a separate .error
[08:41] <pitti> jibel: ah, ok
[08:42] <pitti> jibel: line 5 still has the additional $PACKAGE, that needs to go
[08:42] <jibel> pitti, I dropped it
[08:42] <pitti> jibel: good; LGTM then
[08:42] <pitti> jibel: the [ -f "$resultfile" ] && bit is a Should Not Happen™ though :)
[08:42] <jibel> pitti, I think I'll put all of this in a shell script rather that updating the XML
[08:43] <pitti> jibel: ah good, then we don't need to update the job config to fix the script
[08:44] <jibel> pitti, espacing shell scripts and encapsulating them into xml file makes me sick :)
[08:45] <jibel> that's how it looks after encoding http://paste.ubuntu.com/7328364/ horrible!
[08:45] <davmor2> Morning all
[08:46] <pitti> jibel: yeah, I know; +1 for shell script, also for easier updates
[08:46] <jibel> and if jenkins goes AWOL we don't lose it
[08:52] <jibel> pitti, I pulled the new config on tachash
[08:52] <jibel> and the queue is empty, I think we can do a mass retest
[08:53]  * pitti puts finger in ears
[08:55] <jibel> pitti, 61 pacakges in the queue
[08:55] <jibel> they should appear on jenkins soon
[08:56] <pitti> jibel: ok, let's see what I broke this time :)
[08:58] <jibel> at least the syntax of the shell script seems correct. Jobs start :)
[09:01] <jibel> pitti, is test output buffered? eg. http://d-jenkins.ubuntu-ci:8080/job/utopic-adt-apt/ARCH=i386,label=adt/3/console don't display the output of the run-tests
[09:04] <pitti> jibel: there is no live output yet for adt-virt-qemu, that's rather tricky todo
[09:04] <pitti> jibel: that's on my list, though
[09:04] <jibel> pitti, ok
[09:05] <pitti> jibel: that one just succeeded
[09:05] <jibel> pfff, no fun
[09:05] <pitti> oh the queue length
[09:06] <jibel> we'll immediately see if there is still a race in port allocation :)
[09:08] <pitti> jibel: it properly uses flock() now, that really ought to do :)
[09:09] <pitti> jibel: ok, do you see anything else burning ATM? if not, I'll go ahead with eliminating dh_listpackages (I installed a local copy on the workers which don't have debhelper installed, but that's a temp workaround)
[09:09] <jibel> pitti, no, sounds good. I'll monitor the flow to see if anything else needs fixing
[09:10] <pitti> jibel: I guess all of that didn't affect the bug that results fall off excuses and it sometimes promotes failures, right?
[09:11] <pitti> jibel: I think the other day you said it was a "stupid sorting bug" or so
[09:11] <jibel> pitti, right, I'll push the fix today
[09:11] <pitti> jibel: oh, you have a fix for that? awesome
[09:13] <jibel> pitti, I hope, I simplified the history with just 1 line per tuple package@version/dependency@version and the result
[09:14] <jibel> that'll remove the problem when there are several causes that triggered a build and different results for the same package/dep
[09:15] <jibel> I planned to do it earlier but didn't anticipate I'd have to do so much manual testing this cycle again :(
[09:26] <jibel> pitti, http://d-jenkins.ubuntu-ci:8080/view/Utopic/view/AutoPkgTest/job/utopic-adt-fftw3/ARCH=amd64,label=adt/2/console
[09:27] <jibel> pitti, it generated an error file, so at least it validates thios point
[09:27] <jibel> -o
[09:27] <pitti> jibel: but isn't "echo utopic amd64" missing $PACKAGE?
[09:27] <pitti> jibel: indeed, I'll put that tar error on my list
[09:28] <jibel> I made a typo
[09:49] <pitti> jibel: do you have an idea about http://d-jenkins.ubuntu-ci:8080/job/utopic-adt-gzip/5/ARCH=i386,label=adt/console ?
[09:49] <jibel> pitti, yes, I moved the script from jenkins to a shell script. I'm on it
[09:50] <pitti> jibel: ack, thanks; I need to go offline in 10 mins to change trains
[09:50] <pitti> jibel: anything I need to do urgently?
[09:50] <pitti> I eliminated dh_listpackages and rolled it out, so the workers are once again hack free (i. e. clean git head checkout)
[09:50] <jibel> pitti, no, I'll go through the failures
[09:50] <jibel> and let you know if anything is not right
[09:51] <pitti> jibel: ack, thanks; so for now I know about the tar error (no idea yet)
[09:52] <jibel> uh, that sounds really wrong qemu-img: Could not open '/home/auto-package-testing/cache/disks/adt-trusty-amd64-cloud.img
[09:52] <pitti> but probably better to fix the usage of the shared dir to avoid the tar hack altogether
[09:52] <pitti> trusty??
[09:52] <jibel> on aldebaran. looking
[09:53] <pitti> jibel: oh, perhaps we are running the new job template for trusty SRUs?
[09:53] <pitti> but these jobs should all exist
[09:55] <jibel> on aldebaran: $ distro-info -d
[09:55] <jibel> ubuntu-distro-info: Distribution data outdated
[09:55] <pitti> ah, so that hits the "trusty" fallback
[09:55] <jibel> it's been updated everywhere else
[09:55] <pitti> curious why it doesn't happen on the others
[09:55] <pitti> ah
[09:56] <pitti> ok, AFK for ~ 20 mins or so
[10:44] <pitti> jibel: back online
[11:32] <pitti> jibel: sorry, internet in the Thalys just sucks :/ will need to do offline work
[12:04] <pitti> jibel: ah, I can reproduce the "tar: Unexpected EOF in archive" bug locally \o/
[13:14] <jibel> pitti, for linux {standard input}: Fatal error: can't write drivers/scsi/isci/.tmp_host.o: No space left on device
[13:14] <jibel> pitti, in trusty, there was a list of package to run on disk instead of memory
[13:14] <jibel> packages
[13:15] <jibel> pitti, for mysql, fork failed sleep 1 second and redo: Cannot allocate memory at lib/My/SafeProcess/Base.pm line 53.
[13:17] <jibel> pitti, I'll increase to 2G
[14:16] <jibel> pitti, I reviewed test failures. http://paste.ubuntu.com/7330086/ is the list of tests that potentially fails because of the new test environment. I am not sure about u-drivers-common though
[14:17] <jibel> pitti, I'll fix 1 and 2
[14:18] <jibel> pitti, BTW indicator-datetime doesn't like comments in /etc/timezone. It displays all the content of that file in the indicator
[19:10] <alesage> balloons, where would you send someone to report an upgrade an LTS-to-LTS upgrade bug?
[19:10] <Letozaf_> balloons, elopio hi
[19:28] <balloons> Letozaf_, hello
[19:28] <Letozaf_> balloons, hi
[19:28] <balloons> alesage, against the source package and/or the upgrade tool used
[19:29] <alesage> balloons, unknown, he's having boot-trouble
[19:29] <alesage> balloons, sorry to assail you, just don't know where exactly to take this q :)
[19:29] <alesage> balloons, and he's asking "how to help", etc.
[19:30] <Letozaf_> balloons, I am having the same problem I had time ago in reminders app  on seeing the objectNames I put in the qml not sure if I have to tell you or elopio
[19:30] <balloons> alesage, ohh.. is it live?
[19:31] <balloons> no worries, yes we can troubleshoot. how did he upgrade? and can he get console access?
[19:31] <balloons> I would want to use ubuntu-bug to send a log to lp with th ebug
[19:31] <balloons> Letozaf_, what was the original resolution to that?
[19:31] <balloons> do you remember how we fixed it?
[19:32] <Letozaf_> balloons, I think it was that the reminders binary didn't point at the correct qml file
[19:32] <Letozaf_> balloons, so -q was added to the binary
[19:33] <balloons> Letozaf_, ohh right! yes, it was the launching issue
[19:33] <Letozaf_> balloons, could be I am doing something wrong as it was quite some time ago I used reminders
[19:33] <alesage> balloons, getting some info from him thanks