pittiGood morning04:28
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
pittijibel: FYI, I changed adt-setup-testbed to create utopic VMs (cloud images exist now)07:34
pittijibel: and the first run succeeded07:34
pittijibel: I also pushed a fix for the $ADTTMP permission errors07:34
jibelpitti, good. I aborted an eglibc job that have been running for more than a day. It was stuck on "achiving artifacts" after a timeout07:52
jibelpitti, adt-run doesn't seem to leave old overlay files in /run/shm sometimes. I'll remove obviously old ones.08:01
pittijibel: eglibc> right, I killed some similar ones this morning08:09
pittijibel: overlays> hm, might these be from manual "kill" commands? we might need some cleanup job indeed08:10
pittiah, the fixes mails \o/08:11
jibelpitti, I'll add the cleanup to the wrapper and also fallback to disk when there is no space left in shm08:11
pittijibel: I wonder if we could actually remove the overlay right away08:15
pittijibel: i. e. if qemu ever needs to re-open it; if not, a removed "lingering" inode ought to suffice08:15
* pitti checks that08:15
jibelpitti, there is something with handling of timeouts too, eg chromium http://d-jenkins.ubuntu-ci:8080/job/utopic-adt-chromium-browser/ARCH=i386,label=adt/19/console08:16
pittijibel: oh, looking at that first, then08:17
pittisome zombie adt-virt-qemu processes indeed08:18
jibelsame for libreoffice http://d-jenkins.ubuntu-ci:8080/job/utopic-adt-libreoffice/ARCH=i386,label=adt/41/console08:19
jibelwith 2 zombies08:20
jibelpitti, adt-virt-qemu also leaks directories in /tmp08:30
jibelI think I can safely kill adt-virt processes running since May 2 and April 24 on wazn :)08:37
pittijibel: ah, you cleaned up alderamin already08:55
pittijibel: I tried to locally reproduce the zombies, but it exits cleanly here08:55
jibelpitti, I cleaned them all08:55
pittijibel: nevertheless, I'll improve the timeout handling08:55
jibelI restarted chromium too08:57
pittijibel: which dirs in /tmp/ are leaked?09:02
pittijibel: those are from the same runs as the zombies?09:02
pittijibel: I wonder what happens if you kill a job in jenkins -- is that kill -TERM'ed or kill -KILL'ed ?09:03
pittiin the latter case there's no chance to run the cleanup handlers09:03
pittijibel: http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=commitdiff;h=56039dc1 should help with the zombie processes (hopefully, as I can't reproduce), and http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=commitdiff;h=c733be should help with the /dev/shm/ leaked files09:14
* pitti rolls this out09:14
=== vrruiz_ is now known as rvr
jibelpitti, jenkins sends a Thread.interrupt() to the executor thread which in turns sends a SIGTERM to the child process and all its descendants09:23
pittijibel: ah, thanks09:23
pittijibel: so this probably ought to install a signal handler to run the cleanup bits on SIGTERM09:23
jibelpitti, chromium timed out again09:23
jibelseems to be a timeout during the copy09:25
jibelpitti, this is the last command executed: /usr/bin/python /tmp/adt-virt-qemuWO5seT/runcmd cp -r --preserve=timestamps,links /tmp/apt0-build.bkqTX5/chromium-browser-34.0.1847.116 /autopkgtest/tmp/apt0-tests-tree09:27
pittijibel: ah, I figure that tree will be rather big09:29
pittijibel: I have a FIXME for an optimization to only copy debian/tests/ instead of the full source tree, unless the tree is "build-needed"09:29
pittijibel: if it is build-needed, then I'm afraid we can only increase the copy timeout?09:30
jibelpitti, maybe but same test passed before May 2nd with the same version of chromium09:31
pittijibel: maybe it ran on a host with faster disks?09:31
jibelpitti, same hosts albali and alderamin. it ran on wazn on the 3rd and failed too09:32
jibeland disks shouldn't matter since it runs in memory09:33
pittijibel: last Friday I switched it back from squeezing everything through tar and stdin to 9p09:40
pittibut for most cases this made things a lot faster, hmm09:40
* pitti puts that onto his TODO list, too09:40
elopioalesage: I'm trying to push this:15:32
elopiohave I answered your question?15:32
* alesage clicks15:32
alesageelopio I reviewed15:36
rvrballoons: ping15:38
=== chihchun is now known as chihchun_afk
=== bfiller is now known as bfiller_afk
=== roadmr is now known as roadmr_afk
=== bfiller_afk is now known as bfiller
=== roadmr_afk is now known as roadmr
=== salem_ is now known as _salem

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!