=== bjf is now known as bjf[afk] === jjohansen is now known as jj-afk [10:47] lamont, sigh ... arcorn is dead once again [10:48] i dont get why it suddenly fails so often [11:27] ogra: ditto for hawthorn, it appears [11:27] interestingly, it seems to be the same machines each time. [11:28] yes [11:28] one could almost suspect that it's not exactly workload related [11:28] and i have a feeling that its either related to dove builds or to kubuntu ... it seems to have started when NCommander enabled either [11:28] and they were enabled around the same time [11:29] sycamore is rather rarely affected and doesnt build dove or kubuntu images atm [11:29] Do we know when it crashes, or what it was doing when it crashed? [11:29] * ogra doesnt, i dont get a long and usually only notice it once the machine is gone [11:30] s/long/log/ [11:32] * NCommander runs [11:32] Its not my fault it broke! :-P [11:34] ogra: acorn is up... [11:34] thanks [11:34] heh. up 33 minutes [11:34] oh [11:34] I suspect someone cycled it [11:35] i tried a build about 1h ago :) [11:38] persia: sadly, it is opaque in that regard [11:39] we know that it does not spew upon the serial console during its death throws [11:39] lamont: you have console debug messages on? [11:39] NCommander: and I stuff what where to get that? [11:40] lamont: huh? [11:41] syslog config, I'd think. [11:41] * NCommander kicks stuff [11:43] lamont: you have console debug messages on? <-- how turn on console debug messages? IO [11:44] I'm betting either printk foolery in sysctl.conf [11:45] lamont: remove quiet from the command line and add verbose ;-) [11:45] Oh, heh. [11:54] heh. ok [11:54] NCommander: and then reboot, I bet. :-p [11:55] I'll do that later today [11:55] lamont: acord is omap4, right? [11:55] they're all bbg3 [11:55] we have nothing else online [11:57] lamont: ah, acorn realized it was building the competition and committed suidice! [11:57] NCommander, acorn is omap3 and dove [11:57] sycamore is omap4 [11:57] (target wise) [11:58] *and* we dont build kubuntu for omap4 yet [11:58] so acron is omap3/dove for ubuntu-netbook and kubuntu currently [11:58] dove and kubuntu were both enabled at a similar time [12:00] * ScottK would say that at work Kubuntu or Dove exposed an existing problem. I don't they either actually "caused" a problem. [12:00] yeah, i didnt say they case one :) [12:00] *cause [12:01] we just dont see it on sycamore which onyl builds ubuntu-netbook-omap4 currently, it might as well be just a load issue or something [12:02] except that they all serialize, so you're only ever building one [12:02] I suppose you could trade roles and see if sycamore starts faceplanting [12:03] i could do that but we have an agreement to keep one buildd exclusively for omap4 [12:04] though i dont really expect it to be caused by dove ubuntu builds they are 98% identical to omap3 or 4 (just different kernel packages being unpacked) [12:05] that shouldnt cause more load or anything, whereas kubuntu builds install a completely different set of packages [12:07] it would be good to know the exact time it dies [12:07] then we could easily verify against the crontab [12:11] 1 hr 9 min 20 seconds ago [12:11] or as much as 5 min before that.. [12:19] very very weird [12:20] it should have been idlin at that time [12:20] *idling [12:20] 55 1 * * * ARCHES='armel+omap armel+omap4' OPTIONS="-f ext3" buildlive ubuntu-netbook && ARCHES='armel+omap armel+omap4' for-project ubuntu-netbook cron.ports_daily-preinstalled [12:20] 13 3 * * * ARCHES='armel+omap armel+omap4' OPTIONS="-f ext3" buildlive kubuntu && ARCHES='armel+omap armel+omap4' for-project kubuntu cron.ports_daily-preinstalled [12:21] oh, my copy was outdated [12:21] there is 45 6 * * * ARCHES='armel+omap armel+omap4' OPTIONS="-f ext3" buildlive kubuntu-mobile && ARCHES='armel+omap armel+omap4' for-project kubuntu-mobile cron.ports_daily-preinstalled [12:22] NCommander, why the heck are we building all these omap4 targets ? [12:22] (i'm pretty sure i asked you not to) [12:24] bah [12:24] the crontab in the branch *totally* differs from the actual one [12:24] * ogra sighs deeply [12:25] so kubuntu-mobile is actually the last thing we build, starting at 6:45 [12:26] and according to http://cdimage.ubuntu.com/kubuntu-mobile/ports/daily-preinstalled/20100908/ they finished today at ~8:00 [12:26] so it was actually idle when it died [12:31] lamont, are both systems identical software wise (same cron-daily entries etc) ? [12:31] i wonder if its a system cronjob that makes it die [12:36] as far as I know, yes [12:44] sigh. brainfart. that was when the machine was rebooted. [12:44] I'll be able to give you a time the next time it dies... [12:44] ogra: ?, how does it diff, last time I updated the cron, it matched bzr 100% === rgreening_ is now known as ghost === ghost is now known as rgreening === bjf[afk] is now known as bjf === bjf is now known as bjf[afk] === bjf[afk] is now known as bjf [21:41] I'd appreciate it if someone else on the release team would have a look at Bug #631413. They are reasonably ready to push an upload and IME it's demonstrably better than what we have now, but I'm hardly dispassionate on the question. [21:41] Launchpad bug 631413 in mesa (Ubuntu) "[FFe] Mesa 7.9 (affects: 4) (heat: 26)" [Undecided,New] https://launchpad.net/bugs/631413 === bjf is now known as bjf[afk]