| === JanC_ is now known as JanC | ||
| Odd_Bloke | https://launchpad.net/~daniel-thewatkins/+livefs/ubuntu/wily/cpc/+build/39160 just 'Failed to build' without giving me a buildlog; can anyone see what happened? | 10:34 |
|---|---|---|
| Odd_Bloke | cjwatson: wgrant: ^? | 10:35 |
| wgrant | Let me see. | 10:35 |
| Odd_Bloke | Cheers. | 10:35 |
| wgrant | 2015-09-29 10:30:20+0000 [-] Failure: twisted.internet.defer.CancelledError: | 10:36 |
| wgrant | 2015-09-29 10:30:20+0000 [-] | 10:36 |
| wgrant | 2015-09-29 10:30:20+0000 [-] Judged builder lgw01-53 (1 failures) with job LIVEFSBUILD-39160 (4 failures): None, False | 10:36 |
| wgrant | CancelledError == TCP timeout | 10:36 |
| wgrant | And it did it four times. | 10:37 |
| wgrant | You seem to be causing the buildds some serious grief. | 10:37 |
| wgrant | Are you OOMing or something, possibly? | 10:37 |
| wgrant | There is no general fault; that build is the only one to hit such timeouts in recent hours. | 10:37 |
| Odd_Bloke | Yeah, that's possible. | 10:37 |
| Odd_Bloke | wgrant: Might I see a similar error if I was running out of disk space, or would that still get a build log back? | 10:40 |
| Odd_Bloke | wgrant: Also, is there any harm in doing these builds (with minor modifications for debugging)? | 10:43 |
| wgrant | Odd_Bloke: It's fine, the builder VMs are automatically restarted on failure. | 10:45 |
| wgrant | Odd_Bloke: Disk exhaustion would not cause this. | 10:45 |
| Odd_Bloke | Okie doke. | 10:46 |
| cjwatson | Disk exhaustion can cause other problems that you might not be able to distinguish from this, though. | 11:07 |
| cjwatson | I have some patches lying around somewhere to improve launchpad-buildd's resilience to ENOSPC, but never got round to working out how to test them. | 11:08 |
| cjwatson | It wouldn't cause a TCP timeout, but it could certainly cause launchpad-buildd to crash. | 11:08 |
| wgrant | Right, it could cause another "Failed to build" with no log, but AFAIK it wouldn't cause the CancelledError that we see in buildd-manager logs. | 11:14 |
| Odd_Bloke | Is https://launchpad.net/~daniel-thewatkins/+livefs/ubuntu/wily/cpc/+build/39163 stuck in some way I can't see? It's taking a lot longer than it normally would... | 11:27 |
| Odd_Bloke | cjwatson: wgrant: ^ any ideas? | 12:22 |
| cjwatson | I don't have any magic way to tell. | 12:23 |
| cjwatson | I suggest cancelling it and seeing if you can divine anything from the log afterwards. | 12:24 |
| cjwatson | There might be something unflushed in the log. | 12:24 |
| Odd_Bloke | Done/doing. | 12:24 |
| cjwatson | Odd_Bloke: Hm, not clear, is it? I wonder if there's still some kind of race with loop0p1 creation, though I can't imagine what. Is it repeatable? | 13:12 |
| Odd_Bloke | cjwatson: I think so; this is using GPT partition tables which is the difference from before. | 13:13 |
| Odd_Bloke | cjwatson: The '[ ! -b /dev/mapper/loop0p1 ]' should catch the race condition now. | 13:14 |
| Odd_Bloke | https://launchpad.net/~daniel-thewatkins/+livefs/ubuntu/wily/cpc/+build/39164 was exhibiting the same behaviour, but hasn't ended up with a log. | 13:18 |
| Odd_Bloke | cjwatson: I might try omitting the dd and see if we have something that looks like a sensibly empty disk image at the other end. | 13:20 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!