=== StevenK_ is now known as StevenK === persia__ is now known as persia [10:59] I'm hitting a failed to upload on https://launchpad.net/~daniel-thewatkins/+livefs/ubuntu/wily/cpc/+build/37706 [11:00] I've managed to do it twice now, so I don't think it was a random failure. [11:00] I'm playing around with the files that I output from live-build, so it's probably something to do with that. [11:00] cjwatson: wgrant: Can you shed any light on why I'm hitting ^? [11:00] Let me see. [11:01] UploadFailed: Invalid length: 0 [11:01] Odd_Bloke: You have an empty file, which makes the librarian sad. [11:01] I thought we excluded those in launchpad-buildd, but maybe not. [11:01] OK, that's easy enough to remedy. [11:01] wgrant: Ta. [11:11] wgrant: I think I was a little reluctant to possibly end up papering over problems by ignoring such things. [11:11] Though the OOPS of course isn't ideal. [11:12] Indeed. [12:10] I can't assign/change status on bugs. Known issue? [12:10] OOPS-ca98aa95bdece3ad7bf7accced65c91d is one example [12:10] https://oops.canonical.com/?oopsid=OOPS-ca98aa95bdece3ad7bf7accced65c91d [12:14] the js error also comes back as a html-page, so it's hard to read the error: http://i.imgur.com/8EvY2uU.png [12:17] jgdx: We get occasional reports of this, and don't know the exact cause, but it normally clears up quite quickly [12:18] It's a very slow UPDATE on the BugTask table, probably a long transaction somewhere else ... [12:19] Not exactly: https://lpstats.canonical.com/graphs/AppServer5XXsLpnet/20150908/20150915/ [12:19] Something is up. [12:19] Some Ubuntu bug searches timing out. [12:19] It's possible that a particular scripted search has fallen off a cliff and those long queries are somehow causing contention. [12:19] Worth a pg_stat_activity check? [12:20] Hi! Anyone else noticing issues with launchpad? I'm trying to create a bug under neutron, and get an "Uh oh!" message repeatedly. [12:20] It probably won't show anything, but worth a try. [12:20] cjwatson, okay. I know there's the occational hiccup, but this is now 100% of the time. [12:21] pc_m: That's the current discussion. [12:21] cjwatson: Ah, just joined channel. [12:22] cjwatson: FYI, it complained about load balancer timing out. After 4th try, it created the bug. [12:24] I did the reload button, and once it did work, it created multiple instanced (so looked like each request did work). [12:28] Hammering it isn't going to give us additional information at this point [12:46] jgdx, pc_m: It should be OK now, let us know if you have any more issues. [12:46] One of our DB slaves went a little overboard. [12:51] wgrant, the failing action now completed. Thanks! [12:51] Great, thanks for confirming. [14:22] https://launchpad.net/~daniel-thewatkins/+livefs/ubuntu/wily/cpc/+build/37714 seems to be stuck; can anyone shed light on why that might be? [14:23] (https://launchpad.net/~daniel-thewatkins/+livefs/ubuntu/wily/cpc/+build/37717 is also stuck in the same place, for confirmation) [14:24] wgrant: ^ ? [14:28] Odd_Bloke: Do you have a similar build already that you've cancelled to see what the unflushed tail of the log says? [14:28] cjwatson: I don't; cancelling that one now. [14:34] cjwatson: Cancellation finished a couple of minutes ago, but no build log yet. Am I just being impatient? [14:36] Odd_Bloke: It failed to cancel and gave up. [14:36] So that's unfortunate. [14:37] This usually means that you managed to crash the builder somehow. [14:38] The only difference that this run should have amounts to "cp chroot/boot/vmlinuz-4.2.0-7-generic livecd.vmlinuz-generic.kernel" in a binary hook. [14:38] * cjwatson peers at the custom livecd-rootfs [14:39] Hmm, does the builder blow up on unexpected suffixes? That rings a very vague bell from last time I was playing with this sort of stuff? [14:40] No, my guess is that the copied files aren't readable by the builder. [14:40] Ah, that's probably a better guess. [14:40] Notice that livecd-rootfs/live-build/auto/build does chmod 644 in similar situations. [14:41] In fact, why doesn't the existing stuff in live-build/auto/build work for you? [14:41] It's doing pretty similar stuff to copy out kernels. [14:41] Oh, I must have missed it. [14:42] The only difference is that you need to explicitly list the flavours you want in the project declaration in live-build/auto/config. [14:42] And it also handles initrds and signed images. [14:43] Oh, hmph. [14:43] We _were_ using our own fork of live-build and still doing this outside of live-build. [14:43] So I guess my assumption that we had to do it outside of live-build doesn't hold. [14:44] I think this kind of interaction with the buildd belongs in livecd-rootfs rather than live-build. [14:44] Albeit in its live-build hooks. [14:44] Yeah, that looks precisely like what we need. [14:45] You definitely want to modify live-build/auto/config to set up the config you need. [14:45] Every distinct "project" field passed to a livefs build should have such a thing. [14:45] Oh, but ubuntu-cpc is already in there, OK. [14:46] Yeah, we're already doing most of our stuff there sensibly; I'd just missed that we could do this with configuration rather than custom hooks. [14:46] So it probably just needs to set KERNEL_FLAVOURS. [14:47] Yeah, just hadn't realised it was in trunk rather than in your delta, good. [14:47] Well, at least I've learned more about the difference between binary and chroot hooks this morning. :p [14:47] Heh [15:19] cjwatson: Ah, that isn't working because it's looking in binary/boot which only contains filesystem.ext4, filesystem.packages and filesystem.packages-remove in our builds. [15:26] Which I think would only work if LB_CHROOT_FILESYSTEM was none. [15:26] Because all of the other code paths in lb_binary_rootfs just shove a file there. [15:31] Odd_Bloke: I think it's because you're using --linux-packages=none, which disables the relevant code path in lb_binary_linux-image. [15:31] Aha, that looks promising. [15:31] Surely either you need to copy them out or you have no kernel packages installed, but not both. :-) [15:32] In fact you have KERNEL_FLAVOURS=none in that same section, so those probably need to be changed together. [15:34] --linux-packages defaults to "linux" on Ubuntu, so you should probably instead set KERNEL_FLAVOURS in the per-architecture case statement and drop the manual installation of linux-*. [15:34] I think. [15:34] Though maybe linux-virtual doesn't correspond to the kernel image name, so that might still need some work. [15:35] Maybe that's why you have this stuff to begin with. [15:35] I remember there being some pain around this before, but I don't remember precisely what problems I was seeing. [15:35] But, yeah, I wouldn't be surprised to get a definite reminder in a few minutes time. :p [15:36] So that might in fact be reasonable cause for a cpc-specific hook, but I'd probably do it by doing something in line with what lb_binary_linux-image would do if you could persuade it to DTRT without also changing the behaviour of lb_chroot_linux-image in undesirable ways. [15:36] Since that code is simpler. [15:38] cjwatson: At the moment, we publish the tab-delimited file that is produced by that hook, which is why it was happening in that somewhat circumspect way. [15:43] That seems of dubious utility when the manifest is there as well. [15:43] So, I mean, the trivial fix is to chmod 644, just wondering if it could be done better. [15:47] cjwatson: Yes, indeed. Sadly we have to maintain that interface for trusty. [15:48] I'm building a livecd-rootfs package with some different kernel options, I'll see what that does. [17:51] cjwatson: Yeah, your suspicious is correct; we either fail to find something named *linux-virtual* in /boot or we install linux-generic in the image. [17:51] *suspicion [17:52] So tomorrow I'll look at rewriting that hook in a less hidious fashion. === Guest39525 is now known as anthonyjf === anthonyf is now known as Guest95842 === Guest95842 is now known as anthonyjf [21:36] cjwatson: Hmm, alternatively I could teach the part of live-build that 'virtual' kernels are named 'generic' in /boot. [22:29] Odd_Bloke: I don't know which is better. Your call.