[06:40] hello :), I am wondering if this build is stuck or still working hard in the background? https://launchpad.net/ubuntu/+source/libreoffice/1:7.1.1~rc2-0ubuntu1/+build/21083931 [06:41] same for https://launchpad.net/ubuntu/+source/libreoffice/1:7.1.1~rc2-0ubuntu1/+build/21083932 === icey_ is now known as icey [10:04] ricotz: I can't say, don't know this package's build system well enough [10:05] ricotz: IIRC that sort of thing is normally a pkgbinarymangler bug of some kind [10:49] . [10:57] cjwatson, I see, it seems it hasn't moved in the last hours, could you cancel and restart both? [10:59] ricotz: cancelled, do you want to grab build logs for inspection before I retry? [11:00] cjwatson, grabbed them both, please proceed :) [11:15] ricotz: done [11:37] cjwatson, thank you === hellswor1 is now known as hellsworth === ijohnson is now known as ijohnson|lunch === ijohnson|lunch is now known as ijohnson [22:51] cjwatson, if you are still around, please cancel and restart https://launchpad.net/ubuntu/+source/libreoffice/1:7.1.1~rc2-0ubuntu1/+build/21083933 [22:51] ricotz: can cancel; is there any reason to believe a retry will work? [22:51] cjwatson, the previous retry for amd64 worked [22:52] the other arm64 is still running [22:53] this might be triggered by the recent load on the builders, and calmed down afaics [22:54] ricotz: hm, OK, retried. I think it's more likely to be some kind of occasional race in pkgstripfiles (perhaps when run in parallel?), but not sure [22:55] there are a few KDE packages that sometimes do that 'INFO: pkgstripfiles: waiting for lock' for quite a long time. they do eventually break that loop and finish [22:56] I've also seen it apparently run forever [22:56] it may be subtle in a way I don't currently understand [22:56] I'm glad I have not had that then [22:56] (forever as in I noticed builds that had been running for a week stuck there) [22:56] O_O [22:56] cjwatson, the interesting part of the earlier amd64 log is https://paste.debian.net/plain/1187130 [22:56] I realize this is not an astronomical or theological definition of forever but still :) [22:57] longer that humanly tolerable.... [22:57] *than [22:57] ricotz: ah yes, that looks rather like bad error handling then, one would think it should have given up there [22:58] (leaving aside whatever caused that error to start with, which isn't obvious to me) [23:00] cjwatson, there are disk space issues in the past, which the packaging has a mitigation for, but not sure if this gets hit again due the mangler [23:01] cjwatson, thank you for restarting [23:09] cjwatson, jfyi the other logs contain the "same" error [23:43] Completed builds 290915 successful 49178 failed. [23:44] We certainly don't need all of this data. [23:44] I'm not seeing a way I can just delete old builds - does the Launchpad tooling do this by itself? [23:45] Can you be more specific? [23:46] And why is it important? [23:48] It isn't important, I would just like to be somewhat thoughtful when the Lubuntu CI pumps out many, many builds per day. It's extremely useful for us in Lubuntu development, but we really don't need old build logs and artifacts from more than a week ago at best. [23:48] I'm referring to the PPAs under ~lubuntu-ci. [23:48] Artifacts are automatically garbage-collected after a while when they're no longer published (in public PPAs, anyway). [23:49] We generally keep things like old build logs forever. It's not particularly expensive for us to do so, and they're the sort of thing that's often unpredictably useful years later when doing some kind of archaeology. [23:50] If we didn't GC artifacts that would indeed become a problem for us much more quickly, but build logs are on average rather smaller. [23:50] There's no way to manually delete old builds. [23:50] (Short of SQL surgery on our part, which is much more expensive in terms of human time ...) [23:50] That answers my query; if sysadmins are ever looking at things and it becomes a problem over a couple of years, feel free to purge those old logs. If it's not much of a nuisance then it looks like we're fine for now. [23:51] I understand if it would take more work than it's worth. :) [23:51] I can say with some confidence that build logs are unlikely to be anywhere near the first place we go looking for space :-) [23:51] Sounds good, thanks cjwatson. :) [23:52] Out of curiosity let me see if I can figure out how much space your build logs are currently using ...