| -queuebot:#ubuntu-release- New binary: linux-signed-azure [amd64] (trusty-proposed/main) [4.15.0-1023.24~14.04.1] (kernel) | 00:23 | |
| -queuebot:#ubuntu-release- New: accepted linux-signed-azure [amd64] (trusty-proposed) [4.15.0-1023.24~14.04.1] | 05:34 | |
| -queuebot:#ubuntu-release- New sync: r-cran-rdflib (cosmic-proposed/primary) [0.1.0+dfsg-1] | 07:30 | |
| xnox | tsimonq2, oooh oooh. if there are flakey results, i want to look at them. | 09:27 |
|---|---|---|
| xnox | cking, there appears to be a sysfs ubuntu_stress_smoke_test.stress-smoke-test failure on s390x | 09:30 |
| xnox | is it known? should it be skipped / XFAILED on s390x? | 09:31 |
| xnox | https://paste.ubuntu.com/p/VYfR9StC5Y/ is the relevant bit | 09:31 |
| cking | xnox, it's an on-going issue, I'm working on that at the moment, so let's skip that for now | 09:32 |
| xnox | cking, thank you! | 09:33 |
| xnox | oooooh i think there are bad logs cloning things | 09:34 |
| cking | xnox, can you remind me what kernel that was with, I want to check if it matches what I think should be happening with that test on s390x | 09:38 |
| xnox | cjwatson, does https://paste.ubuntu.com/p/QJXJyYQZCF/ looks at all plausible match for the twisted bug you mentioned the other day? However based on timestamps it looks like it just hang there for half an hour; and it's totally legit to kill half an hour connections..... I wonder if git.launchpad.net can be bypassed in squid.internal, if that matters at all. | 09:39 |
| xnox | cking, that would be 4.18.0-7-generic what's currently in cosmic-release | 09:41 |
| cking | xnox, ah, that's good, phew, that's what I expected | 09:41 |
| cjwatson | xnox: Shortly after I mentioned that Twisted bug, I mentioned that I thought it was a red herring. | 09:50 |
| cjwatson | xnox: It's possible that it was the clone using a lot of memory on the server end and being slow / getting killed. We had some problems with that recently. | 09:50 |
| xnox | cjwatson, ack, thanks. sorry i missed previous red herring. | 09:51 |
| cjwatson | xnox: While it's possible the squid matters somehow, it doesn't seem very likely to me. | 09:51 |
| xnox | well, this clone should not take 25min, even across the ocean. and most of the time it doesn't. it feels like something somewhere is keeping the connection open, with actual pack request lost. | 09:51 |
| xnox | with the connection eventually closed. | 09:52 |
| xnox | cjwatson, can we somehow trace logs on cloning that repo from lp, on the other end? =) | 09:52 |
| cjwatson | What timezone are those logs in? | 09:53 |
| cjwatson | They have a stupid time format so I don't want to assume UTC. | 09:53 |
| xnox | cjwatson, let me correlate those with what i know to be UTC | 09:54 |
| cjwatson | Ah, found it | 09:57 |
| xnox | end time 2018-09-10 03:21:35 UTC, duration 2h 28m 42s; approximate start time 2018-09-10 00:52:51 UTC - that's for all of the tests, let me narrow down the window when this clone was started | 09:57 |
| cjwatson | At that time there was a problem with the backend | 09:57 |
| cjwatson | 2018-09-10 02:25:29+0000 [Uninitialized] Backend connection failed | 09:57 |
| cjwatson | Traceback (most recent call last): | 09:57 |
| cjwatson | Failure: twisted.internet.error.ConnectionRefusedError: Connection was refused by other side: 111: Connection refused. | 09:57 |
| cjwatson | Logs are full of that sort of thing, so everything would've been failing | 09:58 |
| xnox | everything is terrible =))))) oh well | 09:58 |
| cjwatson | And stuff like this from the backend: | 10:00 |
| cjwatson | 2018-09-10 02:25:29+0000 [-] [request-id=898f6e33-d3eb-47c2-bc4c-001540cb3865] [PackBackendProtocol] Connection lost. | 10:00 |
| cjwatson | Traceback (most recent call last): | 10:00 |
| cjwatson | Failure: twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion. | 10:00 |
| cjwatson | Where the "connection" is a Twisted-ish way of saying "pipe to git child process" | 10:00 |
| xnox | thank you for translating =) | 10:01 |
| cjwatson | I think this is just a scaling problem | 10:01 |
| * xnox never did twisted | 10:01 | |
| cjwatson | Unfortunately the git backend is not very scalable right now | 10:01 |
| cjwatson | I already proposed a session for my team in Brussels to work out what we need to do to fix that | 10:01 |
| cjwatson | So this is ammunition | 10:01 |
| xnox | i can actually dig into these at http://autopkgtest.ubuntu.com/packages/linux/cosmic/s390x but most of "x fail" are git clone from launchpad fail, if there is at least one green pass for a given version number. | 10:02 |
| xnox | not all, but roughly half. | 10:02 |
| xnox | hence for like the lifetime of bionic we had s390x blacklisted. | 10:02 |
| cjwatson | It doesn't surprise me that kernel clones would be particularly likely to hit problems | 10:03 |
| xnox | cjwatson, but lp uses git binary right? it doesn't use anything custom like dulwich? i believe github uses libgit2. | 10:03 |
| xnox | apw, cking - i wonder; if git clone of the linux tree fails, if we should be falling back on `pull-lp-source` | 10:03 |
| cjwatson | The backend is the git binary, yes | 10:04 |
| xnox | ack | 10:04 |
| cjwatson | Though libgit2 is involved at other levels, mainly for the internal API | 10:04 |
| cjwatson | git protocol traffic is passed through various layers of custom Twisted-based code, but the actual upload-pack / receive-pack protocol is handled by the git binary | 10:05 |
| apw | xnox, why am i cloning the git repo ? | 10:06 |
| apw | this seems like a strange thing to be doing, unless i need something big to mangle | 10:06 |
| cjwatson | Yeah, pull-lp-source would also be weird considering that autopkgtest *already gives you the source* | 10:07 |
| xnox | apw, to call $ make to compile all the self-tests which are in the linux source tree.... despite being in the linux kernel source tree as provided by autopkgtest already | 10:07 |
| apw | xnox, so it sounds like the correct fix is to add some kind of 'detect you already have the kernel source' | 10:07 |
| xnox | apw, and for some reason, it is done from a git clone of the .../cosmic repo, master branch, rather than what is already unpacked. maybe that's simply how the tests got ported onto autopkgtest. | 10:07 |
| apw | xnox, well one advantage of that is we can 'fix' tests without uploading things | 10:08 |
| xnox | apw, that would be easy to do, there are environment variables that tell you that. Shall I write that up in the bug report to improve the "setup" stage? | 10:08 |
| apw | xnox, yes please | 10:08 |
| xnox | cjwatson, do you use the v2 protocol stuff in git? https://opensource.googleblog.com/2018/05/introducing-git-protocol-version-2.html | 10:08 |
| xnox | somehow it's "better" | 10:09 |
| cjwatson | not yet, no | 10:09 |
| cjwatson | but the scaling problems are mostly memory not bandwidth atm; I don't know whether the new protocol does much about that | 10:09 |
| xnox | i wonder if the fact that we try to do shallow clones here with --depth 1 is actually causing more memory problems here, instead of cloning the full master branch, or even the full repo. | 10:10 |
| cjwatson | I've added a Trello card for the bits I think we need to do to support v2 | 10:21 |
| jbicha | please remove sysprof/s390x/cosmic, it only built because the previous version ignored build tests failures | 10:51 |
| sil2100 | slangasek, infinity: hey! Would be nice if you announced somewhere that some non-release people have been delegated to approve Kubuntu FFe's - I almost switched the plasma one back from 'Triaged' since I thought this is some kind of a illegal activity ;) | 10:52 |
| apw | sil2100, i assume it was approved by one of the project leads for kubuntu | 10:53 |
| apw | those are documented somewhere arn't they ? | 10:53 |
| sil2100 | apw: well, yes, but I guess FFe's normally are only to be approved by ubuntu-release members, not flavour release members from what I know | 10:54 |
| Laney | indeed | 10:55 |
| apw | sil2100, i was under the impression that things in the flavour seed, and not on other images were their responsibility | 10:55 |
| apw | but as there is confusion i defer to infinity | 10:55 |
| sil2100 | Anyway, I'm all good about anything as long as it's made visible to the other release members | 11:00 |
| sil2100 | If not for Laney I wouldn't have known about this | 11:02 |
| sil2100 | And I'd just cause unnecessary trouble | 11:02 |
| tsimonq2 | sil2100: See the IRC discussion from yesterday. | 11:24 |
| tsimonq2 | I proposed it but they determined an announcement to be not necessary. | 11:25 |
| tsimonq2 | I was only given the go-ahead (with Rik) to approve Kubuntu FFEs; Lubuntu FFEs still need Release Team approval. :P | 11:26 |
| sil2100 | tsimonq2: strange decision not to announce, could cause some small chaos by accident | 11:45 |
| sil2100 | tsimonq2: as for Lubuntu, please update the bug description to include what I requested ;p | 11:45 |
| tsimonq2 | sil2100: My backlog is getting very large these days. :P | 11:48 |
| tsimonq2 | sil2100: So, "when I can." | 11:48 |
| sil2100 | tsimonq2: ok! ;) | 12:05 |
| tsimonq2 | sil2100: I just updated the FFe docs; feel free to peer review. | 12:24 |
| tsimonq2 | I'll do that Lubuntu bug next. ;) | 12:24 |
| tsimonq2 | (w.u.c/FreezeExceptionProcess) | 12:25 |
| tsimonq2 | infinity, slangasek: ^ | 12:26 |
| -queuebot:#ubuntu-release- Unapproved: adobe-flashplugin (bionic-proposed/partner) [1:20180814.1-0ubuntu0.18.04.2 => 1:20180911.1-0ubuntu0.18.04.1] (no packageset) | 12:53 | |
| -queuebot:#ubuntu-release- Unapproved: adobe-flashplugin (xenial-proposed/partner) [1:20180814.1-0ubuntu0.16.04.2 => 1:20180911.1-0ubuntu0.16.04.1] (no packageset) | 12:53 | |
| -queuebot:#ubuntu-release- Unapproved: adobe-flashplugin (trusty-proposed/partner) [1:20180814.1-0ubuntu0.14.04.2 => 1:20180911.1-0ubuntu0.14.04.1] (no packageset) | 12:53 | |
| chrisccoulson | hi, could somebody please approve the adobe-flashplugin partner uploads for trusty -> bionic? (and also be around to copy them to the release pocket once they're ready) | 12:54 |
| chrisccoulson | see https://wiki.ubuntu.com/ArchiveAdministration#Handling_updates_to_partner | 12:54 |
| apw | chrisccoulson, ok | 13:15 |
| -queuebot:#ubuntu-release- Unapproved: open-iscsi (bionic-proposed/main) [2.0.874-5ubuntu2.1 => 2.0.874-5ubuntu2.2] (ubuntu-desktop, ubuntu-server) | 13:17 | |
| -queuebot:#ubuntu-release- Unapproved: accepted adobe-flashplugin [source] (bionic-proposed) [1:20180911.1-0ubuntu0.18.04.1] | 13:19 | |
| -queuebot:#ubuntu-release- Unapproved: accepted adobe-flashplugin [source] (xenial-proposed) [1:20180911.1-0ubuntu0.16.04.1] | 13:22 | |
| -queuebot:#ubuntu-release- Unapproved: accepted adobe-flashplugin [source] (trusty-proposed) [1:20180911.1-0ubuntu0.14.04.1] | 13:22 | |
| apw | chrisccoulson, ^ | 13:23 |
| chrisccoulson | apw, awesome, thanks! | 13:24 |
| -queuebot:#ubuntu-release- Unapproved: fwupd (cosmic-proposed/main) [1.1.2-1 => 1.1.2-1] (desktop-core) | 13:47 | |
| -queuebot:#ubuntu-release- Unapproved: fwupd (cosmic-proposed/main) [1.1.2-1 => 1.1.2-1] (desktop-core) | 13:47 | |
| -queuebot:#ubuntu-release- Unapproved: fwupd (cosmic-proposed/main) [1.1.2-1 => 1.1.2-1] (desktop-core) | 13:48 | |
| -queuebot:#ubuntu-release- Unapproved: nvidia-graphics-drivers-340 (bionic-proposed/restricted) [340.106-0ubuntu3 => 340.107-0ubuntu0.18.04.1] (ubuntu-desktop) | 13:53 | |
| -queuebot:#ubuntu-release- Unapproved: fwupd (cosmic-proposed/main) [1.1.2-1 => 1.1.2-1] (desktop-core) | 13:55 | |
| -queuebot:#ubuntu-release- New binary: octave [s390x] (cosmic-proposed/universe) [4.4.1-1] (no packageset) | 16:20 | |
| -queuebot:#ubuntu-release- New binary: octave [ppc64el] (cosmic-proposed/universe) [4.4.1-1] (no packageset) | 16:32 | |
| -queuebot:#ubuntu-release- New source: vulkan-loader (cosmic-proposed/primary) [1.1.82.0-0ubuntu1] | 16:34 | |
| -queuebot:#ubuntu-release- Unapproved: ostree (bionic-proposed/universe) [2018.4-2 => 2018.8-0ubuntu0.1] (ubuntugnome) | 16:49 | |
| -queuebot:#ubuntu-release- Unapproved: flatpak (bionic-proposed/universe) [0.11.7-0ubuntu0.1 => 1.0.1-0ubuntu0.1] (ubuntugnome) | 16:52 | |
| slangasek | LocutusOfBorg: er, you have 10 ppa builds of llvm-toolchain-7 going, in a ppa for which the source has apparently been un-published? | 17:04 |
| slangasek | including 7 for superseded versions, which I am now cancelling | 17:06 |
| -queuebot:#ubuntu-release- New binary: octave [armhf] (cosmic-proposed/universe) [4.4.1-1] (no packageset) | 17:35 | |
| -queuebot:#ubuntu-release- New binary: octave [arm64] (cosmic-proposed/universe) [4.4.1-1] (no packageset) | 17:41 | |
| -queuebot:#ubuntu-release- New: accepted gdcm [amd64] (cosmic-proposed) [2.8.7-2ubuntu1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted gdcm [ppc64el] (cosmic-proposed) [2.8.7-2ubuntu1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted octave [arm64] (cosmic-proposed) [4.4.1-1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted octave [ppc64el] (cosmic-proposed) [4.4.1-1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted r-cran-kmi [sync] (cosmic-proposed) [0.5.4-1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted r-cran-rdflib [sync] (cosmic-proposed) [0.1.0+dfsg-1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted gdcm [i386] (cosmic-proposed) [2.8.7-2ubuntu1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted octave [armhf] (cosmic-proposed) [4.4.1-1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted r-cran-mitools [sync] (cosmic-proposed) [2.3-1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted gdcm [s390x] (cosmic-proposed) [2.8.7-2ubuntu1] | 17:48 | |
| -queuebot:#ubuntu-release- New: accepted octave [s390x] (cosmic-proposed) [4.4.1-1] | 17:48 | |
| -queuebot:#ubuntu-release- New binary: r-cran-mitools [amd64] (cosmic-proposed/none) [2.3-1] (no packageset) | 17:51 | |
| -queuebot:#ubuntu-release- Unapproved: fwupd (bionic-proposed/main) [1.0.6-2 => 1.0.9-0ubuntu1] (desktop-core) | 19:05 | |
| tsimonq2 | slangasek: Thanks for ruby-compass. | 19:46 |
| tsimonq2 | slangasek: thepeg can probably go too. | 19:48 |
| -queuebot:#ubuntu-release- New sync: r-cran-redland (cosmic-proposed/primary) [1.0.17-9-1] | 20:58 | |
| acheronuk | slangasek: I assume 'all-proposed' retries on amd64/ppc64el being unable to install a kernel again is due to linux-signed FTBFS? | 21:23 |
| slangasek | acheronuk: linux-signed FTBFS would certainly trigger it. what series are we talking here? | 21:25 |
| acheronuk | slangasek: cosmic linux-signed (4.18.0-7.8 to 4.18.0-8.9) | 21:25 |
| slangasek | acheronuk: ok. accepting the UEFI binaries now so that we can get that unstuck | 21:58 |
| -queuebot:#ubuntu-release- Unapproved: accepted fwupd [amd64] (cosmic-proposed) [1.1.2-1] | 21:59 | |
| -queuebot:#ubuntu-release- Unapproved: accepted fwupd [armhf] (cosmic-proposed) [1.1.2-1] | 21:59 | |
| -queuebot:#ubuntu-release- Unapproved: accepted fwupd [arm64] (cosmic-proposed) [1.1.2-1] | 21:59 | |
| -queuebot:#ubuntu-release- Unapproved: accepted fwupd [i386] (cosmic-proposed) [1.1.2-1] | 21:59 | |
| acheronuk | slangasek: thank you :) | 22:20 |
| -queuebot:#ubuntu-release- Unapproved: accepted shim [sync] (xenial-proposed) [15+1533136590.3beb971-0ubuntu1] | 22:46 | |
| -queuebot:#ubuntu-release- Unapproved: accepted shim-signed [source] (xenial-proposed) [1.33.1~16.04.2] | 22:47 | |
| -queuebot:#ubuntu-release- New binary: linux-signed [ppc64el] (cosmic-proposed/main) [4.18.0-8.9] (core, kernel) | 22:47 | |
| -queuebot:#ubuntu-release- New binary: linux-signed [amd64] (cosmic-proposed/main) [4.18.0-8.9] (core, kernel) | 22:48 | |
| -queuebot:#ubuntu-release- New: accepted linux-signed [amd64] (cosmic-proposed) [4.18.0-8.9] | 22:56 | |
| -queuebot:#ubuntu-release- New: accepted linux-signed [ppc64el] (cosmic-proposed) [4.18.0-8.9] | 22:56 | |
| -queuebot:#ubuntu-release- Unapproved: plymouth (bionic-proposed/main) [0.9.3-1ubuntu7 => 0.9.3-1ubuntu7.18.04.1] (core) | 23:05 | |
| -queuebot:#ubuntu-release- New sync: ufolib2 (cosmic-proposed/primary) [0.1.3-1] | 23:19 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!