[00:44] -queuebot:#ubuntu-release- New binary: kopanocore [ppc64el] (bionic-proposed/universe) [8.3.4-4ubuntu2] (no packageset) [00:47] -queuebot:#ubuntu-release- New binary: kopanocore [s390x] (bionic-proposed/universe) [8.3.4-4ubuntu2] (no packageset) [00:54] -queuebot:#ubuntu-release- New binary: kopanocore [i386] (bionic-proposed/universe) [8.3.4-4ubuntu2] (no packageset) [00:57] -queuebot:#ubuntu-release- New binary: kopanocore [armhf] (bionic-proposed/universe) [8.3.4-4ubuntu2] (no packageset) [01:04] -queuebot:#ubuntu-release- New binary: kopanocore [amd64] (bionic-proposed/universe) [8.3.4-4ubuntu2] (no packageset) [01:16] -queuebot:#ubuntu-release- New binary: kopanocore [arm64] (bionic-proposed/universe) [8.3.4-4ubuntu2] (no packageset) [05:14] -queuebot:#ubuntu-release- New binary: intel-vaapi-driver-shaders [amd64] (bionic-proposed/none) [2.0.0~pre3+ds1-2] (no packageset) [05:14] -queuebot:#ubuntu-release- New binary: intel-vaapi-driver-shaders [i386] (bionic-proposed/none) [2.0.0~pre3+ds1-2] (no packageset) [07:24] -queuebot:#ubuntu-release- New binary: icu [s390x] (bionic-proposed/main) [60.1-1ubuntu1] (core) [07:26] -queuebot:#ubuntu-release- New binary: icu [amd64] (bionic-proposed/main) [60.1-1ubuntu1] (core) [07:27] -queuebot:#ubuntu-release- New: accepted ocamlbricks [sync] (bionic-proposed) [0.90+bzr456-1] [07:27] -queuebot:#ubuntu-release- New: accepted kopanocore [amd64] (bionic-proposed) [8.3.4-4ubuntu2] [07:27] -queuebot:#ubuntu-release- New: accepted kopanocore [armhf] (bionic-proposed) [8.3.4-4ubuntu2] [07:27] -queuebot:#ubuntu-release- New: accepted kopanocore [ppc64el] (bionic-proposed) [8.3.4-4ubuntu2] [07:27] -queuebot:#ubuntu-release- New binary: icu [ppc64el] (bionic-proposed/main) [60.1-1ubuntu1] (core) [07:27] -queuebot:#ubuntu-release- New: accepted kopanocore [arm64] (bionic-proposed) [8.3.4-4ubuntu2] [07:27] -queuebot:#ubuntu-release- New: accepted kopanocore [s390x] (bionic-proposed) [8.3.4-4ubuntu2] [07:27] -queuebot:#ubuntu-release- New: accepted kopanocore [i386] (bionic-proposed) [8.3.4-4ubuntu2] [07:28] -queuebot:#ubuntu-release- New: accepted intel-vaapi-driver-shaders [amd64] (bionic-proposed) [2.0.0~pre3+ds1-2] [07:28] -queuebot:#ubuntu-release- New: accepted libuvc [amd64] (bionic-proposed) [0.0.6-1] [07:28] -queuebot:#ubuntu-release- New: accepted libuvc [armhf] (bionic-proposed) [0.0.6-1] [07:28] -queuebot:#ubuntu-release- New: accepted libuvc [ppc64el] (bionic-proposed) [0.0.6-1] [07:28] -queuebot:#ubuntu-release- New: accepted intel-vaapi-driver-shaders [i386] (bionic-proposed) [2.0.0~pre3+ds1-2] [07:28] -queuebot:#ubuntu-release- New: accepted libuvc [i386] (bionic-proposed) [0.0.6-1] [07:28] -queuebot:#ubuntu-release- New: accepted libuvc [arm64] (bionic-proposed) [0.0.6-1] [07:28] -queuebot:#ubuntu-release- New: accepted libuvc [s390x] (bionic-proposed) [0.0.6-1] [07:28] -queuebot:#ubuntu-release- New binary: icu [i386] (bionic-proposed/main) [60.1-1ubuntu1] (core) [07:29] -queuebot:#ubuntu-release- New: accepted rsplib [source] (bionic-proposed) [3.0.1-1ubuntu6] [07:30] -queuebot:#ubuntu-release- New binary: ocamlbricks [ppc64el] (bionic-proposed/none) [0.90+bzr456-1] (no packageset) [07:31] -queuebot:#ubuntu-release- New binary: ocamlbricks [amd64] (bionic-proposed/none) [0.90+bzr456-1] (no packageset) [07:31] -queuebot:#ubuntu-release- New binary: ocamlbricks [s390x] (bionic-proposed/none) [0.90+bzr456-1] (no packageset) [07:31] -queuebot:#ubuntu-release- New binary: ocamlbricks [i386] (bionic-proposed/none) [0.90+bzr456-1] (no packageset) [07:31] -queuebot:#ubuntu-release- New binary: rsplib [ppc64el] (bionic-proposed/none) [3.0.1-1ubuntu6] (no packageset) [07:31] -queuebot:#ubuntu-release- New binary: rsplib [s390x] (bionic-proposed/none) [3.0.1-1ubuntu6] (no packageset) [07:32] -queuebot:#ubuntu-release- New binary: ocamlbricks [arm64] (bionic-proposed/none) [0.90+bzr456-1] (no packageset) [07:32] -queuebot:#ubuntu-release- New binary: rsplib [i386] (bionic-proposed/none) [3.0.1-1ubuntu6] (no packageset) [07:32] -queuebot:#ubuntu-release- New binary: rsplib [amd64] (bionic-proposed/none) [3.0.1-1ubuntu6] (no packageset) [07:33] -queuebot:#ubuntu-release- New binary: ocamlbricks [armhf] (bionic-proposed/none) [0.90+bzr456-1] (no packageset) [07:34] -queuebot:#ubuntu-release- New binary: rsplib [arm64] (bionic-proposed/none) [3.0.1-1ubuntu6] (no packageset) [07:35] -queuebot:#ubuntu-release- New binary: rsplib [armhf] (bionic-proposed/none) [3.0.1-1ubuntu6] (no packageset) [07:35] -queuebot:#ubuntu-release- New binary: icu [armhf] (bionic-proposed/main) [60.1-1ubuntu1] (core) [07:42] -queuebot:#ubuntu-release- New binary: icu [arm64] (bionic-proposed/main) [60.1-1ubuntu1] (core) [07:45] -queuebot:#ubuntu-release- New: accepted icu [amd64] (bionic-proposed) [60.1-1ubuntu1] [07:45] -queuebot:#ubuntu-release- New: accepted icu [armhf] (bionic-proposed) [60.1-1ubuntu1] [07:45] -queuebot:#ubuntu-release- New: accepted icu [ppc64el] (bionic-proposed) [60.1-1ubuntu1] [07:45] -queuebot:#ubuntu-release- New: accepted rsplib [amd64] (bionic-proposed) [3.0.1-1ubuntu6] [07:45] -queuebot:#ubuntu-release- New: accepted rsplib [armhf] (bionic-proposed) [3.0.1-1ubuntu6] [07:45] -queuebot:#ubuntu-release- New: accepted icu [arm64] (bionic-proposed) [60.1-1ubuntu1] [07:45] -queuebot:#ubuntu-release- New: accepted icu [s390x] (bionic-proposed) [60.1-1ubuntu1] [07:45] -queuebot:#ubuntu-release- New: accepted rsplib [i386] (bionic-proposed) [3.0.1-1ubuntu6] [07:45] -queuebot:#ubuntu-release- New: accepted icu [i386] (bionic-proposed) [60.1-1ubuntu1] [07:45] -queuebot:#ubuntu-release- New: accepted rsplib [arm64] (bionic-proposed) [3.0.1-1ubuntu6] [07:45] -queuebot:#ubuntu-release- New: accepted ocamlbricks [amd64] (bionic-proposed) [0.90+bzr456-1] [07:45] -queuebot:#ubuntu-release- New: accepted ocamlbricks [armhf] (bionic-proposed) [0.90+bzr456-1] [07:45] -queuebot:#ubuntu-release- New: accepted ocamlbricks [ppc64el] (bionic-proposed) [0.90+bzr456-1] [07:45] -queuebot:#ubuntu-release- New: accepted rsplib [ppc64el] (bionic-proposed) [3.0.1-1ubuntu6] [07:45] -queuebot:#ubuntu-release- New: accepted ocamlbricks [arm64] (bionic-proposed) [0.90+bzr456-1] [07:46] -queuebot:#ubuntu-release- New: accepted ocamlbricks [s390x] (bionic-proposed) [0.90+bzr456-1] [07:46] -queuebot:#ubuntu-release- New: accepted ocamlbricks [i386] (bionic-proposed) [0.90+bzr456-1] [07:46] -queuebot:#ubuntu-release- New: accepted rsplib [s390x] (bionic-proposed) [3.0.1-1ubuntu6] [08:35] -queuebot:#ubuntu-release- Unapproved: gnome-shell-extension-ubuntu-dock (artful-proposed/main) [0.7.1 => 0.8.1] (no packageset) [08:51] hello perl folks [08:51] somebody please enlight me about the restore SIGUNUSED signal patch in perl itself [08:52] I can confirm the patch still work, the config.h is correctly generated [08:52] but... autopkgtest for libpoe-component-client-dns-perl/1:1.054-1ubuntu1: amd64: Regression ♻ , armhf: Regression ♻ , i386: Regression ♻ , ppc64el: Regression ♻ , s390x: Regression ♻ [09:20] @SRU Team - is there any reason I miss why 1726017 was accepted into zesty but not yet into Xenial? [09:20] if there is something I want to fix it, otherwise those should go almost together as usual right? [09:21] I see today RAOF is on SRU duty, not sure if is is too early/late for him atm [09:21] anyway if one can take a look let me know [09:22] cpaelzer, what package is that ? [09:22] dnsmasq [09:22] cpaelzer: I'll be on SRU duty in about 12 hours 😀 [09:29] enick_867: that will be soon enough, I don't mind it takes some time but on this case I was wondering if there is something special to resolve [09:44] -queuebot:#ubuntu-release- Unapproved: nvidia-graphics-drivers-340 (xenial-proposed/restricted) [340.102-0ubuntu0.16.04.2 => 340.104-0ubuntu0.16.04.1] (ubuntu-desktop) [09:58] cpaelzer, i am looking at this dnsmasq change ... after getting a clean diff, because launchpad, there seems to be a discrepency [09:58] oh, interesting [09:58] ok let me know what you find apw [09:58] cpaelzer, between the change as applied in zesty which says 2.77 was applied and xenial which says 2.76 and 2.77 were applied; though the code delta looks to be the same [09:58] maybe that was the reason [09:59] I'm checking my uploads what we should see [10:00] zesty has just the pick of a8509d9076b71dfb30489b684be19eecee01d34f [10:00] checking the debdiff if it agrees [10:00] - 2.77: 68f6312d4b: [10:00] yeah that is it [10:00] got a new sha [10:01] this is "Stop treating SERVFAIL as a successful response from upstream servers." [10:01] ^ that is what is claimed to be applied in zesty, but in xenial you also have "- 2.76: 4ace25c5d6:" listed [10:01] yes it should [10:01] and i would say the source change is the 2.77 change, i cannot see any delta difference however between xenial and zesty [10:02] my git I built the upload from has the second change [10:02] "Treat REFUSED (not SERVFAIL) as an unsuccessful upstream response" [10:02] now checking debdiff [10:02] OH there it is ... it is just my eyes [10:02] - if (forward->forwardall == 0 || --forward->forwardall == 1 || RCODE(header) != SERVFAIL) [10:02] - if (forward->forwardall == 0 || --forward->forwardall == 1 || RCODE(header) != REFUSED) [10:02] my head is telling me those are the same, it is wrong [10:02] ok ... then no i cannot see any reason this is not ok if the zesty one is ok, and there is nothing in the bug [10:03] so i am going to review it again ... [10:04] -queuebot:#ubuntu-release- Unapproved: accepted dnsmasq [source] (xenial-proposed) [2.75-1ubuntu0.16.04.4] [10:04] cpaelzer, ... and ^ [10:04] thanks apw [10:04] essentially it is http://paste.ubuntu.com/25909762/ + http://paste.ubuntu.com/25909763 = http://paste.ubuntu.com/25909768/ [10:05] it seems no one is used to antive packages anymore [10:05] yep, just letting confirmation-bias (that the two diffs are the same) cloud my reading [10:05] but also chaning the same line twice made it hard to spot [10:05] i want the diff to be the same because that means review is simpler, but then the changelog wasn't [10:20] -queuebot:#ubuntu-release- New binary: marionnet [amd64] (bionic-proposed/universe) [0.90.6+bzr508-1] (no packageset) [11:05] -queuebot:#ubuntu-release- New binary: marionnet [i386] (bionic-proposed/universe) [0.90.6+bzr508-1] (no packageset) [11:23] -queuebot:#ubuntu-release- Unapproved: bind9 (xenial-proposed/main) [1:9.10.3.dfsg.P4-8ubuntu1.8 => 1:9.10.3.dfsg.P4-8ubuntu1.9] (core) [11:23] -queuebot:#ubuntu-release- Unapproved: bind9 (zesty-proposed/main) [1:9.10.3.dfsg.P4-10.1ubuntu5.2 => 1:9.10.3.dfsg.P4-10.1ubuntu5.3] (core) [11:24] -queuebot:#ubuntu-release- Unapproved: ubuntu-meta (artful-proposed/main) [1.404 => 1.404.1] (core) [11:28] -queuebot:#ubuntu-release- Unapproved: ubuntu-meta (zesty-proposed/main) [1.379 => 1.379.1] (core) [11:32] please approve marionnet, built only on two archs [11:34] -queuebot:#ubuntu-release- New: accepted marionnet [amd64] (bionic-proposed) [0.90.6+bzr508-1] [11:34] -queuebot:#ubuntu-release- New: accepted marionnet [i386] (bionic-proposed) [0.90.6+bzr508-1] [11:34] LocutusOfBorg, ^ [11:37] <3 [11:41] -queuebot:#ubuntu-release- Unapproved: ubuntu-meta (xenial-proposed/main) [1.361 => 1.361.1] (core) [11:44] -queuebot:#ubuntu-release- Unapproved: ubuntu-meta (artful-proposed/main) [1.404 => 1.404.1] (core) [11:44] There are three ubuntu-meta in artful unapproved queue... please reject the older two, the most recent one combines the other two. https://launchpad.net/ubuntu/artful/+queue?queue_state=1&queue_text=ubuntu-meta [11:47] -queuebot:#ubuntu-release- Unapproved: rejected ubuntu-meta [source] (artful-proposed) [1.404.1] [11:47] -queuebot:#ubuntu-release- Unapproved: rejected ubuntu-meta [source] (artful-proposed) [1.404.1] [11:47] xnox, ^ [11:50] tah [12:37] anyone willing to review libdrm, llvm-5, wayland-protocols on the xenial queue? [12:48] all have been on the x-swat/updates ppa for some weeks now together with new mesa, no complaints so far [12:54] -queuebot:#ubuntu-release- Unapproved: libvirt (xenial-proposed/main) [1.3.1-1ubuntu10.14 => 1.3.1-1ubuntu10.15] (ubuntu-server, virt) [12:55] -queuebot:#ubuntu-release- Unapproved: libvirt (zesty-proposed/main) [2.5.0-3ubuntu5.5 => 2.5.0-3ubuntu5.6] (ubuntu-server, virt) [13:00] -queuebot:#ubuntu-release- Unapproved: rejected ubuntu-fan [source] (xenial-proposed) [0.12.6~16.04.1] [13:14] tjaalton, those are pretty large, remind me why we are updating them [13:16] apw: because of the hwe-16.04 backport stack [13:16] llvm-5 is a no-brainer as it's a new package [13:17] for xenial [13:18] * sil2100 puts his SRU hat on [13:20] these are dependencies for mesa. xserver needs the newer wayland-protocols as well [13:21] w-p just adds new bits [13:21] libdrm is similar [13:21] oSoMoN: hey! Why is libreoffice FTBFS on amd64 for bionic? [13:22] oSoMoN: I was looking at 5.4.2 for artful but I guess we first would need to get it building correctly on bionic, possibly making sure it's migratable [13:24] libreoffice is being handled [13:25] there is a fixed version ready for upload which is being reviewed atm, should be uploaded today [13:32] sil2100, what seb_128 said [13:32] oSoMoN, seb128: thanks [13:34] -queuebot:#ubuntu-release- Unapproved: accepted lxcfs [source] (artful-proposed) [2.0.8-0ubuntu1~17.10.1] [13:50] -queuebot:#ubuntu-release- Unapproved: accepted lxcfs [source] (zesty-proposed) [2.0.8-0ubuntu1~17.04.1] [14:03] hi, I just realized that on bug 1657256 the trusty SRU upload is still in unapproved [14:03] bug 1657256 in percona-xtradb-cluster-5.5 (Ubuntu Trusty) "Percona crashes when doing a a 'larger' update" [High,In progress] https://launchpad.net/bugs/1657256 [14:04] -queuebot:#ubuntu-release- Unapproved: accepted lxcfs [source] (xenial-proposed) [2.0.8-0ubuntu1~16.04.1] [14:04] It seems my question I had this morning with the dnsmasq SRU repeats itself, but is on this upload something special that didn't let it through yet? === heber_ is now known as heber [15:11] -queuebot:#ubuntu-release- Unapproved: grub2 (bionic-proposed/main) [2.02~beta3-4ubuntu7 => 2.02-2ubuntu1] (core) [15:13] -queuebot:#ubuntu-release- Unapproved: accepted ubuntu-fan [source] (artful-proposed) [0.12.7~17.10.1] [15:13] -queuebot:#ubuntu-release- Unapproved: ubuntu-advantage-tools (trusty-proposed/main) [10ubuntu0.14.04.1 => 10ubuntu0.14.04.2] (no packageset) [15:16] -queuebot:#ubuntu-release- Unapproved: accepted ubuntu-fan [source] (zesty-proposed) [0.12.7~17.04.1] [15:21] sil2100, I have re-upload for trusty, and andreas did the verification-done for the existing -proposed pkg ... what can be done to potential consider an early release like previous talked ? [15:22] potentially ^ [15:22] -queuebot:#ubuntu-release- Unapproved: accepted ubuntu-fan [source] (xenial-proposed) [0.12.7~16.04.1] [15:23] slashd: ok, let me review the trusty upload first, then let's think about doing an early release tomorrow - would that be feasible? [15:24] Since I'm driving this, I'd prefer to do it around the start of my day since then I can react in case something bad happens (which shouldn't in this case) [15:24] sil2100, sound good to me [15:24] sil2100, is targetting tomorrow would be reasonnable for you or too soon ? [15:25] slashd: oh, hm, could you re-upload ubuntu-advantage-tools with -v? So that the .changes file has both 0.14.04.1 and 0.14.04.2 included? [15:25] The trusty upload I mean [15:26] -queuebot:#ubuntu-release- Unapproved: grub2 (bionic-proposed/main) [2.02~beta3-4ubuntu7 => 2.02-2ubuntu1] (core) [15:27] e.g. source built with -v10 [15:31] sil2100, not sure to follow what you mean ? I doing the patch on top of what it is in -proposed "10ubuntu0.14.04.1" [15:32] slashd: -v indicates where the changes should start from (the last uploaded version to the corresponding series) [15:32] slashd: see `man dpkg-genchanges` [15:32] slashd: the -v flag is passed via dpkg-buildpackage [15:32] nacc ok [15:34] nacc thanks [15:41] slashd: yeah, as nacc said [15:42] slashd: since without indicating which 'previous' version it is, the new upload would not consider the bugs we fixed in the previous one [15:42] i.e. the SRU report wouldn't ask for verification of the previous bug [15:48] sil2100, ack will do soon [15:56] sil2100, done [15:57] -queuebot:#ubuntu-release- Unapproved: ubuntu-advantage-tools (trusty-proposed/main) [10ubuntu0.14.04.1 => 10ubuntu0.14.04.2] (no packageset) [15:59] -queuebot:#ubuntu-release- Unapproved: accepted mutter [source] (artful-proposed) [3.26.2-0ubuntu0.1] [16:06] slashd: thanks! [16:07] slashd: yeah, this one looks much better [16:07] sil2100, cool [16:07] -queuebot:#ubuntu-release- Unapproved: rejected ubuntu-advantage-tools [source] (trusty-proposed) [10ubuntu0.14.04.2] [16:10] slashd: accepted! [16:10] (and the older one rejected) [16:10] -queuebot:#ubuntu-release- Unapproved: accepted ubuntu-advantage-tools [source] (trusty-proposed) [10ubuntu0.14.04.2] [16:10] sil2100, thanks very appreciated [16:14] apw: I see some fwupdate binaries pending in bionic UNAPPROVED - can I accept those? They're related to the 9-3 upload [16:16] sil2100: good day, looks like RAOF isn't available today for SRU updates. Any idea who I should ping today for the cloud-init 17.1.27 uploads queued for Xenial Zesty and Artful [16:16] blackboxsw: hey! I can take care of those in a minute - I see two uploads of cloud-init in the queue, should I reject the earlier one? [16:18] thanks sil2100 if you need reject 17.1.25 that sounds good. We might be able to just replace it with 17.1.27. I think that's what happened with the previous SRU attempt (a replace instead of reject old and approve new). [16:19] in either case 17.1.27 includes all changes that were in 17.1.25 plus a couple fixes [16:21] gah haskell [16:21] so, now I don't understand https://people.canonical.com/~ubuntu-archive/transitions/html/html/ghc.html - these newly-'bad' packages all seem to install fine for me [16:22] anyone know what I'm missing? [16:25] -queuebot:#ubuntu-release- Unapproved: accepted fwupdate [amd64] (bionic-proposed) [9-3] [16:25] -queuebot:#ubuntu-release- Unapproved: accepted fwupdate [armhf] (bionic-proposed) [9-3] [16:25] -queuebot:#ubuntu-release- Unapproved: accepted fwupdate [arm64] (bionic-proposed) [9-3] [16:25] -queuebot:#ubuntu-release- Unapproved: accepted fwupdate [i386] (bionic-proposed) [9-3] [16:26] it's worth waiting a publisher cycle or two in case it's temporary? [16:29] cjwatson: it's lasted two runs of the tracker, which last updated 10 minutes ago [16:29] otherwise I would just assume it was temporary, yeah [16:31] I don't see anything obvious, although there are possible discrepancies because I think the tracker considers only the newest version of a package across (bionic, bionic-proposed) whereas e.g. chdist will consider the union [16:31] -queuebot:#ubuntu-release- New binary: linux-signed [amd64] (artful-proposed/main) [4.13.0-17.20] (core, kernel) [16:32] cjwatson: indeed, but I spot tested a couple of these and installing them pulled in no haskell-related deps, and a dist-upgrade afterwards didn't want to update or remove anything [16:33] -queuebot:#ubuntu-release- New binary: linux-signed-hwe-edge [amd64] (xenial-proposed/main) [4.13.0-17.20~16.04.1] (kernel) [16:33] e.g. xmobar just pulls in a font and a bunch of C libs [16:34] -queuebot:#ubuntu-release- Unapproved: accepted fwupdate [source] (artful-proposed) [9-2ubuntu0.17.10.1] [16:35] I don't have good answers; it may be worth staring at britney output I guess [16:37] -queuebot:#ubuntu-release- Unapproved: fwupdate (artful-proposed/main) [9-2ubuntu0.17.10.1 => 9-2ubuntu0.17.10.1] (core) [16:38] -queuebot:#ubuntu-release- Unapproved: fwupdate (artful-proposed/main) [9-2ubuntu0.17.10.1 => 9-2ubuntu0.17.10.1] (core) [16:39] yeah.. britney output is a bit confused because of blocked autopkgtests [16:39] -queuebot:#ubuntu-release- Unapproved: fwupdate (artful-proposed/main) [9-2ubuntu0.17.10.1 => 9-2ubuntu0.17.10.1] (core) [16:39] -queuebot:#ubuntu-release- Unapproved: fwupdate (artful-proposed/main) [9-2ubuntu0.17.10.1 => 9-2ubuntu0.17.10.1] (core) [16:40] -queuebot:#ubuntu-release- Unapproved: accepted fwupdate-signed [source] (artful-proposed) [1.14ubuntu0.1] [16:43] -queuebot:#ubuntu-release- Unapproved: fwupdate-signed (artful-proposed/main) [1.14 => 1.14ubuntu0.2] (core) [16:43] ^ mis-click, re-uploading and re-approving a newer fwupdate-signed, this one was missing a bug number [16:44] ETOOMANYBUTTONSONKEYBOARD [16:46] -queuebot:#ubuntu-release- Unapproved: accepted fwupdate-signed [source] (artful-proposed) [1.14ubuntu0.2] [16:46] * ogra_ hands sil2100 https://img-9gag-fun.9cache.com/photo/azVr4Vm_700b.jpg [16:46] heh, those misses are sucky [16:47] ogra_: that's 2 buttons too many [16:47] lol [16:47] blackboxsw: so, back to cloud-init, in this case I'll reject the previous uploads in the queue, since there are two from what I see [16:47] And review the latter [16:47] sounds excellent thx sil2100 [16:48] -queuebot:#ubuntu-release- Unapproved: rejected cloud-init [source] (artful-proposed) [17.1-27-geb292c18-0ubuntu1~17.10.1] [17:03] -queuebot:#ubuntu-release- Unapproved: accepted cloud-init [source] (artful-proposed) [17.1-27-geb292c18-0ubuntu1~17.10.1] [17:06] -queuebot:#ubuntu-release- Unapproved: rejected cloud-init [source] (zesty-proposed) [17.1-27-geb292c18-0ubuntu1~17.04.1] [17:07] -queuebot:#ubuntu-release- Unapproved: accepted cloud-init [source] (zesty-proposed) [17.1-27-geb292c18-0ubuntu1~17.04.1] [17:13] woot! thanks sil2100 [17:13] -queuebot:#ubuntu-release- New binary: armadillo [ppc64el] (bionic-proposed/universe) [1:8.200.0+dfsg-3] (kubuntu) [17:14] -queuebot:#ubuntu-release- New binary: armadillo [i386] (bionic-proposed/universe) [1:8.200.0+dfsg-3] (kubuntu) [17:15] -queuebot:#ubuntu-release- New binary: armadillo [amd64] (bionic-proposed/universe) [1:8.200.0+dfsg-3] (kubuntu) [17:15] -queuebot:#ubuntu-release- New binary: armadillo [s390x] (bionic-proposed/universe) [1:8.200.0+dfsg-3] (kubuntu) [17:16] -queuebot:#ubuntu-release- Unapproved: rejected cloud-init [source] (xenial-proposed) [17.1-27-geb292c18-0ubuntu1~16.04.1] [17:16] -queuebot:#ubuntu-release- New: accepted linux-signed-hwe-edge [amd64] (xenial-proposed) [4.13.0-17.20~16.04.1] [17:17] -queuebot:#ubuntu-release- Unapproved: accepted cloud-init [source] (xenial-proposed) [17.1-27-geb292c18-0ubuntu1~16.04.1] [17:18] blackboxsw: all done, yw! [17:18] -queuebot:#ubuntu-release- New binary: armadillo [arm64] (bionic-proposed/universe) [1:8.200.0+dfsg-3] (kubuntu) [17:18] -queuebot:#ubuntu-release- New binary: armadillo [armhf] (bionic-proposed/universe) [1:8.200.0+dfsg-3] (kubuntu) [17:25] Laney: have you filed a bug report yet on qemu or kernel or talked to the Server or Kernel teams about lcy01? [17:29] hi slangasek, I'm writing a bug report against linux now - was trying to reproduce it on my machine here (failed to do so) [17:29] ok [17:29] I've just poked Server Team privately [17:30] k [17:30] though I could've poked both cpaelzer and rharper here as it turns out [17:30] would be good to sort out lcy01 creds with whoever is going to look [17:30] s/with/for/ [17:30] best case, it's already a known issue and they point us at what needs to be fixed in the ocata cloud archive [17:32] So, the reproducer here should be xenial host, xenial kernel octa qemu backport, reboot artful or bionic image over and over? [17:32] I have a laptop I could maybe sacrifice to that later. [17:33] If the only place we can reproduce it is in a cloud region, that's going to suck. [17:36] I always provoked it on the 1st reboot of an instance [17:36] Not saying it doesn't happen on the n>1th time, but my recipe was - create new instance, reboot it, see what happens, destroy it [17:38] -queuebot:#ubuntu-release- New: accepted armadillo [amd64] (bionic-proposed) [1:8.200.0+dfsg-3] [17:38] -queuebot:#ubuntu-release- New: accepted armadillo [armhf] (bionic-proposed) [1:8.200.0+dfsg-3] [17:38] -queuebot:#ubuntu-release- New: accepted armadillo [ppc64el] (bionic-proposed) [1:8.200.0+dfsg-3] [17:38] -queuebot:#ubuntu-release- New: accepted armadillo [arm64] (bionic-proposed) [1:8.200.0+dfsg-3] [17:38] -queuebot:#ubuntu-release- New: accepted armadillo [s390x] (bionic-proposed) [1:8.200.0+dfsg-3] [17:38] -queuebot:#ubuntu-release- New: accepted armadillo [i386] (bionic-proposed) [1:8.200.0+dfsg-3] [17:41] Laney: But never on first boot? [17:42] Laney: That would seem to imply that version of qemu has a soft reset bug. [17:42] (And you'd see the error on n>1 reboots) [17:42] the first boot has always worked [17:43] well, don't know, I rebooted the first bunch of instances several times and never saw it other than the first time [17:43] but it's racy so could be bad luck [17:43] Well, it's going to rely on memory/register/device state before the reboot, I imagine. [17:43] start a bazillion, do something to them, delete them is easy in the cloud :-) [17:43] So, a bit of luck until one can figure out what needs perturbing. [17:43] (Assuming it's a soft reset bug) [17:45] Also less likely to be a kernel bug, given it never breaks on first boot. The kernel can't be held responsible if the hardware comes back "wrong". [17:46] Unless some workaround for wrongness got dropped. [17:50] well I tried xenial and all of those worked so that might suggest the guest's software is involved somehow [17:50] bug #1730717 anyway [17:50] bug 1730717 in qemu-kvm (Ubuntu) "Some VMs fail to reboot with "watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [systemd:1]"" [Undecided,New] https://launchpad.net/bugs/1730717 [17:51] Laney: Oh! It's literally "on reboot", not post-reboot. [17:52] Laney: Yeah, that's quite likely to be a kernel bug. [17:52] It never shuts down [17:52] Laney: Somehow, I was misreading all of this to be "on next boot, the system is buggered". [17:53] Laney: Might be helpful to confirm with IS what the '-cpu' setting on those VMs is. I vaguely recall it was something like core2duo, but knowing might help with debugging/reproduction. [17:59] infinity: I'm disappearing in a minute, would appreciate handing over overnight duty [18:00] if it helps 0a6a536b-79bf-4651-8177-7dddea9e3408 is a bad instance that I left running [18:00] Laney: Tell me what "overnight duty" entails. [18:00] Laney: Like, is there a way you're monitoring instance death and resetting, so we don't run out of workers? [18:00] Getting any information out of IS [18:00] Oh, sure, I can also attempt to track the RT and bug. [18:00] nah, I'll make the reset job run hourly so we limp along a bit [18:00] and generally working the bug with server or whatever [18:03] yeah OK so every hour the dead workers should revive (and die again eventually, but maybe they'll pick up some jobs before they do so) [18:04] nighty night [18:34] Laney: I've seen this on artful releases in our curtin vmtest runs on a jenkins node running xenial, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1722311 [18:34] Launchpad bug 1722311 in linux (Ubuntu) "softlockups/rcu sched" [Medium,Incomplete] [18:36] currently on the qemu host, I'm applying zone_reclaim_mode=1 and a few other mm knobs to be aggressive at flushing filesystem pagecache as we only get the softlockups when we're memory poor (most memory is in cache of some form and the flushing of dirty pages gets to a peak) [18:38] rharper: Definitely looks like the same bug. [18:39] we run the same tests across all of the releases and only the Artful kernels fall over [18:39] so I tend to agree that it's related to the guest kernel [18:39] but there is some relationship to the host w.r.t cache pressure as we can avoid it by aggressively writing out dirty pages [18:40] we've not yet tried xenial-hwe or xenial-hwe-edge (we're on xenial ga kernel) [18:42] rharper: Can you confirm that this requires a newer qemu to reproduce as well? [18:42] rharper: (In our case, it seems to only happen on hosts with the Octa cloud archive enabled) [18:42] this reproduces on stock qemu from xenial [18:42] on our host [18:43] Huh. Kay. [18:43] softlocks (other than outright bugs in the guest kernel) are almost always host scheduling/resource issues [18:44] it's possible for the kvmclock that there is a regression on the touch to the softlockup watchdog; I've not tried switching guest clock sources yet [18:44] $ apt-cache policy qemu-system-x86 [18:44] qemu-system-x86: [18:44] Installed: 1:2.5+dfsg-5ubuntu10.16 [18:45] so, host scheduling/resource issue would align with the "why is this livefs taking 4x as long to build" issue I saw on lcy01, but I don't know if there have been other observed problems [18:45] is the virt host running xenial or something else? [18:45] xenial. [18:46] indeed [18:46] + octa cloud archive. [18:46] and ga kernel ? [18:46] infinity: y [18:46] Not sure about the host kernel. I think xenial GA, though. [18:46] yeah [18:46] so, this bug is mentioned with OOM, but the "workaround" is what I applied and helped, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1655842 [18:46] Launchpad bug 1655842 in linux-aws (Ubuntu Xenial) ""Out of memory" errors after upgrade to 4.4.0-59" [Undecided,Confirmed] [18:47] folks seeing some issue w.r.t cache flushing on xenial ga kernels; I've not bisected host kernels [18:47] All very curious. I don't think they added "extra" overcommit when reprovisioning these same compute nodes. [18:47] The only difference is that it's a newer openstack. [18:47] and updated host kernel [18:47] I suspect [18:47] a month ago or so, we didn't see these artful tests falling over [18:47] so, either a newer artful kernel in the guest, or an update to the xenial-ga kernel [18:48] Also, see https://bugs.launchpad.net/ubuntu/+source/linux-hwe/+bug/1713751 [18:48] Launchpad bug 1713751 in linux-hwe (Ubuntu) "soft lockup / stall on CPU when shutting down with hwe 4.10 kernel " [Undecided,New] [18:48] Which is on real hardware, not VMs. [18:48] And looks oddly similar. [18:48] interesting [18:48] * rharper loads up the bug [18:56] * infinity pack up his shotgun and goes taco hunting. [19:01] -queuebot:#ubuntu-release- Unapproved: aufs-tools (artful-proposed/universe) [1:4.1+20161219-1 => 1:4.1+20161219-1ubuntu0.1] (no packageset) [19:29] -queuebot:#ubuntu-release- Unapproved: aufs-tools (artful-proposed/universe) [1:4.1+20161219-1 => 1:4.1+20161219-1ubuntu0.1] (no packageset) [19:29] -queuebot:#ubuntu-release- New binary: ldc [ppc64el] (bionic-proposed/universe) [1:1.5.0-1] (no packageset) [19:30] -queuebot:#ubuntu-release- Unapproved: rejected aufs-tools [source] (artful-proposed) [1:4.1+20161219-1ubuntu0.1] [19:31] -queuebot:#ubuntu-release- Unapproved: accepted aufs-tools [source] (artful-proposed) [1:4.1+20161219-1ubuntu0.1] [19:38] -queuebot:#ubuntu-release- New binary: ldc [amd64] (bionic-proposed/universe) [1:1.5.0-1] (no packageset) [19:38] -queuebot:#ubuntu-release- New binary: ldc [i386] (bionic-proposed/universe) [1:1.5.0-1] (no packageset) [19:57] -queuebot:#ubuntu-release- Unapproved: accepted grub2 [amd64] (bionic-proposed) [2.02-2ubuntu1] [19:57] -queuebot:#ubuntu-release- Unapproved: accepted grub2 [arm64] (bionic-proposed) [2.02-2ubuntu1] [20:17] -queuebot:#ubuntu-release- Unapproved: resolvconf (trusty-proposed/main) [1.69ubuntu1.1 => 1.69ubuntu1.2] (core) [21:45] -queuebot:#ubuntu-release- New binary: ldc [armhf] (bionic-proposed/universe) [1:1.5.0-1] (no packageset) [21:48] please accept ldc, so I can make it transition tomorrow, thanks! [22:18] -queuebot:#ubuntu-release- Unapproved: rejected libseccomp [source] (xenial-proposed) [2.3.1-2.1ubuntu2~16.04.1] === NCommander is now known as KD2JRT [23:18] xnox: have any details on why libseccomp was rejected? ^ [23:26] tyhicks: it's going to go into the security queue instead or something [23:29] hmm [23:29] I don't understand why that's needed but I can help with that once someone explains the reasoning to me [23:29] thanks mwhudson [23:33] tyhicks: blame infinity [23:44] ah, he's got his own super powers and can handle it himself [23:44] * tyhicks trusts that he knows what he's doing :) [23:45] I see it in the security-proposed PPA [23:45] thanks