/srv/irclogs.ubuntu.com/2021/04/30/#ubuntu-release.txt

-queuebot:#ubuntu-release- Unapproved: python-pip (focal-proposed/universe) [20.0.2-5ubuntu1.3 => 20.0.2-5ubuntu1.4] (no packageset)00:23
-queuebot:#ubuntu-release- Unapproved: ubuntu-settings (focal-proposed/main) [20.04.5 => 20.04.6] (ubuntu-desktop)07:14
-queuebot:#ubuntu-release- Unapproved: rejected evolution-data-server [source] (focal-proposed) [3.36.5-0ubuntu1]07:24
-queuebot:#ubuntu-release- Unapproved: evolution-data-server (focal-proposed/main) [3.36.4-0ubuntu1 => 3.36.5-0ubuntu1] (i386-whitelist, ubuntu-desktop)07:25
juliankLaney: it seems a lot of xenial arm64 tests fail with grub-install: error: relocation 0x113 is not implemented yet; wondering if we should work around that somehow, e.g. by uninstalling grub08:40
juliankLaney: hang on, they worked a day earlier08:41
Laneyis that a fail fail or a temp fail?08:42
juliankthose seem to fail fail as /unknown08:43
Laneyhmm08:45
seb128bdmurray, hey, could you check what's going on there? lp 1923267 the gjs update was accepted as a SRU in hirsute-proposed but seems it vanished from there?08:47
ubot3Launchpad bug 1923267 in gjs (Ubuntu Hirsute) "Gjs promises on Gio.File operations don't work anymore after upgrading libglib2.0-0 2.67.5-2 to 2.68.0-1 (and indicator-multiload app broke)" [High, Fix Committed] https://launchpad.net/bugs/192326708:47
seb128it's in impish only now?!08:47
seb128hum08:48
seb128Deleted on 2021-04-23 by Iain Lane08:48
seb128moved to impish-proposed08:48
juliankLaney: I see in journal also that "Flavor m1.large could not be found." on  bos01 for arm64, causing a bunch of errors08:48
seb128Laney, ^ could you check what happened to that SRU?08:48
Laneyseb128: Copy it back. We don't have a good way to tell which pre release uploads should be kept, so it's guess work and sometimes we guess wrong08:49
Laneyjuliank: umm yeah, can you check that? if true, get IS to create it08:49
juliankon it08:49
seb128Laney, so pocket copy from impish?08:49
Laneylooks like bos* just went down though08:49
Laneyseb128: yeah08:49
juliankopenstack flavor list is either slow or timing out08:49
Laneyyeah, wait for the cloud to come back08:50
Laney:/08:50
seb128Laney, you don't have a command line in your backlog handy to share so I don't have to figure out the parameters to use again? ;)08:50
Laneyseb128: I shared it to Rik_Mills the other day so it's probably in here, one second08:50
julianksweet waiting time08:50
Laney#ubuntu-release.log-20210427.gz:26/04 10:24:01 <Laney> copy-package --from=ubuntu --to=ubuntu --include-binaries --from-suite=impish-proposed --to-suite=hirsute-proposed gwenview08:51
seb128Laney, thanks!08:51
Laneyjuliank: follow along in the outage channel!08:51
seb128I wonder also if there is some sort of query I could do to get a list of potential other SRUs we lost08:51
Laneyyou could probably find the last proposed-migration run before I did that in https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses/08:52
seb128Laney, thanks08:52
seb128Laney, last question on that, any idea of tweaks we could do the process or archive opening to avoid it happens again in futur cycles?08:53
LaneyI don't know, we have a trello board 'devel unapproved', maybe that could be used somehow, but not super sure atm08:54
rbasakIn that particular case, was there a bug reference with a verification-{needed,done}-hirsute tag? Would filtering on that have helped, if we had appropriate scripting to do it?08:54
Laneyhttps://paste.ubuntu.com/p/kpZGJVVjFp/08:54
Laneythat's the list by the way08:54
seb128rbasak, yes we have a bug with proper tags08:54
rbasakI think maybe the difficulty could be reduced to things still in unapproved then maybe?08:55
seb128bug #192326708:55
ubot3Bug 1923267 in gjs (Ubuntu Hirsute) "Gjs promises on Gio.File operations don't work anymore after upgrading libglib2.0-0 2.67.5-2 to 2.68.0-1 (and indicator-multiload app broke)" [High, Fix Committed] https://launchpad.net/bugs/192326708:55
LaneyI'm sure scripting could help, but I was starting from dists/08:55
-queuebot:#ubuntu-release- Unapproved: gjs (hirsute-proposed/main) [1.67.2-2ubuntu1 => 1.67.2-2ubuntu2] (desktop-core, desktop-extra, i386-whitelist, mozilla) (sync)08:55
Laneyso LP bugs aren't really super easily available there08:55
rbasakWhat do you mean by "starting from dists/"?08:56
LaneyThat's what I used to list -proposed (grep-dctrl). That would need to be input to another script or something which can figure out whatever it is there is to figure out.08:57
LaneyOr instead the script could list proposed itself. Just saying, I wasn't using the LP API at all to do this.08:57
rbasakI see - thanks.08:57
Laneyjuliank: it was funny, I was literally in the middle of typing shell commands on the lxd-armhf machines when they went down08:59
Laneydon't think I've caught an outage that live before :D08:59
juliankLaney: heh09:00
Laneyis there a way to send ~. to an inner ssh btw?09:02
rbasak<Enter>~~.?09:02
rbasakIOW, ~~ is the escape for ~09:02
* Laney tries09:02
Laneyyes! thanks09:02
Laneyjuliank: back09:06
* juliank lists flavors09:06
juliankkeystone still seems down09:07
Laneynah09:07
Laneyit's probably that bug where if you source one file first and then another you can't use the cloud09:07
Laneyquit and go back in09:07
juliankLaney: I only ever sourced that one file :D09:09
juliankLaney: now it works09:10
juliankm1.large is there09:10
juliankmaybe it was the cloud going down that caused the error09:10
Laneyanything is possible in the wonderful world of openstack09:10
Laneygot to say it would have been surprising if the flavor didn't exist09:10
juliankLaney: I wonder if I should try reinstating the port cleanup in cleanup-instances09:14
juliankWe only kept copy-security-group, and it should be enough - at least if the worker restarts09:14
juliankmaybe the worker should purge all ports in its security group before starting a job too09:15
juliankI think workers are down,  autopkgtest@bos01-arm64-6.service: Job autopkgtest@bos01-arm64-6.service/start failed with result 'dependency'.09:18
Laneyyeah I'm running the maintenance thingy09:18
LaneyI dunno if the port thing we're seeing atm is stray ports, did you confirm that?09:18
LaneyI think it's something quotaish on our new users09:18
juliankLaney: I haven't checked, need to write a script and run it I suppose09:19
Laneythere's still something wrong in Boston09:40
LaneyI'm trying to get IS to help out09:40
Laney(without much success so far, it must be said)10:00
juliankI'm digging into bootloader stuff :/10:02
xnoxthe bootloader stuff https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/192674810:19
ubot3Launchpad bug 1926748 in grub2 (Ubuntu Xenial) "regression in xenial updates - grub2 cannot handle new arm64 relocations" [Undecided, New]10:19
Laneyhuh10:24
xnoxhoping that one patch will do it; cause the other relocations need 13 patches =(11:01
xnoxregression-updates can we please downgrade grub2 grub2-unsigned grub-signed from xenial-updates to xenial-proposed please? as otherwise upgrades are broken on aws ec2 arm64.11:06
xnox!regression-alert11:06
xnox!regression11:06
* xnox is not sure what the bot command is11:06
xnoxLaney:  ^^^^11:07
sil2100Ok, on it11:09
xnoxsil2100:  tah.11:10
sil2100(I don't think we have highlights on this btw. ;) )11:10
* sil2100 feels like he is reverting stuff all the time this week11:10
xnoxsil2100:  and previous grub2 & grub2-signed need to put back into xenial-updates, as you obviously know.11:10
-queuebot:#ubuntu-release- Unapproved: ceph (groovy-proposed/main) [15.2.11-0ubuntu0.20.10.1 => 15.2.11-0ubuntu0.20.10.2] (ubuntu-desktop, ubuntu-server)11:13
Ukikiexnox: That first one used to be a factoid, but someone deleted it.11:15
xnoxUkikie:  cool. thanks.11:15
jamespagehi ubuntu-sru - I've just uploaded new ceph versions to focal and groovy proposed - one of the cherry picks included in the last uploads for SRU did not actually fix a UX issue11:16
-queuebot:#ubuntu-release- Unapproved: ceph (focal-proposed/main) [15.2.11-0ubuntu0.20.04.1 => 15.2.11-0ubuntu0.20.04.2] (desktop-core, ubuntu-server)11:16
xnoxsil2100: will go get lunch/coffee and then will test proposed fixed, hopefully we might be able to have a fixed up grub2 sru for xenial-proposed for arm64 at least.11:16
jamespageso I've elected to drop that patch from the other SRU's in flight so we can move forwards11:16
sil2100ACK o/11:18
-queuebot:#ubuntu-release- Unapproved: grub2 (xenial-updates/main) [2.02~beta2-36ubuntu3.31 => 2.02~beta2-36ubuntu3.29] (core) (sync)11:22
-queuebot:#ubuntu-release- Unapproved: accepted grub2 [sync] (xenial-updates) [2.02~beta2-36ubuntu3.29]11:23
sil2100xnox: ok, old versions deleted from -updates, still present in -proposed, old versions copied over to -updates (hopefully the correct ones)11:23
sil2100I hope it's all good regarding signing bits11:24
sil2100I'll go grab a bite now as well and then take a look if all is good11:24
xnoxalso confused how come we have released xenial ahead of bionic.11:26
xnoxand will double check that bionic is not affected otherwise.11:26
sil2100xnox: the reason was that xenial goes ESM today11:49
sil2100I mean, Steve probably should have released both bionic and xenial at this point, but now I'm happy he didn't11:50
sil2100Since we have the liberty of just checking this without rush11:50
xnoxi feel like i don't even know how computers work => none of the commands on xenial work for me, and i have to look up old syntax for all the things. Even like debuild.12:22
-queuebot:#ubuntu-release- Unapproved: accepted google-cloud-sdk [sync] (xenial-release) [335.0.0-0ubuntu1~16.04.1]12:34
-queuebot:#ubuntu-release- Unapproved: grub2 (xenial-proposed/main) [2.02~beta2-36ubuntu3.31 => 2.02~beta2-36ubuntu3.32] (core)12:38
xnoxsil2100:  juliank: please review http://launchpadlibrarian.net/536308504/grub2_2.02~beta2-36ubuntu3.31_2.02~beta2-36ubuntu3.32.diff.gz on amazon ec2 arm64 xenial instance that fixes things for me.12:52
sil2100xnox: looking13:48
sil2100xnox: ok, I don't know much about these parts of grub2 but upload seems sane - from the SRU POV I guess I'd like if it was built with -v to include the previous version and hm, possible since this is to go to -security, maybe actually building it in a security pocket first? Or something13:59
xnoxsil2100:  -v & security is a must14:00
xnoxsil2100: please reject.14:00
xnoxsil2100:  re what this does "it reimplements binutils in grub basically, because it is like lets take these elf objects which are grub .mod modules and lets assemble a core.efi pe/coff binary out of it" it is black magic in binutils, and more so, in grub2. So this is an upstream cherrypick.14:01
xnoxsil2100:  there is also shim signing to review in impish unapproved https://launchpad.net/ubuntu/impish/+queue?queue_state=1&queue_text=shim14:08
xnoxsil2100:  the shim-signed with MS signature is already in impish-proposed (ftbfs, as it is waiting on shim signing to be approved)14:09
xnoxsil2100:  rebuilding grub in bileto14:15
juliankxnox: do you have a ppa? I spent some time building myself to realize that I don't have a local reproducer for the linking issue14:18
juliankxnox: but I can rerun failed autopkgtest in the cloud14:18
Laneywhen did the autopkgtests catch it? like when it needed dist-upgrading in the base system?14:18
juliank[NEEDSBUILD] Needs building Cancel build14:18
Laneynot in an actual triggered test, right?14:18
juliankStart in 2 hours14:18
juliankLaney: all the tests triggered by python-apt yesterday-today/last night14:19
juliankLaney: They all upgrade first, get grub 2.04 which was released yesterday, and then fail14:19
Laneyyeah14:19
Laneyso I guess the question is how can we catch this in a real test triggered by the actual upload14:20
Laneyso it doesn't get released14:20
xnoxjuliank:  it's building14:21
juliankLaney: grub must get an autopkgtest, consisting just of Depends14:22
xnoxjuliank:  i built in amazon ec2 xenial arm64 instance; and tested there.14:22
juliankLaney: and true as test case14:22
xnoxnormal apt upgrade failed to configure grub; but manual call to ./grub-install from the built package worked14:22
juliankLaney: Ok, optimally you do want to run grub-install :D14:22
juliankBut in the cloud as we have it now, it will fail either way14:23
xnoxbut i guess it will not build on arm64 https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4545/+build/21482739 because all builders are borked with that grub2?14:23
xnoxjuliank:  do you want me to build the package in amazon cloud and give you the debs?14:23
juliankxnox: do the builders build on bare (virtual) metal, and not in a chroot?14:24
xnoxjuliank:  well we reverted grub from xenial-updates so all the vms should be fine now?14:24
juliankIf they build in a chroot, they should be fine14:24
xnoxjuliank:  i think it's chroot, on a VM14:24
juliankthat should be fine either way14:24
xnoxbut like some arm64 builders look dead https://launchpad.net/builders14:24
Laneybos02 is looking really bad from where I'm sitting, independent of the grub2 stuff14:26
xnoxcool.14:28
Laney(and bos01, but that's not relevant for launchpad builds)14:29
juliankfun outages14:29
Laneylike https://ubuntu-release.kpi.ubuntu.com/d/76Oe_0-Gz/autopkgtest?orgId=1 check the middle two, doesn't look great]14:32
Laneyseparately there's a lot of instance reboot failures in this latest set of linux tests14:36
Laneyin impish14:36
sil2100xnox: looking!14:37
-queuebot:#ubuntu-release- Unapproved: rejected grub2 [source] (xenial-proposed) [2.02~beta2-36ubuntu3.32]14:37
Laneyeverything is baddddddd14:42
sil2100Everything is terrible14:47
Laneylet's all moan together14:48
julianksigh madness14:51
* xnox has "Everything is awesome" in my head now, as sung by cjwatson14:56
Laneychecking if something in impish-proposed is bad14:57
Laneythere's lots of suspicious things in there :D14:57
Laneytoday I am thankful for lxd VMs14:58
Laneyhttps://paste.ubuntu.com/p/5VZVg7F8z4/15:08
xnoxsil2100:  so it looks like my grub2 xenial sru will not build today, so we shall come back to it like on tuesday. (bank holiday here in the uk)15:13
xnoxsil2100: vorlon: or if https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4545/+packages finishes building feel free to binary copy that into xenial-proposed.15:14
* xnox checks if i uploaded it with the right -v, i think i did not.15:14
xnoxhttps://launchpadlibrarian.net/536318662/grub2_2.02~beta2-36ubuntu3.32_source.changes => i did not15:14
* xnox facepalm15:14
vorlonxnox: why are you building this in bileto instead of just uploading to -proposed?15:16
vorlonbinary copies from bileto are annoying15:17
vorlonxnox: and you say "if it finishes building" but it's ftbfs already on arm64 and ppc64el?15:18
Laneyvorlon: I think your initramfs-tools is bad, it seems to break booting somehow https://paste.ubuntu.com/p/5VZVg7F8z4/15:19
vorlonLaney: caught by autopkgtests?15:19
Laneyvorlon: sort of, yes15:21
vorlonk15:21
vorlonI'll look into it, thanks15:21
LaneyI noticed that all-proposed runs are failing to reboot15:21
vorlonah15:21
Laneyand then bisected until I found that15:21
Laneyso I think we should drop it in the meantime15:21
vorlonack15:21
Laneyyou want to do that, or me?15:21
vorlonI will15:22
Laneycheers15:22
Laneyalso, separately, I've seen that the boston clouds are kinda unhappy, lots of keystone errors and timeouts in lxd15:22
LaneyI've not managed to get IS to bite properly yet (think it's affecting buildds too)15:23
Laneyso if you want to track that it would be helpful; I'll be EODing in 90 minuts15:23
xnoxvorlon:  ftbfs without any build-logs..... cause bos02 is sad15:25
vorlonLaney: ugh it's the damn +x bit being dropped from init again by MoM; I remembered this issue and tried to check for missing +x bits but apparently I mistakenly looked only in the debian/ subdir15:29
Laneyurgh :(15:30
vorlonreuploaded15:30
Laneycheers for the quick fix15:30
vorlonxnox: and now https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4545/+packages has disappeared?15:30
xnoxvorlon:  because wrong -v, so uploaded to 454615:32
vorloninstead of just uploading to the archive15:32
xnoxhttps://launchpadlibrarian.net/536325290/grub2_2.02~beta2-36ubuntu3.32_source.changes => looks better15:32
xnoxvorlon:  must be built against -security pocket15:32
vorlonhmm ok15:33
vorlonbuild scores adjusted15:33
vorlonhmm why are the -proposed links all broken on https://people.canonical.com/~ubuntu-archive/pending-sru.html15:36
sil2100seb128 merged some changes that I think could have caused this15:57
vorlonsil2100: yeah I don't see anything in seb128's diff that obviously explains; I guess I'll dig16:50
LaneyI'm out - hopefully the clouds get fixed so we can make queue progress over the weekend ...17:02
seb128vorlon, I can check if it's my changes17:43
seb128vorlon, ah, I see a typo in there, let me try with that fixed17:44
seb128            sru_item['url'] = rpkg['proposed'] = pkgurl + rpkg['proposed']17:45
seb128vorlon, sil2100, sorry for the inconvenience, I pushed a fix17:50
-queuebot:#ubuntu-release- Unapproved: software-properties (focal-proposed/main) [0.98.9.4 => 0.98.9.5] (core)18:19
-queuebot:#ubuntu-release- New binary: resource-agents [amd64] (impish-proposed/main) [1:4.7.0-1ubuntu2] (ubuntu-server)18:41
-queuebot:#ubuntu-release- New binary: resource-agents [s390x] (impish-proposed/main) [1:4.7.0-1ubuntu2] (ubuntu-server)18:41
-queuebot:#ubuntu-release- New binary: resource-agents [arm64] (impish-proposed/main) [1:4.7.0-1ubuntu2] (ubuntu-server)18:43
-queuebot:#ubuntu-release- New binary: resource-agents [ppc64el] (impish-proposed/main) [1:4.7.0-1ubuntu2] (ubuntu-server)18:43
-queuebot:#ubuntu-release- New binary: resource-agents [armhf] (impish-proposed/main) [1:4.7.0-1ubuntu2] (ubuntu-server)18:45
vorlonseb128: aha, cheers!18:57
-queuebot:#ubuntu-release- Unapproved: python-pip (bionic-proposed/universe) [9.0.1-2.3~ubuntu1.18.04.4 => 9.0.1-2.3~ubuntu1.18.04.5] (no packageset)18:58
-queuebot:#ubuntu-release- New binary: resource-agents [riscv64] (impish-proposed/main) [1:4.7.0-1ubuntu2] (ubuntu-server)19:18
-queuebot:#ubuntu-release- Unapproved: grub2 (xenial-proposed/main) [2.02~beta2-36ubuntu3.31 => 2.02~beta2-36ubuntu3.32] (core) (sync)19:23
kanashiroI'd appreciate if any AA could process resource-agents in impish NEW queue :)19:33
-queuebot:#ubuntu-release- Unapproved: accepted grub2 [sync] (xenial-proposed) [2.02~beta2-36ubuntu3.32]19:36
vorlonxnox: ^^ accepted19:36
vorlonxnox: how did shim migrate to impish release without corresponding shim-signed?20:11
vorlonhmm I guess we no longer have a dep on unsigned shim package, right20:13
vorlonxnox: why did https://launchpad.net/ubuntu/+source/shim-signed/1.47/+build/21482148 download from impish-proposed but https://launchpad.net/ubuntu/+source/shim-signed/1.47/+build/21482149 download from impish?!20:20
xnoxvorlon:  because it checks apt where the candidate for shim is at, and downloads from there. now that shim is not in -proposed anymore it downloads from release21:49
xnoxvorlon:  but what is more confusing is that https://launchpad.net/ubuntu/+source/shim-signed/1.47/+build/21482149 downloads current and gets ubuntu1; wheres when i download the same url from ftpmaster.internal from people.canonical.com i get ubuntu2 signed tarball.21:50
xnoxvorlon:  it's as if arm64 bos02 builders see some different ftpmaster.internal?!21:50
xnoxi don't know if I should ping #launchpad people but not over labor weekened :/21:50
xnoxlet me try #is21:50
vorlonyes, I don't know what's going on there >_<21:52
-queuebot:#ubuntu-release- Unapproved: nvidia-graphics-drivers-340 (focal-proposed/restricted) [340.108-0ubuntu5.20.04.1 => 340.108-0ubuntu5.20.04.2] (kernel-dkms, ubuntu-desktop) (sync)22:01
-queuebot:#ubuntu-release- Unapproved: oss4 (focal-proposed/universe) [4.2-build2010-5ubuntu6~20.04.1 => 4.2-build2010-5ubuntu6~20.04.2] (kernel-dkms) (sync)22:01
-queuebot:#ubuntu-release- Unapproved: r8168 (focal-proposed/multiverse) [8.048.00-1ubuntu0.20.04.1 => 8.048.00-1ubuntu0.20.04.2] (kernel-dkms) (sync)22:02
-queuebot:#ubuntu-release- Unapproved: sysdig (focal-proposed/universe) [0.26.4-1ubuntu0.2 => 0.26.4-1ubuntu0.3] (kernel-dkms) (sync)22:02
-queuebot:#ubuntu-release- Unapproved: virtualbox-hwe (focal-proposed/multiverse) [6.1.16-dfsg-6ubuntu1.20.04.1 => 6.1.16-dfsg-6ubuntu1.20.04.2] (kernel-dkms) (sync)22:02
-queuebot:#ubuntu-release- Unapproved: zfs-linux (focal-proposed/main) [0.8.3-1ubuntu12.8 => 0.8.3-1ubuntu12.9] (core, kernel-dkms) (sync)22:02
-queuebot:#ubuntu-release- Unapproved: rtl8812au (focal-proposed/universe) [4.3.8.12175.20140902+dfsg-0ubuntu13~20.04.1 => 4.3.8.12175.20140902+dfsg-0ubuntu13~20.04.2] (kernel-dkms) (sync)22:02
-queuebot:#ubuntu-release- Unapproved: virtualbox (focal-proposed/multiverse) [6.1.16-dfsg-6~ubuntu1.20.04.1 => 6.1.16-dfsg-6~ubuntu1.20.04.2] (kernel-dkms, ubuntu-cloud) (sync)22:02
-queuebot:#ubuntu-release- Unapproved: v4l2loopback (focal-proposed/universe) [0.12.3-1ubuntu0.3 => 0.12.3-1ubuntu0.4] (kernel-dkms) (sync)22:02
-queuebot:#ubuntu-release- Unapproved: grub2 (xenial-proposed/main) [2.02~beta2-36ubuntu3.32 => 2.02~beta2-36ubuntu3.32] (core) (sync)22:13
xnoxall of the above is from me =)22:13
vorlonkanashiro_: so your resource-agents package split mentions resource-agents-common as a base for both supported and unsupported agents, but I only see an addition of resource-agents-supported, no corresponding -unsupported package?22:22
vorlonthus I'm confused at the split22:22
vorlonalso resource-agents-supported breaks/replaces old resource-agents, but new resource-agents doesn't depend on resource-agents-supported, so this agent (there seems to be only one) disappears on upgrade?22:24
sergiodjvorlon: not Lucas, but I was involved in the review of this change.  there's no corresponding -unsupported package; it is implicit that the resource-agents will contain everything that is not in the -supported package, thus being the unsupported version22:34
vorlonweird but ok?22:35
sergiodjyeah :)22:35
vorlonsergiodj: in that case, why does the unsupported package not depend on the supported one, for upgrades?22:35
sergiodjvorlon: I don't know offhand, sorry22:36
sergiodjI mean, they're two separate packages, and the user can install just one or the other22:37
sergiodjbut maybe I misunderstood your question22:39

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!