| mwhudson | um all glibc triggered autopkgtests failing isn't good? | 00:17 |
|---|---|---|
| vorlon | doko: why are you rebuilding perl for libcrypt1? It's not viable to have to rebuild every package which links against libcrypt to add a new dependency on libcrypt1 | 02:10 |
| vorlon | doko: and the plan that was discussed earlier on irc didn't mention perl, only base-files + libc + libxcrypt | 02:10 |
| mwhudson | vorlon: is there any pump turning i can do to help with glibc? | 02:27 |
| lotuspsychje | good morning to all | 04:27 |
| lotuspsychje | we have a user reporting the same symptons of bug #1860622 in #ubuntu anyone knows whats the status on that? | 04:28 |
| ubot5 | bug 1860622 in pulseaudio (Ubuntu) "Dependency error installing pulseaudio-esound-compat" [Undecided,Fix committed] https://launchpad.net/bugs/1860622 | 04:28 |
| vorlon | mwhudson: I don't know, really; per above I'm not clear on why what's being uploaded is being uploaded | 04:42 |
| -queuebot:#ubuntu-release- New binary: ubuntustudio-look [amd64] (focal-proposed/universe) [0.69] (ubuntustudio) | 05:02 | |
| Eickmeyer | ^ Separated out a metadata file. Probably just needs a rubber stamp. | 05:34 |
| -queuebot:#ubuntu-release- New binary: heimdall-flash [amd64] (focal-proposed/none) [1.4.2+dfsg-1] (no packageset) | 05:43 | |
| -queuebot:#ubuntu-release- New binary: heimdall-flash [s390x] (focal-proposed/none) [1.4.2+dfsg-1] (no packageset) | 05:43 | |
| -queuebot:#ubuntu-release- New binary: heimdall-flash [ppc64el] (focal-proposed/universe) [1.4.2+dfsg-1] (no packageset) | 05:44 | |
| -queuebot:#ubuntu-release- New binary: heimdall-flash [arm64] (focal-proposed/universe) [1.4.2+dfsg-1] (no packageset) | 05:45 | |
| -queuebot:#ubuntu-release- New binary: heimdall-flash [armhf] (focal-proposed/universe) [1.4.2+dfsg-1] (no packageset) | 05:45 | |
| vorlon | I'm not sure what was going on with the autopkgtest queues being stuck, but it seems to be resolved now, (coincidentally?) after I reset the workers | 05:48 |
| -queuebot:#ubuntu-release- New: accepted heimdall-flash [arm64] (focal-proposed) [1.4.2+dfsg-1] | 05:49 | |
| -queuebot:#ubuntu-release- New: accepted heimdall-flash [armhf] (focal-proposed) [1.4.2+dfsg-1] | 05:49 | |
| -queuebot:#ubuntu-release- New: accepted heimdall-flash [amd64] (focal-proposed) [1.4.2+dfsg-1] | 05:50 | |
| -queuebot:#ubuntu-release- New: accepted heimdall-flash [s390x] (focal-proposed) [1.4.2+dfsg-1] | 05:50 | |
| -queuebot:#ubuntu-release- New: accepted qgis [armhf] (focal-proposed) [3.10.3+dfsg-1~exp2] | 05:50 | |
| -queuebot:#ubuntu-release- New: accepted qgis [s390x] (focal-proposed) [3.10.3+dfsg-1~exp2] | 05:50 | |
| -queuebot:#ubuntu-release- New: accepted heimdall-flash [ppc64el] (focal-proposed) [1.4.2+dfsg-1] | 05:50 | |
| -queuebot:#ubuntu-release- New: accepted qgis [ppc64el] (focal-proposed) [3.10.3+dfsg-1~exp2] | 05:50 | |
| -queuebot:#ubuntu-release- New: accepted qgis [amd64] (focal-proposed) [3.10.3+dfsg-1~exp2] | 05:50 | |
| mwhudson | vorlon: hm ok i guess i'll see what doko says when he's around | 06:58 |
| xnox | vorlon: we had all autopkgtest fail, with perl failing to find libcrypt whilst upgrading the autopkgtest workers. | 09:32 |
| xnox | he did say he will look more into that | 09:32 |
| xnox | you can read up backscrol either here or on #ubuntu-devel | 09:32 |
| xnox | let me find a sample log | 09:32 |
| xnox | https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/amd64/3/389-ds-base/20200306_162651_f9385@/log.gz | 09:33 |
| xnox | Setting up libc6:amd64 (2.31-0ubuntu1) ... | 09:33 |
| xnox | /usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory | 09:33 |
| xnox | that was all of glibc triggered autopkgtests, with all of them failing as "unknown" | 09:33 |
| xnox | due to above | 09:33 |
| mwhudson | so everything needs retriggering with perl&glibc from proposed? | 09:34 |
| mwhudson | i notice deboostrap succeeds today | 09:34 |
| xnox | i am assuming somebody already retriggered | 09:39 |
| xnox | because glibc results are half green already | 09:39 |
| xnox | hm yet the queues are empty | 09:40 |
| xnox | and there is lots of red | 09:40 |
| xnox | i'll speak to doko | 09:40 |
| doko | vorlon: perl-base and login are the only packages that pre-depend on libcrypt1, not base-files. and I didn't want to re-introduce another pre dependency | 09:55 |
| doko | no, I didn't retrigger | 10:00 |
| -queuebot:#ubuntu-release- Unapproved: accepted alsa-lib [source] (eoan-proposed) [1.1.9-0ubuntu1.2] | 10:09 | |
| -queuebot:#ubuntu-release- Unapproved: accepted alsa-lib [source] (bionic-proposed) [1.1.3-5ubuntu0.4] | 10:15 | |
| -queuebot:#ubuntu-release- Unapproved: rejected snapd-glib [source] (eoan-proposed) [1.56-0ubuntu0.19.10.0] | 10:20 | |
| xnox | retriggering things triggered by glibc, on amd64 i386 ppc64el s390x, with extra trigger added for perl | 10:26 |
| xnox | 8k of tests | 10:26 |
| xnox | submission in progress | 10:44 |
| -queuebot:#ubuntu-release- Unapproved: rejected ubuntu-dev-tools [source] (bionic-proposed) [0.175ubuntu1~18.04.1] | 10:56 | |
| -queuebot:#ubuntu-release- New binary: linux-signed-oem [amd64] (bionic-proposed/main) [4.15.0-1076.86] (kernel) | 11:22 | |
| doko | vorlon: please could you add llvm-toolchain-10 to the i386 whitelist? | 11:50 |
| doko | xnox: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/ppc64el/c/cross-toolchain-base/20200309_104230_89307@/log.gz | 12:31 |
| doko | Calculating upgrade... | 12:36 |
| doko | The following packages will be upgraded: | 12:36 |
| doko | base-files libc-bin libc6 libcrypt1 libperl5.30 locales login passwd perl | 12:36 |
| doko | perl-base perl-modules-5.30 | 12:36 |
| doko | libcrypt1 is already installed (at least it says it's getting upgraded) | 12:36 |
| -queuebot:#ubuntu-release- New: accepted linux-signed-oem [amd64] (bionic-proposed) [4.15.0-1076.86] | 12:38 | |
| doko | but the crypto library is removed because of the bad breaks/replaces in the old libcrypt1 on the image | 12:38 |
| xnox | doko: thus trigger should include libxcrypt?! | 12:39 |
| xnox | doko: i am wondering if libcrypt1 in shlibs should generate a dep on "libc6 (>= 2.31), libcrypt1 (>= 1:4....) | 12:40 |
| xnox | doko: because really, using libcrypt1 adds an extra requirement on a higher libc6 | 12:40 |
| doko | maybe, but this doesn't help with the current situation | 12:41 |
| xnox | argh | 12:41 |
| xnox | doko: libc6 in it's postinst is using perl. but it is not guaranteed that the new libcrypt1/perl are _configured_ at that point, only unpacked. | 12:42 |
| xnox | doko: i wonder if libc6 maintainer scripts must LD_PRELOAD libcrypt1 or like actually predepend on it. | 12:42 |
| xnox | doko: because debconf | 12:42 |
| doko | no, libcrypt1 is installed before | 12:42 |
| doko | it get's removed by removing libc6 2.30 and installing 2.31 | 12:43 |
| xnox | doko: it really didn't.... | 12:43 |
| xnox | https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/ppc64el/c/cross-toolchain-base/20200309_104230_89307@/log.gz | 12:43 |
| xnox | says that it will be upgraded | 12:43 |
| xnox | does the VM image have the old libcrypt1 without fixed up breaks? | 12:43 |
| doko | yes | 12:43 |
| xnox | in that case we need a new libc6 upload which conflicts with broken libcrypt1 versions | 12:45 |
| xnox | (those without ubuntu-specific breaks/replaces) | 12:45 |
| xnox | or like rebuild our VM images without libcrypt1 | 12:45 |
| xnox | Laney: we have removed libcrypt1 from focal-release; yet our VM images appear to have it. Can we please rebuild autopkgtest VMs without libcrypt1? | 12:46 |
| xnox | or maybe juliank ^^^^ | 12:46 |
| * xnox looks at VM images | 12:46 | |
| doko | xnox: can you confirm the libcrypt1 version on the images? | 12:47 |
| xnox | good point | 12:47 |
| xnox | if only i remembered which machines i need to ssh into | 12:48 |
| doko | ahh, libxcrypt is entirely removed from the release pocket | 12:49 |
| doko | yes, then rebuilding of the autopkg images should solve that issue | 12:49 |
| juliank | xnox: it iss wendigo | 12:50 |
| juliank | and I guess we can do that? | 12:50 |
| xnox | juliank: yeap, ssh-ed in | 12:50 |
| xnox | based on admin docs | 12:51 |
| -queuebot:#ubuntu-release- Unapproved: stress-ng (bionic-proposed/universe) [0.09.25-1ubuntu6 => 0.09.25-1ubuntu8] (no packageset) | 12:52 | |
| doko | and probably it's better to rebuild the buildd images as well | 12:52 |
| doko | cjwatson: ^^^ is this something you could help with? | 12:52 |
| juliank | xnox: So shall I do RELEASE=focal autopkgtest-cloud/tools/build-adt-image-all-clouds autopkgtest/setup-commands/setup-testbed ? | 12:53 |
| xnox | juliank: i want to check that the current images are bad. I am inside the worker, looking up nova creds to download the image | 12:53 |
| xnox | juliank: but then yes something like that. | 12:54 |
| xnox | will need to be done | 12:54 |
| xnox | downloading image, it is taking forever | 12:59 |
| doko | to fix an upgrade from a bad libcrypt1, maybe move libcrypt.so.1 from /lib/<multiarch> in libc6's preinst, run ldconfig, and then remove it in the postinst? | 13:14 |
| xnox | doko: scary, as it is using perl in preinst | 13:16 |
| doko | maybe cp instead of mv | 13:18 |
| xnox | doko: juliank: ii libcrypt1:ppc64el 1:4.4.10-10ubuntu1 is installed in the images | 13:20 |
| xnox | which is bad | 13:20 |
| juliank | ack | 13:21 |
| xnox | we must rebuild images without it | 13:21 |
| juliank | i can start that | 13:21 |
| xnox | juliank: i'm loging out of everything, as i have never rebuild the images from scratch | 13:21 |
| xnox | hm | 13:21 |
| xnox | i should have killed my instance first! | 13:21 |
| xnox | and did that now | 13:22 |
| xnox | doko: so i guess all of my test submissions will still be broken | 13:22 |
| xnox | doko: and tests should just pass, with glibc trigger alone. | 13:23 |
| xnox | after we rebuild our images | 13:23 |
| juliank | xnox: should I remove your test submissions | 13:23 |
| juliank | ? | 13:23 |
| xnox | juliank: i think so, yes please. | 13:23 |
| xnox | everything from xnox submitter | 13:23 |
| juliank | doing so | 13:24 |
| xnox | make it so! | 13:24 |
| doko | yes | 13:24 |
| * xnox giggles | 13:24 | |
| juliank | hmm | 13:25 |
| juliank | I'm hitting a bug somewhere | 13:25 |
| juliank | xnox: also to rebuild the images from scratch, we must hack around a bit, as otherwise, it upgrades the old ones, sigh | 13:26 |
| xnox | juliank: i thought there is a parameter to that script to start off from i.e. eoan stable | 13:26 |
| juliank | xnox: ah really bootstrap from eoan | 13:27 |
| juliank | yes | 13:27 |
| xnox | juliank: yes, use eoan as base image, or the prestine import of focal (the CPC cloud images, which are good) | 13:27 |
| doko | libc6: conficts libcrypt1 (<< 1:4.4.10-10ubuntu3), libcrypt1: breaks libc6 (<< 2.31) | 13:27 |
| doko | juliank: ^^^ how should apt resolve that? | 13:28 |
| xnox | doko: ok great, we just need to eradicate the broken ubuntu1 from the image then. | 13:28 |
| xnox | doko: assuming the breaks libc6 <<2.31 is in ubuntu3, yes. | 13:28 |
| doko | I think the conflct adds an impossible solution to make the upgrade | 13:28 |
| doko | s/solution/requirement/ | 13:28 |
| xnox | doko: let's rebuild autopkgtest VM images; then run tests, I think we will be fine then. | 13:29 |
| xnox | doko: without any extra libxcrypt nor glibc uploads. | 13:29 |
| juliank | I'm investigating | 13:29 |
| xnox | juliank: from bash_history, i see that cpc stock image is often uploaded by hand with glance image-create | 13:30 |
| xnox | however i see that was last done with cosmic | 13:30 |
| xnox | juliank: ./create-nova-image-new-release focal ubuntu/ubuntu-eoan-daily-amd64-server-20191122-disk1.img testbed-`hostname` autopkgtest/setup-commands/setup-testbed | 13:31 |
| juliank | xnox: I want to check if we have older focal images, how many, etc | 13:31 |
| xnox | was run before | 13:31 |
| xnox | juliank: so i see only adt customized images and no stock imports of focal at all. | 13:32 |
| xnox | juliank: https://paste.ubuntu.com/p/TY62S4Fgtj/ | 13:32 |
| xnox | 08/09 adt builds of arm64, ppc64el, s390x without any stock focal images | 13:33 |
| xnox | and those contain broken libxcrypt1 | 13:33 |
| juliank | xnox: i see ubuntu/ubuntu-focal-daily-s390x-server-20200305-disk1.img | 13:34 |
| xnox | juliank: yet there are autosync images of eoan https://paste.ubuntu.com/p/B26J3r8dDY/ | 13:34 |
| juliank | on bos01-s390x | 13:34 |
| xnox | juliank: i'm connected to bos02-ppc64el env | 13:34 |
| juliank | ack | 13:34 |
| xnox | do you have the 0307 eoan daily builds? | 13:34 |
| xnox | i.e. auto-sync/ubuntu-eoan-daily-s390x-server-20200307-disk1.img | 13:34 |
| xnox | i guess we need a common base image that exists in all regions for consistency | 13:35 |
| xnox | or upload a new one | 13:35 |
| juliank | ubuntu/ubuntu-eoan-daily-s390x-server-20200307-disk1.img | 13:35 |
| juliank | yes | 13:35 |
| juliank | xnox: it does not matter much where we start i guess | 13:36 |
| xnox | juliank: i don't see that one in bos02 =( | 13:36 |
| xnox | unless the prefix doesn't matter? | 13:36 |
| xnox | auto-sync/ vs ubuntu/ | 13:36 |
| doko | I'm removing the build-essential upload from -proposed again, it's not needed, because libc6-dev depends on libcrypt-dev | 13:36 |
| juliank | xnox: prefix does nto matter | 13:36 |
| xnox | then rebuild everything against that? but i somehow confused how it will be extrapolated per arch =) cause it should be -$ARCH- right? hopefully the scripts know how to do that | 13:37 |
| xnox | product_name | com.ubuntu.cloud.daily:server:19.10:ppc64el | 13:38 |
| xnox | anyway, i logout! best of luck to you =) | 13:39 |
| juliank | xnox: so it will automatically rebuild with the latest focal base image next round, when was the last save image? | 13:39 |
| juliank | I feel like making sure it was built before March 4 should work | 13:39 |
| xnox | march 4 cpc upstream image is good | 13:40 |
| xnox | adt/* images from 5/6/7/8/9 march are all bad | 13:40 |
| -queuebot:#ubuntu-release- Unapproved: accepted collectd [source] (bionic-proposed) [5.7.2-2ubuntu1.2] | 13:41 | |
| xnox | juliank: or you can modify the rebuild script to temproarly do " apt remove libcrypt1" | 13:41 |
| xnox | juliank: because it is unreferenced in the image today. | 13:41 |
| juliank | xnox: hmm | 13:41 |
| xnox | (release pocket) | 13:41 |
| doko | but remember to remove that line later ... | 13:42 |
| juliank | I just ignore grep -v 20200306 | grep -v 20200307 | grep -v 20200308 | grep -v 20200309 | 13:42 |
| xnox | it used to be, but not anymore. I.e. bad base-files & bad libcrypt1 migrated, which ended up on the image (and persist), but now have been removed from focal-release | 13:42 |
| juliank | Then we start from 2020-03-06 base images or earlier | 13:42 |
| xnox | juliank: more. | 13:42 |
| xnox | juliank: because bad libcrypt1 migrated on the 3rd of march? | 13:43 |
| xnox | https://launchpad.net/ubuntu/+source/libxcrypt/+publishinghistory | 13:43 |
| xnox | 2020-03-03 09:08:17 GMTSupersededFocalreleaseuniverselibs1:4.4.10-10 | 13:43 |
| juliank | ubuntu2 is the one we are worried about | 13:43 |
| xnox | is when the very bad ubuntu1 landed in the image | 13:43 |
| juliank | ubuntu1 did not have breaks/replaces? | 13:43 |
| xnox | juliank: no, ubuntu2 is the first good one | 13:43 |
| juliank | it was just a no-change rebuild | 13:43 |
| xnox | juliank: but ubuntu2 is not installable with libc6 from release pocket. | 13:44 |
| xnox | juliank: ubuntu1 is installable with libc6 in the release pocket, which is bad. | 13:44 |
| juliank | OK, let's just apt purge libcrypt1 | 13:44 |
| doko | xnox: no, the Debian imports are already broken | 13:44 |
| xnox | doko: right, ubuntu1 is the last bad one =) | 13:44 |
| doko | ahh, ok | 13:44 |
| xnox | ubuntu2 is the first good one | 13:44 |
| doko | yes | 13:45 |
| xnox | juliank: but apt purge libcrypt1 might not work, if base-files is too new in the image. because it depends on libcrypt1. So maybe you need to purge libcrypt1 and downgrade base-files, which hopefully apt will figure out.... | 13:45 |
| doko | if it helps, remove base-files from -proposed | 13:46 |
| xnox | juliank: i still feel somehow rebootstrapping adt images from eoan might be better. | 13:47 |
| juliank | xnox: ah | 13:48 |
| juliank | xnox: hmm, it's running now | 13:48 |
| xnox | juliank: ack | 13:48 |
| xnox | let's see if it works | 13:48 |
| juliank | I see E: Unable to locate package libcrypt1 | 13:48 |
| juliank | glib-networking : Depends: gsettings-desktop-schemas but it is not going to be installed | 13:49 |
| juliank | ppc64el | 13:49 |
| juliank | yikes | 13:49 |
| juliank | probably need to really start from eoan again | 13:49 |
| juliank | in any case, it will either produce new images without libcrypt1 or fail | 13:49 |
| juliank | probably should have deleted other 20200309 images so I can know if I finished a new one | 13:50 |
| juliank | some images are good already, and some fail to build | 13:53 |
| juliank | xnox: Rebootstrapping from eoan | 13:56 |
| juliank | xnox: I added a safety check to setup-testbed so it cant't build new images that end up with libcrypt1 installed | 13:58 |
| xnox | juliank: ok | 13:58 |
| xnox | juliank: we will need to remove that safety check after we have good images; because it will become safe after glibc migrates in focal | 13:58 |
| juliank | xnox: yup | 13:58 |
| xnox | (or if images with -proposed are built) | 13:58 |
| juliank | ok I have a syntax error somewhere | 14:00 |
| juliank | take #3 | 14:02 |
| juliank | Apologies for the inconvenience | 14:04 |
| doko | juliank: are the images now updated? | 14:35 |
| juliank | let's check | 14:35 |
| juliank | doko: I don't think so | 14:36 |
| juliank | bos02-arm64 succeeded | 14:37 |
| juliank | but the rest got confused | 14:37 |
| juliank | and spawned multiple instances with the same id | 14:37 |
| juliank | gotta try that again | 14:37 |
| juliank | First gotta delete all the images | 14:37 |
| juliank | s/images/servers/ | 14:37 |
| juliank | doko: this is going to take a few more ages | 14:39 |
| juliank | I'm deleting all servers now | 14:41 |
| juliank | it's slow | 14:41 |
| juliank | I'll also delete the images afterwards, then it should be easy to see | 14:43 |
| juliank | doko: Also, we might be running out of resources on arm, so I'm not super confident it will work | 14:49 |
| juliank | Anyway, deleted all adt-prepare servers and deleting images right now | 14:49 |
| juliank | and then we still need to do the same thing on lxd | 14:50 |
| juliank | this is only arm64+ppc64l+s390x+amd64+i386 | 14:50 |
| juliank | not armhf | 14:50 |
| seb128 | someone there should perhaps send an email to ubuntu-devel@ letting people know that the autopkgtest are out of order on focal and being work on but it's taking a bit | 14:50 |
| juliank | I don't think that's necessary, what are they going to do differently? | 14:51 |
| seb128 | you are arguing that service outage is not worth communicating about? | 14:53 |
| xnox | juliank: i didn't see lxd images to be impacted, but maybe i am wrong | 14:54 |
| juliank | xnox: I think they rebuild from cloud image today | 14:55 |
| juliank | xnox: I'm launcing on | 14:55 |
| juliank | * launching one | 14:55 |
| juliank | lgtm | 14:55 |
| juliank | this is slow | 14:56 |
| juliank | ugh | 14:56 |
| juliank | seb128: i think it's fairly self-evident | 14:56 |
| juliank | seb128: but meh | 14:58 |
| juliank | seb128: sent one | 14:58 |
| seb128 | juliank, it's also self-evident when e.g gmail is down, wonder why google bother communicating about outdates :p | 14:58 |
| seb128 | juliank, thanks | 14:58 |
| juliank | meh | 14:58 |
| juliank | reddit is down all the time and does not tell people about it | 14:58 |
| juliank | autopkgtest is not down per se, just tests fail | 14:59 |
| seb128 | juliank, even when ti's self evident that a piece of infra is down is not evident that it has been reported to the right people so if only that an announce has as effect to don't have 15 different people spending time looking at who to ping and bothering all #is or whatever | 14:59 |
| juliank | seb128: but it's been broken for a week, so fairly diminishing returns by now | 15:00 |
| juliank | :D | 15:00 |
| seb128 | since friday no? | 15:00 |
| seb128 | which means a w.e with a good part of the active contributors out because they don't work on w.e/were travelling back from a sprint | 15:01 |
| juliank | people say the libcrypt1 on march 03 4? was broken too | 15:01 |
| juliank | i don't have followed this closely | 15:01 |
| seb128 | anyway, thanks for sending an email | 15:01 |
| juliank | cloud-init has broken depends, so lcy01 and lgw01 are not building afaict | 15:03 |
| juliank | not sure if the other ones work | 15:04 |
| juliank | they're still doing stuff | 15:04 |
| juliank | but it's possible we're not getting images for everything | 15:04 |
| juliank | hmm | 15:09 |
| juliank | it was trying to build i386 images | 15:09 |
| juliank | I don't know why | 15:09 |
| juliank | ppc64el and s390x are done | 15:09 |
| juliank | should I just completely purge the queue? | 15:13 |
| juliank | I think britney will rerequest stuff if needed? | 15:13 |
| juliank | but not sure | 15:13 |
| juliank | xnox: I only purged your stuff from one queue and forgot the others... | 15:13 |
| juliank | ok, so how do I get amd64 images now? | 15:14 |
| juliank | vorlon: Do we actually still use i386 images? I don't think so, and they fail to build now that we don't upgrade the last one | 15:17 |
| juliank | xnox, doko arm64, ppc64el, s390x are done; building amd64 noew | 15:23 |
| juliank | *now | 15:23 |
| juliank | it's running in a tmux | 15:25 |
| juliank | it's still running | 15:32 |
| juliank | doko, xnox we should be live again | 15:35 |
| juliank | Laney: We might get terrible error messages tomorrow | 15:38 |
| juliank | Laney: I published new images upgraded from eoan that work (have no libcrypt1), but nightly will try to rebuild with latest focal daily - if that ends up containing libcrypt1, setup-testbed will fail its safety check i added | 15:39 |
| juliank | JFYI | 15:39 |
| xnox | juliank: coool | 15:44 |
| xnox | juliank: let me retry just a couple of packages to see if they work fine | 15:44 |
| vorlon | locutus_: was there an FFe bug for this qgis sync starting a transition? | 16:02 |
| vorlon | doko: perl-base and login pre-depending on libcrypt1 doesn't appear to solve the problem of the library previously provided by libc6 moving to a different package. Why did you decide it was not correct to have libc6 pre-depend on libcrypt1? | 16:04 |
| vorlon | juliank: the i386 images are used in launchpad. | 16:05 |
| juliank | vorlon: the i386 adt/ images is what I'm talking about | 16:06 |
| vorlon | juliank: ok. those we aren't using on focal, no | 16:06 |
| juliank | right we should stop building them | 16:06 |
| juliank | gotta add a test for that | 16:06 |
| juliank | aka if [ $arch = i366 -a $RELEASE = focal ]; then continue; fi | 16:07 |
| -queuebot:#ubuntu-release- Packageset: Removed libmng from i386-whitelist in focal | 16:32 | |
| -queuebot:#ubuntu-release- Packageset: Removed qt4-x11 from i386-whitelist in focal | 16:32 | |
| -queuebot:#ubuntu-release- Packageset: Added llvm-toolchain-10 to i386-whitelist in focal | 16:32 | |
| -queuebot:#ubuntu-release- Packageset: Added setuptools to i386-whitelist in focal | 16:32 | |
| locutus_ | vorlon, no transition... | 16:33 |
| locutus_ | I don't remember the rationale for syncing it, let me check | 16:35 |
| vorlon | locutus_: libqgis-server3.4.15 -> libqgis-server3.10.3 is this not a transition? | 16:35 |
| locutus_ | but no reverse-deps... | 16:35 |
| vorlon | et al | 16:35 |
| vorlon | ok | 16:35 |
| locutus_ | its somewhat an internal library I would say | 16:35 |
| locutus_ | unfortunately the diff is huge, but I remember some bugfixes we needed, such as compatibility with proj 6.3.1 and something else | 16:36 |
| cjwatson | doko: I'm off today, but ask me tomorrow. buildd images get built automatically, I just need to push them to LP once builds are available | 16:37 |
| doko | vorlon: I didn't decide ... just questioning why the pre-depends wasn't done in unstable | 18:38 |
| vorlon | doko: didn't decide? you were doing the uploads | 18:40 |
| -queuebot:#ubuntu-release- New sync: pdfchain (focal-proposed/primary) [1:0.4.4.2-1] | 19:23 | |
| -queuebot:#ubuntu-release- New: rejected pdfchain [sync] (focal-proposed) [1:0.4.4.2-1] | 19:24 | |
| -queuebot:#ubuntu-release- New source: pdfchain (focal-proposed/primary) [1:0.4.4.2-1build1] | 19:33 | |
| -queuebot:#ubuntu-release- New: rejected pdfchain [source] (focal-proposed) [1:0.4.4.2-1build1] | 19:34 | |
| -queuebot:#ubuntu-release- New binary: pdfchain [amd64] (focal-proposed/universe) [1:0.4.4.2-1build1] (no packageset) | 19:38 | |
| -queuebot:#ubuntu-release- New binary: pdfchain [s390x] (focal-proposed/universe) [1:0.4.4.2-1build1] (no packageset) | 19:38 | |
| -queuebot:#ubuntu-release- New binary: pdfchain [ppc64el] (focal-proposed/universe) [1:0.4.4.2-1build1] (no packageset) | 19:39 | |
| -queuebot:#ubuntu-release- New binary: pdfchain [armhf] (focal-proposed/universe) [1:0.4.4.2-1build1] (no packageset) | 19:42 | |
| -queuebot:#ubuntu-release- New binary: pdfchain [arm64] (focal-proposed/universe) [1:0.4.4.2-1build1] (no packageset) | 19:42 | |
| Eickmeyer | Is there any chance I can get ubuntustudio-look moved out of NEW? I moved a .xml file into its own package to fix an installation bug, so it's super trivial. | 19:59 |
| Eickmeyer | Besides that, we have our new default wallpaper ready to go, so I want to make sure bug 1866584 gets fixed before I upload more changes. | 20:03 |
| ubot5 | bug 1866584 in ubuntustudio-look (Ubuntu) "ubuntustudio-wallpapers duplicates file in ubuntustudio-wallpapers-focal" [Undecided,Fix committed] https://launchpad.net/bugs/1866584 | 20:03 |
| ahasenack | vorlon: hi, I see debian-installer now is blocked on this new src:libxcrypt | 20:15 |
| ahasenack | vorlon: I also saw juliank's email about the autopkgtest images being broken because of it. Is that the reason for the block? | 20:15 |
| vorlon | ahasenack: no, the reason for the block is in scrollback over the weekend, libxcrypt somehow made it into the release pocket without a corresponding libc6 that pre-depends on it, which broke the world | 20:17 |
| vorlon | Eickmeyer: ubuntustudio-wallpapers depends on ubuntustudio-wallpapers-focal; it does not look like a reasonable solution to me to deal with a file conflict between these two packages by introducing yet another binary package for a single data file | 20:20 |
| ahasenack | vorlon: ok, thanks. What's the event that will unbkreak the world and release the lock | 20:21 |
| ahasenack | ? | 20:21 |
| vorlon | ahasenack: I don't know, doko has been laconic in his responses to me about what he's doing and the uploads I've seen don't match what was discussed. | 20:22 |
| vorlon | I *expected* to see a pre-depends from libc6 to libcrypt1, possibly with some manual ldconfig handling in a preinst, and to unblock all of this once glibc is a candidate | 20:23 |
| vorlon | but then I see no-change perl uploads that look to me like they're treating the symptom instead of fixing the bug | 20:23 |
| ahasenack | vorlon: this glibc predates the breakage? https://launchpad.net/ubuntu/+source/glibc/2.31-0ubuntu2 | 20:23 |
| ahasenack | it says "2 days old" | 20:23 |
| vorlon | ahasenack: it appears to include all of the changes that were discussed. but it needs to be a candidate per p-m, and I also want to know why this change wasn't sufficient without there also being a perl upload | 20:25 |
| ahasenack | I see all its reds point at libcrypt.so.1 | 20:25 |
| vorlon | xnox: you've triggered tests with perl+glibc and these passed on amd64 but not other archs, do you know why? | 20:26 |
| juliank | oh I should send an email that the issue is sorted out i supposer | 20:26 |
| vorlon | ah | 20:29 |
| vorlon | so here's the problem | 20:29 |
| vorlon | libc6 preinst is trying to load debconf before it does the ldconfig | 20:29 |
| vorlon | I think we should just move the ldconfig handling to the front of the maintainer script instead. | 20:30 |
| vorlon | glibc 2.31-0ubuntu3 uploaded | 20:39 |
| vorlon | Eickmeyer: I've looked further and I see where usr/share/gnome-background-properties/ubuntustudio-wallpapers.xml is also shipped in e.g. ubuntustudio-wallpapers-disco, so it makes sense why there is a file conflict. However, the actual contents of this file don't make much sense to me in a common data package, because the xml file /references/ the wallpapers from the various per-series binary | 20:54 |
| vorlon | packages, which are not going to be installed. So what do you expect the behavior to be here if someone installs ubuntustudio-wallpapers, then tries to select the disco wallpaper which is not actually installed? | 20:54 |
| -queuebot:#ubuntu-release- New: rejected ubuntustudio-look [amd64] (focal-proposed) [0.69] | 20:57 | |
| Eickmeyer[m] | vorlon: We | 21:04 |
| Eickmeyer[m] | vorlon: We've been including that file in the packages for YEARS. It's the only way to get gnome to see our wallpapers. I don't appreciate the rejection without explanation. | 21:05 |
| Eickmeyer[m] | But, I guess I'll fix the disco wallpapers, if it's showing up there too. | 21:06 |
| vorlon | Eickmeyer[m]: er, I've given you an explanation for the reject. | 21:06 |
| vorlon | you should have per-series .xml files, with no file conflict, unless you can provide an explanation why this is wrong | 21:07 |
| vorlon | because right now, your all-in-one xml file is going to present impossible options to users in the gui by default | 21:07 |
| Eickmeyer | It causes no issues. If the files aren't there, it simply skips over them. | 21:08 |
| vorlon | (unless gnome happens to filter them out, but even then, this is not a good reason to create a separate -data package) | 21:08 |
| Eickmeyer | Gnome filters them out if they're not there. | 21:08 |
| vorlon | ok | 21:08 |
| vorlon | but even so, having ubuntustudio-wallpapers-focal ship ./usr/share/gnome-background-properties/ubuntustudio-wallpapers-focal.xml with correct contents | 21:09 |
| vorlon | and ubuntustudio-wallpapers-eoan ship ./usr/share/gnome-background-properties/ubuntustudio-wallpapers-eoan.xml with correct contents | 21:09 |
| vorlon | is more correct | 21:09 |
| Eickmeyer | I agree, but you are giving a VOLUNTEER that is sick a ton more work to do, even if it's "correct". | 21:09 |
| vorlon | it is the responsibility of the archive admins to manage the namespace of binary packages in Ubuntu | 21:10 |
| Eickmeyer | We've been shipping that all-in-one file for years, but this is the first time you've raised an issue. | 21:10 |
| vorlon | because this is the first time you've asked me to approve a binary package for a single file | 21:10 |
| -queuebot:#ubuntu-release- New: accepted pdfchain [amd64] (focal-proposed) [1:0.4.4.2-1build1] | 21:10 | |
| -queuebot:#ubuntu-release- New: accepted pdfchain [armhf] (focal-proposed) [1:0.4.4.2-1build1] | 21:10 | |
| -queuebot:#ubuntu-release- New: accepted pdfchain [s390x] (focal-proposed) [1:0.4.4.2-1build1] | 21:10 | |
| -queuebot:#ubuntu-release- New: accepted pdfchain [arm64] (focal-proposed) [1:0.4.4.2-1build1] | 21:10 | |
| -queuebot:#ubuntu-release- New: accepted pdfchain [ppc64el] (focal-proposed) [1:0.4.4.2-1build1] | 21:10 | |
| vorlon | I understand that you are a volunteer, but that does not give you a free pass on binary packages that the archive admins disagree with | 21:11 |
| Eickmeyer | Only overly pedantic ones that like to cause issues with Ubuntu Studio over the past years. | 21:11 |
| Eickmeyer | Sorry, I'm gettin out-of-line. | 21:11 |
| Eickmeyer | Just super frustrated. | 21:11 |
| Eickmeyer | And ill. | 21:11 |
| -queuebot:#ubuntu-release- New: accepted python-tinyalign [amd64] (focal-proposed) [0.2-1ubuntu1] | 21:15 | |
| -queuebot:#ubuntu-release- New: accepted python-tinyalign [armhf] (focal-proposed) [0.2-1ubuntu1] | 21:15 | |
| -queuebot:#ubuntu-release- New: accepted python-tinyalign [s390x] (focal-proposed) [0.2-1ubuntu1] | 21:15 | |
| -queuebot:#ubuntu-release- New: accepted python-tinyalign [arm64] (focal-proposed) [0.2-1ubuntu1] | 21:15 | |
| -queuebot:#ubuntu-release- New: accepted python-tinyalign [ppc64el] (focal-proposed) [0.2-1ubuntu1] | 21:15 | |
| tjaalton | I've got a question about how to handle an upgrade issue with a dkms package which was first put in the OEM archive with '-ubuntuN' version, and then backported to bionic with '-0ubuntuN'. This essentially disables upgrades to the bionic archive version | 21:33 |
| tjaalton | so the options are to bump the epoch (yuck), fake the upstream version somehow, or use '-ubuntuN' in the archive version too.. | 21:34 |
| tjaalton | or something else? | 21:34 |
| tjaalton | I'm leaning towards just using NNNN+1-0ubuntuN | 21:35 |
| tjaalton | so appending +1 to the upstream part | 21:37 |
| tjaalton | the package is backport-iwlwifi-dkms, current version in bionic is 7906-0ubuntu4~18.04.1 | 21:38 |
| vorlon | tjaalton: bumping the upstream version seems reasonable to me | 21:54 |
| vorlon | xnox: the answer for why the perl+glibc triggers were failing, btw, is because the perl no-change rebuild is not a correct fix for the problem and does not guarantee correct ordering on upgrade, so it's a crapshoot whether libcrypt.so.1 is actually available by the time perl starts. Correct fix for glibc preinst uploaded. | 22:14 |
| vorlon | xnox: so debian-installer 20101020ubuntu604, you rolled it back to the release pocket version of the kernel? because the kernel team isn't planning to migrate linux anytime soon? | 22:15 |
| RikMills | vorlon: btw, thanks for Qt4 removal! | 22:16 |
| vorlon | RikMills: my pleasure | 22:16 |
| vorlon | and the fact that it waited until heimdall-flash had a new upstream version available in Debian (with a new maintainer) is surely just coincidence | 22:16 |
| RikMills | of course ;) | 22:17 |
| xnox | vorlon: i agree with moving ldconfig to the top; also we had broken libxcrypt (libcrypt1) in autopkgtest VM base imaged => the one that migrated into release pocket, ended up in the autopkgtest base VM images. | 22:30 |
| xnox | vorlon: yes, and i further rolled back -proposed in 606 | 22:30 |
| xnox | vorlon: thus 606 is build against src:linux-5.4 with bind9 without libcrypt1 | 22:30 |
| xnox | vorlon: the reason being is that debian-installer is completely broken in the release pocket it is 9 abi matching src:linux, whereas the latest linux-generic abi is 14 from src:linux-5.4 | 22:31 |
| xnox | vorlon: because the src:linux => src:linux-5.4 migrated without d-i =( | 22:31 |
| mwhudson | xnox: is this why the d-i server CI is busted? | 22:31 |
| mwhudson | or is that something else | 22:31 |
| xnox | mwhudson: yes, it boots -9 abi, tries to install -14 abi modules, which do not work. | 22:32 |
| mwhudson | xnox: ok good | 22:32 |
| xnox | mwhudson: couldn't be bothered to figure out why it didn't try to install -9 abi modules, but i think this is due to provides. | 22:32 |
| xnox | also it was making our server isos really big with two sets of udebs | 22:32 |
| xnox | vorlon: partially it is a problem that src:linux/release is still published in the archive, despite being "superseeded" by src:linux-5.4/release and NBS not catching that. | 22:33 |
| vorlon | xnox: 606> oh. without 604 migrating? | 22:33 |
| vorlon | ok | 22:33 |
| xnox | vorlon: yes because 606 is built in a bileto ppa, against release pocket only, with bind9/bind9-libs copied in from focal-proposed | 22:37 |
| xnox | vorlon: and i fucked up 605 build =) | 22:37 |
| * xnox points at experiment 636 | 22:38 | |
| xnox | vorlon: once 606 migrates, i will upload a clean rebuild of d-i again which will hold up on both kernel being ready and libxcrypt/glibc being ready | 22:38 |
| vorlon | k | 22:38 |
| xnox | vorlon: however i do wonder if i will need a frankenstein build if either kernel or libxcrypt complete first | 22:38 |
| vorlon | why would you? | 22:39 |
| xnox | cause i don't want to tie up linux & libxcrypt transitions together =) | 22:39 |
| vorlon | it shouldn't tie | 22:39 |
| xnox | 14 is quite an old kernel by now. | 22:39 |
| xnox | a rebuild of d-i picks up libcrypt1-udeb dep | 22:39 |
| vorlon | glibc and libxcrypt should be ready to go "soon" | 22:39 |
| xnox | ok | 22:39 |
| * xnox fingerscrossed | 22:39 | |
| vorlon | only one more britney cycle until glibc is built on armh | 22:40 |
| vorlon | f | 22:40 |
| vorlon | and then the autopkgtests should be virtually instantaneous, with no failures | 22:40 |
| * xnox giggles | 22:40 | |
| xnox | so i do hope that d-i & bind9 migrate in two britney runs | 22:40 |
| * xnox sleeps | 23:37 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!