[00:53] <vorlon> -rw-rw-r-- 1 ubuntu-archive ubuntu-archive 2731 May 17  2017 current-crontab
[00:53] <vorlon> .... not so current
[01:20] <vorlon> ok, I've moved the notest britney runs outside the main loop; they will now run at most once per hour
[03:50] -queuebot:#ubuntu-release- Unapproved: software-properties (bionic-proposed/main) [0.96.24.32.17 => 0.96.24.32.18] (desktop-core, ubuntu-server)
[04:22] <blackboxsw> thanks robert-ancell.    tjaalton if you happen to have time tomorrow for SRU upload reviews into -propsed we've uploaded cloud-init for SRU into bionic focal hirsute and impish per https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1949521
[04:22] <ubottu> Launchpad bug 1949521 in cloud-init (Ubuntu Impish) "sru cloud-init (21.3-1 to 21.4-0) Bionic, Focal, Hirsute and Impish" [Undecided, New]
[04:23] <blackboxsw> also robert uploaded a fix for bionic-proposed for software-properties per https://bugs.launchpad.net/ubuntu/+source/software-properties/+bug/1939732
[04:23] <ubottu> Launchpad bug 1939732 in software-properties (Ubuntu Bionic) "report availability of Ubuntu Advantage ESM services on unattached machines" [Undecided, New]
=== cpaelzer_ is now known as cpaelzer
[10:02] <LocutusOfBorg> doko, hello, I pushed a boost1.74 fix for python3.10 and cmake search
[10:03] <LocutusOfBorg> this fixes e.g. some ros-* cmake based programs to build correctly now
[10:03] <LocutusOfBorg> https://launchpad.net/ubuntu/+source/boost1.74/1.74.0-8ubuntu9
[10:03] <LocutusOfBorg> can you please schedule if possible a build retry for reverse-deps once it builds and its published? do you have a list?
[10:31] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-470 [source] (impish-proposed) [470.82.01-0ubuntu0.21.10.1]
[10:40] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-470 [source] (hirsute-proposed) [470.82.01-0ubuntu0.21.04.1]
[10:43] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-470 [source] (focal-proposed) [470.82.01-0ubuntu0.20.04.1]
[10:49] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-470 [source] (bionic-proposed) [470.82.01-0ubuntu0.18.04.1]
[10:50] -queuebot:#ubuntu-release- Unapproved: ubuntu-drivers-common (focal-proposed/main) [1:0.9.0~0.20.04.1 => 1:0.9.0~0.20.04.3] (desktop-core, ubuntu-server)
[11:33] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-460 [source] (impish-proposed) [460.106.00-0ubuntu0.21.10.1]
[11:34] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-460 [source] (hirsute-proposed) [460.106.00-0ubuntu0.21.04.1]
[11:57] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-460 [source] (focal-proposed) [460.106.00-0ubuntu0.20.04.1]
[12:00] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-460 [source] (bionic-proposed) [460.106.00-0ubuntu0.18.04.1]
[12:35] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-450 [source] (impish-proposed) [450.156.00-0ubuntu0.21.10.1]
[12:37] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-450 [source] (hirsute-proposed) [450.156.00-0ubuntu0.21.04.1]
[12:38] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-450 [source] (focal-proposed) [450.156.00-0ubuntu0.20.04.1]
[12:42] -queuebot:#ubuntu-release- Unapproved: accepted libnvidia-nscq-450 [source] (bionic-proposed) [450.156.00-0ubuntu0.18.04.1]
[12:53] -queuebot:#ubuntu-release- Unapproved: gnome-shell-extension-desktop-icons (focal-proposed/main) [20.04.0-3~ubuntu20.04.4 => 20.04.0-3~ubuntu20.04.5] (ubuntu-desktop)
[13:16] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-470 [source] (impish-proposed) [470.82.01-0ubuntu0.21.10.1]
[13:16] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-470 [source] (hirsute-proposed) [470.82.01-0ubuntu0.21.04.1]
[13:18] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-470 [source] (focal-proposed) [470.82.01-0ubuntu0.20.04.1]
[13:22] -queuebot:#ubuntu-release- Unapproved: fabric-manager-450 (bionic-proposed/multiverse) [450.142.00-0ubuntu0.18.04.1 => 450.156.00-0ubuntu0.18.04.1] (no packageset)
[13:22] -queuebot:#ubuntu-release- Unapproved: fabric-manager-450 (focal-proposed/multiverse) [450.142.00-0ubuntu0.20.04.1 => 450.156.00-0ubuntu0.20.04.1] (no packageset)
[13:22] -queuebot:#ubuntu-release- Unapproved: fabric-manager-450 (hirsute-proposed/multiverse) [450.142.00-0ubuntu0.21.04.1 => 450.156.00-0ubuntu0.21.04.1] (no packageset)
[13:22] -queuebot:#ubuntu-release- Unapproved: fabric-manager-450 (impish-proposed/multiverse) [450.142.00-0ubuntu1 => 450.156.00-0ubuntu0.21.10.1] (no packageset)
[13:22] -queuebot:#ubuntu-release- Unapproved: fabric-manager-460 (bionic-proposed/multiverse) [460.91.03-0ubuntu0.18.04.1 => 460.106.00-0ubuntu0.18.04.1] (no packageset)
[13:22] -queuebot:#ubuntu-release- Unapproved: fabric-manager-460 (hirsute-proposed/multiverse) [460.91.03-0ubuntu0.21.04.1 => 460.106.00-0ubuntu0.21.04.1] (no packageset)
[13:22] -queuebot:#ubuntu-release- Unapproved: fabric-manager-460 (focal-proposed/multiverse) [460.91.03-0ubuntu0.20.04.1 => 460.106.00-0ubuntu0.20.04.1] (no packageset)
[13:22] -queuebot:#ubuntu-release- Unapproved: fabric-manager-460 (impish-proposed/multiverse) [460.91.03-0ubuntu1 => 460.106.00-0ubuntu0.21.10.1] (no packageset)
[13:23] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-470 [source] (bionic-proposed) [470.82.01-0ubuntu0.18.04.1]
[13:25] -queuebot:#ubuntu-release- Unapproved: libnvidia-nscq-450 (bionic-proposed/multiverse) [450.156.00-0ubuntu0.18.04.1 => 450.156.00-0ubuntu0.18.04.1] (no packageset)
[13:25] -queuebot:#ubuntu-release- Unapproved: libnvidia-nscq-450 (focal-proposed/multiverse) [450.156.00-0ubuntu0.20.04.1 => 450.156.00-0ubuntu0.20.04.1] (no packageset)
[13:25] -queuebot:#ubuntu-release- Unapproved: libnvidia-nscq-450 (hirsute-proposed/multiverse) [450.156.00-0ubuntu0.21.04.1 => 450.156.00-0ubuntu0.21.04.1] (no packageset)
[13:25] -queuebot:#ubuntu-release- Unapproved: libnvidia-nscq-450 (impish-proposed/multiverse) [450.156.00-0ubuntu0.21.10.1 => 450.156.00-0ubuntu0.21.10.1] (no packageset)
[13:25] -queuebot:#ubuntu-release- Unapproved: libnvidia-nscq-460 (bionic-proposed/multiverse) [460.106.00-0ubuntu0.18.04.1 => 460.106.00-0ubuntu0.18.04.1] (no packageset)
[13:25] -queuebot:#ubuntu-release- Unapproved: libnvidia-nscq-460 (hirsute-proposed/multiverse) [460.106.00-0ubuntu0.21.04.1 => 460.106.00-0ubuntu0.21.04.1] (no packageset)
[13:25] -queuebot:#ubuntu-release- Unapproved: libnvidia-nscq-460 (focal-proposed/multiverse) [460.106.00-0ubuntu0.20.04.1 => 460.106.00-0ubuntu0.20.04.1] (no packageset)
[13:25] -queuebot:#ubuntu-release- Unapproved: libnvidia-nscq-460 (impish-proposed/multiverse) [460.106.00-0ubuntu0.21.10.1 => 460.106.00-0ubuntu0.21.10.1] (no packageset)
[13:28] -queuebot:#ubuntu-release- Unapproved: rejected fabric-manager-450 [source] (impish-proposed) [450.156.00-0ubuntu0.21.10.1]
[13:28] -queuebot:#ubuntu-release- Unapproved: rejected fabric-manager-460 [source] (impish-proposed) [460.106.00-0ubuntu0.21.10.1]
[13:28] -queuebot:#ubuntu-release- Unapproved: rejected fabric-manager-450 [source] (hirsute-proposed) [450.156.00-0ubuntu0.21.04.1]
[13:28] -queuebot:#ubuntu-release- Unapproved: rejected fabric-manager-460 [source] (hirsute-proposed) [460.106.00-0ubuntu0.21.04.1]
[13:29] -queuebot:#ubuntu-release- Unapproved: rejected fabric-manager-450 [source] (focal-proposed) [450.156.00-0ubuntu0.20.04.1]
[13:29] -queuebot:#ubuntu-release- Unapproved: rejected ubuntu-drivers-common [source] (focal-proposed) [1:0.9.0~0.20.04.2]
[13:29] -queuebot:#ubuntu-release- Unapproved: rejected fabric-manager-460 [source] (focal-proposed) [460.106.00-0ubuntu0.20.04.1]
[13:29] -queuebot:#ubuntu-release- Unapproved: rejected fabric-manager-450 [source] (bionic-proposed) [450.156.00-0ubuntu0.18.04.1]
[13:29] -queuebot:#ubuntu-release- Unapproved: rejected fabric-manager-460 [source] (bionic-proposed) [460.106.00-0ubuntu0.18.04.1]
[13:35] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-460 [source] (impish-proposed) [460.106.00-0ubuntu0.21.10.1]
[13:37] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-460 [source] (hirsute-proposed) [460.106.00-0ubuntu0.21.04.1]
[13:39] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-460 [source] (focal-proposed) [460.106.00-0ubuntu0.20.04.1]
[13:39] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-460 [source] (bionic-proposed) [460.106.00-0ubuntu0.18.04.1]
[13:42] -queuebot:#ubuntu-release- Unapproved: accepted cloud-init [source] (impish-proposed) [21.4-0ubuntu1~21.10.1]
[13:43] -queuebot:#ubuntu-release- Unapproved: lyricue (impish-proposed/universe) [4.0.13.isreally.4.0.12-0ubuntu4 => 4.0.13.isreally.4.0.12-0ubuntu5] (no packageset)
[13:43] -queuebot:#ubuntu-release- Unapproved: accepted cloud-init [source] (hirsute-proposed) [21.4-0ubuntu1~21.04.1]
[13:44] -queuebot:#ubuntu-release- Unapproved: rejected lyricue [source] (impish-proposed) [4.0.13.isreally.4.0.12-0ubuntu5]
[13:44] -queuebot:#ubuntu-release- Unapproved: accepted cloud-init [source] (focal-proposed) [21.4-0ubuntu1~20.04.1]
[13:45] -queuebot:#ubuntu-release- Unapproved: accepted cloud-init [source] (bionic-proposed) [21.4-0ubuntu1~18.04.1]
[13:46] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-450 [source] (impish-proposed) [450.156.00-0ubuntu0.21.10.1]
[13:47] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-450 [source] (hirsute-proposed) [450.156.00-0ubuntu0.21.04.1]
[13:49] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-450 [source] (focal-proposed) [450.156.00-0ubuntu0.20.04.1]
[13:50] -queuebot:#ubuntu-release- Unapproved: accepted fabric-manager-450 [source] (bionic-proposed) [450.156.00-0ubuntu0.18.04.1]
[14:00] <juliank> I rebased cloud's autopkgtest against 5.17 in wip/5.17 branch - Will roll out in staging I suppose, but wait until the queues are down to 1k or so to roll out on half of production
[14:01] <juliank> We could actually upload it to jammy
[14:05] <juliank> It is a bit silly to have the ubuntu specific changes in the cloud instance only and not in the archive
[14:33] -queuebot:#ubuntu-release- Unapproved: lightdm (hirsute-proposed/universe) [1.30.0-0ubuntu4 => 1.30.0-0ubuntu4.21.04.1] (ubuntu-budgie, ubuntu-mate, ubuntukylin, xubuntu)
[14:33] -queuebot:#ubuntu-release- Unapproved: lightdm (impish-proposed/universe) [1.30.0-0ubuntu4 => 1.30.0-0ubuntu4.21.10.1] (ubuntu-budgie, ubuntu-mate, ubuntukylin, xubuntu)
[14:34] -queuebot:#ubuntu-release- Unapproved: lightdm (focal-proposed/universe) [1.30.0-0ubuntu4~20.04.1 => 1.30.0-0ubuntu4~20.04.2] (edubuntu, ubuntu-budgie, ubuntu-mate, ubuntukylin, ubuntustudio, xubuntu)
[15:08] -queuebot:#ubuntu-release- Unapproved: netplan.io (impish-proposed/main) [0.103-0ubuntu7 => 0.103-0ubuntu7.1] (core)
[15:11] -queuebot:#ubuntu-release- Unapproved: netplan.io (hirsute-proposed/main) [0.103-0ubuntu5~21.04.2 => 0.103-0ubuntu5~21.04.3] (core)
[15:12] -queuebot:#ubuntu-release- Unapproved: netplan.io (focal-proposed/main) [0.103-0ubuntu5~20.04.2 => 0.103-0ubuntu5~20.04.3] (core)
[16:35] <vorlon> notest runs were broken since I made the change, for various reasons incl. conflicting lock files.  I'm doing a manual run now that hasn't died yet, so hopefully we'll be at hourly runs shortly
=== Kamilion|ZNC is now known as Kamilion
[16:50] <juliank> autopkgtest queues should be empty in roughly 20 hours (assuming similar test duration for pending tests as last 3 hours)
[16:50] <RikMills> could someone trigger ubiquity tests against itself to confirm if tests are regressed in the release pocket? thanks
[16:50] <juliank> RikMills: sure
[16:51] <RikMills> ty
[16:51] <juliank> RikMills: generally speaking you want to trigger against an unrelated package, though, I like to use hello, in case there is a version in proposed.
[16:51] <juliank> RikMills: Just FYI for the future
[16:51] <juliank> autopkgtest does not really respect the version specified in the trigger
[16:51] <RikMills> juliank: already did that, so was looking for an extra confirm
[16:52] <vorlon> notest run succeeded; output only gets copied to the website when a main run completes
[16:53] <juliank> ooh I should try out some weirdness
[16:54] <RikMills> heh. ubiquity tests also just failed for apt/2.3.11 trigger
[16:54] <RikMills> https://autopkgtest.ubuntu.com/packages/u/ubiquity/jammy/ppc64el
[16:55]  * RikMills will come back to this later
[17:02] <juliank> vorlon: So I hacked around a bit to see what happens if I trigger a test with migration-reference/0 as debian does to reset tests. turns out it doesn't work at all, it will fail to showsrc migration-reference. Not sure where the special sauce lives for that in Debian
[17:04] <juliank> I think because we construct --apt-pocket or whatever it's called  ourselves
[17:06] <vorlon> hmm
[17:07] <vorlon> yeah, could well be a problem with our delta, not Debian secret sauce somewhere?
[17:08] <vorlon> (last comment on notest refactor: now the delay between update_{excuses,output} being generated, and being published where it can be seen and acted on, is ~10m, instead of ~40)
[17:17] <juliank> vorlon: I think it's because debci implements that, and autopkgtest-cloud doesn't
[17:18] <juliank> vorlon: I wonder if we want to, we can then enable that for ~ubuntu-release
[17:18] <juliank> Not sure if useful vs hints
[17:18] <juliank> At least useful for testing
[17:18] <juliank> Then we are also halfway to baseline retesting
[17:30] <juliank> This seems to be it: https://paste.ubuntu.com/p/nbTwfczGQv/
[17:31] <juliank> https://code.launchpad.net/~juliank/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/411450
[17:31] <juliank> ugh multi-charm build is annoying :D
[17:34] <RikMills> ok, can we badtest ubiquity/21.10.10 please?
[17:36] <vorlon> juliank: "not sure if useful vs hints" except the excessive hints have a performance impact on proposed-migration, in addition to the human cost to manually maintain them?
[17:37] <juliank> vorlon: yeah I meant retrying manually with migration-reference/0 until we have the systemd timer that inserts those automatically
[17:37] <juliank> OK the code is buggy: --apt-pocket=proposed
[17:39] <vorlon> well even when we have the automatic retries, there may be cases that a developer wants to jump the queue for retry of a particular package that's blocked
[17:39] <vorlon> so a manual baseline retest is definitely useful
[17:39] <juliank> vorlon: Do you want to allow anyone to trigger them?
[17:40] <vorlon> juliank: anyone with upload rights, yeah
[17:40] <vorlon> because then they can reset the baseline correctly without release team involvement
[17:40] <juliank> ok
[17:41] <vorlon> (today it's a 2 step process, someone does a "baseline" retest, the results are reflected on autopkgtest.u.c, then the release team has to manually add a hint)
[17:47] <juliank> vorlon: OK, branch seems to work now: https://autopkgtest.staging.ubuntu.com/results/autopkgtest-impish/impish/s390x/g/gzip/20211105_174719_4e571@/log.gz
[17:48] <juliank> ready for review: https://code.launchpad.net/~juliank/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/411450 :D
[17:48] <juliank> oh I guess one hunk of those is not needed, but meh, could come in handy later
[17:50] <juliank> the code did not handle the case of "no triggers" properly :D
[17:50] <juliank> I'd guess this just works fine with our britney, as we have merged all the code for it from debian
[17:58]  * juliank drafted an ubuntu-devel announcement for after this is verified :D
[18:09] <juliank> lemme just yolo that
[18:15] <vorlon> juliank: well, I was reviewing and have questions
[18:15] <vorlon> (a question)
[18:17] <juliank> vorlon: ah that question, yeah, historical artefact from the first commit that checked for ubuntu-release I forgot to cleanup
[18:18] <juliank> vorlon: I did not want to push even more tags and charm releases, so I did not clean it back up again
[18:18] <vorlon> hmph :)
[18:18] <juliank> vorlon: might come in handy later
[18:18] <vorlon> ok
[18:19] <juliank> reloads of workers have been scheduled, so this will work sometime in the next hours
[18:20] <juliank> then we can retrigger ubiquity with migration-reference/0 and see what britney thinks about it
[18:22] <vorlon> https://people.canonical.com/~ubuntu-archive/proposed-migration/update_output_notest.txt hurrah
[18:31] <vorlon> Depends: [...] libusrsctp2 (>= 0.9.4-0~) / Recommends: libusrsctp1 (>= 0.9.3.0+20200406) oh what the hell
[18:32] <cjwatson> "sure, whatever, we have horrifying linker code to DWIM"?
[18:42] <juliank> vorlon: I just realized that duh, all workers have a restart queeued still are running tests, so we can actually issue migration-reference/0 tests now
[18:43] <juliank> vorlon: i have done so for ubiquity on amd64
[18:43] <vorlon> cool
[18:43] <juliank> and the other failed archs
[18:44] <juliank> so if it works, we should see ubiquity migrate
[18:44] <juliank> um
[18:44] <juliank> kwin
[18:50] <vorlon> juliank: we should probably change retry-autopkgtest-regressions --no-proposed to use migration-reference/0?
[18:50] <vorlon> (I see no downside)
[18:51] <juliank> vorlon: oh i did not know this existed
[18:51] <vorlon> yeah it's a hack and doesn't work when the source package you're triggering is present in -proposed
[18:51] <juliank> vorlon: and yes, if britney is happy with this, and does not crap out on us, we should :)
[18:51] <juliank> let's see next britney run :D
[18:53] <vorlon> well, britney usefully do anything automatically with the /old/ --no-proposed either
[18:53] <vorlon> so, pushed
[18:54] <vorlon> *doesn't
[18:55] <juliank> added to announcement email
[18:56] <juliank> vorlon: I probably should tighten down the web interface to not allow mixing migration-reference/0 with other triggers
[18:56] <juliank> vorlon: I just got reminded of that because retry-autopkgtest-regressions also adds multiple ones: https://autopkgtest.ubuntu.com/request.cgi?release=impish&arch=amd64&package=pyopencl&trigger=migration-reference%2F0&trigger=migration-reference%2F0
[18:57] <juliank> don't want people to set a migration-reference with something from proposed
[18:58] <vorlon> heh
[18:59] <juliank> vorlon: so I can reduce multiple specifications of migration-reference/0 in web frontend, or you fix retry-autopkgtest-regressions to only add one trigger
[18:59] <vorlon> well the latter should be fixed in any case
[19:00] <vorlon> I'll clean that up
[19:00] <juliank> I'll just add an error if 'migration-reference/0' in triggers and len(triggers) != 1
[19:02] <juliank> Same for ppas and all-proposed
[19:04] <vorlon> juliank: fixed the multiple trigger behavior
[19:06] <juliank> vorlon: thanks, I added the safety checks
[19:23] <vorlon> huge queue is still draining, but I'm going to go ahead and mass-retry all the autopkgtest failures since jammy opened
[19:37] <juliank> vorlon: and then mass retry with --no-proposed for lolz :D
[19:38] <juliank> I retriggered pyopencl in impish actually
[19:39] <juliank> with migration-reference/0
[20:01] <juliank> vorlon: oh, are you doing those thousands of retries over the cgi?
[20:01] <vorlon> juliank: yes
[20:03] <vorlon> juliank: I don't like using the admin-only backdoor; if the cgi is insufficiently performant, we should experience the pain and fix it :)
=== paride4 is now known as paride
[20:04] <juliank> vorlon: I noticed how slow autopkgtest.u.c has become :D
[20:04] <juliank> vorlon: there's a load of 14 on web/1; oddly enough all your requests land on the same server
[20:05] <juliank> vorlon: we should at least give the web workers 2 cpus
[20:05] <vorlon> juliank: afaik each browser client is pinned to a single worker based on cookies
[20:05] <juliank> load factor idling is 4
[20:05] <juliank> vorlon: it makes sense most of the time
[20:06] <juliank> vorlon: maybe we can configure haproxy to distribute request.cgi instead
[20:11] <juliank> oh rabbitmq also needs 2 cpus
[20:12] <juliank> it spikes a lot eating up 100% CPU and it becomes unresponsive/lies about queue content occassionaly
[20:12] <vorlon> does adding more CPU to rabbit fix the problem though? Last time I had to shoot beamer in the head and restart it
[20:13] <juliank> I don't *know*, but it doesn't make all that much sense to have a hyper multi-process thing run on one cpu anyway
[20:22] <juliank> come on britney, start already, I want to watch you migrate stuff
[20:25] <vorlon> heh
[20:27] <vorlon> juliank: fwiw this was already in the last log: I: [2021-11-05T18:44:07+0000] - Fetched test result for ubiquity/unknown/amd64 20211105_171502_ef15c@ (triggers: ['migration-reference/0']): fail
[20:27] <juliank> vorlon: yup thanks I saw that one too
[20:28] <juliank> this was my buggy test run :D
[20:28] <vorlon> oh
[20:28] <vorlon> juliank: right, timestamp is 2 minutes before the successful run ;)
[20:30] <juliank> vorlon: oh I think we need to set ADT_BASELINE = reference in britney config
[20:33] <vorlon> yeah looks like it
[20:34] <vorlon> done, in time for the next 2h britney run by the look of it
[20:40] <juliank> vorlon: :D
[20:46] -queuebot:#ubuntu-release- New binary: ubuntustudio-look [amd64] (jammy-proposed/universe) [22.04.0] (ubuntustudio)
[20:47] <Eickmeyer[m]> Can I get a real quick ack on that one? Just a new wallpaper binary for jammy. ^
[20:48] <vorlon> Eickmeyer[m]: ack
[20:48] -queuebot:#ubuntu-release- New: accepted ruby-uc.micro-rb [amd64] (jammy-proposed) [1.0.5-1]
[20:48] <Eickmeyer[m]> Part of the effort to reduce the ISO size.
[20:48] <Eickmeyer[m]> vorlon: Thanks. :)
[20:48] -queuebot:#ubuntu-release- New: accepted ubuntustudio-look [amd64] (jammy-proposed) [22.04.0]
[20:49] <juliank> vorlon: I found this britney commit to not add triggers already in testing in my branch, hmm: https://code.launchpad.net/~juliank/britney/+git/britney2-ubuntu/+merge/411459
[20:50] <juliank> I don't know what happened a year ago in britneyland :D
[20:50] <vorlon> juliank: is this something that should be upstreamed to Debian or is this all in the Ubuntu-specific code?
[20:51] <juliank> vorlon: I think I might have only upstreamed it
[20:51] <vorlon> ah
[20:51] <juliank> vorlon: yup https://salsa.debian.org/release-team/britney2/-/merge_requests/57
[20:51] <ubottu> Merge 57 in release-team/britney2 "autopkgtest: do not add triggers that are already in target suite" [Merged]
[20:52] <juliank> vorlon: i guess we ought to rebase britney
[20:52] <vorlon> not high on my todo list the Friday before I go on a week's vacation
[20:53] <sarnold> woo :)
[20:54] <juliank> vorlon: yeah that would not be wise
[20:57] <vorlon> juliank: so, consequence of turning on baseline=reference in britney, britney is now triggering baseline tests for all packages which don't already have them
[20:57] <vorlon> across all releases
[20:57] <juliank> vorlon: oh uh
[20:57] <vorlon> not wrong, but cough
[21:00] <juliank> vorlon: only stuff it wants to migrate I hope, though
[21:00] <vorlon> right
[21:00] <juliank> phew
[21:00] <juliank> why across all releases ,though
[21:01] <vorlon> juliank: because we run p-m for stable releases also
[21:01] <vorlon> I: [2021-11-05T21:00:26+0000] - Requesting node-concat-with-sourcemaps autopkgtest on i386 to verify migration-reference/0
[21:01] <juliank> ah they all have the same config file?
[21:01] <vorlon> yeah
[21:01] <vorlon> (rather, the same config template)
[21:02] <vorlon> ^^ so it's asking for baseline retests on things that are ignored failures... this is nice, means we can drop all the i386 hints imminently
[21:03] <RikMills> I must thank you both for the hard work at this point on a Friday. It is appreciated :)
[21:05] <juliank> I: [2021-11-05T20:56:28+0000] - Requesting ubiquity autopkgtest on amd64 to verify migration-reference/0
[21:05] <juliank> odd
[21:05] <juliank> because earlier:
[21:05] <juliank> I: [2021-11-05T20:56:28+0000] - Fetched test result for ubiquity/21.10.10/amd64 20211105_184630_7fc3d@ (triggers: ['migration-reference/0']): fail
[21:05] <juliank> I: [2021-11-05T20:56:28+0000] - -> does not match any pending request for ubiquity/amd64
[21:06] <juliank> maybe it doesn't know?
[21:06] <juliank> keeps a cache file?
[21:06] <vorlon> hmm
[21:07] -queuebot:#ubuntu-release- Unapproved: accepted qemu [source] (impish-proposed) [1:6.0+dfsg-2expubuntu1.1]
[21:07] <vorlon> could be a bug when something other than it has requested a baseline test, yeah
[21:12] -queuebot:#ubuntu-release- Unapproved: rejected libnvidia-nscq-460 [source] (impish-proposed) [460.106.00-0ubuntu0.21.10.1]
[21:14] <juliank> vorlon: I guess we might want to see ADT_REFERENCE_MAX_AGE = 7 then it retries the migration reference after a week
[21:14] <juliank> vorlon: well, when it is getting triggered (and failed?)
[21:18] <juliank> gives us *some* baseline retesting for free
[21:24] <juliank> painful
=== falcojr6 is now known as falcojr
[21:28] <vorlon> juliank: I'd rather not set reference_max_age and instead wait until we can do broader baseline retesting
[21:30] -queuebot:#ubuntu-release- Unapproved: rejected libnvidia-nscq-450 [source] (impish-proposed) [450.156.00-0ubuntu0.21.10.1]
[21:32] -queuebot:#ubuntu-release- Unapproved: accepted lightdm [source] (impish-proposed) [1.30.0-0ubuntu4.21.10.1]
[21:34] -queuebot:#ubuntu-release- Unapproved: accepted lightdm [source] (hirsute-proposed) [1.30.0-0ubuntu4.21.04.1]
[21:35] -queuebot:#ubuntu-release- Unapproved: accepted lightdm [source] (focal-proposed) [1.30.0-0ubuntu4~20.04.2]
[21:58] <juliank> vorlon: I don't understand why britney did not find te trigger and am worried it will retrigger all next run again
[21:59] <juliank> vorlon: the code where it checks the baseline is just a couple lines before where it checks if it ever passed, so um, this should work?
[22:00] <vorlon> yeah... I'll watch it
[22:00] <vorlon> and revert if necessary
[22:02] <juliank> vorlon: in debian they read all new results first thing, so maybe something is missing somewhere
[22:13] <juliank> vorlon: So I think this fixes it: https://code.launchpad.net/~juliank/britney/+git/britney2-ubuntu/+merge/411463
[22:13] <juliank> vorlon: Basically it checks for results too late and hence queues stuff
[22:14] <juliank> vorlon: This should not be a problem in the next run, as the test will be marked pending already, so it won't be requeued
[22:14] <juliank> but it means the run after next one would be the first one to see test results
[22:14] <juliank> Of course, I might have misread somewhere
[22:15] <juliank> This would mean that check_ever_passed results are delayed by one britney run as well
[22:15] <juliank> well maybe that is fine
[22:17] <juliank> oops indentation
[22:17] -queuebot:#ubuntu-release- Unapproved: accepted netplan.io [source] (focal-proposed) [0.103-0ubuntu5~20.04.3]
[22:20] -queuebot:#ubuntu-release- Unapproved: accepted netplan.io [source] (impish-proposed) [0.103-0ubuntu7.1]
[22:26] <juliank> vorlon: oh my, only 16MB of output from britney in 2 hours, last run finished in 2h with 800 MB
[22:27] <vorlon> hmm
[22:28] <vorlon> juliank: well I think we're almost done queuing the new baseline test requests
[22:29] <juliank> I guess those do eat up a lot of time
[22:30] <juliank> vorlon: how do you know?
[22:30] <juliank> ok, I see we are at tk-table trigger tk-table
[22:30] <juliank> but then meson trigger meson
[22:30] -queuebot:#ubuntu-release- Unapproved: accepted netplan.io [source] (hirsute-proposed) [0.103-0ubuntu5~21.04.3]
[22:31] <juliank> anyhow, weekend is a good time to trigger stuff
[22:31] <vorlon> right, I may have incorrectly assumed something about the order
[22:31] <juliank> how many failed were there, 8k?
[22:31] <juliank> ah 4.8k
[22:32] <vorlon> previous update_excuses had 6947 regressions, 4886 ignored failures
[22:32] <juliank> ok
[22:32] <juliank> wwe are at 7.6k so far
[22:33] <juliank> for the migration-reference/0
[22:33]  * vorlon nods
[22:33] <juliank> or 14k if we count dupes?
[22:33] <vorlon> uh we have dupes?
[22:34] <juliank> well if foo failed twice once for bar and once for baz, it will queue one, and then say "already pending" for the other migration-reference/0
[22:34] <juliank> it doesn't queue foo migration-reference/0 twice
[22:34] <vorlon> ah
[22:36] <juliank> vorlon: your(?) sqlite3 branch removed the message where it queries swift for results so it's not super verbose right now
[22:37] -queuebot:#ubuntu-release- Unapproved: accepted livecd-rootfs [source] (hirsute-proposed) [2.719.3]
[22:37] <juliank> vorlon: like if hasattr(self, 'db') branch has no log, but elif uses_swift has
[22:37] <juliank> vorlon: I added logging to both branches in my merge above when I copied it into the baseline determination function
[22:39] <vorlon> not super-verbose, he says in reference to a 1GB text file
[22:40] <juliank> vorlon: :)
[22:44] <juliank> jak@jak-t480s:/tmp$ rg Checking\ hints 20\:42\:23.log  | wc -l
[22:44] <juliank> 20668
[22:44] <juliank> jak@jak-t480s:/tmp$ rg Checking\ hints 18\:29\:29.log  | wc -l
[22:44] <juliank> 6476514
[22:45] <juliank> ah but also this is shorter now
[22:46] <juliank> jak@jak-t480s:/tmp$ rg Checking\ hints.*'\[' 18\:29\:29.log | wc -l
[22:46] <juliank> 26997
[22:46] <juliank> jak@jak-t480s:/tmp$ rg Checking\ hints.*'\[' 20\:42\:23.log | wc -l
[22:46] <juliank> 18924
[22:46] <juliank> this should be more accurate progress indicator
[22:48] <juliank> So the log is so small because we do not detect any regression atm
[22:48] <juliank> hence we never log force-reset-test check hints
[22:49] -queuebot:#ubuntu-release- Unapproved: rejected libnvidia-nscq-460 [source] (hirsute-proposed) [460.106.00-0ubuntu0.21.04.1]
[22:49] <juliank> because all failed tests are now not "regression", but "running reference"
[22:50] -queuebot:#ubuntu-release- Unapproved: rejected libnvidia-nscq-450 [source] (hirsute-proposed) [450.156.00-0ubuntu0.21.04.1]
[22:57] <juliank> 21.4k
[23:01] <juliank> 22.7k
[23:02] <juliank> vorlon: I'm sorry about the file size, most of that was me:
[23:02] <juliank> $ grep -v  force-reset-test 18\:29\:29.log | wc -c 18\:29\:29.log -
[23:02] <juliank> 875192838 18:29:29.log
[23:02] <juliank>  23118516 -
[23:02] <juliank> 898311354 total
[23:04] <juliank> Simply removing the logging in has_higher_force_reset_test or moving it into return True will cut of 850MB from the log
[23:09] <juliank> vorlon: please merge https://code.launchpad.net/~juliank/britney/+git/britney2-ubuntu/+merge/411466 to reduce log output from 875MB to like 30MB or so
[23:10] <juliank> (when we merged force-reset-test the queue was not this large, so did not notice)
[23:12] <juliank> oh was it actually laney who wrote the code, and vorlon who added the caching (which caused the 800 MB log output)
[23:12] <vorlon> I don't believe I added any caching?
[23:12] <juliank> vorlon: 6cd47f352b7eb6f435fe8a0618bdfffeea785394
[23:12] <juliank> vorlon:     In the autopkgtest policy, cache the list of force-reset-test hints once
[23:13] <vorlon> oh that
[23:13] <juliank> vorlon: Remove the package= argument, which suddenly meant displaying all unrelated force-reset-test hints for every package
[23:13] <juliank> anyhow, logging will be a lot nicer with the commit above
[23:14] <juliank> 23.77k
[23:14] <juliank> still like 3k to go
[23:16] <juliank> though I realize now that this does not account for packages that don't have any hints
[23:16] <juliank> so maybe I should go sleep instead
[23:21] -queuebot:#ubuntu-release- Unapproved: accepted barbican [source] (focal-proposed) [1:10.1.0-0ubuntu2]
[23:28] <juliank> oh, there is a new python3-defaults in proposed :/
[23:35] <vorlon> heh
[23:37] <juliank> vorlon: anyway, merge my two merge requests and the next britney run should work correctly with the reference triggers and stop logging a ton of lines
[23:39] <juliank> vorlon: log hasn't moved in 13 minutes, did it die?
[23:39] <vorlon> juliank: I'm watching it scroll by here locally
[23:39] <juliank> oh hmm
[23:40] <vorlon> s/locally/over ssh/
[23:40] <juliank> vorlon: not sure what happened here
[23:40] <juliank> vorlon: I wget -c it every 10s
[23:40] <juliank> not *super* efficient, but works
[23:41] -queuebot:#ubuntu-release- Unapproved: accepted rsync [source] (focal-proposed) [3.1.3-8ubuntu0.1]
[23:42] <juliank> I think my tmux got stuck
[23:42] -queuebot:#ubuntu-release- Unapproved: accepted rsync [source] (bionic-proposed) [3.1.2-2.1ubuntu1.2]
[23:44] <juliank> maybe should insert a sorted() into britney to process packages in order :D
[23:44] <juliank> for progress guessing
[23:47] <sarnold> poor zsh