[05:18] morning [05:58] mborzecki: hi! [05:58] mardy: hey [05:59] not much has landed since yesterday :/ [06:00] mborzecki: but now codecov seems to be working at least :-) [06:00] mborzecki: have you seen this error before? I'm trying to reproduce it locally: https://github.com/snapcore/snapd/pull/10797/checks?check_run_id=3628825501 [06:00] PR #10797: usersession/client: refactor doMany() method [06:01] looks like a timing issue, with the test taking too long [06:01] mardy: yes, it's fixed in master [06:02] really? I just rebased... [06:02] hmmm [06:02] let me pull master again [06:03] I have Pawel's commit in my branch [06:03] ah w8, it's contxt deadline exceeded [06:03] yeah, a timing problem then? [06:03] I think the problem is that locally, it takes less than 5 seconds, and it passes [06:04] there, in CI, it took longer than 5 seconds [06:04] 5.77s [06:12] mardy: reproduced it locally https://paste.ubuntu.com/p/m4p5Gq74dF/ [06:12] with -count 100 [06:14] mborzecki: did you change the timeout? I see it took just 1.02 s [06:14] mardy: i did `cd store && go test -check.f storeActionSuite.TestSnapActionTimeout -count 100 ` [06:14] then it failed after some number of attempts [06:15] i guess ti's either hitting a context deadline set when creating a request, or there's an overarching time.After() when the request is issued somewhere in net/http [06:17] mardy: https://github.com/snapcore/snapd/pull/10800 [06:17] PR #10800: store: one more tweak for the test action timeout [06:18] PR snapd#10800 opened: store: one more tweak for the test action timeout [07:12] morning [07:13] PR snapd#10800 closed: store: one more tweak for the test action timeout [07:13] pstolowski: mvo: morning [07:14] good morning mborzecki and pstolowski [07:14] mvo: can you land https://github.com/snapcore/snapd/pull/10703 ? [07:14] PR #10703: tests/main/security-device-cgroups-strict-enforced: demonstrate device cgroup being enforced [07:16] mborzecki: sure [07:17] thanks! [07:17] mborzecki: does this mean it's time for a git snapshot upload to impiish? [07:18] PR snapd#10703 closed: tests/main/security-device-cgroups-strict-enforced: demonstrate device cgroup being enforced [07:23] PR snapd#10787 closed: tests: add a local snap variant to testing prepare-image gating support [07:27] good morning, I'll be right back, just need to reboot for kernel updates [07:28] PR snapd#10779 closed: tests/nested/manual: use loop for checking for initialize-system task done [07:29] zyga: good morning [07:29] pstolowski: looks like 10765 needs a master merge, failed with the timeout unit test with 1.17 [07:30] good morning mvo :) [07:30] happy Friday! [07:32] mvo: not yet, one more branch [07:33] congratulations on firefox snap everyone! [07:35] mvo: yeah, not just this one, going over my PRs atm [07:43] PR snapd#10801 opened: cmd/libsnap-confine-private: use root when necessary for BPF related operations [07:43] PR snapd#10802 opened: data/selinux: update the policy to allow s-c to manipulate BPF map and programs [07:51] pstolowski: thanks [07:51] mvo: all the branches are up, i've closed the WIP one, also added cgroupv2-impish label for things that I think need to be part of the git snapshot (which is everything that's proposed, sans selinux bits) [07:51] mborzecki: which one is this? [07:51] mborzecki: aha, nice [07:51] mborzecki: thanks so much! [07:52] i'll try to clean up my notes and maybe propose those too as in-source-tree documentation [07:53] PR snapd#10575 closed: [WIP] many: device cgroup v2 support <â›” Blocked> [07:53] PR snapd#10803 opened: tests, interfaces/builtin: introduce 21.10 cgroupv2 variant, tweak tests for cgroupv2, update builtin interfaces [08:01] meh, tests/main/auto-refresh* tests are failing now [08:01] mvo: https://github.com/snapcore/snapd/pull/10799/ can land now, the tests that are affected by the PR are fixed, the failing ones are unrelated [08:01] PR #10799: tests/main/preseed: update for new base snap of the lxd snap <âš  Critical> [08:10] mborzecki: +1 done [08:11] thanks [08:13] PR snapd#10799 closed: tests/main/preseed: update for new base snap of the lxd snap <âš  Critical> [08:27] not bad for 21.10 now https://paste.ubuntu.com/p/v7ZRj8G5JX/ [08:44] mborzecki: awesome.. auto-refresh-gating? what failed there [08:54] pstolowski: afaiu the test expects an auto-refresh change to happen, but there isn't one, store does not return anything [08:54] mborzecki: ah, weird [08:55] from what i can tell test-snapd-refresh-control-provider snap should be refreshed after switching the channels, but perhaps the store is rate limiting refreshes now? [08:57] maybe logs will sched some light [08:57] but that's how the test works and should continue to work [08:59] pstolowski: well, it's failing on master too right now, i'm looking at the logs: https://paste.ubuntu.com/p/V6hhDp6nqC/ [08:59] this is when we expect an auto refresh :) [09:00] hmm [09:00] storehelpers.go:604: no install/refresh information results from the store [09:03] pstolowski: asked in mm [09:04] thanks [09:20] mvo: hi, can you merge https://github.com/snapcore/snapd/pull/10756 please? The test failures are unrelated [09:20] PR #10756: cmd/snap: only log translation warnings in debug/testing [09:22] miguelpires: sure [09:24] PR snapd#10756 closed: cmd/snap: only log translation warnings in debug/testing [10:06] mvo: could you please merge https://github.com/snapcore/snapd/pull/10774 ? unrelated failures [10:06] PR #10774: asserts, snapstate: return full validation set keys from CheckPresenceRequired and CheckPresenceInvalid [10:06] pstolowski: on it [10:07] ty [10:09] PR snapd#10774 closed: asserts, snapstate: return full validation set keys from CheckPresenceRequired and CheckPresenceInvalid [10:22] PR core20#114 closed: Fix riscv64 build [11:28] PR snapcraft#3581 opened: packaging: load the correct libraries on riscv64 [12:05] mvo: could you also land https://github.com/snapcore/snapd/pull/10765 ? [12:05] PR #10765: o/snapstate: enforce validation sets/enforce on InstallMany [12:16] re [12:16] heh that took a while [12:20] mborzecki: all fine? [12:20] yeah, got one more checkup on sunday, but seems to be fine so far [12:26] great [12:48] mvo: can you please use your merging powers on https://github.com/snapcore/snapd/pull/10772? [12:48] PR #10772: kernel/fde: mock systemd-run in unit test [12:49] BTW, the tests there are failing because of some time tolerance issue: https://pastebin.ubuntu.com/p/g76qTyBhC7/ Is anyone working on it (I could take a look, otherwise) [12:54] mardy yeah I've seen that a bunch I think there's a time sync that happens or something because the test definitely is not executing for 45 minutes like the time difference would suggest [12:54] But I'm not working on that failure [12:58] ah, now I see, it's the same test that once failed at midnight [12:59] so, this test does not appear to be doing any refresh on its own :-) [13:00] it just assumes that some other test must have been run before, which triggered a refresh [13:00] but whether that took 10 minutes or more, it depends from the order of the tests [13:14] PR snapd#10765 closed: o/snapstate: enforce validation sets/enforce on InstallMany [13:23] miguelpires: see tests/main/interface-hooks test [13:24] pstolowski: thank you! [13:29] PR snapd#10804 opened: [Feat] Add @squash rule support to snap-seccomp [13:54] PR snapd#10805 opened: overlord/devicestate: make settle wait longer in remodel tests [13:56] trivial pr ^^ [14:20] PR snapd#10806 opened: tests: rename interfaces-hooks-misbehaving spread test to install-hook-misbehaving [14:21] and this ^ [14:34] @jdstrand I'm working on getting per-rule squashing of seccomp denial logs.. I think I've got the snap-seccomp bits done, but it's still logging, so I'm wondering if I need to make changes in libseccomp-go. Any idea what I need to look for to generate a per-rule no-log? [15:21] pstolowski: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1943853 looks like a problem with snapd loosing all connections [15:22] Bug #1943853: Recent update DELETED SNAP PROGRAMS [15:24] mborzecki: oh well, we don't have snap changes, do we? [15:25] pstolowski: there's no way to auto connect things again right? [15:27] mborzecki: no [15:28] mborzecki: and no snaps are broken, very weird [15:28] pstolowski: the connections are clearly gone [15:29] mborzecki: or repo and conns are out of sync for some weird reason, but maybe snap connections uses conns directly these days [15:30] mborzecki: can you ask him for snap changes & snap change if there were errors? [15:32] pstolowski: ok, sent a request [15:33] ty [15:46] pstolowski: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1943853/comments/14 meh, looks like it was garbage collected, any ideas how we can help this poor soul? [15:46] Bug #1943853: Recent update DELETED SNAP PROGRAMS [15:54] mborzecki: dang, I don't think we will be able to figure what happened without changes :( [15:55] I really hate some days the fact that we so aggressively garbage collect changes [15:57] ijohnson[m]: yes [15:58] also, seems like we regressed with snap disconnect and it must happened a very long time ago, filing a bug [16:00] or actually maybe not, but there is a confusing message with snap disconnect foo: [16:08] yeah i think it's just the message problem [16:08] fyi https://bugs.launchpad.net/snapd/+bug/1943987 [16:08] Bug #1943987: snap disconnect : prints a confusing error [16:12] also bit of a bummer right now, as I have no ide how to restore the connections other than just manaully running snap connect for all eligible plugs [16:12] or snap remove & snap install but that would remove their data [16:12] so snap save? [16:12] yeah that's what I was going to suggest them to do to restore their system is just snap connect everything [16:13] ijohnson: but then all connections would be flagged as manual, wouldn't they? [16:13] oh hmm [16:13] but what's the problem with that ? [16:14] ijohnson: afaiu some plugs are not auto connected, so i we iterate blindly and connect everythign the end result will be dierent than what auto connect does [16:15] ijohnson: so what i'm really missing is maybe `snap connect --eligible ` [16:15] PR snapd#10807 opened: tests/nested/manual/core20-cloud-init-maas-signed-seed-data: add gadget variant [16:15] I was just going to install all the snaps manually so the auto-connections process in a known good machine and then just have them manually connect all of those [16:15] snap connect --eligible feels like it's just there to fix a bug that shouldn't happen in the first place [16:17] so I suspect we lost all the connections because at some moment there was a lot of " cannot read snap info of snap "core" at revision 11606: cannot find installed snap "core" at revision 11606: missing file /snap/core/11606/meta/snap.yaml" errors in the log [16:17] but it's no clear how this leads to removing of connections, needs investigation [16:19] fwtw core isn't broken in his snap list, so it recovered [16:19] ijohnson[m]: i wonder if reinstalling core/snapd wouldn't fix it [16:19] how would you reinstall core/snapd though [16:20] without removing all the snaps [16:20] ijohnson[m]: snap refresh --beta snapd etc, i.e. switch channels [16:25] doing this here creates Automatically connect eligible plugs and slots of snap "snapd" task [16:25] Hmm maybe I'm not sure [16:25] Worth a try though, that operation couldn't delete the files at all [16:27] ok, i sugested that, and that's EOW [16:27] have a nice weekend, cu o/ [16:28] bye! you too [16:31] so I don't think snapd can disable logging on a per rule basis until kernel changes are made to not enforce logging which was added at: https://bugs.launchpad.net/snapd/+bug/1721676 [16:31] Bug #1721676: implement errno action logging in seccomp for strict mode with snaps tyhicks> [18:05] PR snapd#10808 opened: cmd/snap-seccomp: add riscv64 support === alan_g__ is now known as alan_g [21:29] PR snapcraft#3582 opened: github: update snapcore/action-build dep [22:31] PR snapd#10809 opened: .github/workflows/test.yaml: bump action-build to 1.0.9 [23:34] PR snapcraft#3582 closed: github: update snapcore/action-build dep