=== amurray_ is now known as amurray === popey9 is now known as popey [07:09] morning [07:18] goooood morning :-) [07:27] good morning pstolowski and zyga! [07:45] cjwatson: hi, do you have any eta for the rollout of your cla fix? [07:53] pstolowski: I think the difference is that the new action doesn't whitelist existing committers, so ends up hitting LP more often [07:53] Setting the accept-existing-contributors input parameter might get it to pass as often as the old workflow [07:54] jamesh: ah, interesting [07:57] jamesh: would that be an argument to github action (not familiar with them)? [07:58] pstolowski: yeah. I'll put together a simple PR so we have something to discuss [07:58] jamesh: thanks [08:14] PR snapd#10104 opened: Use x-gvfs-hide mount option to hide squashfs loopback devices in Gnome gvfs [08:44] PR snapd#10105 opened: ci: set the accept-existing-contributors parameter for the cla-check action === alan_g_ is now known as alan_g [08:45] pstolowski: ^^^ this one will probably get you unstuck [08:48] jamesh: +1, thank you! [08:49] mvo: ^ can you given 2nd review? [09:45] pstolowski: Hopefully today - we have some QA to do on other revisions that are in the way first [09:46] cjwatson: great, thank you. in the meantime we may have a remedy from jamesh to lessen the stress on lp (PR above) [09:46] pstolowski: Well, fewer requests are always good, but it's not a stress thing [09:47] cjwatson: yeah i understand it's a suboptimal query; but at least this should unblock our own landings [09:59] It's interesting the Launchpad API documentation says "If you want a method to check if a given person is a member of a team, you should probably look at IPerson.inTeam()", which is not exposed by the API [10:00] Maybe exposing that would let scripts like this perform simpler queries [10:08] jamesh: That certainly occurred to me, but I wanted to put out the immediate fire first, especially since it's a short week [10:43] * pstolowski lunch [12:33] mvo: can you merge https://github.com/snapcore/snapd/pull/10105 manually? [12:33] PR #10105: ci: set the accept-existing-contributors parameter for the cla-check action [12:51] pstolowski: are you blocked on me atm? I looked at https://github.com/snapcore/snapd/pull/10053 but don't think I can do a full review today [12:51] PR #10053: o/snapstate: helper for getting snaps affected by refresh, define new hook [12:52] pedronis: no i'm not, thanks for asking [12:59] pstolowski,mvo: CLA checks should be fixed now [12:59] \o/ [12:59] cjwatson: thank you! [13:02] \o/ [13:05] PR snapd#10100 closed: github: revert cla-check action [13:31] pstolowski: there are a couple of failure of the degraded test in https://github.com/snapcore/snapd/pull/10077 and I'm not sure if they are related to the PR or just random (degraded does fail sometimes) [13:31] PR #10077: o/configstate: fix panic with a sequence of config unset ops over same path

[13:31] pedronis: ok, i'll investigate, thanks for heads up [13:35] with snapd 2.49.2 we are seeing failures for our juju-db snap to be able to start correctly. did apparmor rules change between 2.49.0 and 2.49.2 ? [13:35] PR snapd#10106 opened: secboot,boot: provide fde-hooks v2 API interface to hooks [13:35] jam: what system are you seeing this on ? [13:36] focal [13:36] jam: with stock ubuntu kernel ? [13:36] yes [13:36] hmm [13:37] jam: if you can share more details on a forum post or on a launchpad bug and we'll take a look [13:37] please include system journal logs from snapd as well as the output of `snap version` and `snap changes` [13:38] also if you are seeing juju-db not starting with 2.49.2, please try reverting with `snap revert` and then see if it works [13:48] pedronis: I will push a PR that *may* help with the prepare issue, I have found an issues here [13:57] pedronis: I opned 10107, let's see if it helps, I will re-run if needed [13:57] pedronis: (if it passes I will re-run to ensure it's not chance but a real effect we see) [14:00] PR snapd#10107 opened: packaging: stop snapd.{socket,service} before purging data [14:08] ijohnson: jam: we also need apparmor denial logs if any in the system log [14:08] yes that too [14:08] thanks, I will try to get some context for you. [14:10] pedronis: i've run degraded test manually on 20.04 and 20.10 and it passed [14:10] mvo: we need to tell sergio not to promote until we have clarified these issues, though maybe it was already 2.49.1 [14:11] jam: you moved from 2.49.0 to 2.49.2, so you don't know if the issue was already in 2.49.1 ? [14:11] pedronis, we know that we are seeing it in 2.49.2, I'm not able to reproduce locally, so I'm proxying for people in AU that were trying and didn't get it working. [14:11] my understanding is they tried reverting to 2.49.1 but that didn't get it to work. [14:54] pedronis, so I've been able to reproduce it, is it just /var/log/syslog that you are looking for the apparmor issues? [14:54] jam: use `journalctl --no-pager | grep DENIED` for denials [14:56] ijohnson, what is the best place to report this, open a launchpad bug and start including logs there, or a forum post (and which forum :) [14:56] jam: let's just use the launchpad, bugs.launchpad.net/snapd [14:56] thanks [14:57] ijohnson, so if I revert snapd, what steps do I need to do to make sure the snap is installed correctly with the old logic? [14:57] jam: what is `snap list` on this machine ? [14:58] I just did 'snap revert snapd' but I might have old/new install of the other snap [14:58] https://paste.ubuntu.com/p/DH6cN2yYJb/ [15:05] jam: ok so you have both the `core` and `snapd` snaps installed, which gets a bit confusing, can you also revert the core snap wit h`snap revert core`, so that you are on 2.49 for both core and snapd, and then see if you can reproduce still ? [15:09] ijohnson, sure. I should also note that 'journalct | grep DENIED' is about 3.8M lines. And while I can just paste the end of it for you, is there a way I can get a reset so I can be sure I'm giving you fresh information? [15:09] jam: what about `journalctl -b 0 --no-pager | grep DENIED` ? [15:09] is that any shorter? [15:10] that should just be the current boot [15:10] 600k [15:10] shorter [15:10] but still a bit unweildy [15:11] but still helpful, yes [15:11] let me figure out hte options to journalctl to limit it to just the past couple of hours [15:11] ijohnson, so juju-db is using core18, presumably I should revert that one as well? [15:11] or maybe you only get snapd from core [15:11] jam: no core18 is fine [15:12] yeah exactly, snapd can come from either the core snap or the snapd snap, whichever is the newest iirc [15:12] though maybe we do have rules to always prefer the snapd snap over the core snap, I don't recall off the top of my head, but I know that re-exec like this is confusing :-| [15:12] core 16-2.49 10859 latest/stable canonical✓ core [15:12] snapd 2.49 11107 latest/stable canonical✓ snapd [15:13] juju-db 4.0.9 29 4.0/stable juju-qa - [15:13] so with those, I am still seeing the issue, though I have re installed a second time juju-db [15:13] after the revert [15:17] jam: ok, here's for the past 1 hour `journalctl --no-pager --since "$(date -d '1 hour ago' "+%Y-%m-%d %H:%M:%S")" | grep DENIED` [15:17] you can change that to however many hours you need to demonstrate when juju-db fails to start [15:18] sure. I can actually use that to 'now' go run the thing, and then give you just the recent content. Thank you [15:20] so just with the reproducer, I get: [15:20] $ sudo journalctl --no-pager --since "$dt" | grep DENIED [15:20] 42:Apr 01 10:20:25 focal audit[3611588]: AVC apparmor="DENIED" operation="open" profile="snap.juju-db.mongod" name="/sys/block/" pid=3611588 comm="mongod" requested_mask="r" denied_mask="r" fsuid=1001 ouid=0 [15:20] 44:Apr 01 10:20:25 focal kernel: audit: type=1400 audit(1617290425.219:322223): apparmor="DENIED" operation="open" profile="snap.juju-db.mongod" name="/sys/block/" pid=3611588 comm="mongod" requested_mask="r" denied_mask="r" fsuid=1001 ouid=0 [15:22] hmm doesn't seem like it would have been related, so to beclear after core at 16-2.49 and snapd at 2.49, you can still reproduce the issue with juju-db ? [15:23] reverting to core 16-2.49 and snapd 2.49 I'm still seeing the failure, yes. so it doesn't seem related to 2.49.2 at least. [15:23] jam: can you confirm the output of `snap version` too ? [15:23] $ snap version [15:23] snap 2.49 [15:23] snapd 2.49 [15:23] series 16 [15:23] ubuntu 20.04 [15:23] kernel 5.4.0-70-generic [15:24] ok, so I think we can confirm this is not related to the 2.49.2 release at least [15:24] but please file a bug on launchpad with all these details and we'll have a look [15:24] also journal of the failing unit would be useful, you can get that for the last hour with: [15:24] `journalctl -u snap.juju-db.* --no-pager --since "$(date -d '1 hour ago' "+%Y-%m-%d %H:%M:%S")" ` [15:27] ijohnson, so just to confirm, the snap I'm running appears to be: 4.0/stable: 4.0.9 2019-06-05 (29) 62MB - [15:28] which means it hasn't been changed for about 1.5years, thus isn't the source of it not working now. [15:28] right agreed, but seeing how it is failing to start will be helpful to understand what else might have changed to break it [15:29] sure. I just wanted to make sure my understanding of what I was running was the right one. :) [15:29] ello folks. I'm looking into WSL2's native gui support - they put the wayland socket at /mnt/wslg/runtime-dir/wayland-0 and the pulseaudio runtime at /mnt/wslg/runtime-dir/pulse (containing native and pid) [15:30] hey diddledan [15:30] if we want to ever support GUI snaps on WSL2 we'll need to allow those locations methinks [15:31] (I have an issue with XWayland via their native support right now, but I think that's a WSL bug rather than because I've started systemd) [15:31] ijohnson, the specific failures seem to revolve around: 2021-04-01T11:29:21.722-0400 W FTDC [initandlisten] Error getting directory iterator '/sys/block': Permission denied [15:32] if you want to play along at home, I enable systemd with https://github.com/diddledan/one-script-wsl2-systemd [15:32] jam: right, certainly seems related [15:32] specifically https://github.com/diddledan/one-script-wsl2-systemd/tree/build-21286+ [15:33] diddledan: so you think the wayland interface needs to also allow permission to read/write to /mnt/wslg/runtime-dir/wayland-0 and the audio-playback interface needs to allow permission to /mnt/wslg/runtime-dir/pulse ? [15:33] that's what I was thinking, yes. The alternative would be to expose those via bind-mounts into the snap world view [15:34] ah right because probably most linux apps are not going to look there they are probably going to look in the existing one in `/run/user/[0-9]*/wayland-[0-9]*` [15:35] right. although WSL does set XDG_RUNTIME_DIR to point to /mnt/wslg/runtime-dir but that gets nuked when we launch a strictly confined snap [15:36] hmm yeah I guess I would be curious to hear what jamesh has to say about that, I admit to being fairly ignorant about wayland specifics [15:36] me too :-) [15:36] enthusiatic amateur covers me ;-p [15:37] PULSE_SERVER=/mnt/wslg/PulseServer is set in the environment of WSL too [15:38] that's another socket file so probably also needs to be considered [15:39] they also look to be starting a dbus daemon and putting it's runtime dir at /mnt/wslg/runtime-dir/dbus-1 and also dconf at /mnt/wslg/runtime-dir/dconf [15:44] ijohnson, I was trying to create a reproducer with a more minimal configuration. however, I can't figure out where the charm is *supposed* to be able to write data. $SNAP_DATA it complains is read-only [15:45] jam: are you running as root? [15:45] $SNAP_DATA is only writable by root, see [15:46] k. Well historically we use 'juju-db.mongod' to initialize our test suite, but any dir under $HOME it is now telling me that it is readonly [15:46] so I was trying to find a better dir for it to play in [15:46] hmm, $HOME should be writable [15:49] so I was able to 'snap run --shell juju-db.mongod mkdir /tmp/foo' and create stuff there (which looks to be /tmp/snap.juju-db/tmp/foo) which at least gets me past the readonly error [15:52] jam if you can't get a minimal reproducer that's fine, the full reproducer should be useful anyways [15:52] I do note that you are using charms, which adds a whole other layer of fun on top of snaps [15:53] ;-) [15:54] ijohnson, so in this case, I'm just running a test suite locally, which means it is *just* snaps [15:54] ah perfect [16:04] ijohnson: I've documented all my findings about the WSL gui support in a bug for snapd https://bugs.launchpad.net/snapd/+bug/1922262 [16:04] Bug #1922262: Add support for WSL2 Native GUI capability [16:04] nice thanks for that [16:35] pedronis, ijohnson 10107 looks promising, first run did not error in prepare anywhere, I triggered a re-run [16:37] Nice === ijohnson is now known as ijohnson|lunch [16:49] * mvo calls it a day but I will check mattermost and tg [19:06] PR snapd#10102 closed: daemon: introduce apiBaseSuite.(json|sync|async|error)Req (and some apiBaseSuite cosmetics) === ijohnson|lunch is now known as ijohnson [22:11] who do I have to sweet-talk into making it so the store won't issue a 500 error when you upload a >~1GB snap?!