[03:44] <amurray> would be good to get snaps listed on this page https://github.com/google/sandboxed-api/blob/master/sandboxed_api/docs/sandbox-overview.md
[06:09] <mborzecki> morning
[06:09] <zyga> Good morning
[06:10] <mborzecki> just been outside, really feels like spring
[06:10] <zyga> amurray: interesting
[06:10] <zyga> mborzecki: I’m just stepping out
[06:10] <zyga> My back is killing me today, way too many hours coding yesterday
[06:11] <zyga> I will need a longer walk today
[06:11] <mborzecki> zyga: buy a standing desk
[06:11] <zyga> Eventually :-)
[06:12] <zyga> I need to buy new NAS disks next month
[06:12] <mborzecki> zyga: every couple of months tehre are some nice discouts at ikea
[06:15] <zyga> Does one go with mdraid or zfs nowadays?
[06:17] <zyga> I plan to get 3x(~5) TB in raid-5 and a 10+ TB drive for weekly snapshot off-case in case the main nas melts
[06:46] <mborzecki> looking at gadget validation again, obviously forgot to check that size must be set at the structure level :/
[07:02] <zyga> back in the office
[07:13] <zyga> mborzecki: could you please look at https://github.com/snapcore/snapd/pull/6636
[07:13] <mup> PR #6636: cmd: prevent umask from breaking snap-run chain <Created by zyga> <https://github.com/snapcore/snapd/pull/6636>
[07:13] <zyga> it's not big but I want a second pair of UNIXy eyes :)
[07:16] <mborzecki> zyga: LOOKING
[07:20] <mborzecki> damn caps
[07:21] <mborzecki> mvo: morning
[07:21] <mvo> hey mborzecki
[07:21] <mvo> mborzecki: good morning
[07:29] <mborzecki> zyga: lgtm, left a comment about trying out umask 000 in the spread test too
[07:30] <mborzecki> after all if it's supposed to be restored fully
[07:38] <zyga> mborzecki: thank you, I will add more tests
[08:02] <zyga> linux is funny
[08:02] <zyga> mborzecki: you can have a process that lives in a current working directory that doesn't exist in its own mount namespace at all
[08:02] <zyga> funny things happen
[08:05] <pstolowski> morning
[08:07] <mborzecki> pstolowski: hey
[08:08] <zyga> hey pawel :)
[08:08] <zyga> hey mvo
[08:14] <mvo> hey zyga and pstolowski
[08:23] <ackk> zyga, hi, I just hit that issue with the snap (under try) reporting not found on executables inside the snap
[08:23] <ackk> zyga, how do I cat mountinfo in the right ns again?
[08:24] <zyga> hey! thank  you for asking: use sudo nsenter -m/run/snapd/ns/foo.mnt
[08:24] <zyga> then inside the new shell run cat /proc/self/mountinfo
[08:24] <zyga> this information will be very useful
[08:24] <zyga> a bit of usabilty warning: there cannot be a space between -m and the path
[08:25] <ackk> ah that's what was hitting me
[08:26] <ackk> zyga, btw strace doesn't seem to give much info: https://paste.ubuntu.com/p/xf3WQRZWyw/
[08:28] <ackk> zyga, what should I look for in the mountifo?
[08:28] <zyga> deleted
[08:28] <zyga> but I would love to see the whole listing
[08:28] <ackk> zyga, http://paste.ubuntu.com/p/zBq4X3TMXb/
[08:28] <ackk> (this is in the ns)
[08:28] <ackk> no deleted
[08:29] <zyga> interesting
[08:29] <zyga> hmm hmm
[08:29] <zyga> can you run snap run --shell maas?
[08:29] <ackk> same errorr (execv failed)
[08:30] <zyga> hmmm
[08:30] <zyga> mborzecki: ^ any ideas
[08:30]  * zyga reads the mountinfo data 
[08:31] <ackk> zyga, ftr I've run the rsync like hundred times yesterday and no issue
[08:31] <ackk> it's always like that, it randomly happens once in a while
[08:31] <zyga> ackk: what I find curious is that you can clearly nsenter
[08:31] <zyga> ackk: what's the base snap?
[08:31] <ackk> zyga, core18
[08:31] <zyga> ackk: what are you tracking (which channel of the base snap)
[08:32] <zyga> ackk: did it refresh recently?
[08:32] <ackk> oh, I'm on core18 edge
[08:32] <mborzecki> hm that strace doesn't have much content
[08:32] <ackk> ah yes esterday
[08:32] <mborzecki> is that a classic snap?
[08:32] <ackk> refresh-date: yesterday at 13:38 UTC
[08:32] <zyga> ackk: did it break at around the same time it refreshed?
[08:33] <zyga> ackk: ahhh
[08:33] <zyga> I know :D
[08:33] <zyga> man
[08:33] <zyga> hold on please
[08:33]  * ackk curious
[08:34] <zyga> ah, no :/
[08:34] <zyga> man, I thought this is it
[08:34] <zyga> we don't support try mode base snaps
[08:34] <zyga> but that's clearly not it
[08:34] <mborzecki> zyga: looks like it faiedl executing snap-confine, unless i'm missing something
[08:34] <zyga> ackk: when you rsync, what do you do exactly?
[08:35] <zyga> Permission denied is curious
[08:35] <ackk> zyga, https://github.com/maas/maas/blob/master/Makefile#L781
[08:35] <zyga> ackk: do you have any apparmor denials?
[08:36] <ackk> zyga, it doesn't seem so
[08:36] <ackk> I mean I have stuff in dmesg but I don't see new errors if I try to run it
[08:36] <mborzecki> ackk: can you run /snap/core/6531/usr/lib/snapd/snap-confine --base core18 snap.maas.maas /usr/lib/snapd/snap-exec maas at all?
[08:36] <zyga> mborzecki: don't forget SNAP_INSTANCE_NAME=maas
[08:37] <ackk> no, same error
[08:37] <zyga> permission?
[08:37] <zyga> check dmesg again please
[08:37] <ackk> $ SNAP_INSTANCE_NAME=maas /snap/core/6531/usr/lib/snapd/snap-confine --base core18 snap.maas.maas /usr/lib/snapd/snap-exec maas
[08:37] <ackk> execv failed: No such file or directory
[08:37] <zyga> wait
[08:37] <zyga> before it was a permission denied error
[08:37] <ackk> no new error
[08:37] <zyga> ackk: since this a  core18 snap
[08:37] <zyga> snapd (and snap-exec) come from another snap
[08:37] <zyga> can you check one more thing
[08:38] <zyga> run that same SNAP_INSTANCE_NAME=... line but run /bin/sh at the end
[08:38] <zyga> instead of snap-exec that is
[08:38] <zyga> once on the inside
[08:38] <zyga> run ls -ld /usr/lib/snapd/snap-exec
[08:38]  * zyga looks at mountinfo list again
[08:38] <zyga> er
[08:38] <mborzecki> zyga: s-c does some work, if it ran i'd expect to see taht in strace log
[08:38] <ackk> $ SNAP_INSTANCE_NAME=maas /snap/core/6531/usr/lib/snapd/snap-confine --base core18 snap.maas.maas /bin/ls -ld /usr/lib/snapd/snap-exec
[08:38] <ackk> /bin/ls: cannot access '/usr/lib/snapd/snap-exec': No such file or directory
[08:39] <zyga> there's no /usr/lib/snapd!
[08:39] <ackk> there is but it's empty
[08:39] <zyga>  SNAP_INSTANCE_NAME=maas /snap/core/6531/usr/lib/snapd/snap-confine --base core18 snap.maas.maas /bin/sh
[08:39] <zyga> then cd /usr/lib
[08:39] <zyga> ls -ld snapd
[08:39] <zyga> ls -l snapd
[08:40]  * zyga checks stuff
[08:40] <ackk> $ ls -lad snapd
[08:40] <ackk> drwxr-xr-x 2 root root 0 Mar 21 11:41 snapd
[08:40] <ackk> $ ls -l snapd
[08:40] <ackk> total 0
[08:40] <ackk> the dir is there, but empty
[08:40] <ackk> is that a mountpoint?
[08:40] <ackk> I mean, does snapd bind-mount stuff on top?
[08:41] <zyga> it should be a mountpoint
[08:41] <ackk> zyga, so you think it could be related to the core refresh?
[08:41] <zyga> that explainswhat you are seeing
[08:41] <zyga> perhaps
[08:42] <zyga> can you collect snap changes and add this to the bug report
[08:42] <ackk> it might well be that's been broken since yesterday,but I haven't restarted the service (nor called the cli) since then, although I think I did
[08:42] <zyga> ackk: do you have snapd.snap?
[08:42] <ackk> zyga, where?
[08:42] <zyga> is snapd the snap installed on your system
[08:43] <ackk> no
[08:43] <zyga> snap list | grep snapd :)
[08:43] <zyga> ok
[08:43] <zyga> thank you
[08:43] <zyga> ackk: I think we have enough to explore this now
[08:43] <zyga> the key question is what happens when core18 + core + app combo has a refresh of core18
[08:43] <zyga> or a refresh of core
[08:43] <zyga> when core refreshes it's different
[08:43] <zyga> actually
[08:43] <zyga> that's a separate bug
[08:43] <zyga> we will never update the tooling
[08:43] <zyga> mvo: core18/core bug
[08:44] <ackk> are core18 apps still somewhat dependent on core?
[08:44] <ackk> I think I had filed a bug a long time ago about it, and I've still seen the issue recently
[08:44] <zyga> mvo: when /usr/lib/snapd comes from core on a core18 base snap app, the core refresh is not reflected in the mount namespace of the app
[08:44] <zyga> that is
[08:44] <mvo> zyga: they shouldn't be - hm, hm
[08:44] <zyga> mvo: future snapd is using old snap-exec
[08:44] <zyga> mvo: it's like a stale base snap prolem
[08:44] <mvo> zyga: oh, snap-exec is not coming from snap?
[08:44] <zyga> mvo: but a stale core snap sub-problem
[08:44] <mvo> zyga: not from the snapd snap?
[08:45] <zyga> mvo: when it comes from core
[08:45] <pedronis> there is snapd snap aroung yet
[08:45] <zyga> mvo: core18 + core (as source of snap tools) + app
[08:45] <pedronis> there is *no*
[08:45] <zyga> mvo: core refreshes
[08:45] <zyga> mvo: now you have stale tools in the app snap
[08:45] <mvo> ok
[08:45] <zyga> mvo: or perhaps that is exactly what ackk observed, an tools just disappear somehow
[08:45] <ackk> zyga, fwiw "core" refreshed 10 days ago
[08:45] <pedronis> zyga: basically  all the snaps that compose the mount namespace can have a staleness prolem no?
[08:45] <pedronis> also content snaps
[08:45] <zyga> mvo: thank you
[08:46] <zyga> pedronis: yes
[08:46] <mvo> zyga: what do we need to do to unstable the mount namespace?
[08:46] <pedronis> zyga: we check only base though?
[08:46] <zyga> pedronis: content snaps are better
[08:46] <zyga> pedronis: because those are new paths that change in the mount profile
[08:46] <zyga> pedronis: so they are correctly updated
[08:46] <zyga> pedronis: it's just the core tooling for now
[08:46] <zyga> pedronis: nothing else exists outside of the snap-update-ns system
[08:47] <zyga> pedronis: yes, we check for base only explicitly because that's the do-I-rebuild-from-scratch decision
[08:47] <zyga> mvo: can you rephrase the question please?
[08:47] <ackk> zyga, should I try refreshing core18 (to stable perhaps)?
[08:47] <zyga> ackk: to aid in debugging? no, to lower the inconvenience for your development: yes
[08:47] <mvo> zyga: what do we need to do to fix this .) ?
[08:47] <zyga> ackk: you can discard the mount namespace to fix it
[08:48] <zyga> mvo: I dont know yet, I don't fully undestand the reason /usr/lib/snapd was empty
[08:48] <zyga> but I think we should be able to reproduce it now
[08:48] <ackk> zyga,  the var/lib/snapd in the snap?
[08:48] <zyga> ackk: sorry?
[08:48] <zyga> mvo: perhaps it's one issue after all
[08:48] <ackk> zyga, oh sorry, discard the namespace I see
[08:49] <ackk> zyga, I thought you meant unmounting something
[08:49] <zyga> mvo: sudo /usr/lib/snapd/snap-discard-ns mass
[08:49] <zyga> er
[08:49] <zyga> ackk: ^ :)
[08:49] <ackk> zyga, thanks
[08:49] <ackk> zyga, oh refreshing to stable also worked
[08:50] <zyga> yes
[08:50] <zyga> because then you have new base
[08:50] <zyga> and you get a fresh mount ns
[08:50] <ackk> I see
[08:50] <zyga> ackk: thank you very much for this
[08:50] <zyga> please do report the bug
[08:51] <zyga> we may split it if this turns out to be a pair of issues
[08:51] <ackk> zyga, thank you for digging it
[08:51] <zyga> I will work on it next week for sure
[08:51] <pedronis> fyi, I need to do some non-PR work before getting back to reviewing
[08:52] <ackk> zyga, is "snap stops working after base refresh" accurate as a title?
[08:52] <zyga> ackk: /usr/lib/snapd unmounted in app + core18 + core (tooling) use case; possibly on core refresh
[08:52] <zyga> that is the summary in my head
[08:53] <ackk> zyga, but it was a core18 refresh, not core
[08:53] <zyga> ah, sorry
[08:53] <zyga> I will check all combinations though
[08:54] <zyga> there are clearly missing bits there
[08:55] <zyga> brb, coffee ran out
[08:59] <ackk> zyga, https://bugs.launchpad.net/snapd/+bug/1821302
[08:59] <mup> Bug #1821302: /usr/lib/snapd unmounted in app + core18 + core (tooling) use case; possibly on core18 refresh <snapd:New> <https://launchpad.net/bugs/1821302>
[09:06] <zyga> ack :)
[09:07] <zyga> this is the week of snap-confine fixes and changes
[09:22] <ackk> zyga, IDK if it could be related, but I still see the issue where if I install a core18-based snap on a clean system (no core snap) it fails to install (the configure hook fails)
[09:22] <ackk> zyga, e.g. snap install --edge maas
[09:22] <zyga> ackk: yes, this is related
[09:22] <zyga> ackk: we track that bug separately now
[09:22] <ackk> zyga, do you have the link?
[09:22] <zyga> ackk: I have a draft solution on my system but it will not be ready yet
[09:24] <zyga> ackk: there's a forum thread aboug this, i have not upgraded it to a bug report yet: https://forum.snapcraft.io/t/problems-installing-a-base-core18-snap-on-bionic/10084/
[09:25] <ackk> zyga, ah cool thanks
[09:28] <zyga> mborzecki: https://www.omgubuntu.co.uk/2019/03/linux-data-cap-survey
[09:29] <mborzecki> aah survey
[09:29] <mborzecki> nice
[09:31] <pstolowski> mvo: hey, added some extra tests you suggested for #6591
[09:31] <mup> PR #6591: overlord/ifacemgr: basic measurements <Created by stolowski> <https://github.com/snapcore/snapd/pull/6591>
[09:33] <mborzecki> zyga:  haha 'RSS feeds' in question 'How would you prioritise these data to download if you had limited internet access?'
[09:33] <zyga> yeah
[09:33] <zyga> who cares about that
[09:33] <zyga> who ordered RSS?
[09:34] <mvo> pstolowski: thanks, looking
[09:35] <zyga> completed the survey
[09:35] <zyga> as someone who is on capped links 24/7 it's important that people care about this
[09:36] <mup> PR snapd#6637 opened: interfaces: deal with the snapd snap correctly for apparmor 2.13 <Created by mvo5> <https://github.com/snapcore/snapd/pull/6637>
[09:38] <pedronis> mvo: there's a card for that https://trello.com/c/1h4hrY3W/180-implement-snapd-snap-support-in-apparmor-backend-dbus-backend-setup-etc  ^
[09:39] <pedronis> also I think there were todo left in other backends too
[09:40] <mvo> pedronis: indeed, added the PR and will see what else needs to be done there
[10:00] <pedronis> mvo: should also be moved to doing :)
[10:31] <pedronis> mborzecki: hi, do you know why we would get a line like this:
[10:31] <pedronis> content                   gnome-calculator:gtk3-themes      gtk-common-themes:gtk3-themes
[10:31] <pedronis> in connections,  there is not content discriminant there
[10:31] <mborzecki> pedronis: that with 2.38?
[10:31] <pedronis> with edge
[10:32] <pedronis> but anyway I get the other ones with []
[10:32] <pedronis> but not that one
[10:32] <pedronis> I thought we always set one internally
[10:32] <pedronis> mborzecki: I'm doing snap connections gnome-logs
[10:32] <mborzecki> pedronis: mhm, installing the snap right now
[10:34] <mborzecki> hmmm pinner with 'Automatically connect eligible plugs and slots of snap "gnome-calculator"', didn'g we have a fix that would switch to showing progress of the download if a dependncy is being installed?
[10:34] <mborzecki> s/pinner/spinner/
[10:36] <mborzecki> pedronis: .38 has 		//
[10:36] <mborzecki> content[gnome-3-28-1804]  gnome-calculator:gnome-3-28-1804  gnome-3-28-1804:gnome-3-28-1804  -
[10:37] <mup> PR snapd#6638 opened: interfaces: add support for the snapd snap in the dbus backend  <Created by mvo5> <https://github.com/snapcore/snapd/pull/6638>
[10:37] <pedronis> mborzecki: I'm talking about gtk3-themes
[10:37] <pedronis> but I don't see that in the snaps
[10:37] <pedronis> mmh
[10:37] <mborzecki> content[gtk-3-themes]     gnome-logs:gtk-3-themes     gtk-common-themes:gtk-3-themes   -
[10:37] <mborzecki> content[icon-themes]      gnome-logs:icon-themes      gtk-common-themes:icon-themes    -
[10:38] <mborzecki> content[sound-themes]     gnome-logs:sound-themes     gtk-common-themes:sound-themes   -
[10:38] <pedronis> mborzecki: it's the issue that we don't remove non-existent connections I suppose
[10:38] <pedronis> mborzecki: it's  not a connections problem, it's other issues
[10:38] <pedronis> mborzecki: if you just installed it you won't see that
[10:39] <mborzecki> pedronis: buf if i disconect, remove and reinstall?
[10:39] <pedronis> yes, a different case
[10:47] <zyga> mvo: fun bug, command-not-found is still in use while in snap run- --hell
[10:47] <zyga> (I meant --shell)
[10:47] <zyga> but the binary is not there anymore
[10:52] <pstolowski> failed to prepare google:opensuse-15.0-64:project, do we have an issue with opensuse in gcloud again?
[10:54] <mup> PR core18#122 opened: hooks: add apt/apt-get/apt-cache warning <Created by mvo5> <https://github.com/snapcore/core18/pull/122>
[11:03] <zyga> pstolowski: what was the failure?
[11:04] <zyga> pstolowski: ah, I see it now
[11:04] <zyga> cachio: ^ we need to change how we install packages
[11:04] <zyga> mborzecki: ^ any ideas, is that a stale cache?
[11:05] <zyga> openSUSE Leap 15.0 package installation issue in spread https://www.irccloud.com/pastebin/Z5UvBD4B/
[11:06] <mborzecki> hmm
[11:07] <pstolowski> zyga: interesting, i didn't see any error output for google:opensuse-15.0-64:project, just failed to prepare
[11:07] <zyga> I saw this in one of my own runs
[11:07]  * zyga needs to take a break
[11:08] <zyga> to avoid the chronic back pain
[11:08] <zyga> mborzecki: there will be an interesting patch later today
[11:08] <zyga> mborzecki: another unixy thing :)
[11:09] <mborzecki> no clue about that opensuse thing, perhaps unfinished mirror sync?
[11:09] <zyga> I think it's like dnf upgrade vs dnf distrosync
[11:10] <pedronis> pstolowski: is there anything left for 6491 that can't be follow ups about improving the test?
[11:10] <zyga> we are probably doing something wrong and should be using "dnf dup" instead of another operation
[11:10] <mborzecki> but it's leap, zypper up should be enough
[11:10] <zyga> mborzecki: up != dup
[11:10] <zyga> mborzecki: but not sure :/
[11:10] <zyga> anyway, I spawned some larger testing for an important snap-confine change, I will break for an hour to move around a bit
[11:10] <mborzecki> zyga: i mean zypper up should be enough for leap, tw has it's own weirdnness :)
[11:10] <zyga> I'm on telegram if anyone needs me urgently
[11:11] <MattJ> \\\\\\!
[11:11] <MattJ> Sorry, words of wisdom from my daughter there
[11:12] <zyga> haha :)
[11:16] <ackk> zyga, just to confirm, I hit the issue again (no core refresh this time) and calling snap-discard-ns fixed it
[11:19] <ackk> zyga, actually not so much, now I get a "read only filesystem" error
[11:21] <ackk> cannot create user data directory: /root/snap/maas/x1: Read-only file system
[11:25] <zyga> ackk: x1 got unmounted somehow?
[11:26] <ackk> zyga, well I suspect it's because of the discard ns ?
[11:26] <zyga> ackk: is your snap "broken" (as in snap list shows broken next to it)
[11:26] <zyga> no, that's not doing that
[11:26] <ackk> no it wasn't
[11:26] <zyga> discard will not change /snap/anything
[11:26] <ackk> (now I actually removed it)
[11:26] <zyga> ah, wait
[11:26] <zyga> I misread that
[11:26] <zyga>  what is /root/snap/maax/x1
[11:26] <zyga> is that a bind mount?
[11:27] <zyga> it would only be a ROFS in case it's a bind mount or if /root inside the mount namespace was unmounted
[11:27] <zyga> but if _that_ is the case then I would really like to know what's going on
[11:28]  * zyga is AFK now
[11:29] <ackk> zyga, it's the user dir
[11:39] <mborzecki> the fun start when offset 0 is a valid value
[11:40] <mborzecki> looks like some of the fields in our gadget structures should be pointers
[11:46] <mborzecki> off to the kids' school for a bit
[12:12] <pstolowski> pedronis: sorry, missed your earlier message; no, nothing left to do in #6491
[12:12] <mup> PR #6491: interfaces: hotplug nested vm test, updated serial-port interface <Hotplug 🔌> <Created by stolowski> <https://github.com/snapcore/snapd/pull/6491>
[12:14] <pstolowski> nb it failed on 14.04 with 'system does not fully support snapd: cannot mount squashfs image' ?!
[12:14] <pstolowski> among 2 other unrelated test failures.. interfaces-daemon-notify seems to be flaky
[12:17] <geodb27> People : hi ! I've come to this issue with the snap version of lxd : https://github.com/lxc/lxd/issues/5590 is that related to snap or to lxd and is there a way to get things work as they are expected to ? To be more specific, I guess that snap don"t like user's home directory to be out of /home
[12:19] <pstolowski> geodb27: correct, user homes need to be in /home currently, no way around that
[12:21] <geodb27> So, my guess was right, thanks for you answer pstolowski. However, to my knowledge, snap is the only one I encounter that relies on this. Is there any technical reason for that (I'm only curious) ? There is an env var set at user login which is filled in with the home directory. Why not use it ?
[12:21] <pedronis> pstolowski: did it have a recent master merge?
[12:21] <pstolowski> pedronis: hmm possibly not, i merged yesterday. doing
[12:26] <pstolowski> geodb27: it's more complicated due to confinement (apparmor profiles etc)
[12:28] <geodb27> oki, I see, thanks again for your kind explanation pstolowski. This will lead me to to many changes on my servers... I'll see how I can go with the less damages.
[12:28] <pstolowski> geodb27: you're welcome!
[12:35] <cachio> zyga, taking a look  to opensuse now
[12:36] <zyga> geodb27: I can refer you to the technical reasons but I am AFK now
[12:36] <zyga> Thanks cachio
[12:53] <mborzecki> re
[12:58] <mup> PR snapd#6639 opened: snap: tweak parsing errors of gadget updates <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6639>
[13:19] <ackk> zyga, why would snapd try to create that directory (/root/snap/maas/x1 when starting a service?
[13:30] <zyga> ackk: we create SNAP USER DATA
[13:30] <zyga> because snaps themselves cannot
[13:31] <ackk> yeah but that dir is there
[13:31] <ackk> why does the snap  suddenly thinks it's readonly
[14:41] <mup> PR snapd#6640 opened: spread: refresh metadata on openSUSE <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6640>
[14:47] <mborzecki> zypper tell me the latest version of libudev is 234-lp150.20.15.1, not 234-lp150.19.1.x86_64
[14:49] <mborzecki> cachio: was opensuse-15.0 image already updated?
[14:57] <diddledan> did the forum fall over? https://usercontent.irccloud-cdn.com/file/kZeiOkOL/image.png
[15:02] <diddledan> seems opening a different tab rectified it
[15:02] <diddledan> ignore me
[15:06] <cachio> mborzecki, no yet
[15:07] <cachio> mborzecki, found more problems
[15:07] <cachio> mborzecki, some package digests are different
[15:20] <cachio> mborzecki,  the image is ready and uploaded
[15:21] <cachio> uploading
[15:28] <mborzecki> cachio: cause before asking i tried running zypper in <packages> that were failing in the test and it worked, so maybe just a mirror sync problem
[15:28] <mborzecki> cachio: still, zypper up show a number of updates avaialble at that point, will try with an updated image now
[15:33] <cachio> mborzecki, yes, the mirrors are giving us problems
[15:33] <cachio> mborzecki, I already updated all the packages to make the update process faster
[15:33] <cachio> I is almost uploaded
[15:36] <cachio> mborzecki, image ready
[15:38] <mborzecki> cachio: it probably was something with mirror sync, #6640 started before an updated image was uploaded and it's green
[15:38] <mup> PR #6640: spread: refresh metadata on openSUSE <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6640>
[15:41]  * cachio lunch
[15:42] <Wimpress> The next Snapcraft Summit has been announced - https://snapcraft.io/blog/snapcraft-summit-montreal
[15:46] <mup> PR snapd#6641 opened: snap-gdb-shim: switch to the SUDO_UID when available <Created by mvo5> <https://github.com/snapcore/snapd/pull/6641>
[15:46] <kenvandine> zyga: we're continuing to see reports on the forum of users not able to run the gnome snaps because the content interface isn't mounted
[15:47] <kenvandine> zyga: it's something i've only seen a couple times and  snap-discard-ns seems to resolve it
[15:48] <kenvandine> zyga: latest reference https://forum.snapcraft.io/t/gnome-calculator-problem/4579
[15:48] <mup> PR snapd#6640 closed: spread: refresh metadata on openSUSE <Created by bboozzoo> <Merged by stolowski> <https://github.com/snapcore/snapd/pull/6640>
[15:50] <kenvandine> zyga: if i can't find an existing bug report, i'll file a new one
[15:51] <pstolowski> pedronis: you've requested John's review on #6591; is it required or can it land now that it has mvo's review?
[15:51] <mup> PR #6591: overlord/ifacemgr: basic measurements <Created by stolowski> <https://github.com/snapcore/snapd/pull/6591>
[15:51] <pedronis> pstolowski: it can land, I asked him in case mvo didn't have time
[15:51] <pstolowski> ack
[15:52] <pstolowski> thx
[15:52] <mup> PR snapd#6591 closed: overlord/ifacemgr: basic measurements <Created by stolowski> <Merged by stolowski> <https://github.com/snapcore/snapd/pull/6591>
[15:54] <kenvandine> zyga: bug 1785410
[15:55] <mup> Bug #1785410: Can not open Gnome snaps as they are not connected to «gnome platform snap» <amd64> <apport-bug> <bionic> <snapd (Ubuntu):Confirmed> <https://launchpad.net/bugs/1785410>
[16:05] <popey> zyga: what do we need to do to bump the priority of that bug? It's pretty harmful as people use it as a reason to remove snaps entirely
[16:05] <kenvandine> mvo: ^^
[16:07] <mvo> kenvandine: in a meeting, adding it to our trello, we get back to you
[16:07] <kenvandine> mvo: thanks
[16:09] <diddledan> user: "compiling this application is difficult with no docs on what it depends upon proving that linux sucks" my reply: "I made a snap of this app so you don't need to compile it!" their retort: "another packaging system isn't the solution. compiling stuff is proving that linux is terrible" <-- I tried to point out that compiling the application on Windows or Mac would be just as difficult, but they insisted that it proves
[16:09] <diddledan> linux is crap
[16:10] <mvo> kenvandine: I have not looked at all at this though, sorry :/ will also check with zyga
[16:10] <diddledan> specifically starruler2 (game) was what they were trying to compile
[16:17] <zyga> re
[16:17]  * zyga reads backlog
[16:18] <zyga> kenvandine: I will investigate several related reports next week,
[16:18] <zyga> kenvandine: thank you for the bug report
[16:19] <zyga> kenvandine: right now no immediate smoking gun but we know of one issue related to core18
[16:19] <zyga> kenvandine: but perhaps there's more magic to uncover there
[16:20] <zyga> kenvandine: anyone with instructions on how to reporduce would be awesome as all of those bugs are "not sure how I got there but..."
[16:21] <zyga> kenvandine: I will write a few new tests next week, specifically when there's a mount namespace consisting of an app snap a base snap (e.g. core18) and the core or snapd snaps providing the tools, each of those can refresh / change
[16:21] <zyga> kenvandine: we don't have full test coverage for that scenario as we learned
[16:21] <zyga> kenvandine: all of the snap-update-ns / snap-confine bugs are my priority and I will investigate them ASAP
[16:22] <kenvandine> zyga: yeah, this is something we've seen randomly for ages now, since the gnome-24 content snap
[16:23] <zyga> kenvandine: we fixed one known issue: base snap refresh was buggy (in two ways actually)
[16:23] <zyga> kenvandine: we know of several more issues: when base snap changes (core -> core18) we do bad stuff if the app is still running
[16:24] <zyga> kenvandine: and when the snapd tooling (snap-exec snapctl) is coming from core/snapd on a snap using base: core18 there is no handling for that becoming stale
[16:24] <zyga> kenvandine: I was discussing with ackk earlier today when his try snap was misbehaving
[16:24] <zyga> kenvandine: and some of the observations indicated that various things got unmounted
[16:25] <zyga> kenvandine: next week I will collect all the bugs and recent forum postings, try to file missing bugs and triage them, then write extra tests to see if anything can be reproduced this way
[16:42] <kenvandine> zyga: thanks
[16:42] <pedronis> pstolowski: I created a meeting for Monday, let me know if it doesn't work
[16:44] <pstolowski> pedronis: it's fine, thank you
[17:24] <cachio> mvo, I see this error on nightly test suite https://paste.ubuntu.com/p/Bp2xw9H8p6/
[17:28] <mvo> cachio: this looks like a store timeout: Client.Timeout
[17:28] <mvo>        exceeded while awaiting headers
[17:28] <cachio> mvo, it is weird because it failed many times
[17:29] <cachio> yesterday and today
[17:29] <mvo> cachio: oh, ok - maybe a real problem on the store, might be best to ask in #snapstore
[17:29] <cachio> ok
[17:34]  * zyga resumes work on the cwd fixes
[18:54] <zyga> first test corrected
[18:55] <LinAGKar> I can't run any snaps on OpenSUSE Tumbleweed. I just get "cannot read mount namespace identifier of pid 1: Permission denied", and prevously I have gotten "libmount.so.1 snap shared library not loading".
[18:56] <zyga> second test also corrected
[18:56] <zyga> jdstrand: do you want to see it?
[18:56] <zyga> LinAGKar: hello
[18:56] <zyga> LinAGKar: what version of snapd are you on?
[18:57] <zyga> LinAGKar: can you look at /var/log/audit/audit.log (as root) and grep for "DENIED"
[18:57] <zyga> LinAGKar: give me the denials please
[18:57] <zyga> and thank you for reporting this!
[18:58] <LinAGKar> I have `type=AVC msg=audit(1553279757.513:482): apparmor="DENIED" operation="ptrace" profile="/usr/lib/snapd/snap-confine" pid=24430 comm="snap-confine" requested_mask="read" denied_mask="read" peer="unconfined"` there.
[18:59] <zyga> LinAGKar: wow, thats' curious
[18:59] <zyga> LinAGKar: uname -a?
[18:59] <LinAGKar> zyga: I have snapd 2.37.4-1.8
[18:59] <zyga> I will upload 2.38 tomorrow
[18:59] <zyga> I wonder if that's something that was fixed there
[18:59] <LinAGKar> zyga: `Linux sudda.kvasta 5.0.2-1-default #1 SMP Thu Mar 14 08:29:17 UTC 2019 (d1f1d19) x86_64 x86_64 x86_64 GNU/Linux`
[18:59] <zyga> 5.0 hmm
[18:59] <zyga> maybe apparmor regression
[19:00] <zyga> LinAGKar: thank you for reporting this, would you mind opening a bug
[19:00] <zyga> LinAGKar: can be either launchpad or opensuse bugzilla
[19:00] <zyga> LinAGKar: I will fix it, if I can, for next week
[19:00] <zyga> and sorry for the trouble
[19:02] <zyga> jdstrand: ^ this is also interestign
[19:02] <zyga> *interesting even :-)
[19:03] <LinAGKar> zyga: I'll go file a bug report.
[19:03] <zyga> thank you very much
[19:08] <jdstrand> the snap-confine profile already has ptrace read peer=unconfined,
[19:09] <zyga> yeah
[19:09] <zyga> maybe some funky 5.0 behavior?
[19:09] <jdstrand> I'm on 5.0
[19:09] <zyga> but with ubuntu sauce
[19:09] <zyga> perhaps the non-upstreamed patches are relevant here
[19:09] <jdstrand> no sauce for ptrace
[19:09] <zyga> aha
[19:09] <zyga> well, that's a good data point
[19:09] <zyga> I will attempt to reproduce this as soon as I'm done with this patch
[19:10]  * jdstrand is still heads down on other things. can't look at cwd today
[19:10] <zyga> jdstrand: sure :)
[19:10] <zyga> jdstrand: we'll chat next week
[19:10] <zyga> I will finish the extra descriptions and open the PR
[19:10] <zyga> enjoy your weekend :)
[19:11] <LinAGKar> zyga: Bug report here: https://bugs.launchpad.net/snapd/+bug/1821396
[19:11] <mup> Bug #1821396: cannot read mount namespace identifier of pid 1: Permission denied, on OpenSUSE Tumbleweed with Linux 5.0 <snapd:New> <https://launchpad.net/bugs/1821396>
[19:11] <zyga> LinAGKar: assigned, thank you
[19:12] <zyga> I will check if 2.38 fixes it and upload ASAP
[19:12] <zyga> if not, I will debug further
[19:12] <LinAGKar> I guess I didn't need to put it her myself.
[19:13] <zyga> LinAGKar: snapd will soon-ish be in the opensuse repository
[19:14] <zyga> LinAGKar: I'm sure this will improve the quality aspect as it will be so much easier to just try it
[19:14] <LinAGKar> zyga: In the official repos?
[19:15] <zyga> yes
[19:15] <zyga> I suspect that 2.37.5 will be first
[19:15] <zyga> and then 2.38 and onwards, I'm not sure what the release schedule impact will be though, yet
[19:39] <zyga> regression test added
[19:49] <zyga> mvo: https://github.com/snapcore/snapd/pull/6642 patch of the day
[19:49] <mup> PR #6642: cmd/snap-confine: prevent cwd restore permission bypass <Created by zyga> <https://github.com/snapcore/snapd/pull/6642>
[19:49] <mup> PR snapd#6642 opened: cmd/snap-confine: prevent cwd restore permission bypass <Created by zyga> <https://github.com/snapcore/snapd/pull/6642>
[19:49] <mvo> zyga: nice
[19:54] <zyga> LinAGKar: I've booted my TW machine, let me update to see if I'm missing something
[19:54] <zyga> LinAGKar: on 5.0.2-1 it is still working but I often use bleeding edge local builds
[19:57] <zyga> jdstrand: I pushed the regression test for TIOCSTI
[19:57] <mup> PR snapd#6643 opened: tests: deny ioctl - TIOCSTI with garbage in high bits <Created by zyga> <https://github.com/snapcore/snapd/pull/6643>
[19:57] <zyga> okay, that's all for today for me
[19:58] <zyga> have a fantastic weekend everyone
[19:58] <zyga> LinAGKar: in case it keeps working for me, can you think of any customization you did on top of the vanilla installation?
[20:01] <LinAGKar> zyga: I've done quite a bit, hard to tell what would affect it. This machine did originally run OpenSUSE 13.2.
[20:02] <zyga> aha
[20:02] <zyga> I will investigate
[20:02] <zyga> not giving up :)
[20:02] <LinAGKar> zyga:  I've also had it suspended, no sure if that affects it.
[20:02] <zyga> but not tonight, need to sleep
[20:02] <zyga> no, it would not
[20:02] <zyga> can you be on IRC next week?
[20:03] <LinAGKar> zyga: I will.
[20:03] <zyga> thank you
[20:03] <zyga> enjoy your weekend too :) see you on Monday
[20:03] <LinAGKar> Enjoy your too
[20:03] <LinAGKar> Thank you
[20:05] <mup> PR snapcraft#2511 opened: build providers: modify the _run signature <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2511>
[21:32]  * cachio EOW
[23:23] <mup> PR snapcraft#2510 closed: Release changelog for 3.3 <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2510>