[07:04] <mborzecki> morning
[08:00] <mborzecki> mvo: hey
[08:03] <mvo> good morning mborzecki
[08:06] <pstolowski> morning
[08:06] <mvo> good morning pstolowski
[08:09] <mborzecki> pstolowski: hey
[08:10] <pstolowski> o/
[09:12] <mborzecki> haha, so i think i've reproduced a strange scenario that was reported in the forum
[09:12] <mborzecki> there's a gnome-platform content snap used by gimp
[09:12] <mborzecki> i ran gimp, then i enabled parallel-instances, installed hello-world_foo and ran it
[09:13] <mborzecki> now gimp does not work
[09:14] <mborzecki> the LD_LIBRARY_PATH is missing entries pointing to gnome-platform snap
[09:15] <mborzecki> the $SNAP/data-dir/gnome-platform directory is empty, there should be a gnome-platform snap content
[09:18] <mborzecki> discarding the mount ns fixed that ofc
[09:37] <mborzecki> degville: hi, i've added a note under the parallel installs doc page: https://forum.snapcraft.io/t/parallel-installs/7679/18 do you think it's something we could put in the admonition box at the top of the page too?
[09:37] <degville> mborzecki: morning! yes, definitely. I'll do that now - thanks!
[09:38] <mborzecki> degville: cool, thank you!
[09:46] <mvo> mborzecki: I updated 9907, a quick look would be cool, it should DTRT now for kernel refreshes, if all of this looks reasonable I will create the manager tests for the refreshes from old-style to new  style next
[09:47] <mvo> (cc pedronis -^)
[09:50] <pedronis> mborzecki: fun, if we turn it on by default we need to decide what to do in general
[09:51] <pedronis> mborzecki: also is this behavior new? it was never reported
[09:52] <mborzecki> it's not new, it's been like this at least for a couple releases, since 2.43 most likely
[09:55] <pedronis> mvo: we need to still land #9859 first?
[09:55] <mup> PR #9859: overlord: add manager gadget refresh test <UC20> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9859>
[10:07] <mvo> pedronis: that would be good yes
[10:32]  * pstolowski dentist
[11:24] <Laney> hey (mvo?), I was just notified that proposed-migration is crashing in focal. There's probably a bug on our side there but I think I could quickly work around it if I could delete snapd's old binaries on riscv64: https://paste.ubuntu.com/p/2ZBtV7K5kP/ - what do you think of that?
[11:24] <Laney> I did try to retry that build but it just fails instantly with no log, have asked LP what's up with that
[11:25] <mvo> Laney: thanks for letting me know. it's fine to delete snapd on riscv64 for now
[11:26] <Laney> \m/
[11:31] <mup> PR snapd#9928 closed: boot: cmd/snap-bootstrap: handle a candidate recovery system <⛔ Blocked> <Created by bboozzoo> <Closed by bboozzoo> <https://github.com/snapcore/snapd/pull/9928>
[12:05] <pedronis> mvo: I'm confused by you rcomment in #9907, updates and install don't need to do the same thing (for now)
[12:05] <mup> PR #9907: gadget,devicestate: perform kernel asset update for $kernel: style refs <UC20> <⛔ Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9907>
[12:05] <pedronis> mvo: I commented there too
[12:07] <mvo> pedronis: thanks, looking
[12:08] <mvo> pedronis: aha, I see. sorry, well, I meant to look at this but I can remove the blocked for now
[12:08] <mvo> pedronis: sorry for the confusion
[12:09] <pedronis> mvo: well, we need to fix that but is relevant only for prepare-image
[12:09] <pedronis> making updates fully multi-volume is a much larger task
[12:11]  * mvo nods
[12:20] <pstolowski> re
[13:35] <pedronis> mvo: I made some comments in #9907
[13:35] <mup> PR #9907: gadget,devicestate: perform kernel asset update for $kernel: style refs <UC20> <⛔ Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9907>
[13:44] <mvo> pedronis: \o/ thank you
[13:49] <pedronis> mvo: it's missing unit tests about the new code also?
[13:49] <pedronis> mvo: I mean I don't see a lot of new tests in gadget and devicestate
[14:01] <mvo> pedronis: yes, lots more needed
[14:09] <abeato> mvo, pedronis fyi, this MR improves apparmor_parser runtime quite a bit, and when it lands, the `-O no-expr-simplify` parameter used by snapd will not be needed anymore (actually removing it will make things faster): https://gitlab.com/apparmor/apparmor/-/merge_requests/711
[14:10] <mvo> abeato: I saw the PR yesterday, it's very cool. what are the chances for this going upstream?
[14:10] <abeato> mvo, well, it's merged now :)
[14:10] <mvo> abeato: woah, nice
[14:36] <zyga> re
[14:36] <zyga> hey degville, long time no see :)
[14:36] <degville> zyga: hello!!! hope you're doing ok :)
[14:41] <zyga> degville I'm quite busy, not the holidays I was expecting :)
[14:41] <zyga> and working from home with kids and working wife is, challenging at times
[14:42] <zyga> but I'm good, learning a lot and driving a lot :)
[14:42] <zyga> I still use your mike :)
[14:59] <mvo> abeato: sorry, was in a meeting. this is great news. I guess we need to wait a bit with the rmeoval of no-expr-simplify until we have apparmor released widely enough with your improvement
[15:01] <abeato> mvo, sure it will take a bit of time indeed. There is some discussion in the MR about vendoring apparmor_parser, does that mean that snapd snap will include its own apparmor_parser binary?
[15:02] <mvo> abeato: I think that is what the security team suggested - to vendor apparmor_parser inside the snapd snap
[15:02] <abeato> mvo, interesting. Of course that will make easy to include this change.
[15:02] <mvo> yeah
[15:03] <abeato> that's great :)
[15:19] <degville> zyga: great to hear the mike is still being used! It's s shame we never got around to the podcast, but maybe one day :) And yeah, it's tough at home with the kids. It's the same here - this week they're holiday which is a useful respite.
[15:19] <zyga> degville I'm still open to doing a podcast at some point but now I'm just too busy to add anything new
[15:20] <degville> zyga: cool. Let's reconvene when everything has settled a little more :)
[15:43] <ijohnson> cachio: so I ran uc20-recovery external with my pi and it worked fine with snapd from stable, should I try candidate or beta of snapd next ?
[15:44] <cachio> ijohnson, yes, I can point you to the image I used
[15:44] <ijohnson> ah ok great
[16:03] <mup> Bug #1915642 changed: snapd: dbus avc permissions denied <Snappy:Fix Released by maciek-borzecki> <https://launchpad.net/bugs/1915642>
[16:37]  * cachio lunch
[16:43] <mup> PR snapd#9792 closed: tests: enable ubuntu 21.04 for spread tests <Squash-merge> <Created by sergiocazzolato> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9792>
[16:48] <mvo> cachio: hm, 9792 still gives me "2021-02-17 14:33:54 Cannot allocate google:ubuntu-21.04-64: cannot allocate new Google server google:ubuntu-21.04-64 (feb171430-514043): cannot find ready marker in console output for google:ubuntu-21.04-64 (feb171430-514043)" in the logs?
[16:52] <zyga> hey mvo
[16:52] <zyga> how is today?
[16:52] <zyga> I had a long day
[16:58] <mvo> zyga: same here
[16:58] <mvo> zyga: and it's not over yet
[16:58] <zyga> oh
[16:58] <zyga> I'm sorry, I'll let you work
[17:16] <cachio> mvo, yes, the problem in the image is not fixed
[17:16] <cachio> I'll ping cloud team
[17:16] <cachio> I already raise that problem
[17:18] <mup> PR snapd#9938 opened: daemon: move single snap querying and ops to api_snaps.go <Created by pedronis> <https://github.com/snapcore/snapd/pull/9938>
[18:06] <mvo> cachio: thank you!
[19:52] <cachio> ijohnson|lunch, hey, could you reproduce the errorr?
[20:32] <ijohnson> cachio: yeah actually it failed, it couldn't restore
[20:32] <ijohnson> https://www.irccloud.com/pastebin/05EN3tMb/
[20:48] <ijohnson> cachio: heh I seem to have finally reproduced that infinite mounting situation that broke the tpe server with this test a long time ago if you recall
[20:49] <ijohnson> maybe that's what is happening for your system too
[20:50] <cachio> nice
[20:50] <cachio> do you need the console for the vm?
[20:50] <ijohnson> no idea how this happened
[20:50] <ijohnson> cachio: I'll let you know, it will probably fail in a few minutes, so not quite yet
[20:50] <cachio> ijohnson, I remember that error
[20:57] <ijohnson> cachio: could you send me the console output from that gce instance now?
[20:57] <ijohnson> I think it should be stuck now, or if not it should be stuck soon
[20:57] <cachio> ijohnson, https://paste.ubuntu.com/p/wbHKhXbc57/
[21:00] <ijohnson> cachio: ah thanks that was super helpful, I think that the virtual disks in gce are weird/unexpected for my code
[21:01] <cachio> ahh, nice
[21:21]  * cachio afk+
[21:31] <ijohnson> cachio: when you get back can you send me the console output for feb172110-529289 ?
[22:15] <cachio> ijohnson, https://paste.ubuntu.com/p/qwpNhHbvPv/
[22:15] <cachio> sorry for the delay
[22:34] <ijohnson> cachio: no worries thanks for that info, also I reflashed my pi with your image and re-ran uc20-recovery and it worked, so I think there's some kind of race condition around the restoring process and if we hit that race condition there is some sort of infinite mount problem that makes the test fail
[22:37] <zyga> that mount be it
[22:37]  * zyga runs
[22:41] <ijohnson> haha
[22:47] <zyga> "infinte" mounts can happen when rbinding something that's not protected with unbidnable
[22:47] <ijohnson> mmm interesting
[22:47] <zyga> it's sadly copied, not "symlinked" so that does tend to explode
[22:48] <zyga> we do that in writable mimic logic IIRC
[22:48] <ijohnson> hmm but I'm not rbinding the mount
[22:48] <ijohnson> all I do is
[22:48] <ijohnson> sudo mount --bind /host/ubuntu-data/user-data/gopath /home/gopath
[22:49] <ijohnson> (where /home is on a tmpfs, not actually on ubuntu-data)
[22:49] <zyga> ijohnson I can only find https://github.com/snapcore/snapd/blob/f9db38ed6d71d2c356bdf55297ddd1358223d397/cmd/snap-confine/mount-support.c#L239
[22:50] <zyga> I thought we also did this from snap-update-ns
[22:50] <ijohnson> this is the code
[22:50] <ijohnson> https://github.com/snapcore/snapd/blob/release/2.49/tests/core/uc20-recovery/task.yaml#L64-L72
[22:50] <ijohnson> oh unless you're saying maybe something from snapd is _also_ doing a bind mount on top of what my sshrc script is doing
[22:50] <zyga> why the /dev/null redirect?
[22:51] <ijohnson> just to not confuse ssh since this is run every time someone logs in over ssh
[22:51] <ijohnson> err rather
[22:51] <ijohnson> just to not confuse spread, since it logs in over ssh
[22:51] <zyga> wait?
[22:51] <zyga> every time?
[22:52] <ijohnson> uh yeah, but there should only be one command run this way, since the test logs in runs a check, then reboots to go back to run mode
[22:52] <zyga> did you try to a dd some guard?
[22:52] <zyga> not sure but I would definitely check
[22:52] <ijohnson> I could try that, but this is the first I've managed to reproduce it
[22:52] <zyga> I mean, it's super hard to know without seeing the kernel console
[22:52] <ijohnson> mmm I think I lost dmesg from that run
[22:52] <zyga> bonus points for forkstat logged over serial
[22:53] <ijohnson> but I can try and save that next time I see it
[22:53] <zyga> yeah, I'd love to have a look