[06:10] <mborzecki> morning
[06:20] <mardy> hi!
[07:06] <pstolowski> morning
[07:29] <mborzecki> pstolowski: hey
[07:29] <mborzecki> pedronis: hi, thanks for merging that PR
[07:37] <zyga-mbp> good morning
[07:56] <mvo> good morning zyga-mbp 
[07:56] <pstolowski> pedronis: hi, do you have a moment to chat about some bits of holding? should be quick
[07:56] <zyga-mbp> hey mvo :)
[07:56] <mvo> and good morning pstolowski and mborzecki 
[07:56] <mvo> and mardy too of course :)
[07:56] <pedronis> pstolowski: in 4-5 minutes?
[07:57] <pstolowski> pedronis: great
[07:57] <mborzecki> mvo: hey, can you take a look at https://github.com/snapcore/snapd/pull/10386 ? once it lands i'll be able to open another bit of the cleanup
[08:00] <mvo> mborzecki: will try, got some more meetings ahead of me :/
[08:00] <mborzecki> mvo: ah, good luck then 😉 make sure you have lots of tea
[08:04] <pedronis> pstolowski: joining the SU
[08:04] <pstolowski> pedronis: ok
[08:06] <mborzecki> mvo: pedronis as for lxd, looks like they switched to core20
[08:06] <mborzecki> but unclear whether that's related in any way
[08:20] <pedronis> mborzecki: I suppose we should add debug/debug what is denied, and what the mount table look at that point?
[08:20] <pedronis> if it's a nodev issue we would see it from the mount table
[08:20] <pedronis> otoh then we would need to dig what has changed about that
[08:21] <mborzecki> pedronis: i indend to take a deeper look, but also trying out lxd on arch, stgraber pinged be to check it after they mvoed to core20 base
[08:21] <pedronis> ok, let us know, this is a big annoyance/issue atm
[08:30] <pedronis> https://github.com/snapcore/snapd/pull/10411 finishes the gci "fixes"
[08:51] <pstolowski> pedronis: thanks for the review of the snapctl --proceed fix, this is the 2nd fix: https://github.com/snapcore/snapd/pull/10413
[09:16] <zyga-mbp> small fix https://github.com/snapcore/snapd/pull/10414
[09:39] <mborzecki> ok, traces uploaded, bugs are filed, https://bugs.archlinux.org/task/71270 kernel probes are awesome
[09:45] <pedronis> pstolowski: I did a pass on https://github.com/snapcore/snapd/pull/10408, some comments/questions
[09:45] <pedronis> thx
[09:48] <pedronis> https://github.com/snapcore/snapd/pull/10308 needs 2nd reviews, it's simple as it's a preparatory PR
[09:50] <mardy> pedronis, jamesh: approved
[09:50] <pedronis> thx
[09:54] <pstolowski> pedronis: thanks
[09:54] <pstolowski> mborzecki: a simple one https://github.com/snapcore/snapd/pull/10412
[10:03] <pstolowski> mvo: could you merge this one? https://github.com/snapcore/snapd/pull/10366
[10:08] <mvo> pstolowski: sure
[10:10] <zyga-mbp> mvo is it expected that test fail on https://github.com/snapcore/snapd/pull/10414?
[10:15] <mborzecki> trivial one: https://github.com/snapcore/snapd/pull/10415
[10:36] <mborzecki> hmm the lxd issues does not make much sense unless i'm missing something
[10:36] <mborzecki> or something in master broke it?
[10:36] <mborzecki> in a clean spread system things seem to work
[10:37] <pstolowski> mborzecki: mvo has been looking at it and afaiu is working on a reproducer
[10:38] <mborzecki> mvo: got any ideas?
[10:38] <mborzecki> pstolowski: the failure reproduces when i run a specific test
[10:39] <pstolowski> mhm
[10:39] <mborzecki> maybe it's something about how we inject snapd, running apt remove --purge which is supposed to do the cleanup
[10:39] <mborzecki> but there shouldn't be any more nested containers in the lxd container yet
[10:40] <pstolowski> mborzecki: mvo was able to reproduce in qemu with nested containers i think
[10:40] <mborzecki> intersting
[10:41] <pstolowski> mvo: can we set it to manual for the time being?
[10:44] <mvo> pstolowski: I think that is ok
[10:44] <mvo> mborzecki: I did use upstream/master as my base branch and ran the spread test in qemu against 20.04 and get the error. it seems it fails in the nested lxd case only
[10:45] <mvo> mborzecki: I wanted to work on a reproducer but had 23423523523 meetings
[10:45] <mvo> mborzecki: I have some minutes now, will try in a clean VM, I hope it's really just a matter of installing edge snapd and then creating a nested vm
[10:49] <pstolowski> https://github.com/snapcore/snapd/pull/10416
[10:49] <pstolowski> mvo, mborzecki ^
[10:51] <mardy> pstolowski: I once had a cgroups related test failing; will you make a PR for that too?
[10:51] <mardy> pity I don't remember which one it was :-)
[10:51] <mborzecki> mvo: pstolowski: i can reproduce it on a spread node with snapd 2.51 on the outside, 2.50.1 on he inside, i'll try refreshing the inside one
[10:52] <pstolowski> mardy: we only do this temorarily for tests that fail all the time
[10:52] <pstolowski> (for reasons that are not our bugs)
[10:54] <mborzecki> mvo: pstolowski: and same with 2.51 in both containers
[10:54] <mborzecki> so not a regression technically? :)
[10:55] <mborzecki> (also refresh app awareness does not seem to have anything to do with it)
[11:00] <mardy> pstolowski: ah, maybe it's exactly the same tests actually, I now see that one of the failures is "google:ubuntu-20.04-64:tests/main/lxd:snapd_cgroup_both", so I guess I just paid more attention to the "cgroup" word than to the "lxd" word :-)
[11:03] <mvo> mborzecki: hm, I did not manage to reproduce outside of spread
[11:03] <mborzecki> mvo: with master only? or 2.51?
[11:04] <mvo> mborzecki: I think I'm just missing steps
[11:04] <mvo> mborzecki: you can reproduce the failure with 2.50.1 you say? if so I think we can hand it to lxd because then it seems something in lxd changed
[11:04] <mvo> mborzecki: is there a reproducer that is not spread :)
[11:04] <mvo> ?
[11:05] <mborzecki> mvo: yes, i had 2.51 on the outside, and 2.50.1 on the inside, and another setup was 2.51 on the outside and 2.51 on the inside
[11:08] <mborzecki> mvo: and i had a spread node on gcp, but using snapd from distro/store
[11:08] <pedronis> mborzecki: but it works with 2.50.1 outside and inside?
[11:11] <mborzecki> pedronis: hm, i'll try to grab snapd 2.50 from lp
[11:11] <mborzecki> the gcp host alredy has 2.51
[11:21] <mborzecki> eh, 2.50.1 was never uploaded as a deb?
[11:27] <mborzecki> nvm, pulled out a snap from the container
[11:28] <mvo> ok
[11:29] <mborzecki> mvo: pedronis: so i see the problem with 2.50.1 on the inside and outside
[11:30] <mborzecki> https://paste.ubuntu.com/p/tQDcVXwGmh/
[11:30] <pedronis> so it's not something that changed in snapd
[11:33] <mvo> nice
[11:35] <mborzecki> pedronis: mvo: and using lxd from 4.14 track on the inside works ok
[11:46] <mborzecki> quick errand, back in 30
[12:17] <mborzecki> re
[12:20] <mborzecki> mvo: pedronis: can you land https://github.com/snapcore/snapd/pull/10415 ? sid is happy again
[12:48] <mardy> some go help: go get -d github.com/mardy/runc@nnp-after-aa-profile-change
[12:48] <mardy> go: cannot use path@version syntax in GOPATH mode
[12:49] <mardy> it's in the context of a snapcraft part, so maybe I should use git directly instead?
[12:56] <mvo> mborzecki: sure, done
[13:00] <mborzecki> mvo: thanks!
[13:42] <mborzecki> mvo: so this is the PR https://github.com/snapcore/snapd/pull/10416 that switches lxd test to manual
[13:45] <mvo> mborzecki: +1
[13:45] <mvo> mborzecki: I merged it and we can just build one on top
[13:46] <mborzecki> great, let me open a quick update now
[13:46] <mvo> ta
[13:51] <mborzecki> mvo: pstolowski: https://github.com/snapcore/snapd/pull/10417
[13:52] <pstolowski> mborzecki: thanks!
[14:18] <pstolowski> pedronis: i implemented pruning of refresh-candidates in check-rerefresh but realized it needs to happen in doLinkSnap (or close to it), as check-rerefresh won't cover manual refreshes..
[14:30] <pedronis> pstolowski: that sounds strange, we should always use it
[14:30] <pedronis> it's needed to do the right thing with epochs
[14:31] <pstolowski> pedronis: ah, because of finalizeUpdate
[14:31] <pstolowski> pedronis: you're right
[14:31] <pstolowski> good
[14:41] <mardy> snapcraft started to fail here when getting the sources... has anyone seen this before? 0% [Connecting to archive.ubuntu.com (2001:67c:1360:8001::24)]
[14:42] <mardy> (building with LXD)
[14:42] <ijohnson[m]> probably just needs to clean the LXD container and try again, could be a apt issue more than a snapcraft issu
[14:50] <mborzecki> heh - Download snap "go" (7736) from channel "latest/stable" (download too slow: 489.60 bytes/sec)
[14:57] <ogra> mardy, people in #ubuntu have complained about an archive outage 
[14:57] <ogra> might be a server side issue
[14:58] <pstolowski> mborzecki: i've never seen it for real ;)
[14:59] <mardy> ogra: ah-ah, I see, thanks
[15:00] <pstolowski> mborzecki: another simple one https://github.com/snapcore/snapd/pull/10413
[15:21] <mborzecki> pedronis: mvo: can you land https://github.com/snapcore/snapd/pull/10082 ? building the snapd snap failed beacuse of a flaky network
[15:24] <mvo> mborzecki: sure
[15:24] <mborzecki> mvo: thanks!
[15:30] <pedronis> mborzecki: \o/ :)
[15:30] <mborzecki> let's see how it fares in practice now
[15:32] <mborzecki> this is all that the linter complains about now: https://paste.ubuntu.com/p/khChwP2yZP/
[15:33] <pedronis> yes, as I said unused and deadcode stuff, that's what I got as well when playing locally
[15:48] <cachio_> ijohnson[m], hi, I created #10419
[15:48] <cachio_> when you have time, could you please take a quick look
[15:48] <cachio_> I need to validate the idea
[15:48] <ijohnson[m]> cachio_: sure I'll have a look, probably not today though
[15:49] <cachio_> tx
[17:39]  * cachio_ afk