[05:04] <mup> PR snapd#7598 closed: test/lib/names.sh: make backslash escaping explicit <Simple 😃> <Created by anonymouse64> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7598>
[05:12] <mborzecki> morning
[05:19] <mvo> mborzecki: good morning
[05:20] <mborzecki> yay 7536 landed :P
[05:20] <mup> PR snapd#7536 closed: gadget: accept system-seed role and ubuntu-data label <Created by cmatsuoka> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7536>
[05:20] <mborzecki> mvo: thanks!
[05:20] <mvo> mborzecki: yes, finally :)
[05:24] <mborzecki> i've updated #7593
[05:24] <mup> PR #7593: recovery-tool: add sfdisk wrapper <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/7593>
[05:24] <mborzecki> and there are some comments from zyga too
[05:24] <mborzecki> hmm, new kernel, i better reboot
[05:24] <mborzecki> brb
[05:28] <mborzecki> and back on 5.3.6 kernel
[06:50] <mup> PR snapd#7599 opened: gadget: refactor ensureVolumeConsistency <Created by mvo5> <https://github.com/snapcore/snapd/pull/7599>
[07:04] <pstolowski> morning
[07:08] <mvo> pstolowski: hey, good morning!
[07:18] <mborzecki> pstolowski: hey
[07:21] <zyga> hello?
[07:21] <zyga> is this thing on?
[07:21] <zyga> I think I connected now
[07:23] <zyga> sorry for starting late, lucy was sleeping hugging me all the time and got grumpy the second I tried to put her to bed
[07:24] <mborzecki> zyga: hey
[07:24] <zyga> mborzecki: hey, question on how to proceed for you
[07:24] <zyga> mborzecki: I removed the guts of snap-update-ns so that I can create other tools with the same logic
[07:25] <zyga> mborzecki: I moved it to system/mount for lack of a better name
[07:25] <zyga> mborzecki: some bits can go to sandbox/cgroup
[07:25] <zyga> mborzecki: but most has no good place to go
[07:25] <zyga> mborzecki: ideas?
[07:26] <zyga> mborzecki: I'm mainly after plan-writable-mimic and execute-writable-mimic
[07:26] <zyga> mborzecki: this pulls in change, assumptions, restrictions and tresspassing
[07:27] <zyga> mborzecki: along with loads of tests
[07:27] <mborzecki> mvo: i'm looking at 7593, perhaps our consistency checks are too strict, in particular the len(volumes) != 0 only counts when constraints are used, but once there's a system-seed structure defines and SsytemSeed is not true, then we bail out, why not allow this case?
[07:28] <mborzecki> zyga: that's part of mount related logic though?
[07:28] <mborzecki> zyga: or hmm we call it namespace (or whatnot) often
[07:28] <zyga> mborzecki: mainly yes
[07:28] <mborzecki> zyga: system/mount/namespace?
[07:29] <zyga> mborzecki: I'd go for system/mount really, it would help abbreviate the API names and would look nice
[07:29] <zyga> mount.Change
[07:29] <zyga> mount.Assumptions
[07:29] <zyga> mount.Restrictions
[07:29] <zyga> etc
[07:29] <mborzecki> system/mounts stgm too
[07:29] <zyga> though
[07:29] <zyga> my main question is: can we use main :)
[07:29] <zyga> er
[07:29] <zyga> system/
[07:29] <zyga> or is that too vague
[07:29] <zyga> I want to have this because otherwise debugging some issues is very hard
[07:30] <zyga> and having an interactive tool where you can do stuff is very useful to understand things misbehaving
[07:30] <zyga> and sadly it's forbidden to import from "main" packages
[07:30] <mborzecki> zyga: hm we have osutil which kind of is system like already, and there's some mount related stuff in there too
[07:31] <zyga> mborzecki: yeah but any existing package may trigger import loops
[07:31] <mborzecki> zyga: osutil/mount ?
[07:31] <zyga> I can tryt
[07:31] <zyga> *try
[07:31] <zyga> let's see if that works
[07:31] <zyga> mborzecki: do you know if go has some rules on what is in a binary
[07:31] <zyga> if there's a package foo/bar
[07:31] <zyga> and I use only bar.Symbol
[07:32] <zyga> does it include all of foo?
[07:32] <mborzecki> zyga: no, bar is bar, foor is foo
[07:33] <zyga> mborzecki: so foo does _not_ bloat a binary using bar?
[07:33] <zyga> mborzecki: and the reverse case?
[07:33] <zyga> mborzecki: is presence of foo/bar affecting a binary that only uses symbols from foo?
[07:33] <mborzecki> zyga: same, no effect unless you import it, or it's imported internally
[07:33] <mborzecki> zyga: or you created an import loop :P
[07:34] <zyga> ok, let's give it a try
[07:37] <mborzecki> mvo: simply put, if i have a gadget yaml with system-seed and system-data, and use an empty ModelConstraints (just want to have consistency checked), this will error out
[07:44] <mvo> mborzecki: I need to look, could you please add a note in the PR? maybe we are too strict .) it seems like the refactor helped to reason about this easier
[07:44] <mvo> mborzecki: I'm in the middle of 7593 right now
[07:44] <mborzecki> mvo: ah ok, started to play with thhis code too, i'll leave a comment in the PR then
[07:48] <mvo> mborzecki: or feel free to propose a pr on top if you feel this should be ok, either way is fine with me :)
[08:02] <mvo> ackk: hey, in snapd 2.42 we improved the fuse performance in lxd, I heard maas is using this a lot, did you see improvements?
[08:06] <Chipaca> 👋
[08:08] <zyga> hello Chipaca
[08:09] <zyga> mborzecki: the easy branch https://github.com/snapcore/snapd/pull/7600
[08:09] <mup> PR #7600: sandbox/cgroup: move freeze/thaw code <Created by zyga> <https://github.com/snapcore/snapd/pull/7600>
[08:09] <mup> PR snapd#7600 opened: sandbox/cgroup: move freeze/thaw code <Created by zyga> <https://github.com/snapcore/snapd/pull/7600>
[08:26] <Chipaca> pedronis: 👋
[08:27] <Chipaca> pedronis: did you see my question about the quotes in the repair script?
[08:27] <pedronis> yes, but I didn't understand it
[08:27] <pedronis> maybe mvo does
[08:28] <Chipaca> pedronis: you didn't understand the question, or the origin of the quotes? :)
[08:29] <mvo> Chipaca: good morning! I did not see the quotes at all
[08:29] <mvo> Chipaca: oh, fun!
[08:29] <Chipaca> mvo: snap known --remote repair brand-id=canonical repair-id=1
[08:29] <Chipaca> mvo: the script has typographic quotes
[08:29] <pedronis> Chipaca: I think it's copying from a gdoc
[08:29] <mvo> Chipaca: haha - I see it now
[08:29] <Chipaca> mvo: that might be intentional, but I doubt it
[08:29] <pedronis> that did that
[08:30] <mvo> Chipaca: yeah, it was not
[08:30] <Chipaca> yeah, that would do it
[08:30] <mvo> Chipaca: we need to improve the way we sign assertions
[08:30] <mvo> Chipaca: or let assertion sign :)
[08:30] <mvo> Chipaca: THANKS for spotting this
[08:30] <Chipaca> it's silly that I worry about this, but I worry because although "echo" doesn't care, there's not a lot more that is as amicable
[08:30] <mvo> Chipaca: oh, totally
[08:31] <mvo> Chipaca: we need to improve our process here
[08:31] <Chipaca> that's all i needed to hear to stop worrying about it :-)
[08:31] <mvo> Chipaca: I'm editing a gdoc now so that we don't use gdocs for this :)
[08:34] <mborzecki> mvo: added a bunch of comments in #7593, i think some relate to how the seed/recovery is supposed to work, can you take a look and maybe we could discuss that with whoever is interested before/after the standup?
[08:34] <mup> PR #7593: recovery-tool: add sfdisk wrapper <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/7593>
[08:36] <zyga> mborzecki: I'm stuck
[08:36] <zyga> mborzecki: the challenge is as follows:
[08:36] <zyga> mborzecki: there's lots of tailored mocking going on
[08:36] <zyga> mborzecki: in osutil/ snap-update-ns and now in osutil/mount
[08:37] <zyga> mborzecki: then there's the general mocking all syscalls that are used in snap-update-ns
[08:37] <zyga> mborzecki: if you try to reconcile all of that it's a big mess
[08:37] <zyga> mborzecki: thinking about it, I see the following way forward and perhaps out of the problem as well
[08:37] <zyga> mborzecki: we can make a new package syscalls/ that has just the system calls and a means to mock them
[08:38] <zyga> mborzecki: as things move out of snap-update-ns, they get updated to use this package over the private mocking code used locally
[08:38] <zyga> mborzecki: then some parts of snap-update-ns, for example all the mk{dir,file,symlink} and openpath code move to osutil/
[08:39] <zyga> (or osutil/mk... if that helps to scope it)
[08:39] <zyga> mborzecki: while the real essence of mount, like secure bind mount, mimic logic, can move to osutil/mount
[08:39] <zyga> or even osutil/mimic
[08:39] <zyga> mborzecki: moving everything in bulk might help in the short term
[08:40] <zyga> mborzecki: but won't change how those things are messy internally when considered along with the rest of osutil
[08:41] <mborzecki> heh, if you move some of the calls to osutil there's probably going to be some funky import dependencies
[08:42] <zyga> mborzecki: there already are, much of the code uses osutil already
[08:43] <zyga> mborzecki: how would you feel about the syscalls package?
[08:43] <zyga> mborzecki: the public API would be various system calls
[08:43] <zyga> mborzecki: and a means to mock them all (e.g. with the recorder already present in another package)
[08:44] <mborzecki> zyga: there's stdlib syscalls already, could those wrappers just live in osutil instead?
[08:44] <zyga> mborzecki: yes but I'd like them not to
[08:44] <zyga> osutils is already kind of large
[08:44] <zyga> but most importantly it already has a number of them and some mocking
[08:45] <zyga> mborzecki: (mock-this, mock-that, not in general)
[08:45] <zyga> mborzecki: the idea is that eventually, over time, code could move to use the new pakcage
[08:45] <zyga> and drop its own mocking
[08:45] <zyga> mborzecki: this is why I think a new package would help
[08:45] <zyga> mborzecki: if we move them over to osutil right now you will see a number of separate mocking systems that need to be reconciled
[08:46] <mborzecki> zyga: osutils/syscalls? with a godoc that 'here lie the syscalls we can mock' ? :)
[08:46] <zyga> :D
[08:46] <zyga> mborzecki: that works for me
[08:46] <zyga> mborzecki: importantly, mock in bulk
[08:46] <zyga> not one by one
[08:47] <Chipaca> mborzecki: osutil/sys
[08:47] <Chipaca> ?
[08:47] <mborzecki> right, avoids a name clash with stdlib's syscalls
[08:48] <ogra> would gadgets on top of debian work or is there any special-casing-code that only allows them on ubuntu classic ?
[08:49] <zyga> Chipaca: would you prefer if this was in osutil/sys?
[08:59] <zyga> brb
[09:12] <Chipaca> zyga: i'm not sure quite what you're asking, but, probably? the difference between osutil/syscall[s] and the existing osutil/sys escapes me
[09:19] <pedronis> zyga: what are the changes to snap-update-ns about ?
[09:28] <zyga> pedronis: being able to expose the internals for interactive debugging
[09:28] <pedronis> zyga: what's the context? are you chasing a bug?
[09:29] <zyga> Yes, correct.
[09:29] <pedronis> which one?
[09:29] <zyga> I did that yesterday and thought that some of those could land
[09:29] <zyga> Not all changes are easy though
[09:31] <pedronis> zyga: I fear I still don't have enough context, maybe you need to propose the mess as is, and I can look what to do with it
[09:31] <pedronis> when I have a bit of time
[09:33] <zyga> I think some bits are easier than others but the main value already exists. In a branch I can build mimic-tool to experiment and see where the kernel behavior is coming from
[09:34] <pedronis> zyga: still no context
[09:39] <zyga> Finishing breakfast, I’ll be back soon
[09:41] <mvo> mborzecki: I addressed a bunch of your comments in 7593, I am looking into using "-apend" now
[09:42] <mborzecki> mvo:  thanks! btw. that's an interesting find about blockdev
[09:43] <mvo> mborzecki: thank you for pointing this out!
[09:43] <mvo> mborzecki: blockdev seems to be part of busybox fwiw
[09:43] <mborzecki> mvo: ah, that makes sens
[09:44] <mvo> mborzecki: also thanks for the comment about -apend, I was wondering about this myself but wasn't aware of -append :)
[09:44] <mup> PR snapd#7601 opened: overlord/ifacestate: use SetupMany in setupSecurityByBackend <Created by stolowski> <https://github.com/snapcore/snapd/pull/7601>
[09:45] <pedronis> mborzecki: what's the status of #7079, is it being absorbed by the snap-recovery work?
[09:45] <mup> PR #7079: [RFC] cmd/snap-image: tool for building UC images <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7079>
[09:47] <mborzecki> pedronis: not really, parts of it probably affected snap recovery work, but iirc claudio did not pull in any patches directly
[09:47] <pedronis> mvo: are we going to do a 2.42.1 ?  mvo, pstolowski: when should we try to land #7092, it's unblocked now?
[09:47] <mup> PR #7092: packaging: use snapd type and snapcraft 3.x <Created by stolowski> <https://github.com/snapcore/snapd/pull/7092>
[09:47] <mborzecki> pedronis: though, the sfdisk parts should be consolidated into the gadget package since we alredy have some code there
[09:48] <pedronis> mborzecki: it sounds like we would redo it on top of the recovery work at this point, no?
[09:48] <pedronis> should it be closed?
[09:48] <mborzecki> pedronis: other than that, the tools are run in different environment and differnt part of the cycle
[09:50] <mborzecki> pedronis: i can close it for now and we can discuss what to do witht he tool separately
[09:50] <pedronis> ok, sounds good
[09:51] <pedronis> mborzecki: to be clear, the medium term plan is still for ubuntu-image to be a thin wrapper around snapd code, but I don't we want to land two codebases doing partions into snapd, we probably want to get to one codebase surface maybe in two slightly diffferent ways
[09:52] <mup> PR snapd#7079 closed: [RFC] cmd/snap-image: tool for building UC images <Created by bboozzoo> <Closed by bboozzoo> <https://github.com/snapcore/snapd/pull/7079>
[09:53] <pstolowski> pedronis: yes i think we should land 7092. going to deconflict it
[09:54]  * Chipaca brb
[09:55] <mvo> mborzecki: I think 7593 is good for now, I want to do a followup with some real sfdisk tests but that will probably become quite a bit bigger
[09:56] <mborzecki> mvo: do you think it'd make sense to move the code that creates a LaidOutVolume out of sfdisk dump into gadget/partition.go ?
[09:57] <mup> PR snapd#7222 closed: tests: show just the last log as part of the debug output when check journal logs <Simple 😃> <Created by sergiocazzolato> <Closed by pedronis> <https://github.com/snapcore/snapd/pull/7222>
[10:00] <pedronis> mvo: mborzecki: volmgr is a strange name, it makes one think LVM
[10:01] <ogra> makes me think of androids vold ... :)
[10:03] <zyga> Voldmgrd
[10:03] <zyga> ;-)
[10:03] <ogra> pedronis, do you knwo off the top of your head if it is seamlessly possible to use gadget snaps on debian systems (i.e. to attach to a brand store or add certain interfaces)  or does snapd currently restrict this to ubuntu systems ?
[10:06] <mvo> pedronis: we could move it to osutil/fdisk ?
[10:06] <mup> PR snapd#7600 closed: sandbox/cgroup: move freeze/thaw code <Created by zyga> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/7600>
[10:07] <pedronis> mvo: probably not osutil given it uses types from gadget
[10:07] <pedronis> it could be under gadget
[10:07] <mvo> pedronis: good point
[10:08] <mvo> pedronis: or just cmd/snap-recovery/fdisk ?
[10:08] <pedronis> that's fine too
[10:08] <pedronis> mostly against the volume manager terminology
[10:09] <mvo> pedronis: cool, I will rename to fdisk for now, I think it will later grow luks and stuff but even that seems to be ok to put under fdisk
[10:10] <pedronis> mvo: gadget/partition ?
[10:11] <pedronis> (I'm fine with fdisk as well)
[10:13] <mvo> pedronis: yeah, partition works for me, shall I keep it under cmd/snap-repair for now or do you prefer gagdet?
[10:13] <pedronis> mvo: as your prefer, discuss with mborzecki
[10:14] <mvo> pedronis: thanks, will do
[10:26] <pstolowski> got ubuntu-core-18-64:tests/core18/snapd-failover failure in one of my PRs: 2019-10-15T10:05:40Z INFO Waiting for restart...
[10:26] <pstolowski> error: cannot perform the following tasks:
[10:26] <pstolowski> - Mount snap "snapd" (unset) (remove /etc/systemd/system/snap-snapd-x2.mount: no such file or directory)
[10:26] <pstolowski> - Automatically connect eligible plugs and slots of snap "snapd" (there was a snapd rollback across the restart)
[10:46] <mup> Bug #1835024 changed: Links triggered within most snap apps open in a separate browser session <snapd:Incomplete> <xdg-utils (Ubuntu):New> <https://launchpad.net/bugs/1835024>
[10:50] <mup> Bug #1801201 changed: Installed snap icons missing if using pixmap <Snapcraft:New> <snapd:New> <https://launchpad.net/bugs/1801201>
[10:50] <mup> Bug #1809729 changed: Removing a snap triggers 'Starting scheduled backup' notification <amd64> <apport-bug> <cosmic> <third-party-packages> <udev> <wayland-session> <snapd:Triaged> <deja-dup (Ubuntu):New> <https://launchpad.net/bugs/1809729>
[10:56] <mup> Bug #1773416 changed: Interface from Thunderbird/Firefox to LibreOffice/VLC <snapd-interface> <snapd:Confirmed> <https://launchpad.net/bugs/1773416>
[10:56] <mup> Bug #1797745 changed: "ln: failed to create symbolic link" in deepin 15.7 <Snappy:Invalid> <https://launchpad.net/bugs/1797745>
[10:59] <zyga> it is frustrating when lxd and snapd use different bug trackers
[10:59] <zyga> and that launchpad does not automatically ping-back the other bug when an URL is mentioned.
[11:00] <zyga> stgraber: hello, could you kindly look at https://bugs.launchpad.net/snappy/+bug/1773044 and if appropriate, convert it to an LXD bug on github please.
[11:00] <mup> Bug #1773044: Can't get snap gui apps (notepadqq and firefox) to run in LXD/LXC container <Snappy:Invalid> <https://launchpad.net/bugs/1773044>
[11:02] <mup> Bug #1773044 changed: Can't get snap gui apps (notepadqq and firefox) to run in LXD/LXC container <Snappy:Invalid> <https://launchpad.net/bugs/1773044>
[11:05] <stgraber> zyga: doesn't look like a bug, just misconfiguration by the user. I suspect they've bind-mounted the X11 socket which works fine for everything but snaps as snaps need the abstract Unix socket instead
[11:05] <mup> Bug #1763071 changed: Error message installing a paid snap as unauthenticated user <snapd:Triaged> <https://launchpad.net/bugs/1763071>
[11:05] <mup> Bug #1765979 changed: snap applications icons missing in launcher wayland <wayland> <snapd:Triaged> <https://launchpad.net/bugs/1765979>
[11:05] <mup> Bug #1766079 changed: ripgrep installed via snap fail to work in /lib/systemd/system folder <Snappy:Invalid> <https://launchpad.net/bugs/1766079>
[11:06] <Eighth_Doctor> zyga: I kind of wish that snapd didn't use launchpad as the bug tracker
[11:06] <Chipaca> diddledan: Dear Tea., … ?
[11:06] <Eighth_Doctor> it's surprisingly difficult to locate bugs in launchpad
[11:06] <zyga> Eighth_Doctor: that makes 1002 of us
[11:07] <zyga> Eighth_Doctor: at least we're looking now :)
[11:07] <zyga> Eighth_Doctor: progress!
[11:07] <Eighth_Doctor> yeah
[11:07] <Eighth_Doctor> the only really useful feature of launchpad is multi-tracker aggregation
[11:07] <Eighth_Doctor> that's actually a powerful feature if it worked more...
[11:08] <mup> Bug #1758465 changed: snapd doesn't upgrade on bionic when a local installed snap mount is failing <snapd:Triaged> <https://launchpad.net/bugs/1758465>
[11:08] <diddledan> hmm?
[11:08] <Chipaca> diddledan: https://forum.snapcraft.io/t/need-to-add-our-product/13694?u=chipaca
[11:08] <diddledan> aha
[11:09] <diddledan> yeah, I think they missed an M
[11:09] <Eighth_Doctor> zyga: but does this mean I need to clone rhbz bugs to LP to get them fixed? ☹️
[11:10] <zyga> Eighth_Doctor: why would you do that?
[11:10] <zyga> ah
[11:10] <zyga> I see
[11:10] <zyga> are you saying that we should look at RHBZ bugs as well?
[11:10] <Eighth_Doctor> yes
[11:11] <mup> Bug #1750856 changed: snapd on s390x tried to run locationd tests, which does not exist on s390x <britney:Fix Released> <Snappy:Fix Released> <snapd (Ubuntu):In Progress> <snapd (Ubuntu Bionic):In Progress> <https://launchpad.net/bugs/1750856>
[11:11] <mup> Bug #1752916 changed: Desktop interface should allow accessing to recent files and xdg dirs <snapd-interface> <snapd:Triaged> <https://launchpad.net/bugs/1752916>
[11:11] <Eighth_Doctor> I've been doing some of the triage for a while now, but if there's an actual code problem, it's a bit difficult to escalate because I don't know how to get people's attention on it
[11:11] <Chipaca> zyga: we do look at ubuntu/+source/snapd bugs, so why not rhbz bugs?
[11:11] <zyga> mvo: could you kindly check if the bionic task of this bug is still valid https://bugs.launchpad.net/snappy/+bug/1750856
[11:11] <mup> Bug #1750856: snapd on s390x tried to run locationd tests, which does not exist on s390x <britney:Fix Released> <Snappy:Fix Released> <snapd (Ubuntu):In Progress> <snapd (Ubuntu Bionic):In Progress> <https://launchpad.net/bugs/1750856>
[11:11] <zyga> Chipaca: I agree, it's just a matter of manpower and gardening the mess first
[11:12] <zyga> Eighth_Doctor: we have a recurring triage work, at some point we'll run out of overflowing NEW bugs and can look at existing bugs
[11:12] <zyga> Eighth_Doctor: can you point me to the list of snapd bugs in RH bugzilla please
[11:12] <zyga> Eighth_Doctor: I'll add that to our laundry list
[11:13] <Eighth_Doctor> http://bugz.fedoraproject.org/snapd
[11:13] <Eighth_Doctor> I need to write a script to mass-close all these SELinux bugs
[11:13] <diddledan> WONTFIX :-p
[11:13] <Eighth_Doctor> I'm pretty sure a large chunk of those are from before the policy rewrite that mborzecki did in May
[11:14] <Chipaca> how do we feel about snaps that tell users to do things that could easily trick them to run things unconfined?
[11:14] <mup> Bug #1748510 changed: shell's test gives false positive on readability of files <Snappy:Won't Fix> <https://launchpad.net/bugs/1748510>
[11:14] <zyga> Chipaca: RUN THIS SNAP WITHOUT PESKY ERRORS WITH THIS ONE TRICK
[11:14] <diddledan> Chipaca: that sounds dirty
[11:14] <diddledan> zyga: you'll never believe what happens next
[11:14] <zyga> diddledan: I know, I know
[11:14] <Eighth_Doctor> zyga: protip, all Fedora packages actually have a BugURL property set on them ;)
[11:14] <zyga> sudo wget | sh
[11:15] <Chipaca> 'loopchain' (brought to my attention by requesting a track) is like this
[11:15] <zyga> it's always like tat
[11:15] <zyga> *that
[11:15] <diddledan> love those instructions appearing everywhere because macOS
[11:15] <Eighth_Doctor> `rpm --query --queryformat "%{BUGURL}" snapd`
[11:15] <Chipaca> it doesn't seem to be immediately bad, but it makes me feel a little uncomfortable
[11:16] <Chipaca> "to install, first snap install this, then apt install rabbitmq, then run thesnap.init, then run the scripts it wrote in $SNAP_COMMON
[11:16] <Chipaca> "
[11:16] <Eighth_Doctor> zyga: thankfully all these Fedora 29 bugs go away when F29 EOLs next month
[11:16]  * Chipaca would NOPE out of that really fast, but some users are perseverant... 
[11:16] <diddledan> or "run `$(thesnap.init)`
[11:17] <mup> Bug #1747200 changed: console conf crashes on dragonboard <subiquity:New> <https://launchpad.net/bugs/1747200>
[11:17] <Chipaca> diddledan: sudo $(thesnap.ok)
[11:17] <Eighth_Doctor> Chipaca: you didn't think snaps would actually stop people from doing dumb things, did you? 😶
[11:17] <zyga> Eighth_Doctor: that's a bit annoying on the user side but that's very convenient for the developer side :)
[11:17] <zyga> diddledan: they will all need to change to curl | zsh
[11:17] <Eighth_Doctor> well, it's the only way I maintained my sanity with snapd bugs
[11:18] <zyga> and sudo won't help since macos is locked down more than ever now
[11:18] <Eighth_Doctor> macOS has become an annoying platform to be a developer on
[11:18] <Chipaca> Eighth_Doctor: https://www.youtube.com/watch?v=TL0dfzK3Aqs
[11:19] <diddledan> love that song
[11:19] <Eighth_Doctor> it's a good song
[11:19] <Eighth_Doctor> and very appropriate here
[11:19] <Chipaca> BUT THE SUDO COMES AT NIGHT
[11:19] <mborzecki> Eighth_Doctor: i'm quite sure there's still a bunch of things popping up when installing snaps :/
[11:20] <zyga> is lxd security.nesting enabling nested apparmor?
[11:20] <mup> Bug #1744584 changed: Exclude Snap .cache from Dejadup backups <Déjà Dup:Fix Released> <Snappy:Invalid> <deja-dup (Ubuntu):Fix Released> <https://launchpad.net/bugs/1744584>
[11:20] <Eighth_Doctor> yeah
[11:20] <Eighth_Doctor> it's a lot less nowadays, thankfully
[11:20] <Eighth_Doctor> and some of the warnings I don't want to fix
[11:20] <mborzecki> Eighth_Doctor: i should try installing some snaps on my wife's lapotp to catch them all :P
[11:21] <Eighth_Doctor> there are cases where it's actually bad behavior it's blocking, and I'm okay with the denial status
[11:21] <mvo> zyga: sure, done
[11:23] <Eighth_Doctor> mborzecki: the issue right now is that Silverblue users are upset that snaps using the "home" perm fail to start because of `/var/home`
[11:24] <Eighth_Doctor> there's a symlink for `/home` to `/var/home`, but that isn't enough for snapd, I guess
[11:24] <zyga> mvo: thank you
[11:24] <zyga> Eighth_Doctor: there's been some patches to make /var/home work already
[11:24] <zyga> Eighth_Doctor: though we obviously lack the full test
[11:25] <Eighth_Doctor> zyga: oh cool, can I cherry-pick that into 2.42 release for Fedora?
[11:25] <zyga> Eighth_Doctor: it's not the full fix yet, just initial patches
[11:25] <Eighth_Doctor> I haven't pushed it yet because this bug just came up as I started working on it
[11:25] <zyga> Eighth_Doctor: base snaps will have /var/home so we can start binding onto it
[11:25] <Eighth_Doctor> ah, I see
[11:26] <Eighth_Doctor> we'll need to keep that in mind for fedora base as well
[11:26] <mup> Bug #1732494 changed: Impossible to manage package after Trusty --> Xenial upgrade <apport-bug> <i386> <third-party-packages> <xenial> <snapd:New> <snapd (Ubuntu):Confirmed> <https://launchpad.net/bugs/1732494>
[11:26] <zyga> mvo: do you have access to launchpad.net/snapd project description?
[11:26] <zyga> mvo: it would be good to change the first line
[11:26] <zyga> mvo: now it reads "code hosted at URL"
[11:26] <zyga> mvo: but if you re-assign bugs between projects all you get is "code hosted at" period
[11:27] <zyga> mvo: we could change it to say something nice, like "snapd is the best way to manage snap packages" :)
[11:28] <mvo> zyga: updted
[11:28] <zyga> mvo: super, thank you
[11:29] <zyga> can you edit again to replace the word "snap ..." with snaps
[11:29] <zyga> the message gets cut off at "snap"
[11:29] <mup> Bug #1647333 changed: adduser misses extrausers support for group management <patch> <snapd:New> <adduser (Ubuntu):Confirmed> <shadow (Ubuntu):In Progress by ogra> <https://launchpad.net/bugs/1647333>
[11:29] <zyga> sorry :)
[11:30] <mvo> zyga: done
[11:30] <zyga> haha, now it says: the best way to manage snaps. Code\n
[11:30] <zyga> and that's all :D
[11:30] <zyga> oh well, it's good enough
[11:31] <ogra> "a good enough way to manage snaps" ?
[11:32] <mup> Bug #1662452 changed: snap install squid-gary failed on Ubuntu Core <snapd:Invalid> <https://launchpad.net/bugs/1662452>
[11:35]  * Chipaca breaks all the tests again
[11:35] <zyga> where are ubuntu-image bugs reported?
[11:35] <zyga> ogra: "a good enough way to"
[11:35] <zyga> mborzecki: do you know? ^
[11:36] <mborzecki> zyga: lp? or github project maybe
[11:36] <Chipaca> zyga: snaps@canonical.com ?
[11:36] <mborzecki> zyga: https://github.com/CanonicalLtd/ubuntu-image/ ?
[11:36] <zyga> thanks
[11:36] <Chipaca> popey: does email to that address reach you?
[11:36] <Chipaca> or who?
[11:36] <mborzecki> zyga: https://launchpad.net/ubuntu-image apaprently
[11:36] <zyga> wait
[11:36] <zyga> no issues there
[11:37] <popey> Chipaca: no, mvo
[11:37] <zyga> oh the horror
[11:37] <zyga> thanks
[11:37] <mborzecki> zyga: yeah, there's this message that points to LP
[11:37] <Chipaca> zyga: it has no bugs! \o/
[11:37] <zyga> Chipaca: I wish it was easier to find :)
[11:37] <mborzecki> Chipaca: no reported bugs :P
[11:37] <zyga> Chipaca: ship it!
[11:37]  * Chipaca ships everyone with everyone
[11:37] <popey> Chipaca: i lamented this back in malta
[11:37] <zyga> Chipaca: do you remember when I first wanted to join the team
[11:37] <zyga> Chipaca: I was supposed to create ubuntu-image
[11:38] <zyga> Chipaca: and maybe help with "resources" aka "capabilities" aka "skills"
[11:38] <Chipaca> zyga: my memory is a two-week round-robin database
[11:38] <zyga> Chipaca: what?
[11:38] <zyga> ;-)
[11:38] <mborzecki> Chipaca: journalctl --vacuum-time=2w ? :P
[11:38] <zyga> <gif of hampster on a treadmill>
[11:38] <mup> Bug #1649839 changed: unknown snap and snapd version when image created via ubuntu-image <Ubuntu Image:New> <https://launchpad.net/bugs/1649839>
[11:38] <mup> Bug #1655060 changed: snapcraft doesn't support the dbus daemon type <Snapcraft:New> <snapd:New> <https://launchpad.net/bugs/1655060>
[11:39] <Chipaca> actually it's more like rrdtool but close enough
[11:40] <Chipaca> popey: was your lament about the fact that the email is there, or that it only goes to the m of the v.o.?
[11:40] <popey> both
[11:40] <popey> it should go to a project github page, or home page
[11:41] <popey> see lxd for the pinnacle of snapcraft store pages
[11:41] <mup> Bug #1655593 changed: chattr code (tests/main/chattr/task.yaml) fails on ppc64el <snapd:Triaged> <https://launchpad.net/bugs/1655593>
[11:41] <mup> Bug #1659523 changed: console-conf doesn't work well if using screen on Mac <subiquity:New for wangliming> <https://launchpad.net/bugs/1659523>
[11:42] <zyga> mvo: can we put the snapcraft logo on snapd?
[11:42] <Chipaca> mvo: WDYT about having snaps@c.c also go to advocacy?
[11:42] <diddledan> zyga: that's far too sensible
[11:43] <Chipaca> mvo: and, WDYT about pointing ubuntu-image's contact to something better than an email address?
[11:43] <Chipaca> zyga: the snapcraft logo but in asciiart
[11:44] <zyga> Chipaca: in black and white
[11:44] <zyga> maybe that's too sad though
[11:44] <Chipaca> we have one of those
[11:44] <Chipaca> nah, sad is grey on grey
[11:44] <Chipaca> or is that depression
[11:44] <cachio> mborzecki, hey
[11:44] <mup> Bug #1679739 changed: System-User Assertions and the system time <snapd:New> <https://launchpad.net/bugs/1679739>
[11:44] <mup> Bug #1683823 changed: snapstore returns multiple instances of the same publication (was snapcraft list-revisions showing multiple publications in the same channel) <apw-snappy> <Snapcraft:New> <https://launchpad.net/bugs/1683823>
[11:44] <cachio> mborzecki, I m with the #7551
[11:44] <mup> PR #7551: tests: fix the how the systemd-journald.service is restarted during the prepare <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7551>
[11:44] <ogra> Chipaca, isnt u-image foundations child now ?
[11:45] <Chipaca> ogra: je ne sais pas, je ne sais plus, je suis perdu, etc
[11:45] <zyga> pedronis: something for your awareness: https://bugs.launchpad.net/snapd/+bug/1679739 -- perhaps assertions should be loaded and only re-validated (and actually take effect) once we have system time synchronized
[11:45] <mup> Bug #1679739: System-User Assertions and the system time <snapd:Triaged> <https://launchpad.net/bugs/1679739>
[11:46] <mborzecki> cachio: do you have more details about those journal failures?
[11:46] <cachio> mborzecki, yes
[11:46] <mborzecki> cachio: i have a hunch that it's the stop that fails
[11:46] <cachio> mborzecki, basically what happens with restart
[11:46] <cachio> is that it stops
[11:47] <cachio> but sometimes it makes a flush
[11:47] <cachio> during stop
[11:47] <cachio> and while it is flushing
[11:47] <mup> Bug #1679210 changed: snap install --revision tracks stable by default <snapd:New> <https://launchpad.net/bugs/1679210>
[11:47] <cachio> it tries to start
[11:47] <cachio> and it can't
[11:47] <mborzecki> yeah, probably when there's a lot of data bc snapd is chatty with debugs enabled
[11:48] <cachio> so retries many times until it reaches the starts limit
[11:48] <mborzecki> cachio: w8, so it didn't stop, is still flushing and nother journald instance starts up?
[11:48] <cachio> and fails
[11:48] <cachio> mborzecki, it doesn't finish the stop and it is trying to start
[11:49] <mborzecki> wow
[11:49] <cachio> mborzecki, if you see the log there, the error is Active: failed (Result: start-limit-hit) since Thu 2019-10-03
[11:49] <cachio> mborzecki, it is just happening on core18 and sometimes on core16
[11:50] <cachio> I could not reproduce is on classic
[11:50] <cachio> mborzecki, I am creating a debug session now
[11:52] <zyga> what on earth is "ubuntu-snappy" source package
[11:53] <cjwatson> Old name for the snapd source package
[11:53] <zyga> ah
[11:59] <mup> Bug #1657552 changed: [spread] install-sideload:reexec0 failure <Snappy:Won't Fix> <https://launchpad.net/bugs/1657552>
[11:59] <mup> Bug #1662772 changed: systemd[1]: Failed to start Set the hostname to the value stored on the writable partition <snappy-set-hostname.service> <Snappy:Won't Fix> <https://launchpad.net/bugs/1662772>
[12:01] <ogra> zyga, dont kill it in case you want to go back to a python based snapd :P
[12:02] <zyga> ogra: maybe one day we can snap install snapd0 ;)
[12:02] <ogra> haha
[12:02] <mup> Bug #1684343 changed: For candicate release is appearing on dragonboard the message "ERROR hal_remove_bsskey response failed err" <subiquity:New> <https://launchpad.net/bugs/1684343>
[12:05] <mup> Bug #1671778 changed: failover:emptysystemd test fails <Snappy:Won't Fix> <https://launchpad.net/bugs/1671778>
[12:05] <mup> Bug #1679432 changed: Bluetooth SCO connections don't work on Dragonboard 410c <Snappy:Invalid> <https://launchpad.net/bugs/1679432>
[12:05] <mup> Bug #1679747 changed: Cannot send bluetooth SCO packets with Raspberry Pi 3 internal bluetooth module. <Snappy:Invalid> <https://launchpad.net/bugs/1679747>
[12:07]  * pstolowski lunch
[12:08] <mup> Bug #1666690 changed: Dependency issues when sharing a library through content interface <personal> <Canonical System Image:New> <snapd:New> <https://launchpad.net/bugs/1666690>
[12:14] <mup> Bug #1705985 changed: snaps fail to install on juju1 deployed lxc container <canonical-bootstack> <snapd:Incomplete> <https://launchpad.net/bugs/1705985>
[12:43] <zyga> kenvandine: hello, could you please enqueue this bug report https://bugs.launchpad.net/snapd/+bug/1661626 -- there's some things related to gsettings and dbus signal for notification on changes. I could use some help in understanding how the desktop stack works in this area.
[12:43] <mup> Bug #1661626: GSettings/dconf reports incorrect values on setting change under confinement <desktop> <snapd:Triaged> <https://launchpad.net/bugs/1661626>
[12:44] <mup> Bug #1661626 changed: GSettings/dconf reports incorrect values on setting change under confinement <desktop> <snapd:Triaged> <https://launchpad.net/bugs/1661626>
[12:44] <mup> Bug #1662962 changed: 'snap find' does not allow channel specification <snapd:Triaged> <https://launchpad.net/bugs/1662962>
[12:48] <mup> Bug #1665683 changed: Strictly confined snapped desktop applications can't toggle Launch at Login <isv> <snapd:Fix Released> <https://launchpad.net/bugs/1665683>
[12:51] <mup> Bug #1689302 changed: Can't set program_usb_boot_mode pi-config option <ce> <snapd:Confirmed> <https://launchpad.net/bugs/1689302>
[12:51] <zyga> sil2100: could you please enqueue this bug to your list https://bugs.launchpad.net/snappy/+bug/1740655 -- I'm not sure what to do about it
[12:51] <mup> Bug #1740655: Using dtoverlay=pi3-disable-bt to disable Bluetooth results in unbootable system <Snappy:Triaged> <https://launchpad.net/bugs/1740655>
[12:59]  * Chipaca stands up
[13:00] <ijohnson> morning folks, I think that https://github.com/snapcore/snapd/pull/7431 is basically ready, if folks want to take a really quick look and see if there's anything obvious that I could split out into different smaller PRs that would be great
[13:00] <mup> PR #7431: overlord/snapstate: don't re-enable and start disabled services on refresh, etc <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/7431>
[13:04] <mup> PR snapd#7602 opened: overlord/many: extend check snap callback to take snap container <Remodel :train:> <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7602>
[13:10] <kenvandine> hey zyga
[13:10] <zyga> hey
[13:11] <zyga> how are you? :)
[13:11] <kenvandine> i'll look at it
[13:11] <mborzecki> weird failure with current master https://paste.ubuntu.com/p/8P6hYK5PcJ/
[13:11] <kenvandine> good, recovering from a long weekend :)
[13:11] <zyga> kenvandine: thank you, it's not urgent, just something that came out of our bug gardening
[13:12] <zyga> mborzecki: two comments
[13:12] <zyga> mborzecki: how big is that deb?!?!
[13:12] <zyga> mborzecki: and maybe systemd became a virtual package somehow?
[13:12] <mborzecki> zyga: idk
[13:16] <zyga> cachio: can you please look at https://bugs.launchpad.net/snapd/+bug/1847665 -- it seems one snap that we get from the store is not published for i386 - perhaps we need to nuke the test or perhaps we need to publish the test snap but it looks like it should work (the test snap name is multi-arch)
[13:16] <mup> Bug #1847665: snapd autopkgtest fails on i386 <snapd:Triaged> <snapd (Ubuntu):Triaged> <https://launchpad.net/bugs/1847665>
[13:16] <zyga> cc mvo ^ not sure who has access to those
[13:18] <kenvandine> marcustomlinson: can you look at bug 1661626 ?
[13:18] <mup> Bug #1661626: GSettings/dconf reports incorrect values on setting change under confinement <desktop> <snapd:Triaged> <https://launchpad.net/bugs/1661626>
[13:19] <zyga> mvo: can you please look at https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1844498 -- you were participating in the discussion there and I'm not sure if the snapd package task is something that is still accurate - should it change to triaged or invalid?
[13:19] <mup> Bug #1844498: 18.10+ cloud images have the LXD group as gid 1000 <id-5d84bb1ca26b0679a708dc40> <cloud-images:New> <cloud-init (Ubuntu):Invalid> <livecd-rootfs (Ubuntu):Fix Released by mwhudson> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1844498>
[13:19] <marcustomlinson> kenvandine: ok
[13:19] <kenvandine> marcustomlinson: thanks
[13:24] <sil2100> zyga: yes, something related to it is actually in the works
[13:24] <jdstrand> pedronis: hey, would you mind giving me an updated list of pulseaudio.snap-ids? if you don't have time, perhaps describe the query to roadmr (or me and I can find someone if he can't do it)?
[13:24] <zyga> sil2100: excellent, thank you
[13:25] <sil2100> zyga: so how it's most probably going to be resolved is for us shipping two different image flavors - one with bluetooth enabled, one with disabled
[13:25] <sil2100> Since from what I've been told on most pi's you can't actually have both serial and bluetooth enabled at the same time
[13:26] <zyga> sil2100: you can, it's just shifting the serial around to other pins
[13:26] <jdstrand> pedronis: I don't need it *immediately* since I can look in the snap usns dump and continue working, but I want to make sure I didn't miss any new ones
[13:26] <zyga> sil2100: AFAIK
[13:26] <zyga> jdstrand: hey :)
[13:27] <zyga> jdstrand: welcome back
[13:27] <sil2100> zyga: I'm not a pi expert so I can't really say, waveform would have more info about that
[13:27] <sil2100> ;)
[13:27] <jdstrand> pedronis: plan is, get through all current ones, propose a pr to manually connect, then right before 2.43 lands, do one more query and fill in any more
[13:27] <zyga> cool, I'm sure he knows more than I do
[13:27] <jdstrand> hey zyga, thanks! :)
[13:27] <zyga> just happy to see progress
[13:28] <pedronis> jdstrand: I can try to look tomorrow morning
[13:28] <cachio> zyga, I'll take a look
[13:28] <jdstrand> pedronis: thanks
[13:28] <zyga> cachio: thanks!
[13:29] <zyga> Chipaca: something that feels like a papercut and I agree with the reporter
[13:29] <zyga> https://bugs.launchpad.net/snapd/+bug/1830183
[13:29] <zyga> Chipaca: related to how we log
[13:29] <mup> Bug #1830183: Absence of updates is labelled a critical error in syslog <snapd:Triaged> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1830183>
[13:29] <Chipaca> zyga: papercut, indeed
[13:30] <zyga> Chipaca: though not sure how to technically solve it
[13:30] <Chipaca> easily :-)
[13:30] <zyga> Chipaca: maybe we should talk to journald natively so that we can send those nice log properties
[13:30] <Chipaca> whoa
[13:31] <Chipaca> zyga: maybe, possibly, if there's a reasonable library for it, but that's not this bug
[13:31] <mup> PR snapd#7603 opened: dpdk and hugepages interfaces <Created by alfonsosanchezbeato> <https://github.com/snapcore/snapd/pull/7603>
[13:31] <Chipaca> this bug is just "don't log no-refreshes-available with INFO" probably :)
[13:31] <Chipaca> zyga: that's what makes it papercut
[13:31] <zyga> Chipaca: IIRC the format is simple so it's just about formatting the error "text" correctly, I think we'd prefer to carry that code instead of depending on it
[13:31] <zyga> Chipaca: yeah :)
[13:31] <zyga> Chipaca: (and talking over the right FD)
[13:32] <Chipaca> yeah, and keeping up with changes, and and and
[13:32] <roadmr> mvo: are you around?
[13:34] <mvo> roadmr: yes
[13:38] <zyga> Chipaca: no, it's not like that actually
[13:39] <zyga> Chipaca: http://0pointer.de/blog/projects/journal-submit.html
[13:39] <Chipaca> zyga: are you really advocating for NIH'ing a systemd library
[13:40] <zyga> Chipaca: not a systemd library, a public protocol
[13:48] <pstolowski> i got lxd spread test failure again on 18.04, fails with "snapd : Depends: systemd but it is not going to be installed", has anyone seen this? 18.04 image needs updating?
[13:53] <zyga> pstolowski: maciek did IIRC
[13:54] <zyga> maybe apt database is empty?
[13:55] <zyga> pstolowski: this is interesting
[13:55] <zyga> https://errors.ubuntu.com/problem/5e58e04cdc647a1fd50f71f02724dd0b5b8c1f4b
[13:55] <zyga> pstolowski: udev netlink hotplug panic
[13:55] <zyga> the bug behind it is https://bugs.launchpad.net/snapd/+bug/1824162
[13:55] <mup> Bug #1824162: /usr/lib/snapd/snapd:11:runtime:runtime:runtime:runtime:runtime <artful> <bionic> <cosmic> <xenial> <zesty> <snapd:New> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1824162>
[13:55] <zyga> can you please enqueue looking at it
[13:56] <zyga> mvo: can I close https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1824237 now?
[13:56] <mup> Bug #1824237: snap 'core' broken/missing and causing autopkgtest failures <snapd:In Progress by zyga> <snapd (Ubuntu):New> <snapd (Ubuntu Cosmic):New> <https://launchpad.net/bugs/1824237>
[13:59] <mvo> zyga: yes, all autopkgtest related bugs can be closed
[14:00] <zyga> mvo: _all_? I found one that looks like a real bug
[14:02]  * zyga is done with triage for the day
[14:02]  * zyga -> break / food
[14:03] <cachio> zyga, I dont have test-snapd-hello-multi-arch under my control
[14:03] <zyga> cachio: it probably belongs to mvo
[14:03] <cachio> zyga, not sure who created this one
[14:04] <zyga> someone from the store can probably help you move it?
[14:04] <pstolowski> zyga: hmm, i think i see where the problem is on the very high level.. go routine running in the background. will check, thanks for pointing out
[14:04] <zyga> roadmr: ^ test-snapd-hello-multi-arch, do you know who is in the collaborators?
[14:04] <roadmr> I can check
[14:04] <zyga> pstolowski, roadmr: cool, thank you for looking
[14:05] <zyga> (one messgage, two topics, take that tracking AI overloards)
[14:05] <roadmr> zyga: nobody 🦀
[14:05] <zyga> oh crab :D
[14:05] <zyga> roadmr: can you please give it to cachio and mvo perhaps, following all protocol
[14:06] <roadmr> zyga: the current owner (the "canonical" account, not sure if this is mvo?) has to add collaborators. I can't do it administratively :(
[14:06] <zyga> I see
[14:06] <zyga> is there an email address?
[14:06] <zyga> mvo: are _you_ secretly all of canonical? :D
[14:07] <ijohnson> does anybody have an example of scanning journald in spread tests for something specific in the logs? is there a helper for "discard everything in the log from x point later on" to just check something in the logs happened after a certain point?
[14:07] <zyga> ijohnson: plenty
[14:07] <zyga> we nuke the journal between tests
[14:07] <zyga> so anything that uses 'journal' helper functions is likely a candidate
[14:07] <ijohnson> right but which journal helper there are several
[14:08] <zyga> I can find a specific example but the main principle applies
[14:08] <zyga> yeah
[14:08] <zyga> but they all do the same thing
[14:08] <zyga> with various quirks on how
[14:08] <zyga> I agree that multitude of functions is confusing
[14:08] <ijohnson> all I need to do is check for a certain message in the log happening after a certain line in the test
[14:08] <ijohnson> thanks btw!
[14:08] <zyga> I think this is a buggy area
[14:08] <zyga> and the helper that we use most often has some loops inside
[14:09] <zyga> some journal versions have a marker concept
[14:09] <zyga> so you can cat the whole log since a marker was emitted
[14:09] <zyga> then grep that
[14:09]  * zyga needs to break for some back pain relief
[14:09] <ijohnson> that feels like the right thing, do you have an example I can look at zyga?
[14:10] <ijohnson> or maybe cachio knows
[14:10] <zyga> ijohnson: sorry, leaving now
[14:10] <zyga> I can later
[14:10] <zyga> if you don't get it before
[14:11] <ijohnson> zyga: np get better I can figure it out
[14:11] <cachio> ijohnson, what we do in some scenarios
[14:11] <cachio> is to count the number of ocurrencies before and after
[14:11] <cachio> and compare the number is iscreased
[14:12] <zyga> ijohnson: just ward of caution, it's all good except on 14.04 :D
[14:12] <zyga> *word
[14:12] <cachio> ijohnson, take a look to test catalog-update
[14:12] <ijohnson> trusty seems not so trustworthy anymore with respect to features :-/
[14:12] <cachio> perhaps it could help
[14:12] <ijohnson> cachio, thanks looking
[14:13] <ijohnson> cachio: that looks like the easiest way to check this, thanks
[14:13] <cachio> ijohnson, yaw
[14:16]  * Chipaca takes a break
[14:34] <mvo> I saw some failures in current PRs related to the lxd test, looks like systemd was not installable in there, anyone know more ?
[14:45] <pedronis> mvo: what's the public interface of 7593 ? most things seem unexported
[14:46] <pedronis> you create a SFDisk manually?
[14:51] <pedronis> mvo: I put some questions there
[14:51]  * cachio lunch
[15:02] <mvo> pedronis: public interface is just partition.NewSFdisk() and you go from there, two exported methods on it, DeviceInfo and CreatePartitions
[15:02] <mvo> pedronis: I will check in the PR
[15:02] <pedronis> mvo: that new is private atm
[15:02] <pedronis> anyway see comments
[15:03] <mvo> pedronis: yeah, everything is spot on - I think we don't really need the interface, *might* be handy for tests later, I check the full branch from claudio - but we can always add it back once we need it so I'm +1 to remove it
[15:04] <pedronis> ok
[15:05] <thresh> what does "grandfathering" mean in context of https://forum.snapcraft.io/t/upcoming-pulseaudio-interface-deprecation/13418 ?
[15:08] <mup> PR snapd#7604 opened: many:  address issues related to explicit/implicit channels for image building <Created by pedronis> <https://github.com/snapcore/snapd/pull/7604>
[15:09] <pedronis> Chipaca: mvo: ^
[15:09] <pedronis> thresh: that a declaration will be setup in the store to keep them working (for now at least) with pulseaudio as they did so far
[15:14] <thresh> pedronis, good, thank you!
[15:14] <thresh> (i suck at idioms)
[15:21] <mvo> thanks pedronis
[15:51] <pstolowski> zyga: i think i found the bug in the upstream go-udev code that causes the crash
[16:16] <pstolowski> commented on the bug
[16:17] <mup> PR snapd#7605 opened: tests: configure the journald service for core systems <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7605>
[16:19] <mup> PR snapd#7551 closed: tests: fix the how the systemd-journald.service is restarted during the prepare <Simple 😃> <Created by sergiocazzolato> <Closed by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7551>
[16:20] <zyga> pstolowski|afk: woot, great
[16:20] <zyga> makes bug gardening worthwhile :)
[16:21] <pstolowski|afk> yes :)
[16:21] <pstolowski|afk> slightly terrible we had this bug for a while
[16:25] <mvo> cachio: can you please have an eye on master in travis? I just noticed some failures in recent PRs related to lxd. would be great if we can get master green again (if its something real and not just transient)
[16:25] <mvo> pedronis: I updated the sfdisk PR now and addressed your points
[16:41]  * ogra notes pstolowski|afk must have looong arms
[16:58] <jdstrand> Chipaca: hey, you gave me a cool command for downloading the snap.yaml (with caveats) of a snap in the public store. do you happen to know a command otoh to get the snap decl for a given snap name?
[16:59] <jdstrand> Chipaca: snap download would do it, but I don't want the snap, just the decl (I'll be doing hundreds of downloads)
[17:09]  * jdstrand looks at 'snap download'
[17:11] <pstolowski|afk> ogra: now you know ;)
[17:14] <pedronis> jdstrand: which revisions do you need ?
[17:15] <jdstrand> pedronis: the latest
[17:15] <jdstrand> pedronis: well, I just need the snap decl, which is shared across revisions
[17:15] <pedronis> jdstrand: ? do you nee the snap.yaml or the snap decl?
[17:15] <pedronis> not the same thing
[17:15] <jdstrand> pedronis: the snap decl
[17:16] <pedronis> jdstrand: do you have the snap name or the snap id ?
[17:16] <jdstrand> pedronis: both
[17:17] <pedronis> jdstrand: something like: snap known --remote snap-declaration series=16 snap-id=KTe2wdAu5JKdRDUgYBuXXGjDXyzobvFI
[17:18] <jdstrand> pedronis: that is perfect, thanks! (cc Chipaca)
[18:01] <mup> PR snapd#7606 opened: tests: run apt update in lxd containers <Created by mvo5> <https://github.com/snapcore/snapd/pull/7606>
[19:33] <ijohnson> cachio: have you been able to figure out the issue with the lxd tests? I see it keeps failing consistently, and mvo's patch apt update still failed
[19:34] <ijohnson> err, mvo's patch to call apt update before doing the install of snapd in the container still fails (see #7606)
[19:34] <mup> PR #7606: tests: run apt update in lxd containers <Created by mvo5> <https://github.com/snapcore/snapd/pull/7606>
[19:34] <cachio>  ijohnson working on that
[19:34] <ijohnson> ack ok
[19:36] <cachio> ijohnson, I just finished a task and started with that one
[19:36] <cachio> ijohnson, currently debugging
[19:38] <ijohnson> cachio: I looked into it a little bit but didn't find anything if you weren't looking into it I was going to, but if you're on top of I'll work on other things
[19:38] <cachio> ijohnson, I'll try to fix it
[19:39] <cachio> otherwise I'll ping you in case I need help
[19:41] <joedborg> jdstrand: will you have some free time this PM?
[19:42] <ijohnson> cachio: ack sounds good
[20:11] <jdstrand> joedborg: if by 'this PM' you mean now, sure :)
[20:12] <jdstrand> zyga: hey, it looks like solus is still at 2.39.3 and parrot os at 2.37. curious when you think they'll get updated to 2.41+
[20:14] <jdstrand> popey, Wimpress: hey, can one of you talk to electron about https://github.com/electron-userland/electron-builder/issues/4234?
[20:18] <joedborg> jdstrand: excellent.  Very basically, I'm still seeing times where containers are getting filesystem permission denied
[20:18] <jdstrand> joedborg: what are the denials? shall I login somewhere?
[20:20] <joedborg> jdstrand: let me copy your ssh keys
[20:23] <joedborg> jdstrand: ubuntu@34.201.205.205
[20:24] <jdstrand> I'm in
[20:25] <joedborg> jdstrand: we can move this to 1:1 if we're worried about spamming.  Basically, this is an EKS node where I've ripped out the EKS provisioned kubelet and stopped docker, replacing it with our snap
[20:25] <jdstrand> ok. yes, let's privmsg since this isn't interesting to others
[20:25] <joedborg> jdstrand: the only pod that's having a problem is the aws-node pod, which tried to create `/host` in its namespace
[20:26] <mup> PR snapd#7607 opened: tests: taunch the lxd images folowing the pattern ubuntu:${VERSION_ID} <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7607>
[20:27] <cachio> ijohnson, if you want to take a look, there is a PR to fix the lxd issue
[20:27] <cachio> #pull
[20:27] <cachio> #7607
[20:28] <mup> PR #7607: tests: taunch the lxd images folowing the pattern ubuntu:${VERSION_ID} <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7607>
[20:28] <jdstrand> joedborg: I privmsg'd you
[20:55] <jdstrand> ijohnson: hi!
[20:56] <ijohnson> jdstrand: hey
[20:56] <jdstrand> ijohnson: joedborg and I are looking at a denial when running kubernetes-worker under strict mode on EKS
[20:56] <ijohnson> ok
[20:56] <jdstrand> Oct 15 20:31:57 ip-192-168-91-85 audit[21757]: AVC apparmor="DENIED" operation="exec" profile="snap.kubernetes-worker.containerd" name="/app/install-aws.sh" pid=21757 comm="sh" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[20:56] <jdstrand> ijohnson: containerd is triying to run /app/install-aws.sh which corresponds to https://github.com/aws/amazon-vpc-cni-k8s/blob/master/scripts/install-aws.sh
[20:57] <jdstrand> ijohnson: the fact that it is at /app (and later /host) means we can't use a layout
[20:57] <ijohnson> right
[20:57] <jdstrand> ijohnson: and I thought I remembered you facing similar issues with greengrass
[20:58] <ijohnson> yeah so first I'll say what we did for the docker snap is that we patched dockerd (in your case probably containerd) to not run pivot_root and instead just do a chroot
[20:58] <joedborg> ijohnson: this is what we were talking about with perhaps needing to patch containerd's app armor template
[20:58] <joedborg> :)
[20:58] <ijohnson> joedborg: jdstrand: is there a reason why you need to do pivot_root?
[20:59]  * ijohnson goes and finds the dockerd patches
[21:00] <jdstrand> ijohnson: I don't know what is doing a pivot_root
[21:00] <jdstrand> joedborg: ^
[21:00] <ijohnson> jdstrand: when the container is launched, it's rootfs is actually somewhere writable like `/var/snap/.../storage-drivers/....`
[21:00] <jdstrand> joedborg: is this a pivot_root or does EKS actually ship /app and /host?
[21:01] <ijohnson> jdstrand: and to start the container containerd/dockerd will do a pivot_root into that storage-driver dir
[21:01] <joedborg> jdstrand: that's what containerd does when the kubelet asks it to run a workload
[21:01] <ijohnson> this is all behind the scense
[21:01] <jdstrand> ijohnson: I should also reiterate this is classic, not core
[21:01] <ijohnson> *scenes
[21:01] <ijohnson> jdstrand: same problem on core as classic though
[21:01] <jdstrand> ah ok
[21:02] <jdstrand> so, in theory, container do could mv /var/snap/.../storage-drivers/..../app, then symlink from /var/snap/.../storage-drivers/..../app to /var/snap/.../storage-drivers/..../somewhere/deeper/app
[21:02] <ijohnson> anyways the issue that happened with the docker snap is that if dockerd ran pivot_root then all of the container fs accesses that the container thought were at `/app/...` were really actually at `/var/snap/.../storage-driver/some-big-uuid/app`
[21:03] <ijohnson> but how we solved that was do instead just not do a pivot_root and instead just do a chroot
[21:03] <jdstrand> ijohnson: right, makes sense
[21:03] <ijohnson> then apparmor doesn't see the container processes as accessing /app it sees /var/snap/.../app
[21:03] <jdstrand> ijohnson: is the chroot a configurable thing?
[21:03] <ijohnson> no :-(
[21:03] <jdstrand> ijohnson: yep
[21:03] <ijohnson> hence why we had to patch it
[21:03] <ijohnson> here's the patch we did: https://git.launchpad.net/~docker/+git/snap/tree/dockerd-patches/snappy-real-chroot.patch
[21:03] <jdstrand> ijohnson: ah, so you patched docker to do that
[21:03] <ijohnson> I'm trying to find where in containerd you would do the same patch
[21:04] <jdstrand> ijohnson: cool, thanks!
[21:05] <ijohnson> it might be in runc actually, it's been a while since I've looked at thsi stuff
[21:06] <joedborg> ijohnson: jdstrand: just as a (probably stupid) question, is there no way for apparmor to know that a process is being containerised and to  apply different rules?
[21:06] <joedborg> ijohnson: yeah it'll almost certainly be runc
[21:07] <ijohnson> well for dockerd it is in fact in github.com/moby/moby aka "real dockerd"
[21:08] <ijohnson> but there has to be an equivalent thing in runc/containerd
[21:08] <jdstrand> joedborg: re apparmor: apparmor absolutely knows stuff, but containerd/runc is doing things to complicate things
[21:09] <jdstrand> joedborg: ie, it is containerdc/runc that is putting the container into the cri-containerd.apparmor.d
[21:09] <ijohnson> joedborg: do you know offhand if containerd just runs the `runc` binary directly? or does it use the opencontainers go library ?
[21:09] <joedborg> jdstrand: ah i mean can't apparmor be told "this proc is being called by containerd so should be allowed free range of the FS"?
[21:09] <jdstrand> joedborg: and that profile is what your snap is loading
[21:09] <joedborg> ijohnson: almost certain that it calls the binary
[21:09] <ijohnson> jdstrand: but for that specific denial you showed me with /app that's for a snap ?
[21:09] <ijohnson> joedborg: ack thanks
[21:09] <joedborg> because the runtime binaries are dropped in via config
[21:10] <jdstrand> joedborg: so you can put whatever is needed for the container. *but* this denial is not for the container, but for containerd, and its doing something post pivot_root(). depending on your pov, apparmor doesn't handle pivot_root well
[21:11] <joedborg> jdstrand: yeah, i guess that was me question; can't it handle pivot root like it does chroot
[21:11] <ijohnson> yeah I can see apparmor's pivot_root handling being a bug right now but being a savior in other cirumstances so yeah ... I dunno
[21:12] <jdstrand> joedborg: it cannot
[21:12] <jdstrand> ijohnson: we have an upstream bug on it
[21:12]  * jdstrand is trying to find it
[21:12] <ijohnson> I think I've seen the bug before
[21:13] <jdstrand> ijohnson: iirc, I think we want full mediation of the whole path with something like an alias mechanism to make it convenient and flexible
[21:13] <ijohnson> yeah that sounds right
[21:17] <ijohnson> I haven't been able to find the bit in containerd you need to patch, but I have to EOD now, I will resume looking tomorrow, if you can't wait, I found that there's already an option to specify disabling pivot_root when creating a container, referenced in the oci options
[21:17] <ijohnson> i.e. https://github.com/containerd/containerd/blob/master/runtime/v2/runc/options/oci.pb.go#L27-L28
[21:17] <ijohnson> I'm not sure how to use that, but that might avoid needing to patch things
[21:17] <jdstrand> this is the bug: https://bugs.launchpad.net/apparmor/+bug/1791711
[21:17] <ijohnson> anyways good luck, I will resume looking tomorrow to try and find the stuff y'all need
[21:17] <mup> Bug #1791711: path-based AppArmor controls for snap-confine are ineffective due to pivot_root <AppArmor:Confirmed> <https://launchpad.net/bugs/1791711>
[21:17] <jdstrand> I thought there was another one
[21:18] <jdstrand> but that is what we reference in the snapd sources too
[21:18] <joedborg> thakns ijohnson, i'll ask on their slack
[21:18] <joedborg> have a good evenign
[21:18] <ijohnson> thanks you too
[21:18] <jdstrand> ijohnson: big thanks!
[21:19] <jdstrand> joedborg: ok, so with the plugs changes we discussed and this, I think we are set for the moment
[21:19] <joedborg> jdstrand: yeah, thanks for pointing out the k8s-support needing to be with containerd too, i'll test that tongiht
[22:24] <Chipaca> yusss
[22:24] <Chipaca> got a PR this PM as promised \o/
[22:24] <mup> PR snapd#7608 opened: o/snapstate, etc: SnapState.Channel -> TrackingChannel, and a setter <Created by chipaca> <https://github.com/snapcore/snapd/pull/7608>
[22:24]  * Chipaca had *more than half an hour* to spare before it was no longer technically PM
[22:24]  * Chipaca now goes to wash the dishes