[00:12] PR snapd#6109 opened: interfaces/apparmor: allow access to /run/snap.$SNAP_INSTANCE_NAME [00:15] PR snapd#6110 opened: spread.yaml: add openSUSE Tumbleweed [00:16] PR snapd#6111 opened: packaging/opensuse: move most logic to snapd.mk [00:18] jdstrand: FYI https://github.com/snapcore/snapd/pull/6111 is the thing I was talking about [00:18] PR #6111: packaging/opensuse: move most logic to snapd.mk [02:28] how do I say "rain is coming", because you saw on the forecast, in a normal conversation? === blackboxsw is now known as blackboxsw_away [06:36] Good morning [06:39] PR snapd#6093 closed: daemon: spool sideloaded snap into blob dir [06:40] hey zyga [06:44] zyga: 6110 has two test failures in the new tumbleweed tests [06:44] zyga: and 6109 has some new comments [06:47] PR snapd#6101 closed: switch travis unit tests to xenial [06:57] Thank you [06:57] Tumbleweed failed for me locally but I wanted to share this with Sergio [06:58] Iโ€™ll get a shower and be back soon [06:59] I posted 6111, curious to know what you think [07:00] zyga: sure, thanks [07:32] Man, I can barely see where Iโ€™m going in this fog === pstolowski|afk is now known as pstolowski [07:59] morning [08:00] hey pstolowski - good morning [08:00] re [08:00] good morning pawel [08:07] mvo: can I push https://github.com/snapcore/snapd/pull/6109 for 2.36.1 as well? [08:07] PR #6109: interfaces/apparmor: allow access to /run/snap.$SNAP_INSTANCE_NAME [08:08] zyga: yeah, please tag it [08:08] zyga: 6096 needs a second review [08:08] looking [08:08] zyga: I will release 2.36.1 once this is green [08:09] done [08:14] ta [08:22] mvo: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/s/snapcraft/20181030_213532_e6942@/log.gz this is in bionic-proposed. could you have a look? that's the python3.6 3.6.7 update [08:23] doko: snapcraft is sergiusens baby - I assume this happens all the time and is not a one-off race or something in their suite? [08:24] doko: hm, hm, "{{{The package python3-dev has unmet dependencies:}}}" [08:24] doko: is/was there some inconsistency in the archive when this ran maybe? [08:28] I'll run it again [08:29] ta [08:54] mvo: shall I squash https://github.com/snapcore/snapd/pull/6096 [08:54] PR #6096: spread.yaml: add more systems to the autopkgtest and qemu backends [08:56] mvo: adopted baby, its your original baby formed with another Michael, you have commit number 3 after all ๐Ÿ˜‰ [08:56] sergiusens: haha [08:56] sergiusens: true [09:03] PR snapd#6096 closed: spread.yaml: add more systems to the autopkgtest and qemu backends [09:22] PR snapd#6112 opened: correct localInstallCleanup doc comment [09:35] hi, I'm having a hard time getting a python-based core18 classic snap to work. if I install it on xenial, I get this: https://paste.ubuntu.com/p/PbPq2bp5md/ [09:37] Chipaca, hi, so I've been trying to fix that snap of mine wrt python3, but I see this issue on xenial: https://paste.ubuntu.com/p/PbPq2bp5md/ [09:37] Chipaca, is snapcraft doing something wrong with python3? [09:37] ackk: looking at your log [09:37] thanks zyga [09:37] Chipaca: hi, #6112 [09:37] PR #6112: correct localInstallCleanup doc comment [09:37] The x3 vs x5 aspect is curious [09:38] Is the python interpreter present in x5 as well? [09:38] ackk: classic snap? [09:38] zyga, ah sorry, that was because I run from history, x5 is the last attempt [09:38] Chipaca, yes, classis, python, core18 [09:38] *classic [09:38] Classic as in classic confinement? [09:38] ackk: snap list --all sshoot ? [09:39] pedronis: ah good catch and thanks [09:40] Chipaca, https://paste.ubuntu.com/p/RvDdMMzW2g/ [09:40] zyga, yes [09:40] is there another type of classic? :) [09:40] Well classic system [09:40] ackk: a snap can be classic, a system can be classic, and there is a devmode snap that's called "classic" [09:40] ah I see [09:41] It is the classic problem of name overloading ;) [09:41] (I'm testing in a xenial lxd if that matters) [09:41] zyga, classic joke, classic :) [09:41] ackk: you built this on 18.04? [09:41] Chipaca, yes [09:42] ackk: on 18.04, what happens when you run python from the snap like that? [09:42] ackk: that is [09:42] ackk: please, snap run --shell sshoot, start python3 in there, let me know how it goes (i'll have more things for you to do in the python) [09:43] pstolowski: didn't we have a way to disable a service froma hook? [09:43] Chipaca, it starts (and "sshoot" also works) [09:44] Chipaca: well, not really, the only way to do something with services from hook is via snapctl [09:46] ackk: import _crypt [09:46] ackk: print(_crypt) [09:46] pstolowski: grmbl [09:46] pstolowski: i'll see if i can fix this :-) [09:46] Chipaca, [09:47] x1! [09:47] zyga, different container, this is bionic [09:47] Ah [09:50] pstolowski: where are hooks documented, for snap developers? [09:51] ackk: can you ldd the python binary in the snap? [09:51] ackk: from inside the snap [09:51] Chipaca: oh you... [09:51] sparkiegeek: i have a developer wanting to use them [09:51] Chipaca: https://forum.snapcraft.io/t/supported-snap-hooks/3795 and https://forum.snapcraft.io/t/interface-hooks/8214 [09:52] pstolowski: thanks [09:52] pstolowski: I'm also going to look into disabling :-) [09:53] nice [09:53] sparkiegeek: or wait, have I continued to break the store [09:53] Chipaca: nah, not today [09:53] Chipaca, https://paste.ubuntu.com/p/kjFbQccfQV/ (bionic and xenial) [09:54] Chipaca, isn't the last entry problematic? it points to the system library [09:55] ackk: yes [09:55] ackk: i think so at least :-) [09:55] ackk: there's this other little-known classic python snap we could look at for inspiration [09:55] ackk: schnappcraft, or something like that, probably about getting drunk [09:56] lol [09:56] Chipaca, isn't snapcraft also classic? :) [09:56] ackk: but not core18 which might be the extra quirk here [09:56] right [09:58] Chipaca, so, is that a core18 bug? 'cause the library isn't there in core18 [09:59] degville: hey, just noticed in https://forum.snapcraft.io/t/interface-hooks/8214 - is the colon at the end of "Interface hooks can read the attributes of the affected plug and slot by using the snapctl command, as defined by the snapโ€™s snap.yaml:" sentence intended? [09:59] PR snapd#6109 closed: interfaces/apparmor: allow access to /run/snap.$SNAP_INSTANCE_NAME [10:00] Chipaca, indeed on xenial that's a broken link [10:00] pstolowski: no - good spot, thanks. [10:01] Chipaca, https://paste.ubuntu.com/p/h7MmBYgjjp/ [10:10] mvo, ^ maybe you have insight [10:13] ackk: I lack context, is the problem that the symlink is absolute? [10:13] mvo, no, the problem is that it's broken on xenial, as the ld.so version is different [10:14] mvo, should core18 perhaps include ld.so as well? [10:14] mvo, it's 2.23 not 2.27 on xenial [10:17] ackk: I think the problem is actually that this link needs to be relative, i.e. /lib64/ld-linux-x86-64.so.2 must point to ../lib... [10:17] ackk: let me look into this [10:17] mvo, ah, so it should point to the snap [10:19] ackk: yeah, the file is there [10:20] ackk: there in the snap [10:20] mvo, ah so it's just a symlink issue [10:21] ackk: sorry i'm at the summit and only have half an ear for this right now ;-) [10:21] ackk: yeah, looking at the fix now [10:21] Chipaca, np it seems mvo has found the culprit [10:22] nice [10:22] * Chipaca hugs mvo [10:22] Chipaca, mvo, thanks [10:25] PR snapd#6113 opened: overlord/ifacestate: handler for hotplug-connect task [10:25] mvo: sudo find /snap/core18/current/ -type l -printf "%l from %p\n" | grep ^/ [10:26] PR core18#88 opened: hooks: make ld-so symlink in /lib64 relative [10:26] Chipaca: hrm, that are many [10:26] ackk: the above PR should fix the issue [10:26] mvo: I don't know if they're all bad though, but yes there are :-) [10:26] pstolowski: "snapctl stop --disable theservice" from within the snap works [10:26] pstolowski: from the install hook i mean [10:26] Chipaca: *sigh* thanks :) [10:27] Chipaca: ah, of course, i forgot stop had the flag /o\... sorry about that [10:28] mvo, cool, thanks [10:35] PR snapd#6112 closed: correct localInstallCleanup doc comment [10:38] mvo: thank you for cherry picking into 2.36 [10:38] Chipaca: I did a first pass on the epochs PR, I understad your focus is elsewhere this week, either way let me know if you disagree when you can look at it [10:39] zyga: my pleasure - we are complete now, I wait for travis and once its green will do 2.36.1 (maybe after lunch, depending on travis) [10:39] super [10:41] zyga: in the meantime I'm packaging/patching fontconfg like its 1998 [10:42] mvo: does current fontconifg support older cache formats/ [10:43] as in, can it write all formats from one binary [10:44] zyga: not afaict [10:44] mvo: tumbleweed fails on gpg key management from snap keys and all [10:45] I wonder if this is a sign of trouble to come in disco with new package sync [10:45] but I won't look more today [10:45] back to user mounts [10:45] I'll do a brief packaging update with 2.36.1 but other than that I'm happy with current state [10:45] zyga: what gnupg does it have? [10:46] https://www.irccloud.com/pastebin/rUQqGjEg/ [10:46] 2.2.10 [10:46] we'll need to dig [10:49] pedronis: thank you. I'll get to it tomorrow, i expect [10:49] tomorrow i'll be staying home [10:51] * zyga is on the road today but trying to focus [11:14] * zyga is in a car dealer shop, on headphones listening to music, writing C on opensuse [11:14] I wonder if people think I'm some kind of evil nasty hacker [11:15] oh, I also have a dog with me [11:16] so maybe they are scared to check :D [11:26] zyga: is you screen flooded with wall of text in a matrix-style all the time? ;) [11:43] is there still no way to remove a snap config option once set? [11:43] snap never forgets? [11:47] ahasenack: correct, best approximation is setting to empty string or emptying a larger context [11:47] what does "emptying a larger context" mean? [11:48] if you have foo.bar and foo.baz setting foo to {}, not saying it's useful in general [11:49] ah, by larger you meant something like parent [11:49] yes [11:49] ok, thx [11:49] we do plan to allow some resetting, still haven't happened yet [12:05] PR snapd#6052 closed: snap, store, overlod/snapstate: always send epochs [12:19] pstolowski: hollywood has a purpose ;-) [12:19] * zyga is back home [12:43] PR snapd#6114 opened: Handler for hotplug-disconnect task [12:54] zyga: got a weird snap-exec error about not finding a profile that went away with removing and reinstalling a snap, is this something you'd be interested in? [12:54] mvo, hey [12:54] mvo, did you see the log I sent you yesterday? [12:54] does it help? [12:55] Chipaca: indeed [12:55] Chipaca: can you tell me more [12:55] zyga: pstolowski: did we end up reusing/abusing addImplicitSlots for hot plug slots as well? [12:55] zyga: i'm waiting for a copy of the problem [12:56] pedronis: yes [12:56] pedronis: I believe so [12:56] :/ [12:56] pedronis: addImplicitSlots now looks also into the state ("hotplug-slots" map) [12:56] what do you think should happen there? [12:57] we should really find a different way to factor that [12:57] it's not the hot plug part I'm not keen on [12:57] it's as usual the addImplicitSlot sprinkled around part [12:57] mixing the two doesn't make things better tough [12:57] I had one idea with keeping well-know snap.Info for core/snapd and only ever handling implicit slots in one single spot at one time [12:58] this is what I did in one of the exploratory branches long ago when core18 was being prepared [12:59] the advantage was that we traded complexity all around snap info [12:59] zyga: http://paste.ubuntu.com/p/KzM4NGdRQs/ [12:59] for special casing one snap [13:00] PR snapcraft#2396 opened: Partial Revert "build providers: fix osx non base and injection" [13:00] pstolowski: zyga: anyway things are as they are, we really need to clean this up at some point [13:00] tough [13:01] I agree, we just need to write down what we are willing to accept [13:01] Chipaca: 1411 was setup-profiles [13:02] "ready-time": "2018-11-08T12:41:47.570494396Z" [13:02] this is when it was ready [13:02] pedronis: yes i agree, i just didn't want to get into rabbit of refactoring before hotplug, similiar to what happened with various refactorings before interface hooks [13:02] *rabbit hole [13:02] pre-refresh hook was 1406 [13:02] it was ready on "ready-time": "2018-11-08T12:41:47.570530073Z" [13:02] so it was after [13:03] do you have kernel logs from the system? [13:03] can you extract the range a second before 12:41:47 [13:03] zyga: person's now in a meeting, but i'll ask [13:03] woa, [13:03] actually [13:03] zyga: probably yes [13:03] its not ! [13:03] I misread [13:03] not ! <- not not [13:04] the sub-second parts show that setup-profiles was after the hook? [13:04] * zyga double checks that [13:04] zyga: do vs undo? [13:04] eh, perhaps [13:04] this sucks, I wish we had better data there [13:04] yes [13:05] kenrel logs will have useful facts but probably not that resolution (though I may be wrong) [13:05] zyga: easiest would be to ask them to 'dmesg -H | pastebinit', would that be enough? [13:05] yes [13:05] asked [13:06] but, they're in a meeting [13:06] so i'll let you know :-) [13:06] zyga: http://paste.ubuntu.com/p/MfKnvp8bD7/ [13:08] [ +0.000004] audit: type=1400 audit(1541680843.472:176): apparmor="DENIED" operation="change_onexec" info="label not found" error=-2 profile="/snap/core/5742/usr/lib/snapd/snap-confine" name="snap.code-oss.hook.pre-refresh" pid=15781 comm="snap-confine" [13:08] [ +0.005188] audit: type=1400 audit(1541681530.284:182): apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.code-oss.hook.pre-refresh" pid=17895 comm="apparmor_parser" [13:09] so we certainly tried to run the hook before we loaded profiles [13:09] pstolowski: ^ can you tell me from the top of your head, the pre-refresh hook, when is it ran in the sequence of snap refresh? [13:09] pstolowski: and if I have a snap with no revisions and install it, would that hook run? [13:10] Chipaca: can someone report a bug about this [13:10] for tracking [13:10] I'll assign it to myself [13:10] zyga: from top of my head not, but let me take a quick look. first off, it only runs if refreshing [13:10] Chipaca: if you can you please at least collect the basic info about the snap (confinement, hooks) and if this can be reproduced (snap remove foo; ... snap install foo # KABOOM) [13:11] pstolowski: I wonder if we can ever do this: [13:11] pstolowski: run pre-refresh hook on fresh install [13:11] pstolowski: when there's no prior revision [13:11] pstolowski: for context, I'm trying to read the already landed hot plug code and also looking at recent hot plug PR that have landed [13:11] pstolowski: and it somehow ends up in a sequence where we haven't installed the snap yet so it doesn't have security setup done [13:12] zyga: it runs after mount-snap, before stop-snap-services, remove-aliases, unlink-current-snap and before switching to the new revision [13:13] pedronis: ok. the biggest chunk that landed was udevmonitor stuff, after that only very small bits to set the stage [13:13] zyga: how can it be not installed if it's a refresh [13:13] pstolowski, pedronis: about implicit slots, I have a feeling it would be good for snapd to track the implicit system slots and hot plug slots explicitly in ifacestate [13:13] as a first step towards refactoring [13:14] isn't that what the current code does? [13:14] pedronis: I don't know yet [13:14] pedronis: but the hook profile was not replaced [13:14] it was _loaded_ [13:14] so it seems like it was loaded for the first time at that stage [13:14] and that can only happen if the snap was never installed (even before because of a bug in profile removal code) [13:15] zyga: btw relatedly remember that you have an open task about merging setup/remove-profiles into *link* tasks [13:15] pedronis: yes, that's true [13:15] pedronis: though awe need to write down TODOs and schedule them [13:15] *we [13:16] zyga: yes, we do need to do that [13:16] this is an long standing open one though, not saying it's a priority now [13:16] pedronis: I have a hunch that we ran the hook from this snap, the one being installed [13:16] just reminding it because it's related to this area [13:16] * zyga nods [13:19] this would be pretty unexpected given that we run it before current link changes and before setup profiles [13:21] right [13:21] but that's how it looks like [13:21] before setup profiles for sure [13:32] pstolowski: I'm seeing some small typoes etc going over the code, if I don't see anything major I will do a small PR with tweaks if it's ok with you [13:35] any idea why 'snap run --strace snap.app args' would just print "exit status 1" [13:35] ? [13:37] pedronis: sure, np, thanks [13:39] Chipaca: hmmm, strace it ;) [13:39] jokes aside, probably no strace? [13:39] pstolowski: I'm confused by ensureUniqureName, it seems to remove "-" when clearing the suffix, but is not adding "-" when attaching the suffix [13:40] PR snapd#6115 opened: cmd: update autogen.sh for opensuse [13:42] zyga: they have strace [13:43] pstolowski: it seems to be by design, it's tested behavior, but is not documented/explained in the comment [13:44] pedronis: https://github.com/snapcore/snapd/pull/6116 is the simplified feature flag storage for snap-confine [13:44] PR snapd#6116 opened: cmd/libsnap: add simplified feature flag checker [13:44] PR #6116: cmd/libsnap: add simplified feature flag checker [13:44] zyga: https://dpaste.de/FNWL/raw [13:44] Chipaca: lookking [13:44] zyga: not helping :-) [13:44] Chipaca: denials [13:44] ? [13:45] Chipaca: if you want I can jump into standup HO and you can walk me through this in the next 15 minutes [13:45] zyga: what's the context for that [13:45] pedronis: indeed, just found the PR https://github.com/snapcore/snapd/pull/5782 ; it was requested in the review - see https://github.com/snapcore/snapd/pull/5782#discussion_r217018767 [13:45] pedronis: we need to learn about enabled experimental features from snap-confine and snap-update-ns [13:45] PR #5782: ifacestate: helpers for generating slot names for hotplug [13:45] pedronis: this is the idea that Gustavo presented (just stat a file) in response to the facts PR [13:46] pstolowski: is not super clear from the discussion tough that it needs to be removed, if it's then not added [13:46] zyga: http://paste.ubuntu.com/p/XVVzYWNHR7/ [13:46] pstolowski: the test for sure look weird [13:47] jdstrand, did you see https://bugs.launchpad.net/bugs/1802124 yet ? [13:47] Bug #1802124: Error installing chromium-mir-kiosk on RPi3/edge [13:48] Chipaca: this is a snap in a container? [13:48] Chipaca: I suspect that may not work fully [13:49] Chipaca: ah, parallels VM [13:50] bug report: on a mac, running parallels, running ubuntu, running lxd as a snap, running another version of ubuntu, running a snap with --strace is not supported [13:50] pstolowski: also I would it work if the base name ends with digits? [13:50] ogra: uhh, there is extra code in this section [13:51] mvo, is "core" needed even if a snap depends on "core18" ? [13:51] pstolowski: we trim only one digit? [13:51] how do we know it's enough [13:51] jdstrand, yeah, sseems to be https://github.com/snapcore/snapd/commit/093b3662047123fe684a0d1dae27775593f1dc1f [13:51] Chipaca: hey, seems you fiddled with ptrace_* for PTRACE_GETFPXREGS? (see https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1802124). looks like there is an armhf case [13:51] Bug #1802124: Error installing chromium-mir-kiosk on RPi3/edge [13:51] pstolowski: this line is very weird: https://github.com/snapcore/snapd/blob/master/overlord/ifacestate/hotplug_test.go#L243 [13:52] ogra: there is a commit after that which is meant to address archs without fp [13:52] ah [13:52] and with fp [13:53] ackk: you shouldn't need core if you just depend on core18 [13:54] mvo, that's weird, we have a (strict) snap that doesn't work if you don't have core [13:54] ackk: oh, please tell me more [13:54] ackk: how can I reproduce this? [13:54] pstolowski: to be clear wasn't my intention to rehash already landed code, but this bit looks weird coming fresh to it [13:54] Chipaca: it looks like the policy isn't conditionally adding/removing the fp variants [13:55] mvo, https://pastebin.canonical.com/p/kTv7H5Q65Q/ [13:55] mvo, i have core18 installed only [13:55] if I install core, it works [13:55] pstolowski: also that function is not used yet, is that right? [13:55] pedronis: no, we take entire numeric suffix with strconv.Atoi(proposedName[len(prefix):]); then increment it until we have unique name as a whole. the point is to generate unique name and the example slot3-5 -> slot36 may look weird but is guaranteed to be unique [13:55] mvo, this is a proprietary snap unfortunately, lemme see if I can get you something that you can try [13:55] ackk: aha, interessting, I think you hit a bug, let me look at this [13:56] ackk: its ok, I can probably trigger it somehow, I suspect its a bug with snap-confine that it does not use the right base when it runs the hooks [13:56] pedronis: it is used in the huge hotplug PR, but not in master yet, no [13:56] mvo, ah I see [13:57] ackk: interesting [13:57] we need to be told of the base explicitly [13:57] another thing we could fix with facts alone :/ [13:57] * zyga looks [13:58] we pass --base every time there's a base defined in the snap [13:58] this is the code path used both by hooks and by apps [13:58] pstolowski: why do we remove the "-" ? [13:58] pstolowski: it just seems a mix of two version of the code, I don't understand how it relates to uniqueness [13:59] zyga: hm, apparently something is not quite wired up in the code path :/ [13:59] zyga: I do remember work on this [13:59] zyga: anyway, will look after the meeting [13:59] k [13:59] mvo zyga thanks [13:59] pstolowski: it also means slot3-5 and slot35 will both produce slot36, I don't know if thats a feature or a bug [13:59] mvo: what is odd is that for hooks we parse the revision [13:59] and use that [13:59] from snap.Revision [14:00] pedronis: that's not going to be a problem [14:00] for apps we just use R(0) [14:00] jdstrand: i thought zyga was looking into fixing the PTRACE thing, but yes [14:00] pedronis: at any given point the function is guaranteed to generate a unique name [14:00] pstolowski: again, can base name end in numbers? [14:00] jdstrand: I fiddled with them because they failed to build [14:00] Chipaca: I don't know-- ogra pinged me about it [14:00] jdstrand: so now we have a runtime failure [14:00] because that's life [14:00] pstolowski: to create, what, slot names? [14:01] pstolowski: those also need not to too confusing [14:01] pedronis: do you mean "slot1234" ? [14:01] mvo: I'm sorry, hooks use --revision from snap run [14:01] not from snap info [14:02] pedronis: sorry, im confused by what do you mean by base? [14:03] Hi snappy people, I am trying core18 because of the layouts and I get this error: "cannot update snap namespace: cannot create writable mimic over "/snap/microk8s/x1/": permission denie" [14:03] pstolowski: you start from a name that might not be unique, I suppose the output of suggestedSlotName ? [14:03] and then you make it unique [14:04] pedronis: yes that's right. btw, standup [14:04] zyga: so, we perhaps probably shouldn't be failing with error here. we ignore unknown syscalls. perhaps we should ignore these unknown PTRACE args. *or* we mock them up like with AF_IB, AF_MLPS, etc [14:04] zyga: honestly, I was surprised we didn't do the latter, but I wasn't involved in that build pr [14:05] (standup) [14:05] that's the extent of my thoughts [14:06] actually that's not true [14:07] zyga: readNumber could return something to signify skipping and continue the loop (silently ignore the rule) [14:08] not sure if this is what we should do, but I'll reply after the call [14:08] jdstrand: I am following the your steps on for a strict microk8s. Got snapcraft 3.0 that has layouts, updated the snapctaft.yaml with base: core18, applied your patch. It seems to compile but failes in the install hook with https://pastebin.ubuntu.com/p/mcXnz52NkQ/ [14:09] kinda 6 of 1. AF_IB we use a #define. we could #define here, but is that #define the same across archs? if not, skip seems reasonable. I don't think I care for browser_support.go having to understand arch-specific details about fp/fpx [14:10] kjackal: sounds like there is an apparmor denial in the snap-update-ns profile. can you check you log? [14:11] zyga: ^ (6 of 1 comment; but also fyi kjackal's) [14:14] jdstrand: hy there! Hey a question - for the purpose of e.g. snap track approval, who other than Gustavo and Mark qualifies as an architect? the post detailing this just says "etc" [14:21] jdstrand: zyga: indeed there is this "audit[22699]: AVC apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13 profile="snap-upda [14:21] te-ns.microk8s" name="/tmp/.snap/snap/microk8s/x1/" pid=22699 comm="3" srcname="/snap/microk8s/x1/" flags="rw, rbind" [14:21] N" that does not tell me much [14:21] roadmr: no one. perhaps pedronis now? if we add pedronis, we should update the docs [14:22] \o/ [14:22] we should add him, he rocks :) [14:22] This looks similar https://forum.snapcraft.io/t/core18-issues/5216 [14:24] kjackal: that is a different issue. I didn't test with core18. this might be a change relevant to that or new PRs in this area. actually, zyga did land some stuff to make it no longer experimental... need his input [14:24] I'm out of touch with this issue now [14:25] still in a call [14:26] is there a report about it anywhere or should I scan the backlog on irc? [14:27] zyga: there are two unrelated issues now. the fp/fpx that we discussed and a sun denial with microk8s using layouts with core18 [14:27] that kjackal brought up [14:28] jdstrand: aha, can we get a paste of the mount profile and of the apparmor profile for sun? [14:28] zyga: afaik, it is all backscroll [14:28] kjackal: ^ [14:28] kjackal: ^ if you know how to collect the two please do [14:28] kjackal: otherwise ask [14:28] kjackal: essentially /var/lib/snapd/apparmor/snap-update-ns.$foo [14:29] and /var/lib/snapd/mount/snap.$foo.fstab [14:29] jdstrand: I wrote a branch where I keep a log of all the ops done by snap-update-ns [14:29] I need to clean it up and propose [14:30] it is very useful for debugging [14:30] kjackal: you can also set SNAPD_DEBUG=1 and run the app to see more debug [14:31] zyga: sounds handy === blackboxsw_away is now known as blackboxsw [14:36] mvo: so the issue [14:36] wait up zyga. My guess is you want something from /var/lib/snapd/apparmor/profiles/ . This is where the snap-update-ns-* is . However I do not see snap-update-ns.microk8s. What is $foo? [14:36] PR snapcraft#2397 opened: cmake plugin: use native primitives [14:36] PR snapcraft#2398 opened: gnome extension [14:36] cachio: please just remind me how that was invoked, [14:36] same for /var/lib/snapd/mount/snap.$foo.fstab [14:36] kjackal: $foo is snap name [14:36] kjackal: did this snap have a hook that exploded on install? [14:37] if so the installation would be reverted [14:37] kjackal: and the corresponding files removed [14:37] kjackal: if so my suggestion would be to remove all hooks to debug this (^ no way to install a snap without hooks or keep it broken) [14:37] mvo: so the issue [14:37] mvo: snap-confine runs snap-device-helper [14:37] mvo: with hardcoded locations [14:37] ok, trying this now [14:37] mvo: that match core and ubuntu [14:39] mvo: this is done after pivot root, so in the mount namespace of the snap already [14:39] zyga: right, don't we bind mount things from the host (/usr/lib/snapd) into the base ? [14:40] mvo: yes that's right [14:40] * ogra hugs kyrofa [14:42] * kyrofa hugs ogra back [14:42] Seriously [14:43] yeah :) [14:43] DEBUG: performing operation: (disabled) use debug build to see details [14:43] * zyga _hates_ that part of snap-confine [14:43] security in the way of usability [14:44] mvo: we don't bind mount it in this case, the base is "core" [14:44] mvo: so the fact that we cannot find it is surprising [14:44] zyga: yeah, that is surprising [14:44] DEBUG: snap-update-ns executable: /usr/libexec/snapd/snap-update-ns [14:44] perhaps we infer the wrong location? [14:45] zyga: yeah, that seems to be it - this runs on a libexec idstro? [14:45] useful debug logs from the fedora29 issue https://www.irccloud.com/pastebin/TTN36845/ [14:45] please have a look, I'm still reading as well [14:45] yes, fedora 29 [14:45] zyga, test-snapd-udev-input-subsystem.slot [14:46] zyga: wow that fedora29 snap is like stitch, destroys everything it touches? [14:46] :D [14:46] roadmr: destroys all bugs [14:46] zyga: true that :) [14:47] https://www.youtube.com/watch?v=JTTwlAT_AwU [14:50] ha [14:50] mvo: I understand now [14:51] ar, wait, no :/ [14:51] hmmm [14:51] so [14:51] google:fedora-29-64 /# /usr/lib/snapd/snap-device-helper [14:51] -bash: /usr/lib/snapd/snap-device-helper: /usr/bin/sh: bad interpreter: No such file or directory [14:52] initially I thought this is PATH [14:52] but [14:52] the file has a wrong shbang [14:52] it says /usr/bin/sh! [14:52] not /bin/sh [14:53] google:fedora-29-64 /root# nsenter -m/run/snapd/ns/test-snapd-udev-input-subsystem.mnt [14:53] -bash: ls: command not found [14:53] -bash: /usr/libexec/grepconf.sh: No such file or directory [14:53] -bash: /usr/libexec/grepconf.sh: No such file or directory [14:53] -bash: sed: command not found [14:53] google:fedora-29-64 /# /bin/ls -l /usr/bin/sh [14:53] /bin/ls: cannot access '/usr/bin/sh': No such file or directory [14:53] google:fedora-29-64 /# /bin/ls -l /bin/sh [14:53] lrwxrwxrwx. 1 root root 4 Nov 8 14:18 /bin/sh -> dash [14:53] mvo: let me propose a quick PR [14:53] fancy [14:53] zyga: gar [14:53] oaoaaah? [14:53] what [14:53] the file has /bin/sh in the tree [14:53] aioueee [14:54] zyga: yeah, I just did a git grep [14:54] zyga: this is strange [14:54] (as long as we're resorting to guttural sounds) [14:54] mvo: [14:54] mvo: sorry, the file is from the host distro [14:54] fedora rewrites the shebang (apparently!) [14:54] but why would it be bind mounted there? [14:55] so we run with fedora29 version of snap-device-helper [14:55] so we must have did the bind mount for /usr/libexec/snapd /usr/lib/snapd [14:55] * zyga confirms [14:56] mountinfo in the app mount ns https://www.irccloud.com/pastebin/qSQmhrPV/ [14:58] PR snapd#6117 opened: overlord/ifacestate: don't remove the dash when generating unique slot name [14:58] pedronis: ^ [14:58] mvo: there is no such mount [14:58] thx [14:58] * zyga wonders [15:00] zyga, mvo I need to go to take my son [15:00] zyga: hm, what snaps is that? [15:00] cachio: ok [15:00] mvo: so it's not mounted [15:00] the core snap is busted [15:00] https://www.irccloud.com/pastebin/3uE4g7eh/ [15:00] if you need a new machine just add fedora-29-64 to the spread.yaml [15:00] theory: in CI when we repackage core and inject snapd [15:01] the locally built snapd.rpm has replaced shebang [15:01] * zyga looks [15:01] * cachio afk [15:01] mvo: this would only break in CI [15:01] mvo: not in real world [15:01] real world doesn't inject snapd from RPM into core snap [15:01] zyga: aha, because we rewrite the core in ci [15:01] actually, I wonder what that was supposed to do [15:01] zyga: indeed [15:02] I sure hope we don't have both /usr/lib/snapd and /usr/libexec/snapd [15:02] * zyga checks that first [15:03] mvo: confirmed, prepare.sh has [15:03] cp -a "$LIBEXECDIR"/snapd/* squashfs-root/usr/lib/snapd/ [15:03] so we take the host's rpm that was mangled [15:03] and inject that into the built package [15:04] this is broken [15:04] not only in the sense that this path is rewritten [15:04] snap-confine compiled on fedora is incompatible with core [15:04] I think we need to take a step back [15:04] without making snap-confine and other C programs compatible with every distribution [15:05] we cannot just insert them into the core snap during testing [15:05] they will expect different paths [15:05] it just happened to work because a path was hard-coded there [15:05] not using @libexecdir@ [15:05] but other parts will not be so useful [15:05] the binary is linked with fedora locations [15:05] it makes no sense to just inject it into a core snap with ubuntu chroot [15:05] mvo: let's have a call about this if you don't mind [15:06] I think our fedora testing story was super lucky so far :) [15:07] snap-confine ldd inside and out https://www.irccloud.com/pastebin/OkT431Pi/ [15:07] mvo: and even if we make it 100% unaware of the distribution configuration [15:07] things like libc version will be at play [15:07] as we move to fedora30 and beyond [15:08] snap-confine linked in fedora30 might not run on ubuntu 16.04 [15:08] I think the extent of testing fedora29 by repackaging core is a bit broken [15:08] we need to repackage core as if it was built on ubuntu [15:08] and test that [15:08] that's the realistic test IMO [15:08] mvo: I'll make coffee and let's have a call, ok? [15:09] zyga: let me finish some things here first, then yes [15:11] zyga: re hook issue & pre-refresh, can I help? what do we know so far (logs, a reproducer?) [15:14] pstolowski: I don't have a reproducer, didn't try [15:15] what we know is: [15:15] http://paste.ubuntu.com/p/MfKnvp8bD7/ [15:15] at the bottom [15:15] lines 865- [15:16] 865 says we didn't have the apparmor profile but we tried to execute a pre-refresh hook [15:16] then line 871 tells us we loaded (not replaced, loaded) the apparmor profile for the pre-refresh hook [15:16] that's all I know [15:17] zyga: Just got the logs you asked https://pastebin.ubuntu.com/p/cRW8jX2XGM/ . Would you prefer a forum topic? [15:17] kjackal: something tracable, yes, may be a bug report or a forum post [15:17] kjackal: too many topics now :) [15:18] kjackal: can you also get the /var/lib/snapd/mount/snap.$foo.fstab please [15:19] zyga: it is on the pastebin line 7, right? [15:19] ah [15:19] perfect [15:19] kjackal: let's have a bug report [15:19] bugs can be assigned [15:19] forum topics are swaaaamped [15:20] kjackal: I'm sorry, the issue with fedora is fundamental to our testing and development story [15:20] where do you keep bug reports? [15:20] we need to tackle it [15:20] kjackal: launchpad.net/snapd please [15:20] yes no worries [15:20] thanks [15:20] mvo: ready when you are [15:21] mvo: https://meet.google.com/mcs-qqzz-idp?authuser=0 [15:23] zyga: thanks, i'll see if i can find anything [15:29] mvo: I wrote a quick summary [15:29] zyga: can we have state.json for the hook issue? [15:29] https://www.irccloud.com/pastebin/QhF9iWhq/ [15:29] pstolowski: yes [15:30] https://www.irccloud.com/pastebin/nYonNbqH/ [15:31] pstolowski: we analysed that and determined that task timestamps are not super useful because of undo [15:31] we should store undo timestamps separately [15:31] pedronis: ^ for debugging we should probably store undo timestamps in the state [15:37] zyga: this state is missing waittasks/halttask and is generally stipped down; do we have a complete state? [15:39] PR snapcraft#2399 opened: Cmake find root build snaps [15:40] ackk: hm, I may need access to your snap afterall, I wrote a testcase with a configure hook and core18 only and it works for me [15:41] mvo, ok, hold on [15:45] ackk: also please pastebin "snap version" [15:50] mvo: silly question that i could probably answer with grep: where's the snapd sanity check? [15:50] Chipaca: sanity/check.go iirc [15:51] mvo: thanks [15:59] zyga or jdstrand, can you help with dbus ownership? on ubuntu transmission-gtk doesn't need a slot but on kubuntu it seems to require a defined slot. also transmission-qt and transmission-gtk both use the same dbus name, so we're wondering if the store allows two snaps to have the same dbus slot definition? [15:59] A tool snapcraft depends on could not be found: 'execstack'. [16:00] is that something I miss from the docker image I build? [16:02] I see that package is in debian, but not in ubuntu [16:04] hm, it's actually in universe, but not shown on packages.ubuntu.com [16:04] and it's actually installed [16:05] packages.ubuntu.com is broken atm [16:05] https://bugs.launchpad.net/pkg-website/+bug/1801518 [16:06] Bug #1801518: Search is broken [16:06] oic [16:07] zyga: Here is the bug on microk8s https://bugs.launchpad.net/snapd/+bug/1802332 [16:07] Bug #1802332: Apparmor complains in strict confinement with base: core18 [16:07] here's the snapcraft output on execstack: https://code.videolan.org/snippets/871 [16:07] kjackal: thank you [16:07] I don't have /usr/sbin in $PATH, too [16:08] jdstrand: I noticed that "kernel-module-observe" is not available but your patch had it there. Should I know anything about it? Merged with something else? Renamed? Not needed? [16:09] I am not in the position yet to see what breaks and what works, just asking because I had to comment it out [16:17] PR snapd#6118 opened: tests: add core18 only hooks test [16:18] pstolowski: where does the stdout/stderr of hooks go? [16:21] mvo: actually [16:22] mvo: on fedora29 running core18 test the same bug will happen; we need to switch the bind mount to use core's or snap's version of /usr/lib/snapd [16:22] the bind mount used for bases [16:22] Chipaca, the journal (at lest i find the output of mine there regulary) [16:22] I realised while cleaning up the notes [16:22] zyga: indeed [16:22] ogra: clue for where abouts? [16:22] zyga: we need the snapd snap [16:22] sparkiegeek, journalctl -f ... then install/rmove7configure the snap in another terminal [16:23] *install/remove/configure [16:23] zyga: we need a bug or forum post on that or it will get lost [16:23] I mean storing undo timestamps [16:23] I'm doing that now [16:23] ah [16:23] ogra: doesn't seem to help :( [16:23] I'm not doing that now [16:24] I will file a bug yes [16:24] I'm writing a post about the issue Sergio found and we just debugged with mvo [16:24] * pedronis was in a longish meeting [16:25] zyga: anyway if undo is involved, then might well be the issue that would be solved by merging *-profiles with *link* [16:25] pedronis: I don't think the issue is related to undo [16:25] ah ok [16:25] it's just that we have fewer bits of data about it to debug clearly from timestamps alone [16:25] we know for sure from kernel logs that we ran a hook before we setup security [16:26] pstolowski: ^ actually this makes sense :D [16:26] yes, but not having security when we should and undo [16:26] that hook should run before we setup security [16:26] because security setup will switch the world for the NEXT revision of the snap [16:26] are the a possible of form of the bug [16:26] Chipaca: to task's log i believe, however there is a catch afair, it only goes there if hook fails [16:26] pedronis: yes, in this particular case we didn't have security because that snap was not installed yet, the bug is that we ran the hook at all, not that we didn't have security [16:26] sparkiegeek, weird, i always see mine ... though i'm typically watching on UC ... not sure if thats any different (i wouldnt expect so though) [16:27] ah, interesting [16:27] ok [16:27] (we had nil security because that's the first install, we ran the pre-refresh hook of the installed snap in this nil security before we proceeded to setup security for the revision being installed) [16:27] I'll add this to the reporty [16:29] zyga: this is super weird, it could only happen if we had a snap in snapstate but it wasn't actually installed (?), in which case IsInstalled() would lie [16:30] but we do [16:30] we are installing it [16:30] do we have a test for this situation [16:30] snap install snap-with-pre-refresh-hook [16:30] perhaps it ends up in snap state too early [16:31] zyga: that would mean we have other serious issues [16:31] zyga: we have a bunch of checks around isInstalled [16:31] pstolowski: it would be good to try this. case [16:31] make a simple snap with pre-refresh hook [16:31] and install it [16:31] zyga: ok, i'll do this [16:33] PR snapd#6119 opened: sanity: refuse to try to do things on WSL [16:33] mvo: ^ my sanity question was about this [16:33] zyga: wait, we do have spread test already [16:33] mvo: https://forum.snapcraft.io/t/issue-with-repackaged-core-and-testing/8394 [16:34] mvo: please sign off that you agree with what it says [16:34] you can edit it I believe [16:34] pstolowski: then I don't know [16:34] pstolowski: chase this with chipaca tomorrow please [16:34] we need to reproduce it really [16:34] zyga: install-refresh-remove-hooks [16:35] pstolowski: maybe interplay with other hooks is important [16:35] try making a quick test with just one hook [16:35] ok [16:35] and base (not sure if relevant) [16:37] zyga: just one hook - pre-refresh - worked. does the hook need to do anything to trigger profile issue? [16:37] no [16:37] it would not be able to execute at all [16:37] right, it's snap-confine erroring early [16:44] zyga: just pre-refresh hook, without base, and with base:core18, both worked [16:44] zyga: chase what? (scrollback from where?) [16:44] zyga: do you have full state.json? [16:44] pstolowski: I don't have anything else, it's all from backlog in this channel [16:44] Chipaca: that bug with pre-refresh-hook [16:45] pedronis, Chipaca: related to this bug I filed https://bugs.launchpad.net/snapd/+bug/1802339 [16:45] Bug #1802339: Tasks should carry separate timestamps for undo paths [16:45] zyga: who reported it originally? [16:45] pstolowski: someone from the sprint Chipaca is at [16:45] Chipaca: wanna see something terrible [16:45] aha [16:46] https://forum.snapcraft.io/t/issue-with-repackaged-core-and-testing/8394 [16:46] kaboom [16:46] we already discussed that ^ in the past [16:46] we also said it was going to bite us at some point [16:47] Chipaca: \o/ [16:48] * zyga needs to break for a while to vent [16:49] zyga: also that post doesn't list snap-exec, doesn't it always come from the base? [16:50] or is that the bind mount bit? [16:51] Chipaca: can you find who that was reporting pre-refresh issue, and get more info? state.json for starters [16:52] pstolowski: which pre-refresh issue? [16:52] gah [16:52] * Chipaca reads backlog again [16:54] ah [16:54] Chipaca: somebody hit an issue where pre-refresh hook was run without security profile (=failed with denials), as if snap wasn't installed. in theory (and according to the code) this hook should run of old revision of the snap, before switching to new revision on refresh [16:55] s/run of/run for/ [16:55] was this related to the dmesg -H pastebin? [16:55] Chipaca: the lof is http://paste.ubuntu.com/p/MfKnvp8bD7/ [16:55] *log [16:56] ah yes [16:56] Chipaca: pastebin from "joao" [16:56] yes [16:57] pstolowski: i'll get the state.json === dkessel_ is now known as dkessel [17:01] pstolowski: zyga: for extra fun, they got the same thing again in the post-refresh hook, afterwards [17:01] pstolowski: zyga: i've send you guys the state json [17:02] niemeyer: github is rate-limiting us which is why sometimes travis statuses don't make it back [17:02] niemeyer: e.g. https://github.com/snapcore/snapd/pull/6107 [17:02] PR #6107: cmd/snap, snapctl: add column selectors to services [17:02] niemeyer: i have logs to prove it [17:02] Chipaca: very curious. can you look around if their installations is sane? we use IsInstalled() check for this hooks, i cannot think of a case where this would lie, except for some inconsistency? [17:02] niemeyer: can youchat with them and ask them to bump our limits a bit? [17:03] niemeyer: i'll fwd the email [17:03] with the logs [17:03] thanks for the state, lookiing [17:04] if it's that, yes IsInstalled returning true for non installed snap would be very weird [17:04] Chipaca: Huh.. curious [17:05] is it possible to delete a configuration key for a snap? i.e. 'snap set mysnap foo=bar; snap set mysnap foo=; snap get mysnap | grep foo' returning no hits [17:05] Chipaca: Thanks, will follow up [17:06] sparkiegeek: not at the moment [17:07] Chipaca, pedronis, zyga from quick glance at the state, they seem to be using try mode quite a bit, which i hinted could be a factor, i think this is the first suspect [17:07] try mode + hooks = bad news? [17:08] that's a big bummer [17:08] what's the situations in which it'd fail? [17:08] pstolowski: ^ [17:08] Chipaca: they should work obviously; is IsInstalled check correct for try mode? [17:09] Chipaca: try on top of a try, is really a refresh [17:09] pedronis: yes [17:09] but if the two use the same dir [17:09] ah [17:09] not sure what it means [17:09] ok, i'll see if this helps [17:09] Chipaca: one thing to try, snap try something without a prerefresh hook [17:09] now you try the same dir but you added a prerefresh hook [17:09] I suppose: boom [17:10] I feel their pain, but is not quite trivial to solve either [17:11] we would need to remember things that atm we don't care to [17:11] where's our CoW support a t [17:11] at* [17:11] the dev isn't blocked on this, fwiw, so there's not that much urgency [17:12] but it does seem to match what we were seeing [17:12] Chipaca: or yes, take a snapshot somehow , unclear when or how [17:13] pedronis: they were doing try+try to test the pre- and post-refresh hooks :-) [17:13] doesn't quite work [17:13] they need a 2nd checkout [17:13] atm [17:13] I fear [17:13] I suppose it can of works [17:13] if there are no interfaces involved [17:14] anyway is not a well supported/understood case by our code [17:14] try is really a best effort big hammer [17:14] I suppose we should at least document this kind of sharp edges somewhere [17:15] PR snapcraft#2400 opened: tests: fix maven spread test [17:16] Chipaca: can we get a bug or forum post on this now that is better understood? [17:16] otherwise it will wash away [17:17] pedronis: i'll give it a try [17:17] * Chipaca would giggle but is too tired [17:17] it's interesting, i've just tested this scenario (snap try without pre-refresh, then snap-try with pre-refresh, same dir), it fails but differently [17:18] Run pre-refresh hook of "snap-hooks" snap if present (run hook "pre-refresh": cannot stat /var/lib/snapd/seccomp/bpf/snap.snap-hooks.hook.pre-refresh.bin: No such file or directory [17:18] and after a few minutes, after timeout [17:18] no denials [17:18] PR snapcraft#2401 opened: ci: reduce spread system declarations [17:19] hmm [17:19] pstolowski: does it have an interface that hook? [17:19] I mean needs to need an interface to get a denial [17:19] ondra: your avahi-client snap - should I be able to see avahi advertised services in the office? [17:19] pedronis: no hook, only home interface [17:20] anyway it's similar but different [17:20] maybe they did enough to have a bf [17:20] bpf [17:20] in their case [17:20] but not the right apparmor profile [17:20] yet [17:21] PR snapcraft#2402 opened: packaging: update requests [17:22] right. anyway, good that we know it's snap try [17:22] * pedronis eoding [17:22] eod as well [17:22] cu === pstolowski is now known as pstolowski|afk [17:24] popey yep [17:24] popey that is exactly it's purpose :) [17:24] pstolowski|afk: pedronis: zyga: https://forum.snapcraft.io/t/snap-try-snap-try-hooks-fun/8400 [17:24] ondra: how do you operate it? [17:25] popey but easier to use snappy-discover [17:25] ooh [17:25] Chipaca: thanks! s/-remove/-refresh/ [17:25] popey then no need to use params [17:25] popey mind, UC device needs to run avahi in first place :) [17:26] d'oh [17:26] not on core [17:26] * Chipaca should've EOWed after lunch [17:26] popey ok then avahi-client [17:26] popey what is device's name> [17:26] ? [17:27] ondra: Volumio [17:28] ah, avahi-client has a bunch of not-automatically-connected interfaces [17:28] popey yeah you need to connect manually [17:28] popey 10.20.65.140, 10.20.65.210 [17:28] what did you do? [17:29] popey on macOS lanscan [17:29] wrong answer :) [17:29] popey it does whole scan [17:29] I'm trying to prove avahi on snap [17:29] we have a broken snap and trying to prove avahi in snap works [17:29] popey OK let me get you right command [17:32] are there instructions for installing the snap system from source? (my distro doesn't ship with a package: Slackware) [17:34] hackedhead: great question. I'd love to see a doc for bootstrapping snapd from sauce [17:34] Chipaca: ^ :D [17:34] 1. get systemd [17:34] 2. ??? [17:34] 3. profit [17:35] * Chipaca wins [17:36] popey avahi-client.browse -a | grep Volumino [17:37] Chipaca, hey, any idea about what could be causing this on rpi 3: https://paste.ubuntu.com/p/B9RsstFFwj/ [17:37] cachio: yes [17:38] Chipaca, :) [17:40] popey, avahi in snaps does not work ... i started a thread about it a month or two ago [17:40] cachio: mozilla broke snapd on non-intel architectures, and we let them [17:40] Chipaca, nice [17:40] cachio: i thought zyga was fixing this but apparently no [17:41] popey, here is a hack in a wrapper script https://github.com/ogra1/dashkiosk-client-browser/blob/master/chromium.launcher ... and here is the thread https://forum.snapcraft.io/t/no-mdns-support-in-snaps-should-core-have-a-modified-nsswitch-conf/7603 [17:41] ogra what? what do you mean? [17:41] oddly my machine can't see any avahi things [17:41] another laptop can! [17:41] on the same network [17:41] ondra, snaps can not use mdns due to nsswitch.conf [17:41] ogra ah, that's when you want to use libc [17:42] Chipaca, good to know, thanks [17:42] * zyga lost track of things to fix [17:42] typo, works now [17:42] ogra avahi client still works [17:42] popey, you can use layouts to overcome that but that means you are forced to snapcraft 3.0 and core18 [17:42] Sorry about that chipaca [17:42] ondra, sure ... [17:42] cachio: ยฏ\_(ใƒ„)_/ยฏ [17:42] ogra core 18? [17:42] cachio: i think this blocks 2.36 [17:42] ondra, we have a broken snap and trying to prove avahi in snap works [17:42] ondra, yes, for layouts without passthrough [17:43] zyga: no problem [17:43] zyga: i mean [17:43] zyga: i'm very tired, i'm probably not the only one [17:43] i might just work half a day tomorrow [17:45] are you fixing the ptrace bug? [17:45] Can you file it please, even a small bug for tracking [17:46] I think we need to fix our ability to store, triage and react to issues [17:46] zyga: you mentioned 2.36.1 will be released today. I don't think it can be released if the fp/fpx ptrace issue is present since if it goes to stable, you'll blow up kiosks [17:46] zyga: perhaps it is best to revert the PRs if you don't have time. then reintroduce in 2.36.2 [17:47] Ack [17:47] zyga: go for it [17:47] I agree but I donโ€™t know if the issue is in edge/master only [17:48] Or also in the release branch [17:48] didn't 2.36 already inlcude the fp/fpx thing? [17:48] Chipaca: ack, the bug is on me now [17:48] zyga: i meant go for filing it [17:48] I donโ€™t remember Chipaca [17:48] zyga: actually, it seems it is only in master. I just checked out 2.36 and don't see the code. sorry [17:49] zyga: remember what [17:49] Iโ€™m taking the dog out now, took a break before. After I am back I will report the bug, check if it present in the release branch and coordinate with mvo [17:49] * Chipaca is confused [17:49] zyga: ah [17:49] zyga: walk away [17:49] zyga: okk [17:49] Chipaca, IRC is slow from phone [17:49] Sorry about that [17:50] zyga: i'll fix tomorrow if you haven't by the time i wake up [17:50] i plan to not wake up early [17:50] Ok [17:50] :-) [17:50] Good plan [17:50] * Chipaca hasn't felt this tired in a while [17:50] I keep doing 1-2am [17:50] :/ [17:50] have i mentioned how being in a room full of people tires me out? :-) [17:50] and that, yes :-D [17:52] Hey [17:52] At least you are not sharing a room this week :-) [17:54] PR snapcraft#2396 closed: Partial Revert "build providers: fix osx non base and injection" [17:58] jdstrand: I don't think, we meant released to stable, 2.36 didn't even make it to candidate [17:58] unless me is very confused [17:59] jdstrand: I think mvo meant pushing .1 to beta [17:59] or so I hope [18:06] re [18:06] pedronis: that's correct [18:12] zyga: ok [18:13] pedronis: yeah, the plan is to push .1 today but I will wait for zyga to get back to me about this issue that jdstrand described (fp/fpx) [18:14] mvo: on that now [18:18] mvo: release branch is not affected [18:18] go for .1 [18:18] I'll fix the master issue [18:18] Chipaca, jdstrand: FYI filed as https://bugs.launchpad.net/snapd/+bug/1802362 [18:18] Bug #1802362: browser-support broken on certain arches due to ptrace [18:18] zyga: ACK [18:19] EOD from me (mostly) [18:26] zyga: ta [18:27] * zyga pours a glass of Jim beam and gets to work [18:29] pedronis: I saw your comment on the forum. I will respond and improve the post tomorrow. Today I just want to wrap up [18:54] mvo, pedronis: yeah I mispoke, 2.36 isn't affected by the fp/fpx [18:55] zyga: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1802124 already existed for that [18:55] Bug #1802124: Error installing chromium-mir-kiosk on RPi3/edge [18:58] zyga: I also think your conclusion in your bug is wrong. just like with AF_IB or or that set_tls is an arm-specific call, the policy shouldn't be arch specific. the backend should be smarter [18:58] I think I swing towards that conclusion as well now [18:59] it's easier to handle on that end [18:59] zyga: yeah. a profiler should only have to worry about policy being safe or not. the backend can just be smart about archs [18:59] zyga: cool, thanks [18:59] thank you :) [19:00] Hello, is the kernel with B+ support released to stable? [19:00] zyga: not sure who will be assigned to that, but ahppy to review it (I can't jump on it right away, otherwise I would) [19:00] I am fixing it now [19:01] just trying to make sure I understand it and can see the fix working [19:23] zyga: how do I reset the mount namespaces again? I suspect I Know why my test is not reproducing the core failure that ack reported [19:24] mvo: just use snap-discar-dns [19:24] discard-ns [19:24] mvo: /usr/lib/snapd/snap-discard-ns snap-name [19:24] zyga: no args? [19:24] note that it does more than it used to long time ago so please use that rather than work on unmounting the mount namespace alone [19:24] just the snap name [19:25] zyga: does discard-ns also undo the device cgroup setup for a snap? [19:25] no [19:25] interesting [19:25] is there a command that does that? [19:26] nope [19:26] we never remove them [19:26] ijohnson: (feels like a bug) [19:26] I currently have had to just reboot my machine to get it to go away after uninstalling a snap with the device cgroup on it [19:26] shall I file a bug about it? [19:27] please [19:27] okay, will do in a bit, about to go out for lunch [19:27] ijohnson: AFAIK that's been true since snapd got started [19:27] ijohnson: I have some related work on refreshes pending [19:27] so I think we can fix it then [19:28] okay cool [19:28] ijohnson: you can stop rebooting [19:28] it should be easy to clean up for development [19:29] yeah I'm sure there's a way I haven't looked into it too much yet though [19:29] * ijohnson -> lunch [19:57] zyga: I'm afraid we need something like 6118 for 2.36 [19:57] looking [19:58] zyga: i.e. on classic we have the quirks code in snap-confine that always assumes core [19:58] mmmm [19:58] I think that should only run on core, not on core18 [19:58] quirks need to die [19:58] they are useless now [19:58] so I suspect we could just ditch them [19:59] lxd is a snap [19:59] zyga: fair enough, I can fix the PR [19:59] I'm sorry that it lived for so long [19:59] let me read the PR in full [20:01] mvo: the patch looks good [20:01] I don't oppose merging it [20:01] we should follow up with removal of quirks for base != "core" [20:01] zyga: thanks, lets see if CI is happy too [20:01] that is, quirks only on classic when running against core [20:01] I suspect we could sync with stgraber and remove it entirely now [20:01] quirks exist for one reason: [20:01] allow lxd from the host to be visible in the snap [20:01] specifically the socket [20:01] but I think the socket moved since [20:02] zyga: oh, nice [20:02] for super compatibility I would probably stick to deprecation [20:02] zyga: well, I can trivially add "if base_snap_name == "core" to the PR [20:02] but maybe we can skip ahead and just drop it [20:02] yeah, plese [20:02] once green [20:02] zyga: what's that? [20:02] I don't want to derail evening [20:02] stgraber: hey, [20:02] stgraber: we have a quirk in snapd that puts /var/lib/lxd from the host into the snap mount namespace [20:03] that wasn't for lxd [20:03] it was added to allow snaps to talk to lxd socket that is otherwise not visible in the core snap (/var/lib/lxd doesn't even exist there) [20:03] no? [20:03] what was it for? [20:03] the LXD snap never accessed /var/lib/lxd [20:03] (mental shortcut: it was for snaps that want to talk to lxd) [20:03] well, lxd.migrate does but it always used /var/lib/snapd/hostfs/var/lib/lxd [20:04] where does lxd keep the socket now? [20:04] yeah, probably was for some other random snaps that wanted to talk to a non-snap LXD on the host, no idea what those would be [20:04] stgraber: that's correct [20:04] it was behind an interface access so we could probably check too [20:05] (across store snaps) [20:05] where does lxd store the socket now? [20:05] these days, they should be using the LXD snap anyway, socket would be at /var/snap/lxd/common/lxd/unix.socket [20:05] ackk: 6118 has a fix, thanks for reporting the core18 only bug [20:05] lxd-client interface gets you access to that [20:05] stgraber: thank you [20:06] mvo: I just checked the interface [20:06] if someone really needs to talk to the non-snap LXD, I guess a lxd-deb-client interface or something could be introduced that gets you /var/lib/snapd/hostfs/var/lib/lxd/unix.socket [20:06] there is no interface that grants access to /var/lib/lxd anymore [20:06] I think we can remove this quirk entirely now [20:07] stgraber: I agree [20:07] jdstrand: FYI, based on the discussion above I think we can remove the quirk for /var/lib/lxd [20:07] mvo: let's go ahead with your fix as-is [20:07] and then clean up by removing the whole hack [20:07] along with extra permissions [20:07] tests and what not [20:34] zyga: ok, lets see what CI says [20:34] I took forever to boot a pi3 [20:35] to ensure that ptrace issue is fixed [20:35] but it's not for the release so no stress [20:39] mvo: offtopic, I think we may strip our snapd bits [20:39] I wonder what's the size of snapd.snap with stripped go [20:41] mvo, great, thank you for the fix [20:42] zyga: good point [20:56] zyga: strip> yeah [20:56] zyga: tests still running, I check the PR in the morning, need sleep :/ [20:56] ok [20:56] ttyl [20:57] * mvo waves [21:10] jdstrand: https://github.com/snapcore/snapd/pull/6120 [21:10] not sure if it will pass [21:10] PR #6120: cmd/snap-seccomp: add full complement of ptrace constants [21:10] it works on armv7 [21:11] (and on amd64) [21:11] PR snapd#6120 opened: cmd/snap-seccomp: add full complement of ptrace constants [21:11] thanks to chipaca's brilliant cross-built test suite we will know soon if it passes on other arhces [21:11] *arches [21:11] jdstrand: anyway, I'll EOD now, please let me know what you think about this patch [21:57] jdstrand: can you look at the update on https://github.com/snapcore/snapd/pull/6120 [21:57] PR #6120: cmd/snap-seccomp: add full complement of ptrace constants [21:57] not sure if you like it still [21:58] ideally I'd say that some things are -1 or NaN or similar [21:58] but it's hard in the current structure since -1 is a valid value at that level [21:58] I could refactor that if you want [21:58] though I don't believe it is required for this specific part (ptrace constants) [21:58] specifically the set we are dealing with here [22:00] jdstrand: not sure if you have the time but I would love a quick look at a simple open, fstatat function; https://github.com/snapcore/snapd/pull/6116 [22:00] PR #6116: cmd/libsnap: add simplified feature flag checker [22:00] the code is at https://github.com/snapcore/snapd/pull/6116/files#diff-034bb7401eb457bc8862f9b4ee97edc8R33 [22:00] the rests are tests [22:01] zyga: if you were going to go that route, I think readNumber would have to return a 3rd value, 'skip' rather than interpreting the value [22:01] yeah [22:01] zyga: but this is like what we do for AF_IB, etc [22:01] it just sucks because we have the knowledge in C as a macro [22:01] what you're doing now [22:01] I'm ok with the approach, just verifying something [22:01] it's all doable, just a bit more uphill [22:01] thank you [22:02] FYI: I ran cross compile tests locally (thanks to chipaca's fantastic cross test suite) [22:02] where locally is really locally invoked spread :) [22:11] PR snapd#6115 closed: cmd: update autogen.sh for opensuse