[00:35] PR snapcraft#1043 closed: tests: fix the CLA launchpadlib install in travis [00:38] PR snapcraft#1043 opened: tests: fix the CLA launchpadlib install in travis [01:10] anybody know the localhost login details for ubuntu snap core ? [01:10] I downloaded the ova from here http://cloud-images.ubuntu.com/snappy/devel/core/current/devel-core-amd64-cloud.ova?_ga=1.214375895.1790561723.1484162457 === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk [03:23] PR snapcraft#1043 closed: tests: fix the CLA launchpadlib install in travis [03:29] PR snapcraft#1044 opened: tests: use python2 to check the CLA === chihchun_afk is now known as chihchun === JanC is now known as Guest39941 === JanC_ is now known as JanC [06:13] Bug #1655834 opened: Support factory reset [07:11] PR snapd#2544 closed: interfaces: implement login-control interface [07:47] o/ [07:49] hey mvo [07:50] hey zyga [08:57] PR snapd#2554 closed: tests: add end-to-end store test for classic confinement [08:57] PR snapd#2605 closed: overlord,overlord/snapstate: have UpdateMany retire/enable auto-aliases even without new revision [09:11] PR snapd#2469 closed: interfaces: upower-observe: refactor to allow snaps to provide a slot [09:14] PR snapd#2616 opened: tests: test classic confinement `snap list` and `snap info` output [09:26] PR snapd#2617 opened: tests: switch more tests to MATCH [09:38] stokachu https://github.com/snapcore/snapcraft/blob/master/snaps_tests/__init__.py#L180 [09:46] sergiusens, https://github.com/juju/juju/blob/staging/snapcraft.yaml [09:50] PR snapcraft#1038 closed: misc: delete bzr ignore [10:03] icey https://bugs.launchpad.net/bugs/1655832 [10:03] Bug #1655832: App name in snapcraft.yaml must match case of .desktop file [10:05] yeah sergiusens? [10:47] PR snapd#2618 opened: tests: run all spread tests with SNAP_REEXEC={0,1} [11:30] PR snapd#2606 closed: overlord/snapstate: share code between Update and UpdateMany, so that it deals with auto-aliases correctly [11:48] sergiusens: I think I got the linker flags in [11:51] PR snapcraft#928 closed: Add missing dependencies to install_requires [12:02] jdstrand: FYI, snap-alter-ns won't be setuid root, it will just be started by snapd (like snap-discard-ns) [12:02] jdstrand: we can choose to confine it but I wanted to make sure you know this [12:03] PR snapd#2567 closed: debian: skip snap-confine unit tests on nocheck [12:04] PR snapd#2619 opened: many: update 14.04 branch to top of master [12:05] PR snapd#2616 closed: tests: test classic confinement `snap list` and `snap info` output [12:06] PR snapd#2620 opened: store: export userAgent. daemon: print store.UserAgent() on startup [12:25] how do i get into the shell of a snap again? [12:25] snap run --shell conjure-up doesn't seem to do what i need === ben_r_ is now known as ben_r === hikiko_ is now known as hikiko [12:42] stokachu: `snap run --shell the-snap` should do what you need [12:42] stokachu: how is it not? [12:42] that is, what do you need, and what are you getting? [12:43] so im building juju inside my snap but I think the problem is hte juju binary isn't getting copied into the snap [12:43] still working that out [12:43] stokachu: that gives me more questions, but doesn't answer mine :-D [12:44] yea sorry still trying to understand what im doing [12:46] Chipaca: http://dpaste.com/361CKWW basically trying to expose the juju binary alongside conjure-up [12:47] mvo: do you know if there is a clear reason why every systemd service unit snapd generates wants the network-online.target? [12:47] stokachu: you did "snap run --shell conjure-up" [12:47] Chipaca, yea [12:47] stokachu: what did that do? and what did you expect it to do? [12:47] it brought me into a new shell and i was looking for the juju binary that wasn't there [12:49] stokachu: i'm missing somemething, i fear [12:50] Chipaca, sorry, i built my conjure-up snap while pulling in the juju part from the wiki, in the end i want to be able to run both conjure-up and juju binaries from that single snap [12:51] stokachu: "snap run --shell conjure-up" is giving you a shell in the environment that apps in the conjure-up snap would see [12:51] stokachu: (modulo connections) [12:51] Chipaca, ok, now im trying to figure out why that juju one isn't showing up in the file list [12:51] Chipaca, using --classic [12:51] morphis: no clear reason [12:51] morphis: if that is a problem we can kill it [12:51] stokachu: when you say "in the file list", you mean in the snap? [12:52] Chipaca, yea [12:53] should it be conjure-up.juju as the command name? [12:53] yes [12:53] how can i make it just juju? with an alias? [12:53] stokachu: yes [12:54] stokachu: but: is juju in conjure-up something users are expected to interact with? [12:54] Chipaca, yea, juju needs to talk to the agent installed inside that snap [12:54] PR snapd#2619 closed: many: update 14.04 branch to top of master [12:55] mvo: not a urgent thing to fix but it conflicts with network-manager when I add a Alias=NetworkManager.Service in the generated unit file [12:55] as it then has a cycle of dependencies === mcphail_ is now known as mcphail [13:19] trying to get this juju binary include but hitting http://dpaste.com/2XNXGRW [13:20] the part im pulling in is https://github.com/juju/juju/blob/juju-2.0.2/snapcraft.yaml === beowulf_ is now known as beowulf === hikiko_ is now known as hikiko|ln [13:35] zyga: any idea about https://paste.ubuntu.com/23786689/? I am using a 3.4 kernel here so could be something not well supported by that ancient kernel but would like to understand this a bit more [13:54] PR snapd#2620 closed: store: export userAgent. daemon: print store.UserAgent() on startup [14:04] morphis: looking [14:04] morphis: 3.4 [14:04] morphis: mount namespaces [14:05] morphis: do you see /proc/self/ns ? [14:05] morphis: if not then 3.4 is too old [14:05] I honestly don't expect 3.4 to work [14:11] https://paste.ubuntu.com/23786828/ [14:12] PR snapd#2618 closed: tests: run some spread tests with SNAP_REEXEC={0,1} [14:12] morphis: there's no mnt entry there [14:12] morphis: the error is confusing but 3.4 is too old [14:12] morphis: and there's no way to solve it easily without a major feature in snap-confine [14:13] (I discussed this a while ago but this is 2-3 weeks to do) [14:13] morphis: what is the HW if you can disclose this? [14:13] zyga: I see [14:14] morphis: if I end up writing this I'd like to know if I have it on my desk === hikiko|ln is now known as hikiko [14:20] zyga: see PM [14:28] morphis: got it now [14:28] morphis: if we chat on rocket I get better notification [14:53] zyga, snapstore supports delta downloads now right? [14:59] PR snapd#2617 closed: tests: switch more tests to MATCH [15:00] zyga: :-) [15:03] stokachu: yes but the client needs a special variable to enable that [15:04] zyga, when will that get enabled by default? [15:04] stokachu: I don't know, that's a question to chipaca (not on IRC now) and nessita [15:04] ok thanks [15:04] zyga, no, *you're* not on irc [15:04] oh [15:04] odd [15:04] tab tab gave me nothing, sorry about that [15:04] :) [15:05] we don't have delta downloads enabled by default [15:05] because of the xdelta3 dependency [15:05] Chipaca, is that something that is in the works to be done soon? [15:06] stokachu, it's paperwork [15:06] but afaik it's moving forwards [15:06] so it'll get done in time [15:06] i mean: the code is there [15:06] Chipaca, ah ok awesome [15:06] and you can even use it if you set the right env var [15:07] cool is it documented anywhere? [15:07] dunno. SNAPD_USE_DELTAS_EXPERIMENTAL=1 in snapd's environ will do it [15:07] cool thanks [15:08] and, you need xdelta3 [15:10] jdstrand: hey [15:12] jdstrand: did you read about the new BFP attachments to cgroups? [15:19] zyga: yes. I assume you are refering to the comment in the 'ip netns' PR. I just commented there [15:19] jdstrand: actually no, I read the article on LWN [15:19] jdstrand: I'll check the PR [15:21] zyga: the pr doesn't mention bpf, but it was talking about network mediation, so I thought you were going there [15:24] hi could i get someone to take a look at https://myapps.developer.ubuntu.com/dev/click-apps/5479/rev/26/? [15:25] stokachu: I will [15:25] jdstrand, ty! [15:28] PR snapd#2621 opened: osutil.GetenvBool now takes an extra, optional, argument [15:29] stokachu: done. you need to press the release button [15:29] jdstrand, perfect ty, do i need to worry about getting additional reviews if i need to push a new snap? [15:30] stokachu: not now (I granted that). check your email [15:30] jdstrand, ok great, thank you again [15:30] np [15:36] jdstrand, sudo snap install conjure-up --edge --classic <- am i missing anything here? [15:36] says snap isn't found [15:37] oh isee [15:37] release button at the bottom [15:38] hmm not sure what im missing now === chihchun is now known as chihchun_afk [15:41] maybe there is a delay [15:44] http://paste.ubuntu.com/23787291/ snap info does show it in the edge channel as well [15:52] stokachu: some confusion but installing classic snaps from the store doesn't work in 2.20 or before (we are fixing it in 2.21) [15:53] pedronis, ah [15:53] pedronis, np i can just work with the blob itself for now [15:55] stokachu: the workaround to try it is to use "snap download snap-name" "snap ack *.assert" "snap install *.snap" [15:55] pedronis, ah perfect, that'll work [15:55] sorry for the incovenience [15:56] np i know it's very new [15:59] stokachu: there is an issue with snapd and the store when using --classic. I don't recall the details. I think sergiusens has a workaround [16:00] ah, I see you already got an answer to that :) [16:00] jdstrand, :D [16:00] i can still corner sergiusens tomorrow unless he's hiding from me [16:03] roadmr: hi! can you pull r824 of the tools? this is just updates to data/ for 2.21 interfaces and whitelisting some approved kernel and gadget snaps [16:04] jdstrand: sure thing [16:04] roadmr: thanks! I *think* this will be it for a while (famous last words) [16:05] PR snapd#2520 closed: many: fix abbreviated forms of disconnect [16:15] PR snapd#2622 opened: asserts: Improve error message when key is not valid at the given time [16:28] hi [16:28] when will rhel/centos be supported [16:30] PR snapd#2621 closed: osutil.GetenvBool now takes an extra, optional, argument [16:34] Cust0sLim3n, perhaps SamYaple or zyga know if thats actually planned ... fedora is surely supported (with limitiations in the confinement) [16:34] hello Cust0sLim3n: https://github.com/snapcore/snapd/wiki/Distributions [16:36] thanks elopio [16:39] Cust0sLim3n: hey [16:39] Cust0sLim3n: CentOS needs some packaging work [16:39] Cust0sLim3n: Fedora has two blocking issues that we've struggled with [16:39] Cust0sLim3n: both are on the backlog but I expect ~1-2 weeks before I start working on that [16:40] zyga, is there some repo with stuff for CentOS / Fedora builds ? [16:40] Cust0sLim3n: please have a look at http://www.opera.com/pl/computer/neon [16:40] er [16:40] paste fail [16:40] http://github.com/snapcore/snapd/wiki/Distributions [16:40] Cust0sLim3n: there was COPR but it is no longer used as development moved to Fedora git [16:41] zyga, cool thanks [16:41] Cust0sLim3n: CentOS should be easier (no selinux on services by default) but it needs separate packaging permissions [16:41] Cust0sLim3n: if you are interested we could work together to revive copr and build a centos package there [16:41] Cust0sLim3n: I cannot commit 100% of my time to it this or next week but I will help you out [16:42] zyga, don't really have time - really need it for RHEL though - but I think in meantime I will just use software collections on rhel or something - still have to see [16:42] zyga, just thought it would be nice to use snappy instead of SCL [16:43] Cust0sLim3n: what software are you most interested in? [16:43] Cust0sLim3n: we'll get there, it's just a busy schedule all the time [16:44] zyga, its for commercial use with our C/C++ software on linux that I want it [16:46] zyga, but its not critical enough for us to dedicate allot of time to it - so it was mainly just if I somehow missed it being available it would have been nice - but otherwise we will just use SCL or package directly [16:46] Cust0sLim3n: how do software collections help you? [16:50] jdstrand: hey - new interfaces API: https://github.com/snapcore/snapd/pull/2613/files [16:50] PR snapd#2613: interfaces: add new interface API [16:51] jdstrand: with initial review from gustavvo [16:51] jdstrand: I think it is likely to land soon [16:51] jdstrand: once it does expect a flag day where all the interfaces and backends are ported to provisional APIs like spec.AddSnippet() (that directly replaces returning a snippet) [16:52] I'll be looking at that in a little bit [16:52] jdstrand: and then dedicated changes to backends like mount, kmod, systemd that are well defined and have just a few interfaces [16:52] jdstrand: then we can discuss apparmor/seccomp [16:52] jdstrand: I think there's no rush on aa/sc but we can see what we'd like to do by porting one or two interfaces over to more semantic APIs [16:53] jdstrand: e.g. in seccomp we could do spec.Allow(seccomp.open) rather spec.AddSnippet("open") [16:53] jdstrand: in apparmor we could do (but in short cases IMHO) spec.Allow("/path/to/file", "r") or even spec.Allow("/path/to/file", apparmor.Read) [16:54] jdstrand: thanks! [16:55] zyga, makes it easier to work with multiple library and application versions and allows us to be less dependent on library versions shipping with OS - so if we want newer boost version for building with than is available on RHEL by default we can package it with SCL [16:56] zyga: keep in mind, I've not looked at this at all yet, but please, please, please keep in mind that the resulting security policy needs to be auditable. that means policy, structure and comments all need to be in place. I maintain my concerns that this will impede security audits since the review will have to do profile dumps to see what is happening [16:56] jdstrand: I keep this in mind and keep reminding everyone about it :-) [16:56] tyhicks: fyi ^ [17:00] PR snapd#2623 opened: tests: add test ensuring manual pages are shipped [18:00] ogra_: hi! I noticed that the bbb kernel is a few versions behind the most recent USNs [18:01] ogra_: is your automated script not working right? [18:14] jdstrand, that one isnt automated ... i'll kick off a new build [18:16] ogra_: fyi, you can automate it if you want. I just added an exception for it in the review tools (though this upload will need a review, when the store syncs it won't any more) [18:16] ok [18:41] PR snapd#2624 opened: Re-associate with pid-1 mount namespace if required [18:46] tyhicks: good news, I think I know what the kernel bug is about :) [18:46] (the one with oops on one thing I did a while ago) [18:46] well, fingers crossed [18:46] at the very least I can give you a way to reproduce easily [18:47] jdstrand, jjohansen: ^^ [18:47] (oops in apparmor) [18:47] on the up side it is *not* Friday yet [18:48] ogra_: thanks for the pi image...one question, anyway we could change pi3 console config to not require ethernet cable? [18:49] i set up wifi fine, but then it loops back and demands i set up via ethernet [18:49] kgunn, i think there is an old bug open for this ... [18:49] ah [18:49] * kgunn thot pi2 didn't have wifi chip [18:49] and that was the reason [18:50] you can set it up initially with eth0, then reboot, ssh in and run "sudo console-conf" and disabe eth0 and switch to wlan0 ... that works fine for me [18:50] it is only the very first boot where it fails [18:50] ogra_: out of interest why are we use pi2 kernel on pi3 are there phyiscally no differences between then? [18:50] (but indeed you need network on the very first boot to set up the user) [18:51] ogra_: you can use the serial console [18:51] ogra_: on a pi :) [18:51] davmor2, the kernel is identical ... but th ebootloaders are not (namely there is a different u-boot.bin ) [18:51] jjohansen: this is not what we talked about yesterday, but zyga is seeing an oops with https://github.com/snapcore/snapd/pull/2624. I have a feeling many of your concerns about changing the mount namespace would apply to this pr [18:51] PR snapd#2624: Re-associate with pid-1 mount namespace if required [18:51] zyga, for what ? [18:51] ogra_: to do the 1st boot dance [18:51] jdstrand: that branch is buggy but perhaps this is causing the oops [18:51] zyga, and that magically pulls the login data via serial ? [18:51] jdstrand: testing again with the fix [18:51] ogra_: oooooho [18:51] :D [18:52] ogra_: I should have gotten that beer [18:52] you need network to be able to finish console-conf once [18:52] hah [18:52] jdstrand: anything I should know about? [18:52] once it is finished, switching networks is easy [18:53] zyga: you should wait for jjohansen. he had concerns regarding changing namespaces and open fds and likely some other concerns [18:54] jdstrand: will he be around today? [18:55] zyga: this came up in the context of snap-alter-ns, but I think they would apply to this [18:55] jdstrand: hmm hmm, can you tell me how would snap-alter-ns be affected? we could presumably do all the open() calls after we re-associate [18:55] (just worried if I should be worried) [18:55] zyga: he should be. he has been sick though. not sure when he'll be online [18:56] zyga: need to wait for jjohansen. we were going to chat today. may as well make it all 3 of us [18:56] (or maybe tomorrow) [18:56] jjohansen: I'm sorry to hear that, I hope you get better [18:57] jdstrand: if he's around can you please PM me on telegram (I get notifications on my phone this way) [18:57] I'd like to listen to understnad this problem better [18:57] zyga: you can listen to irc then? [18:59] jdstrand: if the converstation is on IRC then that's much easier :) [19:00] zyga: it will be, yes [19:02] jdstrand: somewhat offtopic, I think there's a separate bug, kernel doesn't let me bind() the mount namespace FD if I don't run as root despite having all the capabilities; I will need to investigate that in ~10 days. Do you know if this is by design (by any chance)? [19:04] zyga: I don't, sorry [19:04] * zyga will dig through the kernel then [19:05] hi there! How I can submit a snap package both for armhf and amd64? I have both snaps already built. [19:05] alvarolb: just send them to the store [19:05] alvarolb: you can use snapcraft for that [19:05] alvarolb: or use the web UI [19:05] alvarolb: both snaps need to have the same snap name / snap id [19:05] alvarolb: and (hopefully) different architecture entries [19:06] yes, but if I upload the amd64 [19:06] alvarolb, they will both be assigned different revisions and the store will know what arch they are. And snapd won't let a snap for the wrong arch be installed [19:06] then it replaces the armhf [19:06] alvarolb: it doesn't really replace them [19:06] alvarolb: just try it, you will have to publish each one [19:06] alvarolb: then you should be able to install them :) [19:07] Ok, so I upload both versions to the same snap package, and it should work [19:07] ogra_: re eth, ack...except i don't have eth cable access here at my communal office space :-P [19:07] it does not show that the snap is available for both platforms [19:07] alvarolb, then run `snapcraft status ` and you'll see where each arch has its own channel map [19:08] kgunn, ah, damned [19:09] kgunn, well, we dont have a solution beyond this yet [19:10] you could try a wifi dongle, perhaps that works better [19:10] ok, thanks for the support! [19:15] PR snapd#2433 closed: tests: run all snap-confine tests in c-unit-tests task [19:36] jdstrand: can you take a look at this error for me? http://paste.ubuntu.com/23788349/ [19:36] I suspect that it's not really a problem with the limits, but rather the snap's inability to check what the limits are, due to confinement [19:38] mhall119: Jan 12 14:31:59 mhall-thinkpad snap[5671]: /snap/couchbase-server-community/x1/couchbase-server-snap: line 166: exec: erl: not found [19:38] mhall119: limits seem to be a separate issue [19:39] mhall119: look at dmesg | DENIED please [19:39] er [19:39] dmesg | grep DENIED [19:40] zyga: nothing from couchbase there, I have another issue with it not finding the 'erl' binary that I need to fix [19:49] so in a classic snap how would i copy a file to my HOME dir? [19:49] mhall119: then I'd say it dies on that problem now [19:49] not the home dir of the snap [19:50] stokachu: classic snap or in the classic confinement of some snap/ [19:50] zyga, in the classic confinement [19:50] stokachu: snaps that have confinement: classic don't get HOME redirection [19:50] stokachu: HOME is real :) [19:51] stokachu: try it, [19:51] hmm so if i do a cp file ~/test from the snap it doesn't show up in my HOME dir [19:51] stokachu: https://github.com/snapcore/snapd/wiki/Environment-Variables#home [19:51] stokachu: can you use "snap run --shell $SNAP_NAME.appname" and go to $HOME [19:52] stokachu: or just echo $HOME [19:52] stokachu: if that's a bug I can (still) fix it [19:53] zyga, hmm ok $HOME shows my user (ubuntu) /home/ubuntu when i do a snap run --shell conjure-up [19:53] need to check out my code then, because things like os.expanduser in python should work as normal correct? [19:54] yea must be something in my code [19:54] * stokachu back to drawing board [19:58] ogra_: this may be a different bug (wifi), I ran console-conf after first boot like you mentioned but [19:58] wlan0 option dissapears [20:00] stokachu: I hope so, we don't change anything else [20:00] ogra_: I guess a reboot fixed that :) [20:00] zyga, hah [20:05] jdstrand: hey, so I made some progress but got stuck; I don't see any apparmor denials (the process is in devmode), I don't see anything in the kernel log but when I open /proc/1/ns/mnt I get EACCES [20:05] jdstrand: I did an experiment where I just nsenter around and this worked OK [20:05] jdstrand: I suspect apparmor is causing this [20:06] jdstrand: can you please have a look at https://github.com/snapcore/snapd/pull/2624/files#diff-5f1642833244a655796f5f4230f68fdbR207 [20:06] PR snapd#2624: Re-associate with pid-1 mount namespace if required [20:06] jdstrand: and tell me if I need to adjust the apparmor profile in some special way (in the same pull request) [20:06] PR snapcraft#1045 opened: Handle parser errors better [20:06] jdstrand: note, I *don't* get a DENIED anywhere :-( [20:06] jdstrand: (thinking about it now I want to point out that while the process running snap-confine is in devmode, snap-confine itself is obviously confined) [20:07] jdstrand: perhaps my change is not applied to the snap-confine I'm runnning from the (repackaged) core snap? [20:08] jdstrand: still thinking aloud, looking at /sys/kernel/security/apparmor/profiles I see [20:08] http://paste.ubuntu.com/23788499/ [20:10] jdstrand: should snap confine reset the policy (maybe this should be done in the base apparmor template, running snap confine doesn't inerhit the profile but replaces it with the profile for snap-confine) [20:10] jdstrand: insight appreciated, I'll check back later [20:19] jdstrand: I've added some more facts here: https://github.com/snapcore/snapd/pull/2624#issuecomment-272270130 [20:19] PR snapd#2624: Re-associate with pid-1 mount namespace if required [20:19] * zyga EODs [20:22] AlbertA, yeah, wla0 is there and you can even attempt to configure it but it will always fail in the end [20:22] second attempt is stable [20:29] jdstrand: you got lucky and we had a clear store deployment pipeline; tools r824 is now in production [20:30] \o/ [20:30] roadmr: thanks :) [20:41] o/ heyo, trying to use snapcraft from MASTER. My tests pass save for some subclass issues (about 7 failures that seem unrelated) - however when attempting to use snapcraft, i get the following message: Issues while validating snapcraft.yaml: snapcraft validation file is missing from installation path [20:45] lazyPower, how are you running snapcraft? [20:49] kyrofa - i followed the hacking guide to install it (isolated in a lxd container) [20:50] kyrofa - after the predep steps, python setup.py install && cd $my-snaps-path && snapcraft [20:50] s/python/python3/ [20:50] lazyPower, the hacking guide only discusses dependencies, not snapcraft itself [20:51] lazyPower, when it comes time to run it, simply run bin/snapcraft (or add it to your PATH) [21:21] mhall119: your system seems to have 'nofile' set somewhere. you can see your limits by typing 'ulimit'. I did just now on classic, in hello-world on classic and hello-world on all snaps and they all say 'unlimited'. see man limits.conf and /etc/security/limits.conf [21:21] mhall119: are you seeing any security denials? [21:23] zyga: re https://github.com/snapcore/snapd/pull/2624/file if there is no denial, it shouldn't be apparmor. the kernel might be telling you EACCES for another reason. that said, I think we should ask jjohansen about it [21:23] PR snapd#2624: Re-associate with pid-1 mount namespace if required [21:24] looking [21:26] zyga: when you use nsenter, are you doing it from within snap-confine at the point that it is getting the EACCES? ie, nsenter from global namespace to pid 1 namespace is one thing, some snap app running anohter snap app that triggers snap-confine to have it go to pid 1 is another thing [21:27] jjohansen: ^ that is the summary of the issue [21:27] right [21:30] jjohansen: let's wait for zyga to respond on that. do you have time to discuss snap-alter-ns? (what we discussed yesterday) [21:30] we discusssed it? [21:30] well [21:31] we referenced it? [21:31] the thing from yesterday :) [21:31] right [21:31] (which isn't the same thing as the above pr) [21:31] jjohansen: you have time now? [21:32] sure, I guess [21:33] jjohansen: ok. so let me give you the problem and a simple description of what the propsed solution is, then the url for the din depth description of the fix [21:34] jjohansen: today snaps can ship multiple commands. those commands may be daemons or manually started. all commands within a snap share the same mount namespace, so the all see the same /tmp and things that snapd might mount into the namespace [21:36] jjohansen: this is done by when a snap command is first started, a nsfs magic file is checked. it is isn't magic, the full mount namespace is setup, then the magic file is saved. the next command that runs see the magic file and snap-confine enters that mount namespace instead of generating it anew [21:38] jjohansen: on top of that, there is an interface called 'content'. this allows a providing snap to 'export' something in its area so that another snap may 'import' it. this is done via a file that maps the export dir to the import dir that snap-confine reads and it will perform a mount of the exported dir into the calling snaps mount namespace [21:41] jjohansen: iirc, things work fine if the interfaces are connected (ie, snap-confine is supposed to do this mount operation) and the mount namespace is not setup (eg, after a reboot). there have been some issues with if the mount namespace is setup already iirc and adding it after the mount namespace is already setup. (eg, start the command, then connect the interface, etc) [21:41] jjohansen: so zyga came up the the idea of 'live modification of mount namesapces' === smoser` is now known as smoser [21:42] jjohansen: in essence, that is meant to robustly perform the mount of the imported directory on a namespace that is already setup, and remove mounts that have been disconnected [21:43] jjohansen: I mentioned that removing mounts will cause problems with daemons most likely-- I think he will counter that they'll lazy umount. [21:44] jjohansen: the live modification will be done by 'snap-alter-ns', a command that snapd will call when a 'snap connecnt' or 'snap disconnect' is performed. it is meant to mount missing connected imports and unmount ones that are no longer connected [21:44] jjohansen: here is the larger proposal: https://github.com/snapcore/snapd/wiki/Live-Modification-Of-Mount-Namespaces [21:45] jjohansen: [21:47] jjohansen: well, I have one final thought. revocation aside, I thought that simply adding mounts would not be problematic since that isn't really any different than say plugging in a usb key and having it show up. that said, there are many layers of mounts here and that may be a naive way of looking at it [21:48] okay, removing mounts can be problematic for any open fd, but it is probably something that isn't a blocker [21:48] re [21:48] hey [21:48] * zyga watched 2nd episode of 4th season of Sherlock [21:48] jdstrand: replied on your comment [21:49] jdstrand: according to my experiment this only happens when apparmor is in the loop, I could have made a mistake, I was tired already [21:49] 22:26 < jdstrand> zyga: when you use nsenter, are you doing it from within snap-confine at the point that it is getting the EACCES? ie, nsenter from global namespace to pid 1 [21:49] namespace is one thing, some snap app running anohter snap app that triggers snap-confine to have it go to pid 1 is another thing [21:49] additions at first glance look safer, but have the potential to create aliases and rewrite subtree. If tightly controlled it should be okay, if not it is an attack vector [21:50] jdstrand: to reply to this: when I use nsenter I was doing it from the mount namespace created by snap-confine [21:50] I'm happy to discuss anything [21:51] zyga: let's pause on 2624 for a moment and get through snap-alter-ns [21:51] ok [21:51] zyga: can you turn on apparmor debug messages [21:51] echo 1 > /sys/modules/apparmor/parameters/debug [21:51] and see if that turns anything up, also turn off printk rate limiting to make sure printk isn't dropping something [21:51] echo 0 > /proc/sys/kernel/printk_ratelimit [21:52] jjohansen: ack, I'll do that [21:52] jjohansen: wrt aliases and rewrite subtree, how would one trigger that? right now, the snap declares a dir and that is it. is it a malformed dir? a symlink? something in the dir that could trigger it? [21:52] oh I forgot rate limiting [21:52] apparmor shouldn't be denying something without logging it, hmmm unless explicitly told not to log it, better turn off quieting as well [21:52] echo -n noquiet > /sys/module/apparmor/parameters/audit [21:53] jjohansen: we don't actually explicitly much in the snappy policy (I think there are two rules otoh) [21:53] but noquiet never hurts [21:55] just trying to make sure all potential reasons not to log are out of the way [21:55] ok, test in progress [21:55] let's talk about snap-alter-ns [21:56] (fyi, you need sys_admin for entering the namespace and snap-confine has that) [21:56] correct [21:56] jjohansen: did you see my last question? [21:58] * zyga finishes reading backlog [21:58] jdstrand: you mount over an existing dir/tree location that isn't a leaf [21:59] it just has to be controlled is all [21:59] jjohansen: like, twp exported dirs mounted onto the same import dir? [21:59] I'm fine with limiting this so that we can guarantee no "funny" layouts are possible [21:59] jjohansen: it is rather tightly controlled right now. I want to make sure we make sure we get all the bits [21:59] you should also be aware with one more thing we're trying to introduce that may complicate this [22:00] the 'overmount' interface, that within the snap's mount namespace, allows it to almost freely do bind mounts from $SNAP to various places across the fileystem; the use case is shoving stuff into /usr/share so that pre-compiled binaries feel more at home [22:00] zyga: can you add to your list a check to make sure that you can't mount on an already mounted location? ie, no layering at all [22:01] jdstrand: so essentially the target directory cannot itself be a mount point [22:01] zyga: that is how I took jjohansen's comment. let's get his feeback [22:01] jdstrand: will this restriction cause any issues to users? [22:01] ok [22:02] zyga: I think blocking that is a good thing despite this. it would be weird to have two content exports mounted over each other. which would win? poor user experience [22:03] agreed [22:03] are we considering *any* mount points or just those that interfaces created? [22:04] jjohansen: ^ [22:06] /bin/bash: line 44: /sys/modules/apparmor/parameters/debug: No such file or directory [22:06] jdstrand: well sure that is one, doesn't really matter any non-leaf mount can result in weirdness, as you have a shadowing affect [22:06] hmm [22:06] ie. alias [22:06] ah, module vs modules [22:07] with the content interface I think this is ok [22:08] jjohansen: can you give me an example of a mount that would be problematic [22:08] whether its good or bad depends on what you are trying to achieve, but it adds to the complexity of what needs to be analyzed and opens the potential for attacks that you don't see. I think you probably okay, the mounts are being done by snapd not the application [22:08] jjohansen: I'd like to understand what to avoid better [22:08] we are mounting into a dir that shouldn't have any mounts (well, could be on a dir in a squashfs) [22:09] jjohansen: gotcha [22:09] zyga: that is a good question, and the answer is it depends [22:10] zyga: I think we are ok cause the content interface is mounting into a very specific place-- either in SNAP, SNAP_COMMON or SNAP_DATA. there shouldn't be anything else in there if we employ the outlined restriction [22:10] some of it, I just don't know. Aliasing attacks via links etc have been pulled off in the past. I think we largely don't have that problem with snappy [22:11] the other thing to worry about the interaction between the different mechanisms providing your policy, which is a combination of namespaces, dac, apparmor, and seccomp [22:11] (snaps can't mount (excepting a few super restricted interfaces that are only allowed to trusted apps)) [22:11] jdstrand: if the overmount interface lands that comfort will go away, e.g. I can connect overmount and then content and then disconnect overmount, given the right (or evil) interfaces that could do some things we weren't expecting [22:12] zyga: I have quite a few concerns with the overmount interface [22:12] they each have their own set of expectations, I think the interaction between apparmor path mediation and mount namespacs is the most problematic [22:12] again its just a matter of making sure your policy is right for the changes [22:13] based on what we said here, I think we are good. we can deal with overmount if/when it comes up for review [22:14] can you wait 5 more minutes, I will have that debug data [22:14] it is too difficult to think about the policy ramifications of what it could someday do ("it can do anything!" I can't evaluate the impact on policy for 'anything' :) [22:14] I was looking at loaded profiles and I saw that there were "//null" in the profile names, that made me worries somewhat, are those expected? I don't quite understand how profiles stack or what to make of the profile name [22:15] (after the reassociation test failure) [22:15] jjohansen: wrt revocation with snap-alter-ns. today we don't revoke so there isn't an issue. with snap-alter-ns, I suspect that zyga will use a lazy umount. do you see problems there? [22:15] zyga: //null is normal when you are in complain mode [22:16] zyga: //null should only ever showup for profiles that are auto-generated during complain mode [22:17] they will either be //null-#### where #### is unique, or //null-executable name [22:17] dmesg http://paste.ubuntu.com/23789099/ [22:17] kern.log http://paste.ubuntu.com/23789105/ [22:17] we got back to talking about two things at once [22:18] this is the same test with the extra debugging [22:18] we can finish snap-alter-ns if the revocation question is answered [22:18] ah, sorry, I'm always too impacient [22:19] [ 498.595560] apparmor: clearing unsafe personality bits. /usr/lib/snapd/snap-confine label=snap.test-snapd-tools.cmd//null-/usr/bin/snap//null-/usr/lib/snapd/snap-confine [22:19] zyga: so I don't see apparmor directly causing the failure but, it is possible that clearing the unsafe personality bits could lead to an EACCES [22:19] actually [22:19] [ 498.616822] audit: type=1400 audit(1484259403.009:67): apparmor="ALLOWED" operation="open" info="Failed name lookup - disconnected path" error=-13 profile="snap.test-snapd-tools.cmd//null-/usr/bin/snap//null-/usr/lib/snapd/snap-confine" name="" pid=25299 comm="snap-confine" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 [22:19] isn't this exactly what I saw? [22:20] I got errno 13 [22:20] jjohansen: I was reading the log in order, this is the last message [22:20] can we put that on hold? I have a question regarding it that will require discussion [22:20] sure [22:20] there is but one question remainging... [22:20] jjohansen: wrt revocation with snap-alter-ns. today we don't revoke so there isn't an issue. with snap-alter-ns, I suspect that zyga will use a lazy umount. do you see problems there? [22:21] oh, hrmmm, if there is an ALLOWED it should be toggling the error to 0, the logging does report the initial error [22:21] I am not ruling out that there is a bug around complain mode and disconnected paths [22:22] jdstrand: sure, lazy unmount tringgers disconnected paths [22:22] ah [22:22] zyga: so I'm not sure we can do live revocation well with snap-alter-ns [22:23] jdstrand: hmm hmm [22:23] zyga: daemons or anything already running in the mount namespace are going to be unhappy [22:23] jdstrand: with that said, if the fd label is cached it should be fine [22:23] jdstrand: we could ignore that and just carry on (to some extent) [22:23] jdstrand: we could fail the operation and say "gee, cannot disconnect this thing now" (bad UX perhaps, need to re-do some mounts) [22:23] zyga: I think snap-alter-ns will make connect more robust [22:23] the issue would rear its head if new lookups (openat, ...) are done based off of it, or policy is replaced [22:24] policy gets replaced on connect/disconnect [22:24] jdstrand: well, then we are going to have to spend some time looking at it [22:24] zyga: I like the idea of trying to umount non-lazy. if it fails, it just doesn't get unmounted. it gets removed from the saved file. eventually it is gone [22:25] jdstrand: it will depend on several things, and stacking opens up some avenues that might make it not a problem [22:25] zyga: I think more than that will require tyhicks or ratliff in on the conversation so that scope and priority can be discussed [22:25] jjohansen: ack [22:26] jdstrand: we could also consider saying "this doesn't work" and having snapd shutdown all the stuff that runs there [22:26] jdstrand: so in the nice case it just works [22:26] I'm ready to move past revocation if you guys are [22:26] jdstrand: in the non-nice case it just reliably gets closed [22:26] I think we need to try it and meet again to discuss what kind of warts remain [22:26] jjohansen: what do you think? [22:26] zyga: well-- restarting a daemon wouldn't be too bad, but there is no way to restart a non-daemon command that is running [22:26] jdstrand: ah, correct [22:27] jdstrand: I didn't consider this [22:27] jdstrand: we could use cgroups to do this though [22:27] freeze and kill [22:27] zyga: that said, snap-alter-ns and its way of managing the saved file sounds fine and will make the connect more robust. we can ponder disconnect and defer [22:27] zyga: yeah, meeting again sounds good [22:27] eh [22:27] jdstrand: sounds good, I think most people will struggle with connect more :) [22:27] I'd be pretty miffed if I lost all my state [22:28] zyga: I do too [22:28] jdstrand: connect is interactive [22:28] jdstrand: we could do something smart like "are you *really* sure you want this?" [22:28] zyga: so if you add attach_disconnected to your profile, and test again that should tell us whether the failure is a bug in how apparmor is handling disconnected paths wrt complain mode [22:29] jjohansen: this is attach_disconnected already I think [22:29] jjohansen: there are only two profiles at play: (and a hat): snap-confine, base policy all snaps get and the hat in snap-confine [22:29] * zyga checks [22:29] I have a debug shell so I can experiment if you have ideas [22:29] zyga: nope, you won't get info="Failed name lookup - disconnected path" [22:30] well not unless that is a bug too ... [22:30] zyga: ok, so now my question-- why is snap-confine calling snap to call snap-confine from within the profile? I think the snap command needs to be the thing that sets everything straight. ie, snap can talk to snapd which can notice if the comamnd is running under confinement, then check if the confinement allows calling the command, then fork/exec to call snap itself [22:30] you are focusing on devmode, but I don't know how this is expected to work in strict mode policy-wise [22:31] jjohansen: snap-confine has that, looking at the generated profile now [22:31] so snapd is a trusted helper [22:31] jdstrand: this is a feature for CE, in devmode only they need to be able to run a snap command from another snap command, before mount namespace tricks it use to work so they treat it as a regression [22:32] I know that has problems with fds, etc, etc, but so does this method [22:32] jdstrand: this is only expected to work in devmode (for them) [22:32] jjohansen: I confirmed that all profiles are attach_disconnected [22:33] I can pastebin them if you like [22:33] qemu:ubuntu-16.04-64 .../tests/regression/lp-1644439# cat /sys/kernel/security/apparmor/profiles |pastebinit [22:33] zyga: does /sys/kernel/security/apparmor/policy/raw_data/ exist? [22:33] http://paste.ubuntu.com/23789168/ [22:33] checking [22:34] yes [22:34] looking at line 91 and the lines after, I feel like maybe snap-confine is trying to transition to the hat but can't find it [22:34] but empty [22:34] jdstrand: note that this is _before_ the hat [22:34] gah, never mind. I've got it. The null child profile is not picking up the flags of the parent, so it is not attach_disconnected [22:34] jdstrand: the reassociation is done before we even attempt to fork [22:35] zyga: can you open a bug [22:35] jjohansen: sure, just tell me where [22:35] this is a funky denial: apparmor="ALLOWED" operation="open" info="Failed name lookup - disconnected path" error=-13 profile="snap.test-snapd-tools.cmd//null-/usr/bin/snap//null-/usr/lib/snapd/snap-confine" name="" pid=25299 comm="snap-confine" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 [22:35] name=""? [22:35] jjohansen: I can boot other kernels, test anything, this is easy to reproduce [22:35] well, it doesn't know what the name is [22:35] zyga: drop your failure and log in the bug, open it again linux or apparmor it doesn't really matter [22:35] zyga: do you have a reproducer for jjohansen that doesn't involve spread? [22:36] zyga: name="" means that its going to attach to / [22:36] jdstrand: spread just runs those few shell commands, all you need to do is to build s-c from that branch and then you can do it easily [22:36] jjohansen: I'll file one shortly [22:37] disconnected paths don't have a leading / and currently the only connection point is / [22:37] jdstrand: and this is in qemu, [22:37] (pretty easy to set up) [22:37] zyga: can you make sure that the bug has very clear instructions for reproducing and building s-c? [22:37] jdstrand: sure [22:37] jdstrand: I'll try to simplify it if I can [22:37] but this is pretty robust, just clone that branch and run spread with a given argument [22:37] zyga: jjohansen is trying very hard to hit a merge window for upstreaming apparmor and he doesn't hack on snap-confine or snapd :) [22:37] and you have shell in the qemu box :) [22:38] point taken [22:38] btw, how is that progressing? [22:38] zyga: I don't need the reproducer, just the bug. I should have a kernel for you in a few hours [22:38] zyga: yeah, don't expect that to be fast for him :) he doesn't have any of that setup. a shell script or .c file demontrating the problem is way better than bootstrapping the snapd dev environment [22:38] jjohansen: looking forward to that [22:39] sounds like jjohansen has it under control and I've said my bits, so I'm out [22:39] I'll report the bug now [22:39] thanks guys, it was good to talk to you again :) [22:40] np [22:40] jjohansen: thanks for all your time! :) [22:47] Question: I have a user that built a custom kernel and they are trying to install it with the snap install command but it fails saying it is not signed. They used the --dangerous and equivilent options to no avail. Is there any other command line flags to use when trying to do this? [22:47] wililupy: you need to build an image with this but I don't believe this option is supported in ubuntu image yet [22:49] zyga, is there no way to use a custom kernel, then? [22:49] zyga: Thats what I thought. I told them that and sent them instructions on building a custom image but I also told them I would get a difinitive answer on this as well. Thanks. [22:49] jmm [22:49] maybe I'm confusing building an image with devmode snaps and building with a unsigned kernel [22:49] * zyga doen't know [22:50] barry, help? [22:51] kyrofa: when I build custom images, I use custom kernels and it works. I never tried snap install kernel.snap but they did and told me what happened. [22:51] wililupy, ah, so ubuntu image _does_ support this? [22:52] wililupy, was the install attempted on classic ubuntu, or in ubuntu core? [22:52] kyrofa, yes. in the assertion I use my custom kernel's name value and then use --extra-snaps and the kernel.snap and it builds and runs no problems. [22:53] kyrofa: ubuntu-core. [22:53] wililupy, okay good, glad to know that works. barry unping [22:53] wililupy, I kinda feel like you should be able to install --dangerous on core though... otherwise testing a custom kernel is rough, no? [22:54] I wonder about the reason behind that [22:54] kyrofa, thats what I thought as well, but they came back saying it didn't work. [22:54] zyga, can you think of why that wouldn't work? [23:03] sorry, I'm semi off now [23:03] kyrofa: you probably cannot install a kernel now, not sure [23:04] you may need to build an image with it [23:04] but I'm just throwing ideas at 4 minutes past midnigth [23:04] * zyga postpones filing the bug till tomorrow [23:12] jjohansen, jdstrand: https://bugs.launchpad.net/apparmor/+bug/1656121 [23:12] Bug #1656121: unexpected errno=13 and disconnected path when trying to open /proc/1/ns/mnt from a unshared mount namespace [23:12] please tell me if I should provide more data [23:13] zyga: thanks, I'm just kicking off a kernel build [23:18] PR snapcraft#1046 opened: godeps plugin: work when GOBIN is set