[00:48] jdstrand: hey [00:48] jdstrand: it works :) [00:48] jdstrand: all skills based, I'll bug you with details tomorrow :) [00:49] https://github.com/zyga/snappy/tree/skills-demo-i2c <- somewhat hacky, to a small degree === timchen1` is now known as timchen119 [04:04] howdy [04:28] i'm trying to install snappy core on i386 and the file is .img and i've tried converting to .iso but it's failing. What gives? [04:50] X86 as in 32bit? [04:56] 64 === Guest82926 is now known as pitti === pitti is now known as Guest48131 === Guest48131 is now known as pitti === pitti` is now known as pitti === pitti is now known as Guest54153 === Guest54153 is now known as pitti [07:45] good morning [07:46] * zyga has made great progress towards adding lots of more, complex skills [07:47] hey zyga [07:57] good morning [08:02] good morning dholbach! === kickinz1|afk is now known as kickinz1 [08:08] salut didrocks === faenil_ is now known as faenil === chihchun_afk is now known as chihchun === faenil is now known as faenil_ === faenil_ is now known as faenil [09:52] Good morning all! Happy Wednesday, and happy World Human Spirit Day! 😃 === chihchun is now known as chihchun_afk [10:22] kyrofa, resize is fixed with the latest kernel snap in the store, yes (you can use mvo's u-d-f to build one or wait til later today when i release a new image) === chihchun_afk is now known as chihchun === kickinz1 is now known as kickinz1|afk [11:32] No snappy images for 15.10, is there a roadmap that states if a 16.04 release will appear? [11:39] msvb-out, snappy is a rolling release, after release of the "normal" 16.04 ubuntu the stable channel will switch to it [11:46] Hi [11:47] I'm running 16.04 snappy core on an RPi2 [11:48] I want to build an app using snapcraft but my laptop outputs an arm64 package, which is not copatible with the RPi2 (ARMv7) [11:48] What can I do? [11:49] you should use the classic dimension on your rpi to build it [11:49] sudo snappy enable-classic [11:49] snappy shell classic [11:49] then install snapcraft with apt-get, pull your snapcraft.yaml in and build it [11:50] ok ill try it, thanks! [11:51] zyga: do we have a list of skills that we will do yet? [11:51] or skill types rather [11:52] wow, my desktop just locks itself after 10 seconds without me touching anything :/ [11:53] wonder if that means i need to restart X :)? [11:56] asac: I don't know anything more than last week, sorry [11:57] asac: maybe something will be decided at the sprint [11:57] asac: I'm not there, I don't know [11:57] asac: I explored adding a new skill type this week (i2c) to iron out issues with the security layer [12:00] geez .... will that netsplit ever stop today ? [12:00] asac, you should reboot after the libc bug anyway [12:01] hmm. ok :) [12:01] * asac turns down productivity [12:01] and goes for reboot [12:04] * zyga needs to reboot as well, thanks for the tip ogra_ [12:05] :) [12:06] hmm, seems to not have landed in xenial yet http://people.canonical.com/~ogra/core-image-stats/20160217.changes [12:07] ogra_: I see it in my apt-get changelog [12:07] yeah, but it landed after the daily image build [12:07] glibc (2.21-0ubuntu6) xenial; urgency=medium [12:25] hm, what does this mean when I run "snapcraft snap" on a very simple package (figlet):- http://paste.ubuntu.com/15099808/ is the yaml, here's the 'error message' "Command '['/bin/sh', '/tmp/tmpom31_khu', '/bin/sh', '/tmp/tmp6lhcweja']' returned non-zero exit status 1" [12:26] it clears up after itself so i can't even tell what /tmp/tmpom31_khu & /tmp/tmp6lhcweja were. [12:27] * ogra_ has never used the nil plugin ... [12:29] i used it in my cowsay one and it worked fine [12:29] don't understand it [12:29] where would figlet.sh come from ? [12:29] i dotn see it being copied or anything [12:29] ah [12:29] balls, i missed that out [12:29] good spot, thanks 😃 [12:30] :) === yp is now known as ypwong === Michaela is now known as Ciblia [12:34] ogra_: Ah, a rolling release. But why didn't the snappy stable channel follow the 15.10 release, was it not 'normal'? [12:35] it was a snapshot of 15.04 to have a non-moving base but has a lot of additional updates from an additional repo [12:38] all new development is in 16.04 though, 15.04 was kind if a "stable pre-release" [12:38] *of === retrack is now known as Guest83882 [12:38] but we arent really tied to the normal release schedules after all [12:41] ogra_: Okay, that's what I gathered. It would be easier for users if a intuitive release schedule and regular image filenames exist. === tbr_ is now known as tbr [12:41] ogra_: But maybe that will follow once a larger userbase is established. [12:41] the release process is totally different for snappy [12:41] a schedule wouldnt make sense [12:42] fixes go in if they are ready and will always show up in the daily rolling builds ... [12:42] also, since you can update apps and os completely independently the os isnt really that important (apart from feature completeness) [12:43] there is some raw schedule though ... [12:45] http://www.olli-ries.com/t-242d/ [12:46] ogra_: interesting, hadn't thought of the release philosophy being different. What you say makes sense. [12:46] Hi guys, I'm trying to install nginx. I do that with deb packages but when it tries to start it killed itself again :s [12:46] I guess this was more relevant when snappy on ARM builds had no implementation of snappy update. [12:47] yeah, the switch to the all-snaps model in 16.04 will chnage that [12:48] noizer, via stage-packages in your snapcraft.yaml ? [12:48] you most likely need a wrapper script to adjust config and paths etc [12:57] Ah, ogra_ thanks for getting back to me regarding the resize thing :) [12:58] Good morning #snappy, by the way [12:59] kyrofa, if you want to resize it on your PC http://paste.ubuntu.com/15099918/ might help (adjust DISK to point to your SD reader and run with sudo) [12:59] yea with stage packages ogra_ [12:59] i'm working that snippet into the initrd today (and clean it up a bit) [13:06] noizer, so if you take a look at the startup script i use for lighttpd ... http://bazaar.launchpad.net/~ogra/+junk/upnp-server/view/head:/run-lighthttpd between line 12 and 44 i create/update the config (you probably dont need that if you have a fixed config pointing to the right dirs) and in line 50 i run lighttpd with adjusted options to run from the snap dir [13:07] you likely want something similar for ngnix fi you use the deb, since the deb defaults wont work for snappy [13:08] (note this is for 15.04 though ... the env vars have changed in 16.04 and you should use the new ones) [13:18] * ogra_ triggers a xenial build to get the fixed app-launcher and libc before rolling new ubuntu-core snaps [13:18] hmmmm strange i think i do that [13:18] when the service is running can you see it with ps -A [13:18] ? [13:19] yes [13:20] ogra_, yeah good idea [13:23] (RaspberryPi2)ubuntu@localhost:~$ ps ax|grep lighttpd [13:23] 5373 ? S 0:00 /apps/upnp-server-armhf.ogra/0.1.0/usr/sbin/lighttpd -D -f /var/lib/apps/upnp-server-armhf.ogra/0.1.0/lighttpd.conf -m /apps/upnp-server-armhf.ogra/0.1.0/usr/lib/lighttpd [13:23] noizer, ^^^ [13:23] ogra_ dammed xD stupid nginx xD [13:26] ogra_ But kyrofa told me to compile it myself nginx. But i think that would be overkill [13:31] noizer: why, this is the best way to integrate [13:32] noizer: and if you really care about it, you can have a branch with extra snappy integration features that tracks upstream [13:32] noizer: it really is sensible IMHO [13:37] noizer, well, you have twoo options, eiter use the deb and invest all the work to adjust the paths via wrapper or you compile from source and set the right prefixes and paths in there ... either will be fine and both will likely be equally much work [13:39] ogra_: note; there is no valid value for the prefix as $SNAP_APP_foo is not fixed in stone [13:39] ogra_: you really have to load and re-loate at runtime [13:40] (load environment variables and use them to find stuff) [13:40] well, luckily we still have backwards compatibility [13:40] the old values should all still work afaik [13:41] ogra_: yeah but I mean that stuff like configure --prefix=/snaps/something is not going to work naively, there is no really good way to configure application that uses prefix to load its data [13:42] well ... --prefix=./ might work though :) [13:43] heh :) maybe but I doubt autotools will respect that [13:43] heh [13:43] the good news is that patching it is usually quite easy [13:43] the bad news is that then you have to carry a patch [13:43] * ogra_ is in the deb camp though ... [13:43] Right, Snappy is a bit odd in that you need to build in one place, tell it it'll run in another place, and actually run in yet another place :P [13:43] but thats just because i love shell [13:44] i still have to hit my first package where a wrapper doesnt work (there are surely a lot though) === ysionnea1 is now known as ysionneau [14:03] ogra_ when he tries to start now my nginx i got following error: Failed with result 'start-limit'. [14:03] well, no idea what that means ... i never used ngnix === JamesTai1 is now known as JamesTait [14:20] kyrofa: master static tests are currently failing? [14:21] asac, hmm... looks like an lxd error. Restarting [14:21] zyga: think for the time being you could use an x-plugin to have a hacked autotolls one? [14:22] asac, to take care of the patching? [14:22] the other thing could be to have variable expansion in configflags [14:22] kyrofa: to not require to patch the buggy upstream build system that relies on --prefix for data install instead of also honouring DESTDIR [14:23] better than adding a patch to autotools plugin imo that will auto add prefix to install dir... but thats just my feel [14:23] asac: yeah, that works too [14:24] asac: I think now you can fake most with a makefile [14:24] jdstrand: hey, I have a security question for you [14:24] you can also make a makefile wrapper for sure [14:24] but then thats a bit nasty to do too as you need multiple parts i guess [14:24] zyga: shoot [14:24] jdstrand: old hw-assign world ran a chmod so that "granted" file can be world/group read/writable [14:24] one with local source and one with remote source and then the local one needs to build the one from remote [14:25] it did? [14:25] jdstrand: this made sense since those files were off-limits anyway [14:25] jdstrand: yep, [14:25] jdstrand: so that at the end of the day a non-root process can open the file [14:25] I don't think I remember that [14:25] jdstrand: I can find that, one sec [14:29] I imagine this was done for something like /dev/video where udev is setting up strict groups and permissions that the non-root user was not part of [14:29] but, I don't really like just making the device world writable [14:29] coundnt we hook into udev-acl here ? [14:30] groups are pretty dead everywhere anyway [14:30] hmmmmmm [14:30] I don't know what that is, but it sounds promising [14:30] I cannot find that yet, that's not rightr [14:30] so actually this is quite related [14:30] I wrote a hacky i2c skill [14:31] it allows ioctl (though it's on by default) via seccomp [14:31] actually, if udev-acl is doing what I think it is, I don't think I like that either [14:31] it allows open via apparmor on the affected /dev/i2c-1 file [14:31] now the last thing I want to understand [14:31] is systemd /dev namespace and udev [14:32] (without the final chmod my demo app didn't have right to open the i2c controller) [14:32] jdstrand, well, afaik it hooks into logind and manages acls based on polkit ( pitti may correct me here, i'm only a consumer usually) [14:32] note - this is not a service/framework [14:33] jdstrand: are we on the same page or am I missing something with regards to things in /dve? [14:33] in /dev [14:33] ogra_: right, but what I was getting at is that while it might (arguably) be fine to grant the device access to the non-root user within the context of a confined process, either method gives the non-root user access to the devices when it starts processes that aren't confined [14:34] why would anything from a snap start something thats non confined ? [14:34] zyga: with hw-assign we didn't use a systemd /dev namespace [14:34] (beyond the granted skills) [14:34] ogra_: nothing from a snap would. a login starts an unconfined shell [14:35] but a login also gets me sudo access anyway :) [14:35] (at least currently) [14:35] ogra_, jdstrand: not polkit, it really adds ACLs to the device node (you can't control file acess with polkit) [14:35] jdstrand: https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/types/i2c.go [14:35] ogra_: but sudo doesn't equal root. if the login account is hacked, the password is unknown [14:35] pitti: oh, where do we do that? [14:36] jdstrand, ah, indeed [14:36] zyga: /lib/udev/rules.d/70-uaccess.rules [14:36] zyga: so, remember, the way hw-assign works is that isn't just apparmor [14:36] pitti, oh, right, i'm living in the past ... i remember it hooked into consolekit once [14:36] ogra_: right, that part got replaced with logind which tracks the foreground console, but the ACL part is still the saem [14:37] zyga: because we didn't want to have to recompile apparmor policy for ever changing device names (until our profile composition feature is finished (not 16.04) [14:37] pitti: uaccess? is it something like this: https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/types/i2c.go#L98 [14:37] ah, k [14:37] ) [14:37] zyga: they are both tags, yes; snappy-assign was meant for a different purpose, though [14:37] zyga: so we tag devices via the file pitti mentioned and then the launcher adds any devices that are tagged into a device cgroup for the app [14:37] zyga: IIRC we do that via apparmor [14:37] uaccess means "accessible to the current foreground console" [14:38] right, devices cgroup, not apparmor, sorry [14:38] it's been a while [14:38] it *should* be apparmor [14:38] but, aforementioned unimplemented feature [14:39] jdstrand: okay, I actually asked the wrong question up front; I don't want to design the security system; let me re-state the question differently: how should I implement access to files in /dev (e.g.g /dev/i2c-1 here) for snappy skills for 16.04? [14:39] zyga: the same general method. add them to a device cgroup [14:40] zyga: via the udev tagging [14:40] jdstrand: how? [14:40] jdstrand: like what I linked to? [14:40] https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/types/i2c.go#L98 [14:40] or something else [14:40] zyga: something like that, yes [14:40] zyga: ie, just port the old hw-assign technique to skills [14:41] jdstrand: that gets written to https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/security.go#L176 [14:41] yes [14:41] jdstrand: yeah, I'm trying to ensure I didn't miss anything, that old code is somewhat convoluted [14:42] hmm, so with this i still had to chmod /dev/i2c-1 [14:42] zyga: two things need to happen: *if* there is a /dev assignment, create a udev rules file in /etc/udev/rules.d *and* add a /dev/** rwk rule to the apparmor profile if it isn't already there [14:43] zyga: if there is not a /dev assignment, make sure there are no /etc/udev/rules.d files for the snap and remove the /dev/** rwk apparmor rule [14:43] zyga: what this is doing is transferring control from apparmor to the devices cgroup for /dev assignement [14:43] zyga: and giving it back with unassignment [14:44] zyga: this has nothing to do with device ownership btw [14:45] in the sense of UNIX perms [14:45] jdstrand: hmmm, so currently I do both [14:45] jdstrand: I have the apparmor rule and the /dev "assignment" via udev file [14:45] zyga: you don't want to do the apparmor part when we release, but it is ok for now [14:46] jdstrand: so what happens when I drop that today? [14:46] zyga: this is a boot time optimization [14:46] jdstrand: I mean, just use /dev assignment [14:46] jdstrand: (don't worry about boot time, everything will change before 16.04) [14:46] zyga: which are you dropping? [14:47] jdstrand: I'm trying to see what to do to make it right, out of the three security systems (seccomp, apparmor, "udev") I can give any kind of security configuration for the i2c skill [14:47] jdstrand: you just said I should not do apparmor [14:47] it might be more than boot time-- the point is, that if we assign a usb camera to the snap, every time the camera device name changes, we don't want to have to recompile apparmor policy [14:47] ogra_, dholbach: I've seen this a bunch now but it only just dawned on me to ask, why in snap do you have to do in popey's example cowsay.cowsay to make the command actually run? [14:47] jdstrand: let's not worry about that, I'm pretty sure we will do a full transition at that time [14:47] jdstrand: I want to focus on the simple mechanics, optimize later if we can (I suspect we won't) [14:48] davmor2, because there could be a cowsay.ogra and a cowsay.davmor2 too [14:48] jdstrand: the state engine in snappy will change how many things happe [14:48] *happen [14:48] zyga: if assigning something to a snap from /dev, use the udev method and add a blanket /dev/** rwk apparmor rule. hw-assign does that apparmor rule via an .additional file [14:48] davmor2, snaps actually replace PPAs on a very subtle level ;) [14:48] zyga: you should do the same [14:48] jdstrand: ok [14:48] ogra_: so the second is just hte dev namespace right? [14:48] yeah [14:48] or company or whatever [14:48] jdstrand: we're switching to skills controlling all of the security files, the .additional file doesn't exist in this world tw [14:49] btw [14:49] why not? [14:49] davmor2, like: libreoffice.microsoft ;) [14:49] jdstrand: each app has one apparmor profile, we don't write anything to disk ourselves [14:49] lulz ogra_ [14:49] zyga: that file makes it so you don't have to recompile the policy all the time [14:50] ogra_: that's evil dude seriously get ready for the letters ;) [14:50] jdstrand: how does that work if we still have to load the updated profile into the kernel? [14:50] zyga: you do write out to disk-- or are you saying that you removed /var/lib/snappy/{apparmir,seccomp}/profiles? [14:50] jdstrand: I kept that but I write fewer files: [14:51] https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/security.go#L59 [14:51] so, to be clear, hw-assign was doing exactly what it needed to wrt /dev files [14:51] https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/security.go#L138 [14:51] no more, no less [14:51] ogra_: so say for example libreoffice.libreoffice is the official snap and then libreoffice.davmor2 is mine with all the QA report templates added right [14:51] if you are removing a bunch of stuff, we need to discuss that, cause there are some pretty major implications [14:51] jdstrand: don't assume that what hw-assign did stays the same, snappy is changing in interesting ways, we cannot keep some of the old stuff around [14:52] davmor2, the point is that you can have the same app/binary multiple times installed and compare them, or use feature a from one and b from the other [14:52] zyga: I don't care about the high level. I'm talking about the lowest level security enforcement [14:52] davmor2, right [14:52] jdstrand: I can write to other files but it's not exactly clear that we want to; IMHO we'll start with empty security on boot (nothing has extra permissions) and build a fully dynamic changing system as things happen (snaps get activated, skills get granted) [14:53] ogra_: okay cool thanks for that makes a lot more sense now :) [14:53] :) [14:53] zyga: at the highest level, that may make sense, but if you are dynamically generating apparmor policy all the time, you'll find boot will be slow, assignements will be slow, etc [14:54] jdstrand: I think this is unavoidable given that the system has to be fully dynamic (skill hooks) [14:54] zyga: we have to utilize caching and be smart about updating the apparmor policy otherwise the system will feel slow [14:54] no it doesn't [14:54] we can avoid the slowness [14:54] if you listen to me :) [14:54] jdstrand: please tell me :) [14:54] initial boot from factory, there is nothing [14:54] (fine) [14:54] jdstrand: generating the policy is cheap, compiling one is slow but I hope we can cache *that* [14:55] write [14:55] right* [14:55] think of it like this: [14:56] write the policy to /var/lib/snappy/apparmor/profiles, then compile from that and write a cache file from that to that on next load, /var/lib/snappy/apparmor/profiles is used. ok, now you can [14:56] when the skills change or anything else, create a temporary policy generation somewhere [14:57] then compare that to what is in /var/lib/snappy/apparmor/profiles [14:57] if they are the same, do nothing [14:57] if different, copy to /var/lib/snappy/apparmor/profiles, recompile and update the cache [14:57] that is how you can be fully dynamic [14:57] that's somewhat similar to what I'm trying to build, I'll maintain a set of compiled policies [14:58] without losing caching benefits [14:58] probably hashed by contents [14:58] that does not cover device names that change all the time though (which is where the cgroups come in) [14:58] my point was that on boot we'll still do the dynamic bring-up (nothing stays granted after boot as a hook has to decide that), if the decisions are the same we just get cached binaries [14:58] hashed is fine, point is, just don't overwrite /var/lib/snappy/apparmor/profiles unless things actually change [14:59] that's another thing [14:59] we may end up changing the directory sturcutre [14:59] so, if you are regenerating the profile all the time and comparing it, then yes, you don't need an .additional file [14:59] for better transactionality, right now it is not transactional at all [15:00] the .additional file was just a technique to maintain a few additions rather than snappy having to maintain the profile itself [15:00] for apparmor the actual file is not interesting much, just the loaded profile in the kernel, we can keep cache on disk and actual profiles in temporary files that exist just long enough to load the profile into the kernl [15:00] *kernel [15:00] zyga: not quite [15:00] no? [15:00] please tell [15:01] no, because what if apparmor itself changes? [15:01] apparmor itself? [15:01] os-snap update? [15:01] yes, we add features to it [15:01] we reboot [15:01] yes [15:01] rules change, etc [15:01] start form scratch [15:01] cache won't be used because binary content will differ [15:01] all of security files are ephemeral [15:01] we re-construct them (we can) on boot [15:02] right? [15:02] I think that you should maintain the files in /var/lib/snappy/apparmor/profiles [15:02] the reason why is you won't know if the cache file changed without recompiled the policy [15:02] put more simply [15:02] hmm? [15:03] you mean things like ? [15:03] when you generate a cache file, its mtime is later than the policy file it was generated from [15:03] (and abstractions it includes) [15:03] only if the mtime of the profile is later than the cache will the policy be recompiled and the cache updated [15:04] (or the abstractions it includes) [15:04] hmmm [15:04] I see [15:04] so if always regenerate dynamically, then your mtime is always newer than the cache and you always recompile [15:04] I'll think about it [15:04] so you must have the files in /var/lib/snappy/apparmor/profiles [15:04] does apparmor have a preprocessor? [15:04] yes [15:05] so that we can just get the full raw output that we want to compile and stick into the kernel [15:05] and that is what you need to be using with my technique of seeing if the tmp profile and the one in /var/lib/snappy/apparmor/profiles are different [15:05] the reason I'm leaning that way is that all of security blobs seem to migrate to tmpfs in the world we're going to [15:05] is the perprocessor also slow? [15:05] or just the compiler? [15:05] no [15:06] it doesn't matter though [15:06] it is the same problem [15:06] I'd much rather compute the full output of the preprocesosr and manage just the perprocessed src -> binary cache [15:06] then it is independent of mtime [15:06] (time is generally pesky) [15:06] can you explain how you will do that? [15:07] basically, you are taking control away from apparmor on when and how to use its cache [15:07] jdstrand: perhaps I'm missing something, I'd like to treat apparmor profiles as preprocessed .c files [15:07] jdstrand: and use the cache as a ccache [15:07] and this is an area of intense thought and evolution over the years [15:08] jdstrand: the reason for all of this is that we need to make snappy able to recover from various things, to make it more transactional so to speak [15:08] zyga: the preprocessor doesn't have everything though. it has the rules, but not what the kernel or the parser supports [15:08] jdstrand: and the numer of things that can fail in applying a new set of security permissions is large, we'd like to be able to "go back" to a known consistent state [15:09] zyga: I don't see how having a profile in /var/lib can't be made to work with transactions [15:09] if you "go back" simply invalidate the cache and remove the profiles [15:09] jdstrand: we need to be able to keep arbitrairly many good states around [15:09] jdstrand: and pick the one we want to be in at will [15:09] not for this [15:09] jdstrand: today that's involving rewriting many separate files [15:09] you yourself said it is dynamic [15:10] yes, but the application process can fail [15:10] then remove the file [15:10] you are going to have to write out stuff regardless [15:10] so going from state A to state B can fail and we want to be able to go to A (if possible) or to a new state C with as few error paths as possible [15:10] ie, maintain your preprocessor cache [15:10] easy, always invalidate [15:11] you have to anyway [15:11] you are just moving the problem around [15:11] in ephermeral mode it's easier, nothing stays behind [15:11] not quite [15:11] by the nature of using a cache, you have to do certain things [15:11] I'd like to make "switch" operation impossible to fail [15:11] jdstrand: so I can have two entire sets of security files [15:12] jdstrand: and switch between them with a very primitive operation that has minimal chance of failure [15:12] we can do that for seccomp easily [15:12] I fail to see how that can't be done with using profiles in /var [15:12] sure, there is no compilation for seccomp [15:13] well, we can keep them in /var but that just needs more gardening [15:13] (well, sorta-- the launcher is doing the compile) [15:13] perhaps I'm trying to avoid the inevitable [15:13] right [15:13] udev is more complex, we cannot switch that atomicall [15:13] atomically* [15:13] same with dbus [15:13] zyga: I think this is a really thorny problem that needs very careful thought and there are more pressing matters to attend to [15:13] apparmor we sort of can, we can have 2 profiles loaded and use the launcher to pick the one we want [15:14] jdstrand: but that's the matter we are attending, that's the state engine [15:14] zyga: if you want atomicity, in case of any error, blash everything from /var and the cache files and regenerate everything [15:14] jdstrand: we have to solve this, now [15:14] blast* [15:14] (now as in before 16.04) [15:15] zyga: I fail to see how pushing the problem into the launcher makes the system better [15:15] I don't think that I'm pushing the problem into the launcher [15:15] if the launcher is picking which one, I don't think that is necessarily better [15:16] the fact today is that security changes are not atomic and failure leaves the system in inconsistent state, I'm trying to build something that has better properties [15:16] plus, remember, the profile names are already versioned [15:16] well, the launcher just makes it happen, currently we _tell_ it which to pick [15:16] no [15:16] they are not [15:16] only by snap name [15:16] and app name and revision [15:16] not by grant/revoke iteration count [15:16] so if I'm building state for "after grant" it clobbers some of the state "before grant" [15:17] it seems like I've been out of a few conversations here and that I probably shouldn't have been [15:17] those are just my observations, there are no background conversations on this [15:17] the files in /var are versioned [15:17] the state engine is the big thing that is changing [15:17] currently [15:17] can you tell my what is the version key? [15:18] they are not versioned by things that matter to skills (snap version is not important here) [15:18] pkgname_appname_revision [15:18] ogra_ can i see the yaml file of you application with lighthttpd? [15:18] that's not important here, that's my point [15:18] we aren't talking about skills [15:18] jdstrand: when I say "snap grant foo bar" [15:18] noizer, just go one level up [15:18] we are talking about apparmor caching [15:18] same tree [15:18] zyga: this is why we had .additional files [15:19] that will clobber some of the state, perhaps even before the process is finished (because something failed) we already overwrote the state of that snap-version-app [15:19] jdstrand: how do .additional files improve that? [15:19] zyga: there are other ways to do it without additional files, but they were unversioned and the profile was versioned. the profiles was versioned precisely to cover moving from one state to another [15:19] ie, rollbacks, forwards, whatever [15:20] please differentiate snap updates from changes to granted skills [15:20] in a state where snaps don't change [15:20] skill grants can change [15:20] and that is currently not transactional in any way [15:20] zyga: the additional file was the only thing snappy managed wrt hw-assign [15:20] correct? [15:20] zyga: like I said, that could be changed [15:21] right, okay, sorry if I seem confused sometimes, I'm trying to ensure I still grok the picture as it exists today [15:21] hw-assign is going away [15:21] skills replace it entirely [15:21] yes [15:21] I'm just looking for the best way to do that [15:21] * jdstrand understands that [15:21] and transactionality is big part of that [15:21] (to the extent that snappy is transactional in general) [15:21] zyga: perhaps it would help me if you said exactly what qualities you want for your transactionality [15:22] right, that's a good question [15:22] right now it's somewhat open as a sprint is under way and perhaps new things will happen there [15:23] jdstrand: but what I believe to be true is a world where snappy has a set of compoentns internally [15:23] jdstrand: each component having a way to veto an arbitrary examined state [15:23] jdstrand: one state is effective (as in true on disk and in memory as observed by other processes) [15:23] jdstrand: snappy can try to transition from one state to another by computing a new state [15:23] jdstrand: asking each of the components (managers) to sanitize that state [15:24] jdstrand: perhaps veto it [15:24] jdstrand: or perhaps sanitize it with some modifiations [15:24] jdstrand: then if all of the managers agree, we can apply that state to make it effective [15:24] jdstrand: that can fail as well so we need a way to roll back [15:24] jdstrand: each manager has a different notion of a rollback [15:24] jdstrand: (snaps, assertions and skills) [15:25] jdstrand: but in general it is to return to a state that is consistent, that all managers agree on [15:25] it sounds like the security system may be a manager as well [15:25] yes [15:25] jdstrand: I think so [15:25] let me know when you are done with your synopsis [15:25] jdstrand: currently skills are one such manager but it's just being built so perhaps it will grow [15:26] jdstrand: (ok) for skills my "state" is the set of intents-to-grant [15:26] jdstrand: so set of tuples (snap-skill-name, skill-name, snap-slot-name, slot-name) [15:26] jdstrand: that state is persistent [15:26] jdstrand: at runtime, the manager will try to satisfy those intents by handing out appropriate permissionto apps, running skill hooks [15:27] jdstrand: inspecting hook output and iterating until the system is stable [15:27] jdstrand: the effective state of the system is comprised of the state of the launcher files, systemd units, udev rules, seccomp and apparmor profiles and dbus filters [15:28] are there here people that installed nginx succesfully into a snap? [15:28] (are dbus filters the same as dbus bus policy?) [15:28] jdstrand: I'd like to be able to "apply" and "recover" from a state / to state with little surface for failure as possible [15:28] jdstrand: yes, I'm sorry, I didn't know the right name [15:28] jdstrand: so that's the picture we're trying to build [15:28] ok [15:28] that's all fine [15:29] 09:24 < zyga> jdstrand: then if all of the managers agree, we can apply that state to make it effective [15:30] all you need to do wrt apparmor is before making it 'effective', generate a preprocessed profile, then compare it to /var/lib/snappy/apparmor/profiles (after running it through a preprocessor [15:30] ) [15:30] if they are the same, do nothing (ie, it is effective) [15:31] if they are different, create the new profile in /var/lib/snappy/apparmor/profiles and apply the cache [15:32] you also have to manage the file in /etc/udev and /etc/dbus/system.d (but those are the same as with seccomp, just a file, no cache) [15:33] jdstrand: yes but those are a bit harder to work with, we're racing with things using them, right? [15:33] ubuntu@localhost:~$ sudo snappy update [15:33] Updating ubuntu-core (16.04.0-4.arm64) [15:33] ... [15:33] jdstrand: udev monitors /etc/udev/rules.d/ ? [15:33] which those are you referring to? [15:33] :D [15:33] jdstrand: udev and dbus [15:33] ogra_: wow, so it does work after all :) [15:33] zyga, rebooting ... lets see [15:34] well, that is a different discussion point, but, ideally these things would happen before the service started [15:34] ogra_: same here, updating now [15:34] if it changes while the service is running, the service needs to be notified [15:34] jdstrand: ah, that's a very good observation [15:35] jdstrand: notified how? I was thinking that we really should restart the service after revoke affecting it, perhaps also after grant that fiddled with udev [15:35] ogra_: magic [15:35] geez, shutdown takes half a century [15:35] zyga: a restart would be an excellent default choice [15:35] ogra_: update uses nearly 2W [15:35] jdstrand: that's good to hear, I think that's the best we can do for 16.04 [15:36] zyga: perhaps someday we an app could opt into being notified with a HUP signal (for example) [15:36] s/we an/an/ [15:36] zyga, well, wifi download i guess [15:36] jdstrand: there are two concerns, notifying cooperating services and ensuring that malicious service is not abusing permissins [15:36] *permission [15:36] *permissions [15:36] geez, my keyboard [15:37] yay, update worked [15:37] ha ! and arm64 is the first one to have the libc fix ! [15:37] http://people.canonical.com/~ogra/core-image-stats/20160217.1.changes [15:38] zyga: I'd also like to point out that if an apparmor profile isn't already loaded, you can't apply it to a running process (you can update the profile for a running process). This is part of why we have cache files as well-- the cache files are loaded before any services so they are guaranteed to be running under confinement [15:38] zyga: 'ensuring that malicious service is not abusing permissions' - how are you planning on ensuring that? [15:39] jdstrand: restarting it after changing the profile [15:40] zyga: you mean revocation? [15:40] jdstrand: as for sequencing things in right order I'd have to talk to mvo first, there are some simple things about snaps (activated/deactivated) but activated snaps have systemd units that are more complicated to control, I'd like to ensure that we correctly shut down all the services around transition points [15:40] jdstrand: yes, revocation [15:41] zyga: cause yes if you want revocation of a device to happen, you are going to have to kill the process and have it restart, otherwise an open file handle won't be closed [15:41] jdstrand: but as I said above, perhaps also on grant if a new /dev namespace entry is needed and systemd manages that [15:41] jdstrand: right, that's why I mentioned that [15:41] we aren't using /dev namespaces with systemd [15:41] jdstrand: no revoke() [15:41] to be clear [15:41] sorry, I keep confusing that [15:41] that is something different from what we are currently doing [15:42] so the udev rule does what? [15:42] tags the device [15:42] is that for foreground apps only? [15:42] right, but the launcher is interpreting those tags [15:42] the udev rule tags the device to be used with a particular snap [15:43] the launcher looks up the tags for that snap and if it finds any, adds them to an app-specific device cgroup [15:43] zyga: it is for anything that runs under the launcher [15:43] so all 'apps' (ie, formerly services and binaries) [15:43] jdstrand: ah, so cgroup that's managed by the launcher, I see [15:43] yes [15:44] jdstrand: I haven't looked at background apps yet, are they also using the launcher? [15:44] yes [15:44] ah, great [15:44] look in /etc/systemd/system [15:44] hmm, one concern, can we tag one device to more than one snap? [15:44] all of them use ubuntu-core-launcher as part of the Exec [15:45] that's great, my demo snap (https://github.com/zyga/snappy-pi2-piglow) is not using services yet [15:45] zyga: I believe the launcher can handle that. if it can't today, it could be made to [15:45] jdstrand: ok [15:46] correction, ExecStart [15:46] jdstrand: I think we mostly agree, I have to handle apparmor pre-processor, cache and delta computations [15:46] but yes, all services and commands use the launcher [15:46] jdstrand: but mostly integrate into the state engine being built [15:48] zyga: I'd like to review those changes. I think all this will work fine and we can continue to use caching effectively and avoid slowness [15:48] jdstrand: I will ask you to review all the patches affecting skills [15:48] cool [15:48] jdstrand: and I think we should cooperate more daily [15:49] yes, I'm getting to the point where I think I can do that [15:49] beuno and I went through the backlog of cards and I'm finishing up review tools and store things and then it is skills [15:51] jdstrand: that sounds great, I think we'll have a storm of skills-under-development as soon as the foundations are in place [15:51] jdstrand: it took me ~24 hours to build the i2c skill from scratch plus another 24 hours to build demo snaps [15:51] jdstrand: I expect to see a lot of similar new skills sprout [15:53] zyga: I'm going to have a bunch of stuff to talk about for existing frameworks moving to only skills (which among other things gets into dbus bus policy and what I'll refer to as native security skills) [15:54] jdstrand: we can have a call at any time, I'd love to know what you are building so that we are aligned [15:54] also [15:55] we're discussing a lot of that at the sprint [15:55] so we'll have some updates on what we think the default skills should look like [15:55] zyga: well, not building anything yet. I'm collecting frameworks (ie, helping others use the old system so I can see what we have to deal with on the new) [15:55] more on that soon [15:55] beuno: hey! [15:55] :) [15:55] beuno: hi! [15:55] heya jdstrand! [15:55] o/ zyga [15:55] beuno: tell us, I'm dying of curiosity [15:56] zyga, oh, no (incomplete) spoilers [15:56] beuno: who reviews / approves snaps in the store, and what's the process? Is it something that needs more eyeballs ? [15:56] more importantly, I don't actually have the details [15:56] it's all Gustavo [15:56] beuno: fyi, mvo asked a bunch of questions after I was eod. I have answers. the most important is that it seems you guys were looking at 15.04 caps. 16.04 has the caps that were enumerated in brazil already on the device [15:56] beuno: sure, I understand [15:56] popey, it's usually me or jdstrand, hoping we don't need eyeballs and instead keep automating [15:57] ie, in Brazil we said what security skills should be present. I implemented those as caps [15:57] * beuno nods [15:57] that's great [15:57] jdstrand: I'd like to implement the migration skill as an actuall skill this week [15:57] and what zyga and I were discussing at the end was converting those to skills (among other things) [15:57] jdstrand: so that it's not a special case using the old code [15:58] zyga: it will still look the same to developers? you are just talking about under the hood [15:58] ? [15:58] Hello, on xenial for Pi2 I should enable I2C and SPI support. In raspbian should be like `modprobe i2c-dev / i2c-bcm2708` What's the corresponding in snappy? [15:58] jdstrand: under the hood [15:58] jdstrand: nothing on the outside should change [15:58] beuno: fyi, 'snappy-debug.security list -i' on 16.04 system will show you what is there [15:58] zyga: ack [15:58] jdstrand: but in the end we'll be able to drop code from snappy/ (the package) [15:58] beuno: but I'm going to talk to mvo when he comes online [15:59] (just fyi) [15:59] beuno: ah okay. i put a couple of stupid snaps up, wondered if they'd pass. [15:59] popey, atm, anything targeted at 16.04 will get stuck [15:59] for a brief time, I think we'll fix that really soon [16:00] popey: I've been furiously working on getting 16.04 snaps reviewable by the store. if you update the review tools in trunk, you can have all but security checks [16:00] cool. [16:00] popey, reviewing now [16:00] oh, thanks :) [16:00] there is another branch for those, but don't use it until tomorrow-- it isn't ready yet [16:01] popey: once this lands in the store, 15.04 *and* 16.04 snaps can be reviewed just like clicks [16:01] ie, by any of the reviewers [16:02] popey, so [16:02] https://myapps.developer.ubuntu.com/dev/click-apps/4540/rev/1/ [16:02] + signs in version numbers break our system atm [16:02] cc/ matiasb ^ [16:02] popey, so, I'll need you to re-upload with a non +'s version [16:02] sorry [16:02] we're fixing it, etc [16:03] beuno, ack [16:03] heh, okay [16:04] beuno, fwiw the fix is expected to be deployed during today [16:04] zyga, you have piglow2 waiting to be published [16:04] zyga, are you aware [16:04] * beuno hugs matiasb [16:04] you should thank pindonga, really [16:05] one hug per day, sorry [16:05] you have to hug him now [16:05] heh [16:05] beuno: yep [16:06] beuno: I pushed it to see if I can start having more complete demos working from the store later [16:06] beuno: I'm not blocked by it though [16:06] zyga, it's approved, but not published [16:06] so it won't be accessible [16:06] just an FYI [16:06] beuno: ah, I'll have to check that [16:06] thanks [16:19] beuno: ok, uploaded a new package with no + sign [16:21] elopio: sorry, I have this weird thing where my hangout always seems to crash at the end of a call... which is good timing I guess, but it looks like I cut off at the end [16:21] popey, yeah, beuno's store only supports subtraction, not addition :P [16:22] How can I do a modprobe on snappy ? [16:23] opny, have a look at "sudo snappy config ubuntu-core" [16:23] it supports setting modules to be loaded [16:23] ogra_, oh thanks! [16:25] ogra_, does modprobe expects a yaml list or spaced list? How to R/W config? ... Is there any docs? [16:26] ogra_: that's good, I subtracted loads from those packages :) [16:27] opny, you can just "sudo snappy config ubuntu-core >core.yaml" ... edit the modprobe line and then call: sudo snappy config ubuntu-core core.yaml [16:28] (edit in the core.xaml file it produced indeed) [16:28] *yaml [16:28] ogra_, ok, hope there is some kind validation :) [16:29] ubuntu@localhost:~$ sudo snappy config ubuntu-core |grep modprobe [16:29] modprobe: foo [16:29] not for the content [16:29] :) [16:29] (but there is syntax checking i think) [16:31] ogra_, I put modprobe: "i2c-dev i2c-bcm2708" let's see waht happens [16:31] opny, take a look at /etc/modprobe.d/ubuntu-core.conf afer you ran snappy config with the new yaml [16:32] thats the file where they end up [16:36] * zyga has published his first snap into the store! [16:36] with a vimeo video no less :) [16:52] where can I find information about docker in snappy? [17:56] Hey elopio, you around? [17:56] kyrofa: yes. [17:57] elopio, can you sanity check me and tell me if I'm being stupid? [17:59] kyrofa: :) [17:59] kyrofa: what's your problem? [18:00] elopio, I've been operating under the assumption that, if I ask snapcraft.repo to fetch the same package twice, it'll just say "yup" the second time [18:02] elopio, however, that doesn't seem to be the case. But it doesn't seem to be the case in a very weird way: http://pastebin.ubuntu.com/15101603/ [18:02] Notice first get() call, stuff is fetched. Second get() call, failure. Third get() call, it says "yup" [18:03] (fourth get() call, failure, fifth get() call success, you get the idea) [18:03] I don't understand what's happening there [18:05] kyrofa: hum, I can't reproduce it because there's a hash sum mismatch in the archive. [18:05] let me try in a different machine. [18:06] I think Daniel was the one who touched the cache. But he's away. Let me try to debug and see if I find something useful. [18:08] I just can't explain the success/failure flip-flop [18:21] elopio, ah, I got it [18:22] elopio, it's the consistency check. Since we unmark some packages, the consistency check fails. Since we know it'll fail, I figure disabling the consistency check is probably fine [18:23] kyrofa: makes sense. [18:25] elopio, thanks for taking a look :) [18:40] hey. [19:48] kgunn: are u there? [19:54] jerryG: yes [19:57] Hi all, can someone direct me to any resource that explains how to change the default size of the System-A/B partitions in a snappy appliance (if this is possible)? [20:09] alex_wri, hi. asac, ricmm or ogra might be able to help [20:16] kgunn: any way to hide MIR cursor? [20:16] beuno: Thank you. [20:17] ricmm: Is there a way to change the default partition size of the System-A/B partitions? [20:31] alex_wri: no, its hardcoded [20:31] why? [20:34] ricmm: Well, maybe it's a conceptual issue. I want to deploy a large software package as part of my appliance. This makes the system larger than 1 GB. Can this be deployed through the oem snap? [20:37] jerryG: yes, there is...do you mean the server itself? [20:38] jerryG: might ask in #ubuntun-mir [20:40] jerryG: i'm kinda assuming you mean in the context that you are using mir_demo_server ? [20:40] or something else? [20:41] the server used in the recent snap posts i made, is not the demo server, but a simple server very much like it [20:48] alex_wri, note that the system-a/b partitioning is already obsolete, with 16.04 this has been moved to squashfs images (signed and with 0 free byte) ... you need to package your project as a snap, then you can define the inclusion of this snap in your gadget [20:50] ogra_: Where can I find up to date documentation or information on the new architecture? [20:51] sadly i'm not sue this is properly documented yet ... (it will be once we release 16.04) [20:52] there are images at http://people.canonical.com/~mvo/all-snaps/ in case you want to inspect it though [20:53] in these images everything is a snap ... [20:53] Har har. Btw, those images are getting stale. [20:53] Two weeks old. [20:53] qengho, i'll push new rootfs snaps to the store tomorrow (only updated arm64 today) [20:54] the images auto-update already ;) [20:54] * ogra_ goes back afk [20:54] ogra_: Is there a concept of a snap that has unfettered access to the system by default, or do all permissions need to be explicitly stated and whitelisted? [21:22] wgrant, hey, beuno mentioned you wanted to speak/write to me [21:24] sergiusens: Hi hi. I'm interested in working on a snap for classic. What can you tell me about classic progress, dependencies, etc? === bigcat_ is now known as bigcat [21:34] qengho, classic is totally under mvo's wing; but maybe robert_ancell can provide more insight than myself === chihchun is now known as chihchun_afk === sergiusens_ is now known as sergiusens [23:30] kgunn: yes, that's correct [23:31] jerryG: yeah, so first you can check mir_demo_server --help there might be some args to look at there [23:31] otherwise as in #ubuntu-mir [23:32] duflu will be coming on in a bit...and he's pretty good with that sort of thing [23:32] kgunn: kk thx [23:32] kgunn: hey, while I have you here [23:32] do you have any pointers to that dragonboard demo you mentioned? [23:53] Hi guys are there some people available here? [23:54] idk [23:55] noizer: u might want to try in the morning [23:57] jerryG ok xD [23:57] jerryG see you tomorrow