/srv/irclogs.ubuntu.com/2016/02/17/#snappy.txt

zygajdstrand: hey00:48
zygajdstrand: it works :)00:48
zygajdstrand: all skills based, I'll bug you with details tomorrow :)00:48
zygahttps://github.com/zyga/snappy/tree/skills-demo-i2c <- somewhat hacky, to a small degree00:49
=== timchen1` is now known as timchen119
encryptedchickenhowdy04:04
encryptedchicken i'm trying to install snappy core on i386 and the file is .img and i've tried converting to .iso but it's failing. What gives?04:28
aatchisonX86 as in 32bit?04:50
encryptedchicken6404:56
=== Guest82926 is now known as pitti
=== pitti is now known as Guest48131
=== Guest48131 is now known as pitti
=== pitti` is now known as pitti
=== pitti is now known as Guest54153
=== Guest54153 is now known as pitti
zygagood morning07:45
* zyga has made great progress towards adding lots of more, complex skills07:46
didrockshey zyga07:47
dholbachgood morning07:57
didrocksgood morning dholbach!08:02
=== kickinz1|afk is now known as kickinz1
dholbachsalut didrocks08:08
=== faenil_ is now known as faenil
=== chihchun_afk is now known as chihchun
=== faenil is now known as faenil_
=== faenil_ is now known as faenil
JamesTaitGood morning all!  Happy Wednesday, and happy World Human Spirit Day! 😃09:52
=== chihchun is now known as chihchun_afk
ogra_kyrofa, resize is fixed with the latest kernel snap in the store, yes (you can use mvo's u-d-f to build one or wait til later today when i release a new image)10:22
=== chihchun_afk is now known as chihchun
=== kickinz1 is now known as kickinz1|afk
msvb-outNo snappy images for 15.10, is there a roadmap that states if a 16.04 release will appear?11:32
ogra_msvb-out, snappy is a rolling release, after release of the "normal" 16.04 ubuntu the stable channel will switch to it11:39
TomerHi11:46
TomerI'm running 16.04 snappy core on an RPi211:47
TomerI want to build an app using snapcraft but my laptop outputs an arm64 package, which is not copatible with the RPi2 (ARMv7)11:48
TomerWhat can I do?11:48
ogra_you should use the classic dimension on your rpi to build it11:49
ogra_sudo snappy enable-classic11:49
ogra_snappy shell classic11:49
ogra_then install snapcraft with apt-get, pull your snapcraft.yaml in and build it11:49
Tomerok ill try it, thanks!11:50
asaczyga: do we have a list of skills that we will do yet?11:51
asacor skill types rather11:51
asacwow, my desktop just locks itself after 10 seconds without me touching anything :/11:52
asacwonder if that means i need to restart X :)?11:53
zygaasac: I don't know anything more than last week, sorry11:56
zygaasac: maybe something will be decided at the sprint11:57
zygaasac: I'm not there, I don't know11:57
zygaasac: I explored adding a new skill type this week (i2c) to iron out issues with the security layer11:57
ogra_geez .... will that netsplit ever stop today ?12:00
ogra_asac, you should reboot after the libc bug anyway12:00
asachmm. ok :)12:01
* asac turns down productivity12:01
asacand goes for reboot12:01
* zyga needs to reboot as well, thanks for the tip ogra_ 12:04
ogra_:)12:05
ogra_hmm, seems to not have landed in xenial yet http://people.canonical.com/~ogra/core-image-stats/20160217.changes12:06
zygaogra_: I see it in my apt-get changelog12:07
ogra_yeah, but it landed after the daily image build12:07
zygaglibc (2.21-0ubuntu6) xenial; urgency=medium12:07
popeyhm, what does this mean when I run "snapcraft snap" on a very simple package (figlet):- http://paste.ubuntu.com/15099808/ is the yaml, here's the 'error message' "Command '['/bin/sh', '/tmp/tmpom31_khu', '/bin/sh', '/tmp/tmp6lhcweja']' returned non-zero exit status 1"12:25
popeyit clears up after itself so i can't even tell what /tmp/tmpom31_khu & /tmp/tmp6lhcweja were.12:26
* ogra_ has never used the nil plugin ... 12:27
popeyi used it in my cowsay one and it worked fine12:29
popeydon't understand it12:29
ogra_where would figlet.sh come from ?12:29
ogra_i dotn see it being copied or anything12:29
popeyah12:29
popeyballs, i missed that out12:29
popeygood spot, thanks  ðŸ˜ƒ12:29
ogra_:)12:30
=== yp is now known as ypwong
=== Michaela is now known as Ciblia
msvb-outogra_: Ah, a rolling release. But why didn't the snappy stable channel follow the 15.10 release, was it not 'normal'?12:34
ogra_it was a snapshot of 15.04 to have a non-moving base but has a lot of additional updates from an additional repo12:35
ogra_all new development is in 16.04 though, 15.04 was kind if a "stable pre-release"12:38
ogra_*of12:38
=== retrack is now known as Guest83882
ogra_but we arent really tied to the normal release schedules after all12:38
msvb-outogra_: Okay, that's what I gathered. It would be easier for users if a intuitive release schedule and regular image filenames exist.12:41
=== tbr_ is now known as tbr
msvb-outogra_: But maybe that will follow once a larger userbase is established.12:41
ogra_the release process is totally different for snappy12:41
ogra_a schedule wouldnt make sense12:41
ogra_fixes go in if they are ready and will always show up in the daily rolling builds ...12:42
ogra_also, since you can update apps and os completely independently the os isnt really that important (apart from feature completeness)12:42
ogra_there is some raw schedule though ...12:43
ogra_http://www.olli-ries.com/t-242d/12:45
msvb-outogra_: interesting, hadn't thought of the release philosophy being different. What you say makes sense.12:46
noizerHi guys, I'm trying to install nginx. I do that with deb packages but when it tries to start it killed itself again :s12:46
msvb-outI guess this was more relevant when snappy on ARM builds had no implementation of snappy update.12:46
ogra_yeah, the switch to the all-snaps model in 16.04 will chnage that12:47
ogra_noizer, via stage-packages in your snapcraft.yaml ?12:48
ogra_you most likely need a wrapper script to adjust config and paths etc12:48
kyrofaAh, ogra_ thanks for getting back to me regarding the resize thing :)12:57
kyrofaGood morning #snappy, by the way12:58
ogra_kyrofa, if you want to resize it on your PC http://paste.ubuntu.com/15099918/ might help (adjust DISK to point to your SD reader and run with sudo)12:59
noizeryea with stage packages ogra_12:59
ogra_i'm working that snippet into the initrd today (and clean it up a bit)12:59
ogra_noizer, so if you take a look at the startup script i use for lighttpd ... http://bazaar.launchpad.net/~ogra/+junk/upnp-server/view/head:/run-lighthttpd between line 12 and 44 i create/update the config (you probably dont need that if you have a fixed config pointing to the right dirs) and in line 50 i run lighttpd with adjusted options to run from the snap dir13:06
ogra_you likely want something similar for ngnix fi you use the deb, since the deb defaults wont work for snappy13:07
ogra_(note this is for 15.04 though ... the env vars have changed in 16.04 and you should use the new ones)13:08
* ogra_ triggers a xenial build to get the fixed app-launcher and libc before rolling new ubuntu-core snaps 13:18
noizerhmmmm strange i think i do that13:18
noizerwhen the service is running can you see it with ps -A13:18
noizer?13:18
ogra_yes13:19
kyrofaogra_, yeah good idea13:20
ogra_(RaspberryPi2)ubuntu@localhost:~$ ps ax|grep lighttpd13:23
ogra_ 5373 ?        S      0:00 /apps/upnp-server-armhf.ogra/0.1.0/usr/sbin/lighttpd -D -f /var/lib/apps/upnp-server-armhf.ogra/0.1.0/lighttpd.conf -m /apps/upnp-server-armhf.ogra/0.1.0/usr/lib/lighttpd13:23
ogra_noizer, ^^^13:23
noizerogra_ dammed xD stupid nginx xD13:23
noizerogra_ But kyrofa told me to compile it myself nginx. But i think that would be overkill13:26
zyganoizer: why, this is the best way to integrate13:31
zyganoizer: and if you really care about it, you can have a branch with extra snappy integration features that tracks upstream13:32
zyganoizer: it really is sensible IMHO13:32
ogra_noizer, well, you have twoo options, eiter use the deb and invest all the work to adjust the paths via wrapper or you compile from source and set the right prefixes and paths in there ... either will be fine and both will likely be equally much work13:37
zygaogra_: note; there is no valid value for the prefix as $SNAP_APP_foo is not fixed in stone13:39
zygaogra_: you really have to load and re-loate at runtime13:39
zyga(load environment variables and use them to find stuff)13:40
ogra_well, luckily we still have backwards compatibility13:40
ogra_the old values should all still work afaik13:40
zygaogra_: yeah but I mean that stuff like configure --prefix=/snaps/something is not going to work naively, there is no really good way to configure application that uses prefix to load its data13:41
ogra_well ... --prefix=./ might work though :)13:42
zygaheh :) maybe but I doubt autotools will respect that13:43
ogra_heh13:43
zygathe good news is that patching it is usually quite easy13:43
zygathe bad news is that then you have to carry a patch13:43
* ogra_ is in the deb camp though ... 13:43
kyrofaRight, Snappy is a bit odd in that you need to build in one place, tell it it'll run in another place, and actually run in yet another place :P13:43
ogra_but thats just because i love shell13:43
ogra_i still have to hit my first package where a wrapper doesnt work (there are surely a lot though)13:44
=== ysionnea1 is now known as ysionneau
noizerogra_ when he tries to start now my nginx i got following error: Failed with result 'start-limit'.14:03
ogra_well, no idea what that means ... i never used ngnix14:03
=== JamesTai1 is now known as JamesTait
asackyrofa: master static tests are currently failing?14:20
kyrofaasac, hmm... looks like an lxd error. Restarting14:21
asaczyga: think for the time being you could use an x-plugin to have a hacked autotolls one?14:21
kyrofaasac, to take care of the patching?14:22
asacthe other thing could be to have variable expansion in configflags14:22
asackyrofa: to not require to patch the buggy upstream build system that relies on --prefix for data install instead of also honouring DESTDIR14:22
asacbetter than adding a patch to autotools plugin imo that will auto add prefix to install dir... but thats just my feel14:23
zygaasac: yeah, that works too14:23
zygaasac: I think now you can fake most with a makefile14:24
zygajdstrand: hey, I have a security question for you14:24
asacyou can also make a makefile wrapper for sure14:24
asacbut then thats a bit nasty to do too as you need multiple parts i guess14:24
jdstrandzyga: shoot14:24
zygajdstrand: old hw-assign world ran a chmod so that "granted" file can be world/group read/writable14:24
asacone with local source and one with remote source and then the local one needs to build the one from remote14:24
jdstrandit did?14:25
zygajdstrand: this made sense since those files were off-limits anyway14:25
zygajdstrand: yep,14:25
zygajdstrand: so that at the end of the day a non-root process can open the file14:25
jdstrandI don't think I remember that14:25
zygajdstrand: I can find that, one sec14:25
jdstrandI imagine this was done for something like /dev/video where udev is setting up strict groups and permissions that the non-root user was not part of14:29
jdstrandbut, I don't really like just making the device world writable14:29
ogra_coundnt we hook into udev-acl here ?14:29
ogra_groups are pretty dead everywhere anyway14:30
zygahmmmmmm14:30
jdstrandI don't know what that is, but it sounds promising14:30
zygaI cannot find that yet, that's not rightr14:30
zygaso actually this is quite related14:30
zygaI wrote a hacky i2c skill14:30
zygait allows ioctl (though it's on by default) via seccomp14:31
jdstrandactually, if udev-acl is doing what I think it is, I don't think I like that either14:31
zygait allows open via apparmor on the affected /dev/i2c-1 file14:31
zyganow the last thing I want to understand14:31
zygais systemd /dev namespace and udev14:31
zyga(without the final chmod my demo app didn't have right to open the i2c controller)14:32
ogra_jdstrand, well, afaik it hooks into logind and manages acls based on polkit ( pitti may correct me here, i'm only a consumer usually)14:32
zyganote - this is not a service/framework14:32
zygajdstrand: are we on the same page or am I missing something with regards to things in /dve?14:33
zygain /dev14:33
jdstrandogra_: right, but what I was getting at is that while it might (arguably) be fine to grant the device access to the non-root user within the context of a confined process, either method gives the non-root user access to the devices when it starts processes that aren't confined14:33
ogra_why would anything from a snap start something thats non confined ?14:34
jdstrandzyga: with hw-assign we didn't use a systemd /dev namespace14:34
ogra_(beyond the granted skills)14:34
jdstrandogra_: nothing from a snap would. a login starts an unconfined shell14:34
ogra_but a login also gets me sudo access anyway :)14:35
ogra_(at least currently)14:35
pittiogra_, jdstrand: not polkit, it really adds ACLs to the device node (you can't control file acess with polkit)14:35
zygajdstrand: https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/types/i2c.go14:35
jdstrandogra_: but sudo doesn't equal root. if the login account is hacked, the password is unknown14:35
zygapitti: oh, where do we do that?14:35
ogra_jdstrand, ah, indeed14:36
pittizyga: /lib/udev/rules.d/70-uaccess.rules14:36
jdstrandzyga: so, remember, the way hw-assign works is that isn't just apparmor14:36
ogra_pitti, oh, right, i'm living in the past ... i remember it hooked into consolekit once14:36
pittiogra_: right, that part got replaced with logind which tracks the foreground console, but the ACL part is still the saem14:36
jdstrandzyga: because we didn't want to have to recompile apparmor policy for ever changing device names (until our profile composition feature is finished (not 16.04)14:37
zygapitti: uaccess? is it something like this: https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/types/i2c.go#L9814:37
ogra_ah, k14:37
jdstrand)14:37
pittizyga: they are both tags, yes; snappy-assign was meant for a different purpose, though14:37
jdstrandzyga: so we tag devices via the file pitti mentioned and then the launcher adds any devices that are tagged into a device cgroup for the app14:37
pittizyga: IIRC we do that via apparmor14:37
pittiuaccess means "accessible to the current foreground console"14:37
pittiright, devices cgroup, not apparmor, sorry14:38
pittiit's been a while14:38
jdstrandit *should* be apparmor14:38
jdstrandbut, aforementioned unimplemented feature14:38
zygajdstrand: okay, I actually asked the wrong question up front; I don't want to design the security system; let me re-state the question differently: how should I implement access to files in /dev (e.g.g /dev/i2c-1 here) for snappy skills for 16.04?14:39
jdstrandzyga: the same general method. add them to a device cgroup14:39
jdstrandzyga: via the udev tagging14:40
zygajdstrand: how?14:40
zygajdstrand: like what I linked to?14:40
zygahttps://github.com/zyga/snappy/blob/skills-demo-i2c/skills/types/i2c.go#L9814:40
zygaor something else14:40
jdstrandzyga: something like that, yes14:40
jdstrandzyga: ie, just port the old hw-assign technique to skills14:40
zygajdstrand: that gets written to https://github.com/zyga/snappy/blob/skills-demo-i2c/skills/security.go#L17614:41
jdstrandyes14:41
zygajdstrand: yeah, I'm trying to ensure I didn't miss anything, that old code is somewhat convoluted14:41
zygahmm, so with this i still had to chmod /dev/i2c-114:42
jdstrandzyga: two things need to happen: *if* there is a /dev assignment, create a udev rules file in /etc/udev/rules.d *and* add a /dev/** rwk rule to the apparmor profile if it isn't already there14:42
jdstrandzyga: if there is not a /dev assignment, make sure there are no /etc/udev/rules.d files for the snap and remove the /dev/** rwk apparmor rule14:43
jdstrandzyga: what this is doing is transferring control from apparmor to the devices cgroup for /dev assignement14:43
jdstrandzyga: and giving it back with unassignment14:43
jdstrandzyga: this has nothing to do with device ownership btw14:44
jdstrandin the sense of UNIX perms14:45
zygajdstrand: hmmm, so currently I do both14:45
zygajdstrand: I have the apparmor rule and the /dev "assignment" via udev file14:45
jdstrandzyga: you don't want to do the apparmor part when we release, but it is ok for now14:45
zygajdstrand: so what happens when I drop that today?14:46
jdstrandzyga: this is a boot time optimization14:46
zygajdstrand: I mean, just use /dev assignment14:46
zygajdstrand: (don't worry about boot time, everything will change before 16.04)14:46
jdstrandzyga: which are you dropping?14:46
zygajdstrand: I'm trying to see what to do to make it right, out of the three security systems (seccomp, apparmor, "udev") I can give any kind of security configuration for the i2c skill14:47
zygajdstrand: you just said I should not do apparmor14:47
jdstrandit might be more than boot time-- the point is, that if we assign a usb camera to the snap, every time the camera device name changes, we don't want to have to recompile apparmor policy14:47
davmor2ogra_, dholbach: I've seen this a bunch now but it only just dawned on me to ask, why in snap do you have to do in popey's example cowsay.cowsay to make the command actually run?14:47
zygajdstrand: let's not worry about that, I'm pretty sure we will do a full transition at that time14:47
zygajdstrand: I want to focus on the simple mechanics, optimize later if we can (I suspect we won't)14:47
ogra_davmor2, because there could be a cowsay.ogra and a cowsay.davmor2 too14:48
zygajdstrand: the state engine in snappy will change how many things happe14:48
zyga*happen14:48
jdstrandzyga: if assigning something to a snap from /dev, use the udev method and add a blanket /dev/** rwk apparmor rule. hw-assign does that apparmor rule via an .additional file14:48
ogra_davmor2, snaps actually replace PPAs on a very subtle level ;)14:48
jdstrandzyga: you should do the same14:48
zygajdstrand: ok14:48
davmor2ogra_: so the second is just hte dev namespace right?14:48
ogra_yeah14:48
ogra_or company or whatever14:48
zygajdstrand: we're switching to skills controlling all of the security files, the .additional file doesn't exist in this world tw14:48
zygabtw14:49
jdstrandwhy not?14:49
ogra_davmor2, like: libreoffice.microsoft ;)14:49
zygajdstrand: each app has one apparmor profile, we don't write anything to disk ourselves14:49
popeylulz ogra_14:49
jdstrandzyga: that file makes it so you don't have to recompile the policy all the time14:49
davmor2ogra_: that's evil dude seriously get ready for the letters ;)14:50
zygajdstrand: how does that work if we still have to load the updated profile into the kernel?14:50
jdstrandzyga: you do write out to disk-- or are you saying that you removed /var/lib/snappy/{apparmir,seccomp}/profiles?14:50
zygajdstrand: I kept that but I write fewer files:14:50
zygahttps://github.com/zyga/snappy/blob/skills-demo-i2c/skills/security.go#L5914:51
jdstrandso, to be clear, hw-assign was doing exactly what it needed to wrt /dev files14:51
zygahttps://github.com/zyga/snappy/blob/skills-demo-i2c/skills/security.go#L13814:51
jdstrandno more, no less14:51
davmor2ogra_: so say for example libreoffice.libreoffice is the official snap and then libreoffice.davmor2 is mine with all the QA report templates added right14:51
jdstrandif you are removing a bunch of stuff, we need to discuss that, cause there are some pretty major implications14:51
zygajdstrand: don't assume that what hw-assign did stays the same, snappy is changing in interesting ways, we cannot keep some of the old stuff around14:51
ogra_davmor2, the point is that you can have the same app/binary multiple times installed and compare them, or use feature a from one and b from the other14:52
jdstrandzyga: I don't care about the high level. I'm talking about the lowest level security enforcement14:52
ogra_davmor2, right14:52
zygajdstrand: I can write to other files but it's not exactly clear that we want to; IMHO we'll start with empty security on boot (nothing has extra permissions) and build a fully dynamic changing system as things happen (snaps get activated, skills get granted)14:52
davmor2ogra_: okay cool thanks for that makes a lot more sense now :)14:53
ogra_:)14:53
jdstrandzyga: at the highest level, that may make sense, but if you are dynamically generating apparmor policy all the time, you'll find boot will be slow, assignements will be slow, etc14:53
zygajdstrand: I think this is unavoidable given that the system has to be fully dynamic (skill hooks)14:54
jdstrandzyga: we have to utilize caching and be smart about updating the apparmor policy otherwise the system will feel slow14:54
jdstrandno it doesn't14:54
jdstrandwe can avoid the slowness14:54
jdstrandif you listen to me :)14:54
zygajdstrand: please tell me :)14:54
jdstrandinitial boot from factory, there is nothing14:54
jdstrand(fine)14:54
zygajdstrand: generating the policy is cheap, compiling one is slow but I hope we can cache *that*14:54
jdstrandwrite14:55
jdstrandright*14:55
jdstrandthink of it like this:14:55
jdstrandwrite the policy to /var/lib/snappy/apparmor/profiles, then compile from that and write a cache file from that to that on next load, /var/lib/snappy/apparmor/profiles is used. ok, now you can14:56
jdstrandwhen the skills change or anything else, create a temporary policy generation somewhere14:56
jdstrandthen compare that to what is in /var/lib/snappy/apparmor/profiles14:57
jdstrandif they are the same, do nothing14:57
jdstrandif different, copy to /var/lib/snappy/apparmor/profiles, recompile and update the cache14:57
jdstrandthat is how you can be fully dynamic14:57
zygathat's somewhat similar to what I'm trying to build, I'll maintain a set of compiled policies14:57
jdstrandwithout losing caching benefits14:58
zygaprobably hashed by contents14:58
jdstrandthat does not cover device names that change all the time though (which is where the cgroups come in)14:58
zygamy point was that on boot we'll still do the dynamic bring-up (nothing stays granted after boot as a hook has to decide that), if the decisions are the same we just get cached binaries14:58
jdstrandhashed is fine, point is, just don't overwrite /var/lib/snappy/apparmor/profiles unless things actually change14:58
zygathat's another thing14:59
zygawe may end up changing the directory sturcutre14:59
jdstrandso, if you are regenerating the profile all the time and comparing it, then yes, you don't need an .additional file14:59
zygafor better transactionality, right now it is not transactional at all14:59
jdstrandthe .additional file was just a technique to maintain a few additions rather than snappy having to maintain the profile itself15:00
zygafor apparmor the actual file is not interesting much, just the loaded profile in the kernel, we can keep cache on disk and actual profiles in temporary files that exist just long enough to load the profile into the kernl15:00
zyga*kernel15:00
jdstrandzyga: not quite15:00
zygano?15:00
zygaplease tell15:00
jdstrandno, because what if apparmor itself changes?15:01
zygaapparmor itself?15:01
zygaos-snap update?15:01
jdstrandyes, we add features to it15:01
zygawe reboot15:01
jdstrandyes15:01
jdstrandrules change, etc15:01
zygastart form scratch15:01
zygacache won't be used because binary content will differ15:01
zygaall of security files are ephemeral15:01
zygawe re-construct them (we can) on boot15:01
zygaright?15:02
jdstrandI think that you should maintain the files in /var/lib/snappy/apparmor/profiles15:02
jdstrandthe reason why is you won't know if the cache file changed without recompiled the policy15:02
jdstrandput more simply15:02
zygahmm?15:02
zygayou mean things like <include> ?15:03
jdstrandwhen you generate a cache file, its mtime is later than the policy file it was generated from15:03
jdstrand(and abstractions it includes)15:03
jdstrandonly if the mtime of the profile is later than the cache will the policy be recompiled and the cache updated15:03
jdstrand(or the abstractions it includes)15:04
zygahmmm15:04
zygaI see15:04
jdstrandso if always regenerate dynamically, then your mtime is always newer than the cache and you always recompile15:04
zygaI'll think about it15:04
jdstrandso you must have the files in /var/lib/snappy/apparmor/profiles15:04
zygadoes apparmor have a preprocessor?15:04
jdstrandyes15:04
zygaso that we can just get the full raw output that we want to compile and stick into the kernel15:05
jdstrandand that is what you need to be using with my technique of seeing if the tmp profile and the one in /var/lib/snappy/apparmor/profiles are different15:05
zygathe reason I'm leaning that way is that all of security blobs seem to migrate to tmpfs in the world we're going to15:05
zygais the perprocessor also slow?15:05
zygaor just the compiler?15:05
jdstrandno15:05
jdstrandit doesn't matter though15:06
jdstrandit is the same problem15:06
zygaI'd much rather compute the full output of the preprocesosr and manage just the perprocessed src -> binary cache15:06
zygathen it is independent of mtime15:06
zyga(time is generally pesky)15:06
jdstrandcan you explain how you will do that?15:06
jdstrandbasically, you are taking control away from apparmor on when and how to use its cache15:07
zygajdstrand: perhaps I'm missing something, I'd like to treat apparmor profiles as preprocessed .c files15:07
zygajdstrand: and use the cache as a ccache15:07
jdstrandand this is an area of intense thought and evolution over the years15:07
zygajdstrand: the reason for all of this is that we need to make snappy able to recover from various things, to make it more transactional so to speak15:08
jdstrandzyga: the preprocessor doesn't have everything though. it has the rules, but not what the kernel or the parser supports15:08
zygajdstrand: and the numer of things that can fail in applying a new set of security permissions is large, we'd like to be able to "go back" to a known consistent state15:08
jdstrandzyga: I don't see how having a profile in /var/lib can't be made to work with transactions15:09
jdstrandif you "go back" simply invalidate the cache and remove the profiles15:09
zygajdstrand: we need to be able to keep arbitrairly many good states around15:09
zygajdstrand: and pick the one we want to be in at will15:09
jdstrandnot for this15:09
zygajdstrand: today that's involving rewriting many separate files15:09
jdstrandyou yourself said it is dynamic15:09
zygayes, but the application process can fail15:10
jdstrandthen remove the file15:10
jdstrandyou are going to have to write out stuff regardless15:10
zygaso going from state A to state B can fail and we want to be able to go to A (if possible) or to a new state C with as few error paths as possible15:10
jdstrandie, maintain your preprocessor cache15:10
jdstrandeasy, always invalidate15:10
jdstrandyou have to anyway15:11
jdstrandyou are just moving the problem around15:11
zygain ephermeral mode it's easier, nothing stays behind15:11
zyganot quite15:11
jdstrandby the nature of using a cache, you have to do certain things15:11
zygaI'd like to make "switch" operation impossible to fail15:11
zygajdstrand: so I can have two entire sets of security files15:11
zygajdstrand: and switch between them with a very primitive operation that has minimal chance of failure15:12
zygawe can do that for seccomp easily15:12
jdstrandI fail to see how that can't be done with using profiles in /var15:12
jdstrandsure, there is no compilation for seccomp15:12
zygawell, we can keep them in /var but that just needs more gardening15:13
jdstrand(well, sorta-- the launcher is doing the compile)15:13
zygaperhaps I'm trying to avoid the inevitable15:13
zygaright15:13
zygaudev is more complex, we cannot switch that atomicall15:13
zygaatomically*15:13
zygasame with dbus15:13
jdstrandzyga: I think this is a really thorny problem that needs very careful thought and there are more pressing matters to attend to15:13
zygaapparmor we sort of can, we can have 2 profiles loaded and use the launcher to pick the one we want15:13
zygajdstrand: but that's the matter we are attending, that's the state engine15:14
jdstrandzyga: if you want atomicity, in case of any error, blash everything from /var and the cache files and regenerate everything15:14
zygajdstrand: we have to solve this, now15:14
jdstrandblast*15:14
zyga(now as in before 16.04)15:14
jdstrandzyga: I fail to see how pushing the problem into the launcher makes the system better15:15
zygaI don't think that I'm pushing the problem into the launcher15:15
jdstrandif the launcher is picking which one, I don't think that is necessarily better15:15
zygathe fact today is that security changes are not atomic and failure leaves the system in inconsistent state, I'm trying to build something that has better properties15:16
jdstrandplus, remember, the profile names are already versioned15:16
zygawell, the launcher just makes it happen, currently we _tell_ it which to pick15:16
zygano15:16
zygathey are not15:16
zygaonly by snap name15:16
jdstrandand app name and revision15:16
zyganot by grant/revoke iteration count15:16
zygaso if I'm building state for "after grant" it clobbers some of the state "before grant"15:16
jdstrandit seems like I've been out of a few conversations here and that I probably shouldn't have been15:17
zygathose are just my observations, there are no background conversations on this15:17
jdstrandthe files in /var are versioned15:17
zygathe state engine is the big thing that is changing15:17
jdstrandcurrently15:17
zygacan you tell my what is the version key?15:17
zygathey are not versioned by things that matter to skills (snap version is not important here)15:18
jdstrandpkgname_appname_revision15:18
noizerogra_ can i see the yaml file of you application with lighthttpd?15:18
zygathat's not important here, that's my point15:18
jdstrandwe aren't talking about skills15:18
zygajdstrand: when I say "snap grant foo bar"15:18
ogra_noizer, just go one level up15:18
jdstrandwe are talking about apparmor caching15:18
ogra_same tree15:18
jdstrandzyga: this is why we had .additional files15:18
zygathat will clobber some of the state, perhaps even before the process is finished (because something failed) we already overwrote the state of that snap-version-app15:19
zygajdstrand: how do .additional files improve that?15:19
jdstrandzyga: there are other ways to do it without additional files, but they were unversioned and the profile was versioned. the profiles was versioned precisely to cover moving from one state to another15:19
jdstrandie, rollbacks, forwards, whatever15:19
zygaplease differentiate snap updates from changes to granted skills15:20
zygain a state where snaps don't change15:20
zygaskill grants can change15:20
zygaand that is currently not transactional in any way15:20
jdstrandzyga: the additional file was the only thing snappy managed wrt hw-assign15:20
zygacorrect?15:20
jdstrandzyga: like I said, that could be changed15:20
zygaright, okay, sorry if I seem confused sometimes, I'm trying to ensure I still grok the picture as it exists today15:21
zygahw-assign is going away15:21
zygaskills replace it entirely15:21
jdstrandyes15:21
zygaI'm just looking for the best way to do that15:21
* jdstrand understands that15:21
zygaand transactionality is big part of that15:21
zyga(to the extent that snappy is transactional in general)15:21
jdstrandzyga: perhaps it would help me if you said exactly what qualities you want for your transactionality15:21
zygaright, that's a good question15:22
zygaright now it's somewhat open as a sprint is under way and perhaps new things will happen there15:22
zygajdstrand: but what I believe to be true is a world where snappy has a set of compoentns internally15:23
zygajdstrand: each component having a way to veto an arbitrary examined state15:23
zygajdstrand: one state is effective (as in true on disk and in memory as observed by other processes)15:23
zygajdstrand: snappy can try to transition from one state to another by computing a new state15:23
zygajdstrand: asking each of the components (managers) to sanitize that state15:23
zygajdstrand: perhaps veto it15:24
zygajdstrand: or perhaps sanitize it with some modifiations15:24
zygajdstrand: then if all of the managers agree, we can apply that state to make it effective15:24
zygajdstrand: that can fail as well so we need a way to roll back15:24
zygajdstrand: each manager has a different notion of a rollback15:24
zygajdstrand: (snaps, assertions and skills)15:24
zygajdstrand: but in general it is to return to a state that is consistent, that all managers agree on15:25
jdstrandit sounds like the security system may be a manager as well15:25
zygayes15:25
zygajdstrand: I think so15:25
jdstrandlet me know when you are done with your synopsis15:25
zygajdstrand: currently skills are one such manager but it's just being built so perhaps it will grow15:25
zygajdstrand: (ok) for skills my "state" is the set of intents-to-grant15:26
zygajdstrand: so set of tuples (snap-skill-name, skill-name, snap-slot-name, slot-name)15:26
zygajdstrand: that state is persistent15:26
zygajdstrand: at runtime, the manager will try to satisfy those intents by handing out appropriate permissionto apps, running skill hooks15:26
zygajdstrand: inspecting hook output and iterating until the system is stable15:27
zygajdstrand: the effective state of the system is comprised of the state of the launcher files, systemd units, udev rules, seccomp and apparmor profiles and dbus filters15:27
noizerare there here people that installed nginx succesfully into a snap?15:28
jdstrand(are dbus filters the same as dbus bus policy?)15:28
zygajdstrand: I'd like to be able to "apply" and "recover" from a state / to state with little surface for failure as possible15:28
zygajdstrand: yes, I'm sorry, I didn't know the right name15:28
zygajdstrand: so that's the picture we're trying to build15:28
jdstrandok15:28
jdstrandthat's all fine15:28
jdstrand09:24 < zyga> jdstrand: then if all of the managers agree, we can apply that state to  make it effective15:29
jdstrandall you need to do wrt apparmor is before making it 'effective', generate a preprocessed profile, then compare it to /var/lib/snappy/apparmor/profiles (after running it through a preprocessor15:30
jdstrand)15:30
jdstrandif they are the same, do nothing (ie, it is effective)15:30
jdstrandif they are different, create the new profile in /var/lib/snappy/apparmor/profiles and apply the cache15:31
jdstrandyou also have to manage the file in /etc/udev and /etc/dbus/system.d (but those are the same as with seccomp, just a file, no cache)15:32
zygajdstrand: yes but those are a bit harder to work with, we're racing with things using them, right?15:33
ogra_ubuntu@localhost:~$ sudo snappy update15:33
ogra_Updating ubuntu-core (16.04.0-4.arm64)15:33
ogra_...15:33
zygajdstrand: udev monitors /etc/udev/rules.d/ ?15:33
jdstrandwhich those are you referring to?15:33
ogra_:D15:33
zygajdstrand: udev and dbus15:33
zygaogra_: wow, so it does work after all :)15:33
ogra_zyga, rebooting ... lets see15:33
jdstrandwell, that is a different discussion point, but, ideally these things would happen before the service started15:34
zygaogra_: same here, updating now15:34
jdstrandif it changes while the service is running, the service needs to be notified15:34
zygajdstrand: ah, that's a very good observation15:34
zygajdstrand: notified how? I was thinking that we really should restart the service after revoke affecting it, perhaps also after grant that fiddled with udev15:35
zygaogra_: magic15:35
ogra_geez, shutdown takes half a century15:35
jdstrandzyga: a restart would be an excellent default choice15:35
zygaogra_: update uses nearly 2W15:35
zygajdstrand: that's good to hear, I think that's the best we can do for 16.0415:35
jdstrandzyga: perhaps someday we an app could opt into being notified with a HUP signal (for example)15:36
jdstrands/we an/an/15:36
ogra_zyga, well, wifi download i guess15:36
zygajdstrand: there are two concerns, notifying cooperating services and ensuring that malicious service is not abusing permissins15:36
zyga*permission15:36
zyga*permissions15:36
zygageez, my keyboard15:36
ogra_yay, update worked15:37
ogra_ha ! and arm64 is the first one to have the libc fix !15:37
ogra_http://people.canonical.com/~ogra/core-image-stats/20160217.1.changes15:37
jdstrandzyga: I'd also like to point out that if an apparmor profile isn't already loaded, you can't apply it to a running process (you can update the profile for a running process). This is part of why we have cache files as well-- the cache files are loaded before any services so they are guaranteed to be running under confinement15:38
jdstrandzyga: 'ensuring that malicious service is not abusing permissions' - how are you planning on ensuring that?15:38
zygajdstrand: restarting it after changing the profile15:39
jdstrandzyga: you mean revocation?15:40
zygajdstrand: as for sequencing things in right order I'd have to talk to mvo first, there are some simple things about snaps (activated/deactivated) but activated snaps have systemd units that are more complicated to control, I'd like to ensure that we correctly shut down all the services around transition points15:40
zygajdstrand: yes, revocation15:40
jdstrandzyga: cause yes if you want revocation of a device to happen, you are going to have to kill the process and have it restart, otherwise an open file handle won't be closed15:41
zygajdstrand: but as I said above, perhaps also on grant if a new /dev namespace entry is needed and systemd manages that15:41
zygajdstrand: right, that's why I mentioned that15:41
jdstrandwe aren't using /dev namespaces with systemd15:41
zygajdstrand: no revoke()15:41
jdstrandto be clear15:41
zygasorry, I keep confusing that15:41
jdstrandthat is something different from what we are currently doing15:41
zygaso the udev rule does what?15:42
jdstrandtags the device15:42
zygais that for foreground apps only?15:42
zygaright, but the launcher is interpreting those tags15:42
jdstrandthe udev rule tags the device to be used with a particular snap15:42
jdstrandthe launcher looks up the tags for that snap and if it finds any, adds them to an app-specific device cgroup15:43
jdstrandzyga: it is for anything that runs under the launcher15:43
jdstrandso all 'apps' (ie, formerly services and binaries)15:43
zygajdstrand: ah, so cgroup that's managed by the launcher, I see15:43
jdstrandyes15:43
zygajdstrand: I haven't looked at background apps yet, are they also using the launcher?15:44
jdstrandyes15:44
zygaah, great15:44
jdstrandlook in /etc/systemd/system15:44
zygahmm, one concern, can we tag one device to more than one snap?15:44
jdstrandall of them use ubuntu-core-launcher as part of the Exec15:44
zygathat's great, my demo snap (https://github.com/zyga/snappy-pi2-piglow) is not using services yet15:45
jdstrandzyga: I believe the launcher can handle that. if it can't today, it could be made to15:45
zygajdstrand: ok15:45
jdstrandcorrection, ExecStart15:46
zygajdstrand: I think we mostly agree, I have to handle apparmor pre-processor, cache and delta computations15:46
jdstrandbut yes, all services and commands use the launcher15:46
zygajdstrand: but mostly integrate into the state engine being built15:46
jdstrandzyga: I'd like to review those changes. I think all this will work fine and we can continue to use caching effectively and avoid slowness15:48
zygajdstrand: I will ask you to review all the patches affecting skills15:48
jdstrandcool15:48
zygajdstrand: and I think we should cooperate more daily15:48
jdstrandyes, I'm getting to the point where I think I can do that15:49
jdstrandbeuno and I went through the backlog of cards and I'm finishing up review tools and store things and then it is skills15:49
zygajdstrand: that sounds great, I think we'll have a storm of skills-under-development as soon as the foundations are in place15:51
zygajdstrand: it took me ~24 hours to build the i2c skill from scratch plus another 24 hours to build demo snaps15:51
zygajdstrand: I expect to see a lot of similar new skills sprout15:51
jdstrandzyga: I'm going to have a bunch of stuff to talk about for existing frameworks moving to only skills (which among other things gets into dbus bus policy and what I'll refer to as native security skills)15:53
zygajdstrand: we can have a call at any time, I'd love to know what you are building so that we are aligned15:54
beunoalso15:54
beunowe're discussing a lot of that at the sprint15:55
beunoso we'll have some updates on what we think the default skills should look like15:55
jdstrandzyga: well, not building anything yet. I'm collecting frameworks (ie, helping others use the old system so I can see what we have to deal with on the new)15:55
beunomore on that soon15:55
zygabeuno: hey!15:55
beuno:)15:55
jdstrandbeuno: hi!15:55
beunoheya jdstrand!15:55
beunoo/ zyga15:55
zygabeuno: tell us, I'm dying of curiosity15:55
beunozyga, oh, no (incomplete) spoilers15:56
popeybeuno: who reviews / approves snaps in the store, and what's the process? Is it something that needs more eyeballs ?15:56
beunomore importantly, I don't actually have the details15:56
beunoit's all Gustavo15:56
jdstrandbeuno: fyi, mvo asked a bunch of questions after I was eod. I have answers. the most important is that it seems you guys were looking at 15.04 caps. 16.04 has the caps that were enumerated in brazil already on the device15:56
zygabeuno: sure, I understand15:56
beunopopey, it's usually me or jdstrand, hoping we don't need eyeballs and instead keep automating15:56
jdstrandie, in Brazil we said what security skills should be present. I implemented those as caps15:57
* beuno nods15:57
beunothat's great15:57
zygajdstrand: I'd like to implement the migration skill as an actuall skill this week15:57
jdstrandand what zyga and I were discussing at the end was converting those to skills (among other things)15:57
zygajdstrand: so that it's not a special case using the old code15:57
jdstrandzyga: it will still look the same to developers? you are just talking about under the hood15:58
jdstrand?15:58
opnyHello, on xenial for Pi2  I should enable I2C and SPI support. In raspbian should be like `modprobe i2c-dev / i2c-bcm2708` What's the corresponding in snappy?15:58
zygajdstrand: under the hood15:58
zygajdstrand: nothing on the outside should change15:58
jdstrandbeuno: fyi, 'snappy-debug.security list -i' on 16.04 system will show you what is there15:58
jdstrandzyga: ack15:58
zygajdstrand: but in the end we'll be able to drop code from snappy/ (the package)15:58
jdstrandbeuno: but I'm going to talk to mvo when he comes online15:58
jdstrand(just fyi)15:59
popeybeuno: ah okay. i put a couple of stupid snaps up, wondered if they'd pass.15:59
beunopopey, atm, anything targeted at 16.04 will get stuck15:59
beunofor a brief time, I think we'll fix that really soon15:59
jdstrandpopey: I've been furiously working on getting 16.04 snaps reviewable by the store. if you update the review tools in trunk, you can have all but security checks16:00
popeycool.16:00
beunopopey, reviewing now16:00
popeyoh, thanks :)16:00
jdstrandthere is another branch for those, but don't use it until tomorrow-- it isn't ready yet16:00
jdstrandpopey: once this lands in the store, 15.04 *and* 16.04 snaps can be reviewed just like clicks16:01
jdstrandie, by any of the reviewers16:01
beunopopey, so16:02
beunohttps://myapps.developer.ubuntu.com/dev/click-apps/4540/rev/1/16:02
beuno+ signs in version numbers break our system atm16:02
beunocc/ matiasb ^16:02
beunopopey, so, I'll need you to re-upload with a non +'s version16:02
beunosorry16:02
beunowe're fixing it, etc16:02
matiasbbeuno, ack16:03
popeyheh, okay16:03
matiasbbeuno, fwiw the fix is expected to be deployed during today16:04
beunozyga, you have piglow2 waiting to be published16:04
beunozyga, are you aware16:04
* beuno hugs matiasb 16:04
matiasbyou should thank pindonga, really16:04
beunoone hug per day, sorry16:05
beunoyou have to hug him now16:05
matiasbheh16:05
zygabeuno: yep16:05
zygabeuno: I pushed it to see if I can start having more complete demos working from the store later16:06
zygabeuno: I'm not blocked by it though16:06
beunozyga, it's approved, but not published16:06
beunoso it won't be accessible16:06
beunojust an FYI16:06
zygabeuno: ah, I'll have to check that16:06
zygathanks16:06
popeybeuno: ok, uploaded a new package with no + sign16:19
plarselopio: sorry, I have this weird thing where my hangout always seems to crash at the end of a call... which is good timing I guess, but it looks like I cut off at the end16:21
ogra_popey, yeah, beuno's store only supports subtraction, not addition :P16:21
opnyHow can I do a modprobe on snappy ?16:22
ogra_opny, have a look at "sudo snappy config ubuntu-core"16:23
ogra_it supports setting modules to be loaded16:23
opnyogra_, oh thanks!16:23
opnyogra_, does modprobe expects a yaml list or spaced list? How to R/W config?   ...   Is there any docs?16:25
popeyogra_: that's good, I subtracted loads from those packages :)16:26
ogra_opny, you can just "sudo snappy config ubuntu-core >core.yaml" ... edit the modprobe line and then call: sudo snappy config ubuntu-core core.yaml16:27
ogra_(edit in the core.xaml file it produced indeed)16:28
ogra_*yaml16:28
opnyogra_, ok, hope there is some kind validation :)16:28
ogra_ubuntu@localhost:~$ sudo snappy config ubuntu-core |grep modprobe16:29
ogra_    modprobe: foo16:29
ogra_not for the content16:29
ogra_:)16:29
ogra_(but there is syntax checking i think)16:29
opnyogra_,  I put modprobe: "i2c-dev i2c-bcm2708" let's see waht happens16:31
ogra_opny, take a look at /etc/modprobe.d/ubuntu-core.conf afer you ran snappy config with the new yaml16:31
ogra_thats the file where they end up16:32
* zyga has published his first snap into the store!16:36
zygawith a vimeo video no less :)16:36
noizerwhere can I find information about docker in snappy?16:52
kyrofaHey elopio, you around?17:56
elopiokyrofa: yes.17:56
kyrofaelopio, can you sanity check me and tell me if I'm being stupid?17:57
zygakyrofa: :)17:59
elopiokyrofa: what's your problem?17:59
kyrofaelopio, I've been operating under the assumption that, if I ask snapcraft.repo to fetch the same package twice, it'll just say "yup" the second time18:00
kyrofaelopio, however, that doesn't seem to be the case. But it doesn't seem to be the case in a very weird way: http://pastebin.ubuntu.com/15101603/18:02
kyrofaNotice first get() call, stuff is fetched. Second get() call, failure. Third get() call, it says "yup"18:02
kyrofa(fourth get() call, failure, fifth get() call success, you get the idea)18:03
kyrofaI don't understand what's happening there18:03
elopiokyrofa: hum, I can't reproduce it because there's a hash sum mismatch in the archive.18:05
elopiolet me try in a different machine.18:05
elopioI think Daniel was the one who touched the cache. But he's away. Let me try to debug and see if I find something useful.18:06
kyrofaI just can't explain the success/failure flip-flop18:08
kyrofaelopio, ah, I got it18:21
kyrofaelopio, it's the consistency check. Since we unmark some packages, the consistency check fails. Since we know it'll fail, I figure disabling the consistency check is probably fine18:22
elopiokyrofa: makes sense.18:23
kyrofaelopio, thanks for taking a look :)18:25
magizianhey.18:40
jerryGkgunn: are u there?19:48
kgunnjerryG: yes19:54
alex_wriHi all, can someone direct me to any resource that explains how to change the default size of the System-A/B partitions in a snappy appliance (if this is possible)?19:57
beunoalex_wri, hi. asac, ricmm or ogra might be able to help20:09
jerryGkgunn: any way to hide MIR cursor?20:16
alex_wribeuno: Thank you.20:16
alex_wriricmm: Is there a way to change the default partition size of the System-A/B partitions?20:17
ricmmalex_wri: no, its hardcoded20:31
ricmmwhy?20:31
alex_wriricmm: Well, maybe it's a conceptual issue. I want to deploy a large software package as part of my appliance. This makes the system larger than 1 GB. Can this be deployed through the oem snap?20:34
kgunnjerryG: yes, there is...do you mean the server itself?20:37
kgunnjerryG: might ask in #ubuntun-mir20:38
kgunnjerryG: i'm kinda assuming you mean in the context that you are using mir_demo_server ?20:40
kgunnor something else?20:40
kgunnthe server used in the recent snap posts i made, is not the demo server, but a simple server very much like it20:41
ogra_alex_wri, note that the system-a/b partitioning is already obsolete, with 16.04 this has been moved to squashfs images (signed and with 0 free byte) ... you need to package your project as a snap, then you can define the inclusion of this snap in your gadget20:48
alex_wriogra_: Where can I find up to date documentation or information on the new architecture?20:50
ogra_sadly i'm not sue this is properly documented yet ... (it will be once we release 16.04)20:51
ogra_there are images at http://people.canonical.com/~mvo/all-snaps/ in case you want to inspect it though20:52
ogra_in these images everything is a snap ...20:53
qenghoHar har. Btw, those images are getting stale.20:53
qenghoTwo weeks old.20:53
ogra_qengho, i'll push new rootfs snaps to the store tomorrow (only updated arm64 today)20:53
ogra_the images auto-update already ;)20:54
* ogra_ goes back afk20:54
alex_wriogra_: Is there a concept of a snap that has unfettered access to the system by default, or do all permissions need to be explicitly stated and whitelisted?20:54
sergiusenswgrant, hey, beuno mentioned you wanted to speak/write to me21:22
qenghosergiusens: Hi hi. I'm interested in working on a snap for classic. What can you tell me about classic progress, dependencies, etc?21:24
=== bigcat_ is now known as bigcat
sergiusensqengho, classic is totally under mvo's wing; but maybe robert_ancell can provide more insight than myself21:34
=== chihchun is now known as chihchun_afk
=== sergiusens_ is now known as sergiusens
jerryGkgunn: yes, that's correct23:30
kgunnjerryG: yeah, so first you can check mir_demo_server --help there might be some args to look at there23:31
kgunnotherwise as in #ubuntu-mir23:31
kgunnduflu will be coming on in a bit...and he's pretty good with that sort of thing23:32
jerryGkgunn: kk thx23:32
ricmmkgunn: hey, while I have you here23:32
ricmmdo you have any pointers to that dragonboard demo you mentioned?23:32
noizer_Hi guys are there some people available here?23:53
jerryGidk23:54
jerryGnoizer:  u might want to try in the morning23:55
noizer_jerryG ok xD23:57
noizer_jerryG see you tomorrow23:57

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!