[00:12] <Chipaca> ogra_: noice
[08:02] <dholbach> good morning
[08:17] <fgimenez> good morning
[08:23] <mvo> hey fgimenez, good morning!
[08:24] <mvo> ogra_: this armhf build failure is quite annoying, it takes forever to run the tests on a porter box
[08:24] <fgimenez> hi mvo!
[08:27] <fgimenez> mvo, btw with the new cloud resources in place we should redirect the github webhook, let me know when you have a minute and i'll send you the new ip
[08:32] <mvo> fgimenez: sure, I have a minute
[08:32] <mvo> fgimenez: trying to figure out an armhf build failure right now
[08:36] <fgimenez> mvo, once i finish fixing the snapd integration tests we should have green builds, i'll fire up builds every 15 minutes to check
[08:37] <mvo> fgimenez: sweet. and then we can add a user that leaves comments too, right?
[08:39] <fgimenez> mvo, yep, we should create a new github user and add it to the organization as collaborator, then add its credentials to jenkins, and that's it :)
[10:10] <Chipaca> jdstrand: could we add 'less' to the list of things we can exec when confined?
[10:17] <JamesTait> Good morning all! Happy Giving Tuesday, and happy Bifocals at the Monitor Liberation Day! 😃
[10:24] <Chipaca> JamesTait: monitors had a liberation day, and bifocals were present, and it was an event that needs commemorating?
[10:24] <Chipaca> JamesTait: the mind is boggled
[10:28] <JamesTait> Chipaca, it's a tribute to those poor souls who wear bifocals and use a monitor daily, and keep bobbing their heads up and down trying to figure out which lens to use.
[10:29] <Chipaca> yeah. fear that day.
[10:29] <Chipaca> i wouldn't be able to look at the monitor with my head tilted, letting ideas settle around the clot
[10:46] <Chipaca> longsleep: ping
[11:09] <longsleep> Chipaca: pong
[11:13] <longsleep> Chipaca: i added a hack to u-d-f, moving the services to writable - i know it is crappy but works for now http://paste.ubuntu.com/13597040/
[11:14] <Chipaca> longsleep: heh. i'm working on that right now (on the snappy side)
[11:14] <Chipaca> broke all the tests of course
[11:14] <longsleep> Chipaca: you are moving the stuff to first boot now?
[11:14] <Chipaca> longsleep: the ping was to ask you for your oem package.yaml, if that was shareable
[11:14] <Chipaca> longsleep: yes
[11:14] <longsleep> Chipaca: ok good
[11:15] <Chipaca> longsleep: making install not activate from u-d-f (u-d-f passes in a particular flag), then making firstboot activate them
[11:15] <longsleep> Chipaca: uhm sure i can share the oem snap, though the snaps are not in the store
[11:15] <Chipaca> yep, no worries about that
[11:16] <Chipaca> and feel free to munge the names if they are sensitive
[11:16] <longsleep> Chipaca: http://paste.ubuntu.com/13597081/
[11:16] <Chipaca> just looking for a real-world example, as what i've got access to is either too bare, or theoretical
[11:16] <Chipaca> perfect
[11:16] <longsleep> Chipaca: yeah, this is what we currently use
[11:52] <pindonga> jdstrand, hi there, there is no schedule... which revno do you want to release? I'll do my best to get it out asap
[12:01] <fgimenez> hi Chipaca, ive got a question about http.chipaca
[12:01] <Chipaca> fgimenez: shoot
[12:02] <fgimenez> Chipaca, it doesn't let me send files http://paste.ubuntu.com/13597723/, am i doing it right?
[12:03] <fgimenez> Chipaca, this is the response with debug enabled http://paste.ubuntu.com/13597757/
[12:06] <Chipaca> fgimenez: it can't read /tmp/pp, because it'll have its own private /tmp
[12:06] <Chipaca> fgimenez: two questions:
[12:06] <Chipaca> fgimenez: first, what's in /tmp/pp?
[12:07] <fgimenez> Chipaca, nothing, just touched it for trying, the real test tries to use a snap file from ADT_ARTIFACTS, under /tmp
[12:08] <fgimenez> Chipaca, i can try to move it to wherever http needs
[12:08] <Chipaca> fgimenez: would `cat /tmp/pp | sudo http.POST snapd:///1.0/packages` work?
[12:09] <Chipaca> otherwise, put it under $SNAP_APP_DATA_PATH
[12:09] <Chipaca> i.e., /var/lib/apps/http.chipaca/current/
[12:09] <fgimenez> Chipaca, yep, works fine :) ok, i'll try putting it there, thanks!
[12:10] <Chipaca> pipes are easy from the shell, but tricky in a test, so
[14:59] <fgimenez> Chipaca, is it also possible to pass http a json string literal, as in http://paste.ubuntu.com/13600302/ ?
[15:00] <Chipaca> fgimenez: yes, but
[15:03] <Chipaca> fgimenez: https://github.com/jkbrzt/httpie#request-items
[15:06] <Chipaca> fgimenez: so in particular for doing a multi-config PUT,
[15:07] <Chipaca> gah. gimme a bit.
[15:07] <jdstrand> Chipaca: re less, sure
[15:08] <Chipaca> jdstrand: whee :-)
[15:08] <jdstrand> Chipaca: when do you need it?
[15:08] <Chipaca> jdstrand: less has exec-this-other-thing, but that'll just fail because confinement, right?
[15:08] <Chipaca> jdstrand: no hurry, backburner project needs it (or me reimplementing a pager in python ;-) )
[15:08] <jdstrand> Chipaca: it will fail to exec the other thing, yes
[15:09]  * Chipaca will run it with that feature disable, because usability++, but still
[15:11] <Chipaca> fgimenez: sudo http.PUT snapd:///1.0/packages 'ubuntu-core.ubuntu:="config: {ubuntu-core: {autoupdate: true}}"'
[15:11] <Chipaca> fgimenez: note the quotes in the quotes
[15:13] <dholbach> sergiusens, so you're landing 0.6 soon? :)
[15:14] <Chipaca> fgimenez: the thing on the right of the := is a json string literal (hence the double quotes there). Of YAML.
[15:14] <sergiusens> dholbach, yeah
[15:14] <Chipaca> not the nicest of things, but ¯\_(ツ)_/¯
[15:14] <sergiusens> jdstrand, is there a cap I can use for capname="net_admin" on 15.04?
[15:15] <fgimenez> Chipaca, great, in my case 'ubuntu-core.ubuntu:="config: {basic-config.sideload: {key: value}}"' prevents the 400 and creates the operation, i'll put that into the test, thanks!
[15:16] <Chipaca> ummm
[15:16] <Chipaca> fgimenez: that should kinda not work
[15:16] <Chipaca> that is, the operation should fail
[15:16] <Chipaca> because ubuntu-core.ubuntu mismatch with basic-config.sideload
[15:16] <dholbach> sergiusens, awesome :)
[15:17] <Chipaca> fgimenez: (yes, config is redundant like that; needs revamp)
[15:18] <fgimenez> Chipaca, yes i'm getting "invalid ubuntu-core configuration", ok, then 'basic-config.sideload:="config: {basic-config.sideload: {key: value}}"', right?
[15:18] <elopio> plars: are you here?
[15:20] <plars> elopio: sort of, feeling pretty sick today. What's up?
[15:20] <fgimenez> Chipaca, yes, that works fine, thanks :)
[15:20] <elopio> plars: I need your help to collect the SPI output. But I can wait, lets talk tomorrow, get some rest.
[15:20] <elopio> plars: I hope you get better soon.
[15:21] <Chipaca> fgimenez: \o/ :)
[15:21] <jdstrand> sergiusens: yes. either network-admin or network-firewall
[15:21] <sergiusens> jdstrand, thanks
[15:22] <jdstrand> sergiusens: are you aware of 'sudo snappy-debug.security scanlog' :)
[15:22] <jdstrand> ?
[15:22] <plars> elopio: no, it's fine. was this from a job that just ran?
[15:22] <sergiusens> jdstrand, I wasn't, no I am :-)
[15:22] <elopio> plars: latest ran yesterday midnight. Want me to run another one?
[15:22] <jdstrand> Chipaca: ok, you are getting less{,file,pipe} and more
[15:23] <Chipaca> jdstrand: whee :-)
[15:24] <elopio> mvo: do you have time to talk about the godd snap?
[15:24] <plars> elopio: I think I found it, one sec
[15:25] <sergiusens> jdstrand, are all three suggestions to be followed? http://paste.ubuntu.com/13600633/
[15:26] <jdstrand> sergiusens: no. pick one
[15:26] <sergiusens> jdstrand, I already put network-admin in there, which is why I ask
[15:27] <jdstrand> sergiusens: is that a new denial or a previous denial?
[15:27] <sergiusens> jdstrand, it is a fresh boot
[15:28] <jdstrand> sergiusens: I'm guessing that /var/lib/snappy/seccomp/profiles/gopaste.sideload_... doesn't reflect the change
[15:28] <mvo> elopio: yes
[15:28] <plars> elopio: https://pastebin.canonical.com/145186/plain/
[15:28] <jdstrand> sergiusens: or it didn't at the time it started
[15:28] <Chipaca> mvo: Kalnischkiesfile
[15:29] <jdstrand> sergiusens: check if setsockopt is in /var/lib/snappy/seccomp/profiles/gopaste.sideload_...
[15:29] <Chipaca> mvo: in https://mvogt.wordpress.com/2015/11/30/apt-1-1-released/ in english
[15:29] <sergiusens> jdstrand, it is not
[15:29] <plars> elopio: that seems to be all that came through
[15:30] <jdstrand> sergiusens: how did you install the snap?
[15:30] <sergiusens> jdstrand, `snapcraft run` :-)
[15:30] <elopio> plars: so it's running! but being killed.
[15:31] <mvo> Chipaca: hm? did I misspell something somewhere?
[15:31] <mvo> Chipaca: in that blog post?
[15:31] <Chipaca> mvo: yes
[15:31] <Chipaca> mvo: I don't know if it's misspelt; it's just not very englishy
[15:31] <plars> elopio: kick off another one and I'll see if it does anything different
[15:31] <jdstrand> sergiusens: what is in the generated meta/package.yaml on the device?
[15:32] <elopio> mvo: I hw-assigned /dev/null to the snapp but I am still getting:
[15:32] <elopio> godd.godd
[15:32] <elopio> /apps/godd.sideload/current/godd_1.0_amd64.snap /dev/null
[15:32] <elopio> failed to dd: open /proc/self/mountinfo: permission denied
[15:32] <elopio> what else should I give to the snap to be able to run the bin?
[15:32] <sergiusens> jdstrand, http://paste.ubuntu.com/13600718/ this file looks strange though
[15:32] <mvo> Chipaca: haha now I see what you mean, this is actually kind of funny
[15:34] <elopio> plars: triggered.
[15:34] <jdstrand> sergiusens: what is the version of ubuntu-core-security-seccomp on the device?
[15:34] <mvo> elopio: hm, can you he-assign /proc/self/mountinfo? just for fun and testing?
[15:35] <mvo> elopio: it needs this to prevent you from shooting yourself in the food
[15:35] <sergiusens> jdstrand, hah, hard to know given the dpkg cache is wiped
[15:35] <sergiusens> mvo, do you know? ^
[15:35] <mvo> elopio: I think for this to be a proper snap it needs a customized appamor profile, i.e. it needs to be able to write to /dev/** by default
[15:36] <mvo> sergiusens: latest rolling? I can check on the builds. or what version of the image do you use?
[15:36] <sergiusens> mvo, 15.04 stable
[15:36] <mvo> sergiusens: we have the dpkg info on 15.04 :)
[15:36] <jdstrand> sergiusens: acutally, that shouldn't matter. snappy-debug.security looks at what is installed on the device and if it is suggesting network-admin, then network-admin should have it
[15:36] <sergiusens> mvo, oh right
[15:38] <jdstrand> sergiusens: how does snapcraft run work? is it just doing a snappy install of a generated snap or is it doing something else?
[15:38] <sergiusens> jdstrand, it scp's the snap and installs it
[15:38] <jdstrand> sergiusens: can you give me the snap?
[15:38] <sergiusens> jdstrand, with `snappy install`
[15:38] <sergiusens> jdstrand, sure
[15:39] <sergiusens> jdstrand, http://people.canonical.com/~sergiusens/snappy/gopaste_1.0_amd64.snap
[15:39] <elopio> mvo: I can't hw-assign mountinfo. It says invalid hardware device.
[15:39] <jdstrand> sergiusens: btw, you are almost certainly hitting https://bugs.launchpad.net/snappy/+bug/1465724 and don't actually need network-admin
[15:40] <mvo> elopio: aha, indeed, there is a check for this
[15:41] <jdstrand> (snappy-debug.security should mention that, but you ran it after you added the cap I think)
[15:41] <mvo> elopio: hm, then we need to update the security profile of this to make it work on snappy itself. whats you use-case (just curious)?
[15:42] <elopio> mvo: there is a godd snap in the snapcraft examples. I want to run the binary and check the output.
[15:42] <Chipaca> jdstrand: is /proc/self/mountinfo unreadable for a reason?
[15:42] <sergiusens> jdstrand, for every snap I build I restart the vm though
[15:43]  * Chipaca is not trying to ddos jdstrand 
[15:43] <sergiusens> unless the log persists, which I think it doesn't, it shouldn't be the root cause
[15:43] <sergiusens> mvo, we just want the examples in snapcraft to actually run and not just build ;-)
[15:44] <jdstrand> Chipaca: yes, it is an information leak
[15:44] <Chipaca> aww
[15:44] <jdstrand> sergiusens: I can confirm this. it is a bug in snappy-debug.security
[15:44] <Chipaca> mvo: so i guess godd needs fixing to not need that
[15:45] <sergiusens> jdstrand, how so?
[15:45] <jdstrand> sergiusens: it is suggesting network-admin and network-admin doesn't have setsockopt
[15:46] <jdstrand> sergiusens: use network-service or network-client
[15:47] <sergiusens> jdstrand, ah, thanks; will try and report back
[15:47] <mvo> sergiusens, elopio: let me look at this
[15:48] <jdstrand> oh heh
[15:48] <jdstrand> silly bug
[15:48] <jdstrand> fix is easy. here is the output:
[15:48] <jdstrand> Syscall: setsockopt
[15:48] <jdstrand> Suggestions:
[15:48] <jdstrand> * add 'setsockopt' to seccomp policy
[15:48] <jdstrand> * add one of 'network-client, network-firewall, network-service' to 'caps'
[15:48] <jdstrand> * add 'setsockopt' to seccomp file in 'security-policy'
[15:48] <elopio> thanks mvo
[15:48] <jdstrand> sergiusens: ^
[15:50] <sergiusens> jdstrand, yay, no I have a defunct process, let's see why :-)
[15:51] <plars> elopio: my bbb tests on rolling seem to be failing since last week when I was away also. Looks like the last one that passed was 237
[15:53] <plars> elopio: well, maybe... that one might be a fluke. Out of curiosity, do you have it saving your output to a location that you cleanup? or is it still a tmpdir?
[15:54] <elopio> plars: I'm using the pwd to put the resulting files, and mktemp -d to get the code. I delete the temp dir on exit.
[15:55] <mvo> elopio, sergiusens: sorry, looks like I can not easily fix it, snapcraft does not understand the new security-override syntax it seems
[15:55] <sergiusens> jdstrand, oh goodie, I now ran into the sqlite fchown issue ;-)
[15:56] <mvo> sergiusens: it appears that https://github.com/mvo5/godd/commit/38dfb37c34db7c1314ee1fe6a730c53565de9914 is making snapcraft unhappy
[15:56] <sergiusens> jdstrand, is that going to be allowed soon or should I fix it in the snap
[15:56] <jdstrand> how is it that everything is using sqlite all of a sudden...
[15:56] <plars> elopio: well, there was no output from it that time, and the dir has already been cleaned up so I can't see anything useful from it
[15:57] <elopio> mvo: oh, I was thinking this was a bad snapcraft example because it wasn't showing anything useful. But now with the security override it's interesting.
[15:57] <jdstrand> sergiusens: it should be fixed in the snap or someone should fix it in the archive
[15:57] <plars> elopio: all I can see is that it exited with rc=1
[15:57] <sergiusens> mvo, notice that this version of snapcraft does not play nice at all with 16.04 syntax
[15:57] <elopio> plars: but looking at this output you pasted, it seems that something is killing my script, right?
[15:58] <plars> elopio: not unless you are, or it times out and spi is killing it or something
[15:58] <plars> elopio: that last one exited with the same rc
[15:59] <elopio> sergiusens: we need to support pinning, to keep the sources in a known good version.
[15:59] <plars> elopio: it's possible that I just don't have *all* the output. Getting output from the script this way is not really the best. It's more intended for logging messages from provisioning rather than as a debugging tool for your scripts
[16:00] <elopio> plars: hum, that's possible. There is a lot of output.
[16:00] <sergiusens> elopio, you can pin with source-tag, source-branch to some extent
[16:00] <sergiusens> mvo, you seem to be using 16.04 syntax
[16:00] <jdstrand> sergiusens: fyi, snappy-debug 0.7 fixes the issue and is in the store
[16:00] <plars> elopio: have you tried running this from a bbb locally to make sure it works?
[16:00] <elopio> plars: yes.
[16:01] <elopio> I have made many tiny changes trying to collect some output, so I will try again. But it's great news to see that the tests are starting.
[16:02] <sergiusens> jdstrand, so for fchown it is not going to be allowed and I need to fix it with a security.override?
[16:02] <sergiusens> s/./-/
[16:02] <fgimenez> elopio, it seems that we don't have access to the store (or the internet for that matter) from the instances http://10.55.32.74:8080/job/github-snappy-integration-tests-cloud/10/consoleFull
[16:02] <elopio> plars: the troubling situation is that this should be bullet proof now. It doesn't have -e, so it won't stop the execution on failure. And I do  > "$pwd"/output.txt 2>&1 which should collect any possible error.
[16:02] <elopio> that's where I'm lost. I don't understand why I get no output at all.
[16:03] <fgimenez> elopio, Get https://system-image.ubuntu.com//ubuntu-core/rolling/edge/generic_amd64/index.json: dial tcp 91.189.88.142:443: i/o timeout
[16:03] <jdstrand> tyhicks: hey, so, chown seems to be coming up all the time now
[16:03] <elopio> fgimenez: yes, I bet the scalingstack firewall is too strick. We need it open for a lot of things. Would they get mad if we just ask to remove it? :)
[16:04] <mvo> sergiusens: is that bad :) ?
[16:04] <tyhicks> jdstrand: chown to what?
[16:04] <mvo> sergiusens: aha, ok. I see that it is bad, I will use a different syntax
[16:04] <plars> elopio: let me take a look
[16:04] <sergiusens> mvo, no, it just won't work
[16:04] <ogra_> sergiusens, is that sqlite ?
[16:04] <jdstrand> tyhicks: like, 4 times in a week. our stance has been that we won't allow it until we have per-app uids
[16:04] <sergiusens> ogra_, yes
[16:04] <ogra_> sergiusens, if so, i have a patch ;)
[16:04] <tyhicks> jdstrand: right
[16:04] <jdstrand> tyhicks: sqlite is doing a superfluous chown
[16:04] <plars> elopio: what do you get on the spi side? do you get your results json uploaded there?
[16:04] <jdstrand> tyhicks: chown root:root /path/to/db
[16:04] <ogra_> jdstrand, it isnt actually superfluous
[16:04] <jdstrand> tyhicks: and it is getting killed
[16:05] <elopio> plars: no. empty result_payload
[16:05] <jdstrand> ogra_: it is, cause the db is already root:root
[16:05] <ogra_> jdstrand, it does that to make sure the db file matches the master file ... with the UID the master file was created with
[16:05] <fgimenez> elopio, :) with 91.189.88.142:443 will do for now IMO, we are just accessing the store?
[16:05] <ogra_> so it is valuable in non-snappy
[16:05] <jdstrand> and the master file will be root:root
[16:05] <elopio> fgimenez: aren't we accessing also docker servers?
[16:05] <jdstrand> I understand the non-snappy case and why they are doing it
[16:05] <ogra_> but since it tries to chown to a user thats not you usually it only does that when run as root
[16:06] <jdstrand> I'm talking about the snappy case, it is superflous because it does the chown unconditionally whether it needs it or not
[16:06] <ogra_> right
[16:06] <jdstrand> anyway
[16:06] <sergiusens> elopio, https://github.com/ubuntu-core/snapcraft/pull/130
[16:06] <jdstrand> tyhicks: so all of a sudden everyone is hot on sqlite
[16:06] <fgimenez> elopio, no, that's from canonistack, where the infra lives. the instances just need the store
[16:07] <jdstrand> tyhicks: and I'm getting pings left and right on this
[16:07] <elopio> fgimenez: great. Then yes, please make an RT to request opening the firewall to the store.
[16:07] <jdstrand> tyhicks: I've suggested we fix sqlite in the archive so that it does the chown conditionally
[16:08] <ogra_> sergiusens, http://paste.ubuntu.com/13601278/
[16:08] <jdstrand> tyhicks: but that hasn't happened yet and that doesn't fix vivid and doesn't fix something else that might do the same
[16:08] <ogra_> sergiusens, for the upstream sqlite tarball ...
[16:08] <mvo> elopio: ok, can you try now? I updated the git tree to use the unconfined template
[16:08] <ogra_> sergiusens, or just pull the binaries out of my upnp-server packages if you are lazy ;)
[16:09] <sergiusens> ogra_, I am lazier, a lot lazier
[16:09] <jdstrand> tyhicks: I've been thinking that when we have syscall arg filtering, we will allow 'chown 0' (or whatever the syntax) in addition to 'chown 12345' for per-app, if/when we do that
[16:09] <elopio> mvo: trying.
[16:09] <ogra_> (after all you only need the sqlite3 executable and the lib)
[16:09] <ogra_> sergiusens, bah
[16:09] <ogra_> hacking security ! so evil :P
[16:09] <jdstrand> tyhicks: per-app aside, when we have syscall arg filtering, the 'chown 0' one would fix sqlite
[16:10] <jdstrand> tyhicks: but that won't happen until after the new year
[16:10] <plars> elopio: ok, there seem to be some garbage characters in the output. I wonder if that's why things are getting a bit messed up. It's definitely failing though
[16:10] <plars> elopio: https://pastebin.canonical.com/145191/
[16:10] <jdstrand> tyhicks: on rolling, there is the saying things can break at any time
[16:11] <jdstrand> tyhicks: I've been reluctant to allow chown in the seccomp policy since that opens the floodgates to all uids, only to reign it in again later and potentially break stuff
[16:11] <jdstrand> tyhicks: but maybe that is actually ok for 16.04
[16:12] <jdstrand> tyhicks: and 15.04 is simply what it is
[16:12] <elopio> mvo: still permission denied on mountinfo.
[16:12] <elopio> now I didn't have to hw-assign /dev/null
[16:13] <jdstrand> tyhicks: more clearly-- what do you think of me adding chown family to 16.04 policy now with a TODO to adjust when seccomp arg filtering exists?
[16:13] <tyhicks> jdstrand: if an app can chown in its data directory, what does that mean for this policy:
[16:13] <tyhicks>   # Writable home area for this version.
[16:13] <tyhicks>   owner @{HOMEDIRS}/*/apps/@{APP_PKGNAME}/@{APP_VERSION}/   w,
[16:13] <tyhicks>   owner @{HOMEDIRS}/*/apps/@{APP_PKGNAME}/@{APP_VERSION}/** wl,
[16:13] <plars> elopio: here's the json it spits out for spi to get, which is not valid and would probably fail for several reasons: http://paste.ubuntu.com/13601335/
[16:13] <tyhicks> jdstrand: do we drop owner?
[16:13] <mvo> elopio: hm, thats confusing, let me dive deeper
[16:13] <elopio> plars: right. Let me pastebinit.
[16:14] <tyhicks> jdstrand: that's from the default template
[16:14] <jdstrand> yes, I recognize the rule
[16:14]  * tyhicks is trying to think through what it means to allow chown
[16:15] <jdstrand> fyi, this is exactly the sort of thing I have been saying could cause unintended interactions
[16:15] <jdstrand> and hence my reluctance
[16:15]  * tyhicks agrees
[16:15] <jdstrand> tyhicks: that rule is actually terribly problematic, but that is another story
[16:16] <elopio> mvo: this is not urgent. I'm just trying to make sense of the examples.
[16:16] <jdstrand> tyhicks: the launcher is really not smart about that directory and will create ~/apps/... as the root user
[16:16] <mvo> elopio: hm, my latest git seems to work in my snappy now, what git revno is the one you have?
[16:16] <jdstrand> and then running stuff as non-root is broken
[16:16] <mvo> elopio: just did "snapcraft assemble --force" and copied it to a snappy system
[16:16] <jdstrand> but that is another issue (there is a bug for that)
[16:16] <elopio> mvo: 6901dc912162b05fa381ead0e6d344152a68bbe9
[16:16] <jdstrand> I think we *must* keep the owner checks there
[16:17] <elopio> I did snapcraft clean and snapcraft. Copied it to 15.04 stable #10.
[16:17] <tyhicks> jdstrand: I guess sqlite is not chowning files in that directory?
[16:17] <jdstrand> tyhicks: it is doing it in /var/lib/apps
[16:17] <tyhicks> ah
[16:18] <mvo> elopio: thanks, let me try that too. the version looks correct
[16:18] <jdstrand> well, I'm assuming. everyone is talking about services
[16:18] <jdstrand> and the services should be using SNAP_APP_DATA_PATH
[16:18] <jdstrand> not SNAP_APP_USER_DATA_PATH
[16:19] <sergiusens> jdstrand, correct
[16:20] <sergiusens> mvo, elopio we use git upstream, but our example tree has its own snapcraft.yaml
[16:20] <jdstrand> tyhicks: adding the chown now would mean that an app could perform a DoS on the SNAP_APP_USER_DATA_PATH
[16:20] <jdstrand> tyhicks: but I think that will be true when we add 'chown 0'
[16:20] <jdstrand> hrm
[16:20]  * sergiusens will bbl
[16:20] <tyhicks> jdstrand: but that DoS is only for itself
[16:21] <tyhicks> jdstrand: it wouldn't affect other snaps
[16:21] <jdstrand> for itself but for another user
[16:21] <tyhicks> oh, right
[16:21] <tyhicks> that's a problem
[16:21] <ogra_> doesnt the launch wrapper prevent you from that ?
[16:22] <jdstrand> that will be true of 'chown 0' and 'chown 12345' too though
[16:22] <jdstrand> when we have syscall arg filtering
[16:22] <ogra_> (how would you mangle SNAP_APP_USER_DATA_PATH to be for another user ? )
[16:22] <ogra_> (given the launcher will set it after you touched it)
[16:23] <tyhicks> ogra_: a malicious app doesn't have to use SNAP_APP_USER_DATA_PATH, it would be able to chown any path it chooses
[16:23] <ogra_> ah
[16:23] <jdstrand> ogra_: you wouldn't-- but you can read /etc/passwd (necessarily) to get the paths of other users
[16:23] <ogra_> well, /var/lib/extrausers/passwd but yeah, i get what you mean :)
[16:24] <ogra_> (/etc7passwd is rather useless)
[16:24] <jdstrand> right. both are readable in the default policy (they must be)
[16:24] <ogra_> yeah
[16:25] <tyhicks> jdstrand: I think that I need some more time to think about this
[16:26] <jdstrand> tyhicks: I think to properly support @{HOME} with chown we need fine-grained mediation of chown in apparmor and something more than 'owner', or a kernel side var to expand @{HOME} in some manner
[16:26] <jdstrand> tyhicks: I agree. this is the sort of thing I have been saying is problematic with chown
[16:28] <jdstrand> ogra_, sergiusens: I'm fairly certain we aren't going to just add chown to the policy any time soon. we may be able to fix it this cycle. I suggest someone update sqlite in the archive to make the check conditional
[16:28] <jdstrand> tyhicks: fyi ^
[16:28] <ogra_> well, i'm just using upstreams tarball now ...
[16:28] <ogra_> would be nice if snapcraft had a concept of patching though ... so i dont need to build in two steps
[16:28] <jdstrand> most people will surely opt to snapcraft the deb though, no?
[16:29] <ogra_> yeah
[16:29] <ogra_> well my hackish patch from above is definitely not suited for the package
[16:29] <Chipaca> jdstrand: tyhicks: so we need to change things so that SNAP_APP_USER_DATA_PATH is under the home of the effective user, right?
[16:29] <ogra_> (it blindly just rips out all chown calls)
[16:29] <Chipaca> ogra_: using ld_preload?
[16:30] <ogra_> Chipaca, for what ?
[16:30] <Chipaca> ogra_: for ripping out the chown calls
[16:30] <Chipaca> well, replacing them
[16:30] <jdstrand> not for sqlite
[16:30] <ogra_> well, sqlite consiste of two binary files ... i just drop them in the right place with the copy plugin
[16:31] <tyhicks> Chipaca: he patched out the calls to chown(2)
[16:31] <ogra_> usually i have some debs involved so i get the wrapper setup that gives me LD_PRELOAD anyway
[16:31] <ogra_> but having to patch the code means i need to build it separately from the snapcraft run
[16:31] <jdstrand> Chipaca: the problem with the launcher is that $HOME of the real user is being used for the euid of 0 under sudo
[16:32] <Chipaca> jdstrand: yes
[16:32] <jdstrand> Chipaca: which is why we get root owned dirs in $HOME
[16:32] <Chipaca> yes
[16:32] <jdstrand> this isn't about fixing all that, though that is an incredibly annoying bug
[16:32] <Chipaca> http.chipaca comes up against this a bit
[16:32] <Chipaca> do you know the bug number offhand?
[16:32] <jdstrand> let me find the bug
[16:32] <Chipaca> heh
[16:32] <jdstrand> I'll find it
[16:33] <jdstrand> https://bugs.launchpad.net/snappy/+bug/1466234
[16:33] <jdstrand> Chipaca: ^
[16:34] <jdstrand> tedg suggests in comment #2 that we shouldn't set SNAP_APP_USER_DATA_PATH for root
[16:35] <Chipaca> yes, but agreed with setting it because there are valid reasons for using it
[16:35] <tedg> Root isn't a user :-)
[16:35] <jdstrand> Chipaca: that is one idea. someone told me to not include /root in the policy, which is why we don't now. another option is setting SNAP_APP_USER_DATA_PATH to SNAP_APP_DATA_PATH
[16:36] <Chipaca> wrt stgraber, I suspect this was running as root, not with sudo
[16:36] <jdstrand> when running as root
[16:37] <jdstrand> Chipaca: I think the lxc command is a binary in the lxd package
[16:37]  * jdstrand checks
[16:37] <jdstrand> Chipaca: yes, it is. so he used 'sudo lxc' at the time
[16:39] <jdstrand> plus, only the launcher does the mkdir, not the services (unless they wrote the service to do it)
[16:39] <jdstrand> well, actually, scratch that
[16:40] <jdstrand> I forgot for a moment the luancher is used by the systemd service
[16:42] <jdstrand> so, from a traditional security perspective, it is probably wrong to have a process running under sudo to read files from the real user's home directory, so the current behavior of setting SNAP_APP_USER_DATA_PATH to the real user's home is wrong
[16:42] <jdstrand> tyhicks: what do you think? ^ (different topic from chown-- looking at 1466234 now)
[16:44] <tyhicks> just a minute - I'm in a meeting
[16:46] <jdstrand> oh, I'm supposed to be in that meeting
[16:46] <jdstrand> actually, I'm optional there
[16:47] <tyhicks> jdstrand: it just ended
[16:47] <tyhicks> jdstrand: yes, I dislike the idea of root mucking with files in the user's home dir
[16:47] <jdstrand> me too
[16:47] <jdstrand> Chipaca: ^
[16:48] <tyhicks> jdstrand: with chmod being allowed now, there's nothing stopping services from dropping setuid root binaries in a user's home dir :/
[16:48] <jdstrand> Chipaca: I think the choice is to either not set SNAP_APP_USER_DATA_PATH when running as root or set it to SNAP_APP_DATA_PATH
[16:48] <Chipaca> jdstrand: not to /root/apps/yadda/yadda?
[16:49] <jdstrand> well, that is another choice, yes. I was explictly told *not* to allow that, and I asked in the bug for why, and got no response
[16:50] <jdstrand> asac: hey, do you recall why we didn't want to allow the launcher to create /root/apps/pkgname/version ?
[16:50] <jdstrand> mvo: do you? ^
[16:51] <ogra_> is /root writable ?
[16:51] <jdstrand> it is these days
[16:51] <ogra_> ah, i see
[16:51] <jdstrand> it is in /etc/system-image/writable-paths
[16:52] <jdstrand> it may not have always been
[16:52] <ogra_> yeah, i think it hasnt in the past
[16:52] <jdstrand> I don't know how this would fit into all snaps either
[17:06] <elopio> plars: can you check once again, please?
[17:08] <plars> elopio: http://paste.ubuntu.com/13602227/
[17:09] <elopio> plars: nice!
[17:09] <elopio> now I don't understand why the payload is still empty, but things are improving.
[17:11] <elopio> plars: can you please install the subunit package?
[17:14] <plars> elopio: done
[17:14] <elopio> thanks
[17:21] <sergiusens> elopio, can you check my last comment on https://github.com/ubuntu-core/snapcraft/pull/130
[17:24] <elopio> sergiusens: how could it be running on 80 if you are calling it with 8080}
[17:24] <elopio> ?
[17:25] <elopio> I don't get a defunct service.
[17:30] <sergiusens> elopio, just run `sudo netstat -putan` (as a spanish speaker it is hard to forget those netstat switches :-P)
[17:30] <sergiusens> elopio, I might have messed up the flags, that is why I ask ;-)
[17:31] <elopio> sergiusens: it's not in 80, nor in 8080. But snappy service status says it's active and running.
[17:32] <sergiusens> elopio, snappy service is wrong; check `ps -ef|grep gopaste`
[17:32] <kyrofa> sergiusens, pretty sure they're order independent :P
[17:32] <elopio> sergiusens: yes, defunct.
[17:32] <elopio> sergiusens: I agree that it's better than before, fwiw.
[17:32] <jdstrand> Chipaca: ok, typos aside, I laid everything out in https://bugs.launchpad.net/snappy/+bug/1466234/comments/6
[17:33] <sergiusens> elopio, let's merge this then and I'll think of how to get sqlite working
[17:33] <sergiusens> kyrofa, they are; but you know how it goes :-)
[17:33] <jdstrand> Chipaca: I requested an architect comment, so hopefully we can get a direction on the bug
[17:34] <jdstrand> Chipaca: one problem with /root/apps is upgrades/roolbacks
[17:34] <Chipaca> jdstrand: why?
[17:34] <sergiusens> elopio, ok, I'm preparing the changelog again
[17:35] <sergiusens> Chipaca, btw, if a service goes defunct, systemd/snappy still think it is in a good state
[17:35] <sergiusens> we are probably missing a service toggle
[17:35] <Chipaca> sergiusens: depends how it goes defunct
[17:36] <Chipaca> sergiusens: as per the table in systemd.unit
[17:36] <Chipaca> sorry, no, systemd.service
[17:36] <sergiusens> Chipaca, oh, it sort of just gives up on living due to the jail apparmor and seccomp created for it
[17:36] <Chipaca> sergiusens: man systemd.service, look for 'table 1'
[17:36] <sergiusens> :-)
[17:40] <elopio> our examples are depressed :(
[17:43] <ogra_> elopio, no worries, we'll give them free psycotherapy vouchers
[17:43] <elopio> we need an eliza snap :)
[17:44] <ogra_> +1 !
[17:44]  * ogra_ actually wants two and have them play nethack against each other ;)
[17:45] <kgunn> hey sergiusens, getting an error with snapcraft
[17:45] <kgunn> https://pastebin.canonical.com/145201/
[17:46] <kgunn> what's strange, is...i'm pretty sure this worked and i got a snap with no errors...
[17:46] <kgunn> so not really sure what happened..
[17:47] <sergiusens> kgunn, oh, 0.6 fixes that and I am releasing right now
[17:47] <ogra_> kgunn, i get that when i crate a file like "config.sh" with no content
[17:47] <ogra_> (i.e. no shebang)
[17:47] <sergiusens> kgunn, if in a hurry git clone https://github.com/ubuntu-core/snapcraft.git
[17:48] <sergiusens> kgunn, then $PATH_TO_SNAPCRAFT_GIT/bin/snapcraft
[17:48] <sergiusens> elopio, https://github.com/ubuntu-core/snapcraft/pull/127
[17:50] <elopio> sergiusens: go for it. And hopefully no 0.7 branch anytime soon.
[18:03]  * Chipaca suddenly realises there's a hole in the cloud, and strikes it rich by offering "service as a service"
[18:03] <Chipaca> sergiusens: btw, snapcraft is still broken wrt things not in the first plane
[18:03] <Chipaca> sergiusens: because pyyaml is broken that way
[18:04] <sergiusens> Chipaca, you talking about encoding I guess
[18:04] <Chipaca> sergiusens: yes
[18:04] <Chipaca> sergiusens: just saw you close bug 1518150
[18:04] <sergiusens> Chipaca, shouldn't that work?
[18:05] <Chipaca> sergiusens: here, try: 𠀡
[18:05] <sergiusens> Chipaca, hah, xchat is also broken :P
[18:06] <Chipaca> sergiusens: hit ctrl+shift+u, then type 20021, then hit enter or space
[18:06] <sergiusens> Chipaca, the particular bug report is fixed though
[18:06] <sergiusens> Chipaca, I just downloaded the snapcraft.yaml from librarian and tried
[18:07] <Chipaca> sergiusens: ok. But there's a lot of cjk codepoints, not all of them in the first plane
[18:07] <sergiusens> Chipaca, yeah, that is broken
[18:07] <Chipaca> sergiusens: and pyyaml incorrectly reports things in higher planes as "special characters"
[18:07] <sergiusens> Chipaca, yeah, seems like it
[18:08] <sergiusens> Chipaca, I'll add a task for pyyaml on l
[18:08] <sergiusens> lp
[18:08] <Chipaca> it's broken in some subtle way; the spec says some codepoints are not allowed, and these characters utf8 starts with the byte sequence that corresponds to those characters codepoints
[18:08] <Chipaca> i.e. this character in utf8 is 0xF0 0xA0 0x80 0xA1
[18:09] <Chipaca> *codepoint* 0xF0 is invalid
[18:09] <Chipaca> no, wait, that's wrong also
[18:10] <Chipaca> "The allowed character range explicitly excludes the surrogate block #xD800-#xDFFF, DEL #x7F, the C0 control block #x0-#x1F (except for #x9, #xA, and #xD), the C1 control block #x80-#x9F, #xFFFE, and #xFFFF."
[18:10] <Chipaca> I have no idea why it thinks U+20021, 0xF0 0xA0 0x80 0xA1, is invalid
[18:10] <sergiusens> Chipaca, care to add that to the bug?
[18:11] <Chipaca> point me at the bug
[18:11] <sergiusens> Chipaca, https://bugs.launchpad.net/ubuntu/+source/pyyaml/+bug/1518150
[18:11] <Chipaca> and i'll paste random facts at it until somebody fixes it :-p
[18:11] <sergiusens> Chipaca, yeah, lets just bug barry ;-)
[18:11]  * barry hides
[18:14] <Chipaca> barry: sergiusens: there, https://bugs.launchpad.net/snapcraft/+bug/1518150/comments/9
[18:14] <barry> Chipaca: i'll look later.  i'm on a deep bug hunt right now
[18:15] <Chipaca> barry: yeh, i don't think this is urgent, but it'll byte somebody at some point :-)
[18:15] <sergiusens> barry, you took too long, the bugs now hunt and HAUNT you ;-)
[18:16] <barry> sergiusens: bugception
[18:17] <Chipaca> ok, stopping now. barry, if i write a patch for this thing, should it be against xenial, or upstream?
[18:19] <barry> Chipaca: i don't know upstream too well, but if the xenial version is equiv to latest upstream, probably xenial will be fine (as quilt patch) and i'll submit it upstream
[18:38] <jdstrand> Chipaca: re why> /home/$user/apps/... is (presumably) already handled by upgrades and rollbacks since it was always in the design. /root is not (currently) integrated into that, but would have to be if we supported /root/apps
[18:38] <jdstrand> Chipaca: it isn't that it can't be done, it is whether or not it should be done
[18:49] <elopio> look plars, we have output again! :) http://paste.ubuntu.com/13602944/
[18:49] <elopio> thanks, without you I would have never noticed it was because of the subunit binary.
[19:09] <plars> elopio: great! I'll add subunit to the list of dependencies to deploy
[19:46] <jdstrand> Chipaca: fyi, I'll have the /root fix for apparmor in a little bits
[19:46] <jdstrand> bit*
[19:48] <mvo> jdstrand: I don't know why /root/apps/pkgname/version is not allowed, sorry
[19:49] <jdstrand> mvo: that's fine. there are now actionable items in the bug report. I'm fixing the apparmor bits now. it sounds like Chipaca may be interested in the change to wrapper generation
[19:49] <jdstrand> mvo: are we doing anything special with upgrades and rollbacks with $HOME?
[19:50] <jdstrand> mvo: cause if so, will need to do it for /roots/apps too
[19:51] <mvo> jdstrand: I need to look, I don't think so, but worth checking
[19:55] <Chipaca> mvo: jdstrand: on upgrade we copy the home datadir over
[19:56] <jdstrand> mvo: uh oh
[19:56] <Chipaca> mvo: jdstrand but it's easy to fix to also copy the root datadir
[19:56] <jdstrand> mvo: ImportError: No module named 'LibAppArmor'
[19:56] <jdstrand> mvo: when did we drop python3-apparmor?
[19:56] <mvo> jdstrand: where do you see that
[19:56] <mvo> jdstrand: on the image? I don't think we did drop that intentionally
[19:56] <jdstrand> mvo: sorry, python3-libapparmor
[19:57] <jdstrand> mvo: snappy-debug.security scanlog
[19:58] <mvo> jdstrand: hm, I can have a look tomorrow why but I'm pretty sure this did not happen on purpose
[19:58] <jdstrand> mvo: this then gets into the question of how big do we want snappy-debug to be
[19:59] <jdstrand> mvo: maybe I should just add it back to the seed and then we can discuss at a later date?
[19:59] <mvo> jdstrand: +1
[20:49] <sergiusens> elopio, kyrofa I've created https://github.com/ubuntu-core/snapcraft/tree/2.x
[20:50] <kyrofa> sergiusens, why not create the stable 1.0 and use master for 2.0 development?
[20:52] <sergiusens> kyrofa, I have thought about that; if we don't change documentation for the time being that could be good
[20:52] <sergiusens> kyrofa, btw, 1.x or 1.0?
[20:53] <kyrofa> sergiusens, ah, right, 1.x. But good point about the documentation
[20:54] <sergiusens> kyrofa, they auto pull from master; I thought I'd work on 2.x for a bit and then do some swapping
[20:54] <kyrofa> sergiusens, good call
[20:58]  * sergiusens heads to aikido practice
[21:54] <Chipaca> barry: so, i've got a fix for pyyaml, but i'm not sure how to add tests to it -- the test suite is rather intractable for driveby fixes
[21:55] <Chipaca> barry: and 'tis tricky to give you a diff without this being in some kind of ... something :-)
[21:55] <barry> Chipaca: i've never looked at the code.  maybe there's an upstream repo somewhere?
[21:55] <Chipaca> barry: subversion
[21:56]  * barry has a sad
[21:56] <Chipaca> i guess i'll file a bug and do the patch upstream
[21:57] <Chipaca> um, the trac seems to be readonly, or i've forgotten how to use it
[21:57] <barry> Chipaca: yep.  we can always add the patch to quilt for ubuntu and/or debian
[21:57] <barry> you might need to be logged in
[21:57] <Chipaca> http://pyyaml.org/report
[21:57] <Chipaca> sigh
[21:58] <barry> yeah, it sucks that all these tracs are silos
[22:04] <Chipaca> barry: are short unicode builds still a thing?
[22:04] <barry> Chipaca: not in python3 anymore
[22:04] <Chipaca> barry: and 2.7?
[22:05] <barry> Chipaca: yes in 2.7
[22:05] <barry> but who cares about 2.7? <wink>
[22:06] <Chipaca> barry: do you remember what maxunicode was in a short unicode build?
[22:06] <barry> Chipaca: i don't remember.  i could probably figure it out if needed
[22:06] <Chipaca> barry: nm, got it
[22:07] <barry> ah cool
[22:07] <Chipaca> 65535
[22:07] <barry> makes sense!
[22:07] <Chipaca> and that's the max unicode as far as pyyaml is concerned
[22:07] <Chipaca> anything higher than that and it barfs
[22:07] <barry> even in a wide build?
[22:18] <Chipaca> barry: yes, it explicitly checks
[22:18] <Chipaca> barry: the spec says "the following are not allowed [list of codepoints]"
[22:18] <Chipaca> barry: pyyaml negated that list against a non-wide build, and uses that regexp to check
[22:19] <barry> Chipaca: ah
[22:20] <Chipaca> anyway, got a quilt patch. now what :)
[22:23] <barry> Chipaca: probably best to open a bug on the debian package and either attach the quilt patch to that, or attach a debdiff.  this is a DPMT maintained package, so i am happy to take a look and shepherd the patch through debian (which should autosync to xenial)
[22:24] <Chipaca> barry: to use debdiff i'd have to add a changelog and get a fresh .dsc, right?
[22:25] <barry> Chipaca: yep, but i'm happy to do that work if you have a bug # and a quilt patch
[22:25] <Chipaca> ok, i'll go with that :)
[22:50] <Chipaca> barry: bug 806826
[22:50] <barry> Chipaca: got it
[22:50] <Chipaca> ubottu: lulz
[22:50] <barry> debian bug 806826
[22:50] <barry> :)
[22:53] <Chipaca> barry: and I managed not to mention U+1F4A9 in the whole report!
[22:53] <Chipaca> I must be growing up
[22:54] <barry> :)
[23:11] <barry> Chipaca: i see what you mean about the test suite.  the patch would be better with tests, but if none of the existing tests regress and you vouch that it fixes your problem, that'll have to be good enough.
[23:12] <Chipaca> I tried adding to the tests, and something completely nonrelated failed, so i gave up
[23:13] <Chipaca> the test is easy: yaml.load("\N{PILE OF POO}")
[23:13] <Chipaca> does it explode? y/n :)
[23:14] <barry> Chipaca: cool.  at the very least i can add a dep-8 test for that
[23:20] <Chipaca> barry: dep-8?
[23:20] <barry> Chipaca: autopkgtests
[23:20] <Chipaca> cool
[23:21] <Chipaca> barry: well, yaml.dump() of the same should also give the same, fwiw
[23:23] <barry> Chipaca: hmm, that doesn't fail for me with the unpatched pyyaml (neither does the yaml.load() with py2.7)
[23:24] <barry> Chipaca: this does though: python3 -c "import yaml; yaml.load('\N{PILE OF POO}')"
[23:25] <Chipaca> barry: no, it doesn't fail, but it prints "\U0001F4A9"
[23:26] <barry> Chipaca: what should it print?
[23:26] <Chipaca> print(yaml.dump("\N{PILE OF POO}", allow_unicode="very yes"))
[23:27] <Chipaca> barry: ^ try that in one and t'other
[23:27] <barry> Chipaca: ok.  gotta run out for a bit.  i'll either finish this tonight or tomorrow
[23:42] <jdstrand> Chipaca: lucky you-- you get 'less' in the next stable update :) (I wanted to do the apparmro change for /root/apps there too so fixed your less issue :)
[23:43] <Chipaca> hehe