/srv/irclogs.ubuntu.com/2015/12/01/#snappy.txt

Chipacaogra_: noice00:12
=== chihchun_afk is now known as chihchun
=== bigcat_ is now known as bigcat
=== chihchun is now known as chihchun_afk
dholbachgood morning08:02
fgimenezgood morning08:17
mvohey fgimenez, good morning!08:23
mvoogra_: this armhf build failure is quite annoying, it takes forever to run the tests on a porter box08:24
fgimenezhi mvo!08:24
fgimenezmvo, btw with the new cloud resources in place we should redirect the github webhook, let me know when you have a minute and i'll send you the new ip08:27
mvofgimenez: sure, I have a minute08:32
mvofgimenez: trying to figure out an armhf build failure right now08:32
fgimenezmvo, once i finish fixing the snapd integration tests we should have green builds, i'll fire up builds every 15 minutes to check08:36
mvofgimenez: sweet. and then we can add a user that leaves comments too, right?08:37
fgimenezmvo, yep, we should create a new github user and add it to the organization as collaborator, then add its credentials to jenkins, and that's it :)08:39
=== chihchun_afk is now known as chihchun
Chipacajdstrand: could we add 'less' to the list of things we can exec when confined?10:10
JamesTaitGood morning all! Happy Giving Tuesday, and happy Bifocals at the Monitor Liberation Day! 😃10:17
ChipacaJamesTait: monitors had a liberation day, and bifocals were present, and it was an event that needs commemorating?10:24
ChipacaJamesTait: the mind is boggled10:24
JamesTaitChipaca, it's a tribute to those poor souls who wear bifocals and use a monitor daily, and keep bobbing their heads up and down trying to figure out which lens to use.10:28
Chipacayeah. fear that day.10:29
Chipacai wouldn't be able to look at the monitor with my head tilted, letting ideas settle around the clot10:29
Chipacalongsleep: ping10:46
longsleepChipaca: pong11:09
longsleepChipaca: i added a hack to u-d-f, moving the services to writable - i know it is crappy but works for now http://paste.ubuntu.com/13597040/11:13
Chipacalongsleep: heh. i'm working on that right now (on the snappy side)11:14
Chipacabroke all the tests of course11:14
longsleepChipaca: you are moving the stuff to first boot now?11:14
Chipacalongsleep: the ping was to ask you for your oem package.yaml, if that was shareable11:14
Chipacalongsleep: yes11:14
longsleepChipaca: ok good11:14
Chipacalongsleep: making install not activate from u-d-f (u-d-f passes in a particular flag), then making firstboot activate them11:15
longsleepChipaca: uhm sure i can share the oem snap, though the snaps are not in the store11:15
Chipacayep, no worries about that11:15
Chipacaand feel free to munge the names if they are sensitive11:16
longsleepChipaca: http://paste.ubuntu.com/13597081/11:16
Chipacajust looking for a real-world example, as what i've got access to is either too bare, or theoretical11:16
Chipacaperfect11:16
longsleepChipaca: yeah, this is what we currently use11:16
=== chihchun is now known as chihchun_afk
pindongajdstrand, hi there, there is no schedule... which revno do you want to release? I'll do my best to get it out asap11:52
=== chihchun_afk is now known as chihchun
fgimenezhi Chipaca, ive got a question about http.chipaca12:01
Chipacafgimenez: shoot12:01
fgimenezChipaca, it doesn't let me send files http://paste.ubuntu.com/13597723/, am i doing it right?12:02
fgimenezChipaca, this is the response with debug enabled http://paste.ubuntu.com/13597757/12:03
Chipacafgimenez: it can't read /tmp/pp, because it'll have its own private /tmp12:06
Chipacafgimenez: two questions:12:06
Chipacafgimenez: first, what's in /tmp/pp?12:06
fgimenezChipaca, nothing, just touched it for trying, the real test tries to use a snap file from ADT_ARTIFACTS, under /tmp12:07
fgimenezChipaca, i can try to move it to wherever http needs12:08
Chipacafgimenez: would `cat /tmp/pp | sudo http.POST snapd:///1.0/packages` work?12:08
Chipacaotherwise, put it under $SNAP_APP_DATA_PATH12:09
Chipacai.e., /var/lib/apps/http.chipaca/current/12:09
fgimenezChipaca, yep, works fine :) ok, i'll try putting it there, thanks!12:09
Chipacapipes are easy from the shell, but tricky in a test, so12:10
fgimenezChipaca, is it also possible to pass http a json string literal, as in http://paste.ubuntu.com/13600302/ ?14:59
Chipacafgimenez: yes, but15:00
Chipacafgimenez: https://github.com/jkbrzt/httpie#request-items15:03
Chipacafgimenez: so in particular for doing a multi-config PUT,15:06
Chipacagah. gimme a bit.15:07
jdstrandChipaca: re less, sure15:07
Chipacajdstrand: whee :-)15:08
jdstrandChipaca: when do you need it?15:08
Chipacajdstrand: less has exec-this-other-thing, but that'll just fail because confinement, right?15:08
Chipacajdstrand: no hurry, backburner project needs it (or me reimplementing a pager in python ;-) )15:08
jdstrandChipaca: it will fail to exec the other thing, yes15:08
* Chipaca will run it with that feature disable, because usability++, but still15:09
Chipacafgimenez: sudo http.PUT snapd:///1.0/packages 'ubuntu-core.ubuntu:="config: {ubuntu-core: {autoupdate: true}}"'15:11
Chipacafgimenez: note the quotes in the quotes15:11
dholbachsergiusens, so you're landing 0.6 soon? :)15:13
Chipacafgimenez: the thing on the right of the := is a json string literal (hence the double quotes there). Of YAML.15:14
sergiusensdholbach, yeah15:14
Chipacanot the nicest of things, but ¯\_(ツ)_/¯15:14
sergiusensjdstrand, is there a cap I can use for capname="net_admin" on 15.04?15:14
fgimenezChipaca, great, in my case 'ubuntu-core.ubuntu:="config: {basic-config.sideload: {key: value}}"' prevents the 400 and creates the operation, i'll put that into the test, thanks!15:15
Chipacaummm15:16
Chipacafgimenez: that should kinda not work15:16
Chipacathat is, the operation should fail15:16
Chipacabecause ubuntu-core.ubuntu mismatch with basic-config.sideload15:16
dholbachsergiusens, awesome :)15:16
Chipacafgimenez: (yes, config is redundant like that; needs revamp)15:17
fgimenezChipaca, yes i'm getting "invalid ubuntu-core configuration", ok, then 'basic-config.sideload:="config: {basic-config.sideload: {key: value}}"', right?15:18
elopioplars: are you here?15:18
plarselopio: sort of, feeling pretty sick today. What's up?15:20
fgimenezChipaca, yes, that works fine, thanks :)15:20
elopioplars: I need your help to collect the SPI output. But I can wait, lets talk tomorrow, get some rest.15:20
elopioplars: I hope you get better soon.15:20
Chipacafgimenez: \o/ :)15:21
jdstrandsergiusens: yes. either network-admin or network-firewall15:21
sergiusensjdstrand, thanks15:21
jdstrandsergiusens: are you aware of 'sudo snappy-debug.security scanlog' :)15:22
jdstrand?15:22
plarselopio: no, it's fine. was this from a job that just ran?15:22
sergiusensjdstrand, I wasn't, no I am :-)15:22
elopioplars: latest ran yesterday midnight. Want me to run another one?15:22
jdstrandChipaca: ok, you are getting less{,file,pipe} and more15:22
Chipacajdstrand: whee :-)15:23
elopiomvo: do you have time to talk about the godd snap?15:24
plarselopio: I think I found it, one sec15:24
sergiusensjdstrand, are all three suggestions to be followed? http://paste.ubuntu.com/13600633/15:25
jdstrandsergiusens: no. pick one15:26
sergiusensjdstrand, I already put network-admin in there, which is why I ask15:26
jdstrandsergiusens: is that a new denial or a previous denial?15:27
sergiusensjdstrand, it is a fresh boot15:27
jdstrandsergiusens: I'm guessing that /var/lib/snappy/seccomp/profiles/gopaste.sideload_... doesn't reflect the change15:28
mvoelopio: yes15:28
plarselopio: https://pastebin.canonical.com/145186/plain/15:28
jdstrandsergiusens: or it didn't at the time it started15:28
Chipacamvo: Kalnischkiesfile15:28
jdstrandsergiusens: check if setsockopt is in /var/lib/snappy/seccomp/profiles/gopaste.sideload_...15:29
Chipacamvo: in https://mvogt.wordpress.com/2015/11/30/apt-1-1-released/ in english15:29
sergiusensjdstrand, it is not15:29
plarselopio: that seems to be all that came through15:29
jdstrandsergiusens: how did you install the snap?15:30
sergiusensjdstrand, `snapcraft run` :-)15:30
elopioplars: so it's running! but being killed.15:30
mvoChipaca: hm? did I misspell something somewhere?15:31
mvoChipaca: in that blog post?15:31
Chipacamvo: yes15:31
Chipacamvo: I don't know if it's misspelt; it's just not very englishy15:31
plarselopio: kick off another one and I'll see if it does anything different15:31
jdstrandsergiusens: what is in the generated meta/package.yaml on the device?15:31
elopiomvo: I hw-assigned /dev/null to the snapp but I am still getting:15:32
elopiogodd.godd15:32
elopio/apps/godd.sideload/current/godd_1.0_amd64.snap /dev/null15:32
elopiofailed to dd: open /proc/self/mountinfo: permission denied15:32
elopiowhat else should I give to the snap to be able to run the bin?15:32
sergiusensjdstrand, http://paste.ubuntu.com/13600718/ this file looks strange though15:32
mvoChipaca: haha now I see what you mean, this is actually kind of funny15:32
elopioplars: triggered.15:34
jdstrandsergiusens: what is the version of ubuntu-core-security-seccomp on the device?15:34
=== kickinz1_ is now known as kickinz1
mvoelopio: hm, can you he-assign /proc/self/mountinfo? just for fun and testing?15:34
mvoelopio: it needs this to prevent you from shooting yourself in the food15:35
sergiusensjdstrand, hah, hard to know given the dpkg cache is wiped15:35
sergiusensmvo, do you know? ^15:35
mvoelopio: I think for this to be a proper snap it needs a customized appamor profile, i.e. it needs to be able to write to /dev/** by default15:35
mvosergiusens: latest rolling? I can check on the builds. or what version of the image do you use?15:36
sergiusensmvo, 15.04 stable15:36
mvosergiusens: we have the dpkg info on 15.04 :)15:36
jdstrandsergiusens: acutally, that shouldn't matter. snappy-debug.security looks at what is installed on the device and if it is suggesting network-admin, then network-admin should have it15:36
sergiusensmvo, oh right15:36
jdstrandsergiusens: how does snapcraft run work? is it just doing a snappy install of a generated snap or is it doing something else?15:38
sergiusensjdstrand, it scp's the snap and installs it15:38
jdstrandsergiusens: can you give me the snap?15:38
sergiusensjdstrand, with `snappy install`15:38
sergiusensjdstrand, sure15:38
sergiusensjdstrand, http://people.canonical.com/~sergiusens/snappy/gopaste_1.0_amd64.snap15:39
elopiomvo: I can't hw-assign mountinfo. It says invalid hardware device.15:39
jdstrandsergiusens: btw, you are almost certainly hitting https://bugs.launchpad.net/snappy/+bug/1465724 and don't actually need network-admin15:39
ubottuLaunchpad bug 1465724 in Snappy "net_admin apparmor denial when using Go" [High,Confirmed]15:39
mvoelopio: aha, indeed, there is a check for this15:40
jdstrand(snappy-debug.security should mention that, but you ran it after you added the cap I think)15:41
mvoelopio: hm, then we need to update the security profile of this to make it work on snappy itself. whats you use-case (just curious)?15:41
elopiomvo: there is a godd snap in the snapcraft examples. I want to run the binary and check the output.15:42
Chipacajdstrand: is /proc/self/mountinfo unreadable for a reason?15:42
sergiusensjdstrand, for every snap I build I restart the vm though15:42
* Chipaca is not trying to ddos jdstrand 15:43
sergiusensunless the log persists, which I think it doesn't, it shouldn't be the root cause15:43
sergiusensmvo, we just want the examples in snapcraft to actually run and not just build ;-)15:43
jdstrandChipaca: yes, it is an information leak15:44
Chipacaaww15:44
jdstrandsergiusens: I can confirm this. it is a bug in snappy-debug.security15:44
Chipacamvo: so i guess godd needs fixing to not need that15:44
sergiusensjdstrand, how so?15:45
jdstrandsergiusens: it is suggesting network-admin and network-admin doesn't have setsockopt15:45
jdstrandsergiusens: use network-service or network-client15:46
sergiusensjdstrand, ah, thanks; will try and report back15:47
mvosergiusens, elopio: let me look at this15:47
jdstrandoh heh15:48
jdstrandsilly bug15:48
jdstrandfix is easy. here is the output:15:48
jdstrandSyscall: setsockopt15:48
jdstrandSuggestions:15:48
jdstrand* add 'setsockopt' to seccomp policy15:48
jdstrand* add one of 'network-client, network-firewall, network-service' to 'caps'15:48
jdstrand* add 'setsockopt' to seccomp file in 'security-policy'15:48
elopiothanks mvo15:48
jdstrandsergiusens: ^15:48
sergiusensjdstrand, yay, no I have a defunct process, let's see why :-)15:50
plarselopio: my bbb tests on rolling seem to be failing since last week when I was away also. Looks like the last one that passed was 23715:51
plarselopio: well, maybe... that one might be a fluke. Out of curiosity, do you have it saving your output to a location that you cleanup? or is it still a tmpdir?15:53
elopioplars: I'm using the pwd to put the resulting files, and mktemp -d to get the code. I delete the temp dir on exit.15:54
mvoelopio, sergiusens: sorry, looks like I can not easily fix it, snapcraft does not understand the new security-override syntax it seems15:55
sergiusensjdstrand, oh goodie, I now ran into the sqlite fchown issue ;-)15:55
mvosergiusens: it appears that https://github.com/mvo5/godd/commit/38dfb37c34db7c1314ee1fe6a730c53565de9914 is making snapcraft unhappy15:56
sergiusensjdstrand, is that going to be allowed soon or should I fix it in the snap15:56
jdstrandhow is it that everything is using sqlite all of a sudden...15:56
plarselopio: well, there was no output from it that time, and the dir has already been cleaned up so I can't see anything useful from it15:56
elopiomvo: oh, I was thinking this was a bad snapcraft example because it wasn't showing anything useful. But now with the security override it's interesting.15:57
jdstrandsergiusens: it should be fixed in the snap or someone should fix it in the archive15:57
plarselopio: all I can see is that it exited with rc=115:57
sergiusensmvo, notice that this version of snapcraft does not play nice at all with 16.04 syntax15:57
elopioplars: but looking at this output you pasted, it seems that something is killing my script, right?15:57
plarselopio: not unless you are, or it times out and spi is killing it or something15:58
plarselopio: that last one exited with the same rc15:58
elopiosergiusens: we need to support pinning, to keep the sources in a known good version.15:59
plarselopio: it's possible that I just don't have *all* the output. Getting output from the script this way is not really the best. It's more intended for logging messages from provisioning rather than as a debugging tool for your scripts15:59
elopioplars: hum, that's possible. There is a lot of output.16:00
sergiusenselopio, you can pin with source-tag, source-branch to some extent16:00
sergiusensmvo, you seem to be using 16.04 syntax16:00
jdstrandsergiusens: fyi, snappy-debug 0.7 fixes the issue and is in the store16:00
plarselopio: have you tried running this from a bbb locally to make sure it works?16:00
elopioplars: yes.16:00
elopioI have made many tiny changes trying to collect some output, so I will try again. But it's great news to see that the tests are starting.16:01
sergiusensjdstrand, so for fchown it is not going to be allowed and I need to fix it with a security.override?16:02
sergiusenss/./-/16:02
fgimenezelopio, it seems that we don't have access to the store (or the internet for that matter) from the instances http://10.55.32.74:8080/job/github-snappy-integration-tests-cloud/10/consoleFull16:02
elopioplars: the troubling situation is that this should be bullet proof now. It doesn't have -e, so it won't stop the execution on failure. And I do  > "$pwd"/output.txt 2>&1 which should collect any possible error.16:02
elopiothat's where I'm lost. I don't understand why I get no output at all.16:02
fgimenezelopio, Get https://system-image.ubuntu.com//ubuntu-core/rolling/edge/generic_amd64/index.json: dial tcp 91.189.88.142:443: i/o timeout16:03
jdstrandtyhicks: hey, so, chown seems to be coming up all the time now16:03
elopiofgimenez: yes, I bet the scalingstack firewall is too strick. We need it open for a lot of things. Would they get mad if we just ask to remove it? :)16:03
mvosergiusens: is that bad :) ?16:04
tyhicksjdstrand: chown to what?16:04
mvosergiusens: aha, ok. I see that it is bad, I will use a different syntax16:04
plarselopio: let me take a look16:04
sergiusensmvo, no, it just won't work16:04
ogra_sergiusens, is that sqlite ?16:04
jdstrandtyhicks: like, 4 times in a week. our stance has been that we won't allow it until we have per-app uids16:04
sergiusensogra_, yes16:04
ogra_sergiusens, if so, i have a patch ;)16:04
tyhicksjdstrand: right16:04
jdstrandtyhicks: sqlite is doing a superfluous chown16:04
plarselopio: what do you get on the spi side? do you get your results json uploaded there?16:04
jdstrandtyhicks: chown root:root /path/to/db16:04
ogra_jdstrand, it isnt actually superfluous16:04
jdstrandtyhicks: and it is getting killed16:04
elopioplars: no. empty result_payload16:05
jdstrandogra_: it is, cause the db is already root:root16:05
ogra_jdstrand, it does that to make sure the db file matches the master file ... with the UID the master file was created with16:05
fgimenezelopio, :) with 91.189.88.142:443 will do for now IMO, we are just accessing the store?16:05
ogra_so it is valuable in non-snappy16:05
jdstrandand the master file will be root:root16:05
elopiofgimenez: aren't we accessing also docker servers?16:05
jdstrandI understand the non-snappy case and why they are doing it16:05
ogra_but since it tries to chown to a user thats not you usually it only does that when run as root16:05
jdstrandI'm talking about the snappy case, it is superflous because it does the chown unconditionally whether it needs it or not16:06
ogra_right16:06
jdstrandanyway16:06
sergiusenselopio, https://github.com/ubuntu-core/snapcraft/pull/13016:06
jdstrandtyhicks: so all of a sudden everyone is hot on sqlite16:06
fgimenezelopio, no, that's from canonistack, where the infra lives. the instances just need the store16:06
jdstrandtyhicks: and I'm getting pings left and right on this16:07
elopiofgimenez: great. Then yes, please make an RT to request opening the firewall to the store.16:07
jdstrandtyhicks: I've suggested we fix sqlite in the archive so that it does the chown conditionally16:07
ogra_sergiusens, http://paste.ubuntu.com/13601278/16:08
jdstrandtyhicks: but that hasn't happened yet and that doesn't fix vivid and doesn't fix something else that might do the same16:08
ogra_sergiusens, for the upstream sqlite tarball ...16:08
mvoelopio: ok, can you try now? I updated the git tree to use the unconfined template16:08
ogra_sergiusens, or just pull the binaries out of my upnp-server packages if you are lazy ;)16:08
sergiusensogra_, I am lazier, a lot lazier16:09
jdstrandtyhicks: I've been thinking that when we have syscall arg filtering, we will allow 'chown 0' (or whatever the syntax) in addition to 'chown 12345' for per-app, if/when we do that16:09
elopiomvo: trying.16:09
ogra_(after all you only need the sqlite3 executable and the lib)16:09
ogra_sergiusens, bah16:09
ogra_hacking security ! so evil :P16:09
jdstrandtyhicks: per-app aside, when we have syscall arg filtering, the 'chown 0' one would fix sqlite16:09
jdstrandtyhicks: but that won't happen until after the new year16:10
plarselopio: ok, there seem to be some garbage characters in the output. I wonder if that's why things are getting a bit messed up. It's definitely failing though16:10
plarselopio: https://pastebin.canonical.com/145191/16:10
jdstrandtyhicks: on rolling, there is the saying things can break at any time16:10
jdstrandtyhicks: I've been reluctant to allow chown in the seccomp policy since that opens the floodgates to all uids, only to reign it in again later and potentially break stuff16:11
jdstrandtyhicks: but maybe that is actually ok for 16.0416:11
jdstrandtyhicks: and 15.04 is simply what it is16:12
elopiomvo: still permission denied on mountinfo.16:12
elopionow I didn't have to hw-assign /dev/null16:12
jdstrandtyhicks: more clearly-- what do you think of me adding chown family to 16.04 policy now with a TODO to adjust when seccomp arg filtering exists?16:13
tyhicksjdstrand: if an app can chown in its data directory, what does that mean for this policy:16:13
tyhicks  # Writable home area for this version.16:13
tyhicks  owner @{HOMEDIRS}/*/apps/@{APP_PKGNAME}/@{APP_VERSION}/   w,16:13
tyhicks  owner @{HOMEDIRS}/*/apps/@{APP_PKGNAME}/@{APP_VERSION}/** wl,16:13
plarselopio: here's the json it spits out for spi to get, which is not valid and would probably fail for several reasons: http://paste.ubuntu.com/13601335/16:13
tyhicksjdstrand: do we drop owner?16:13
mvoelopio: hm, thats confusing, let me dive deeper16:13
elopioplars: right. Let me pastebinit.16:13
tyhicksjdstrand: that's from the default template16:14
jdstrandyes, I recognize the rule16:14
* tyhicks is trying to think through what it means to allow chown16:14
jdstrandfyi, this is exactly the sort of thing I have been saying could cause unintended interactions16:15
jdstrandand hence my reluctance16:15
* tyhicks agrees16:15
jdstrandtyhicks: that rule is actually terribly problematic, but that is another story16:15
elopiomvo: this is not urgent. I'm just trying to make sense of the examples.16:16
jdstrandtyhicks: the launcher is really not smart about that directory and will create ~/apps/... as the root user16:16
mvoelopio: hm, my latest git seems to work in my snappy now, what git revno is the one you have?16:16
jdstrandand then running stuff as non-root is broken16:16
mvoelopio: just did "snapcraft assemble --force" and copied it to a snappy system16:16
jdstrandbut that is another issue (there is a bug for that)16:16
elopiomvo: 6901dc912162b05fa381ead0e6d344152a68bbe916:16
jdstrandI think we *must* keep the owner checks there16:16
elopioI did snapcraft clean and snapcraft. Copied it to 15.04 stable #10.16:17
tyhicksjdstrand: I guess sqlite is not chowning files in that directory?16:17
jdstrandtyhicks: it is doing it in /var/lib/apps16:17
tyhicksah16:17
mvoelopio: thanks, let me try that too. the version looks correct16:18
jdstrandwell, I'm assuming. everyone is talking about services16:18
jdstrandand the services should be using SNAP_APP_DATA_PATH16:18
jdstrandnot SNAP_APP_USER_DATA_PATH16:18
sergiusensjdstrand, correct16:19
sergiusensmvo, elopio we use git upstream, but our example tree has its own snapcraft.yaml16:20
jdstrandtyhicks: adding the chown now would mean that an app could perform a DoS on the SNAP_APP_USER_DATA_PATH16:20
jdstrandtyhicks: but I think that will be true when we add 'chown 0'16:20
jdstrandhrm16:20
* sergiusens will bbl16:20
tyhicksjdstrand: but that DoS is only for itself16:20
tyhicksjdstrand: it wouldn't affect other snaps16:21
jdstrandfor itself but for another user16:21
tyhicksoh, right16:21
tyhicksthat's a problem16:21
ogra_doesnt the launch wrapper prevent you from that ?16:21
jdstrandthat will be true of 'chown 0' and 'chown 12345' too though16:22
jdstrandwhen we have syscall arg filtering16:22
ogra_(how would you mangle SNAP_APP_USER_DATA_PATH to be for another user ? )16:22
ogra_(given the launcher will set it after you touched it)16:22
tyhicksogra_: a malicious app doesn't have to use SNAP_APP_USER_DATA_PATH, it would be able to chown any path it chooses16:23
ogra_ah16:23
jdstrandogra_: you wouldn't-- but you can read /etc/passwd (necessarily) to get the paths of other users16:23
ogra_well, /var/lib/extrausers/passwd but yeah, i get what you mean :)16:23
ogra_(/etc7passwd is rather useless)16:24
jdstrandright. both are readable in the default policy (they must be)16:24
ogra_yeah16:24
tyhicksjdstrand: I think that I need some more time to think about this16:25
jdstrandtyhicks: I think to properly support @{HOME} with chown we need fine-grained mediation of chown in apparmor and something more than 'owner', or a kernel side var to expand @{HOME} in some manner16:26
jdstrandtyhicks: I agree. this is the sort of thing I have been saying is problematic with chown16:26
jdstrandogra_, sergiusens: I'm fairly certain we aren't going to just add chown to the policy any time soon. we may be able to fix it this cycle. I suggest someone update sqlite in the archive to make the check conditional16:28
jdstrandtyhicks: fyi ^16:28
ogra_well, i'm just using upstreams tarball now ...16:28
ogra_would be nice if snapcraft had a concept of patching though ... so i dont need to build in two steps16:28
jdstrandmost people will surely opt to snapcraft the deb though, no?16:28
ogra_yeah16:29
ogra_well my hackish patch from above is definitely not suited for the package16:29
Chipacajdstrand: tyhicks: so we need to change things so that SNAP_APP_USER_DATA_PATH is under the home of the effective user, right?16:29
ogra_(it blindly just rips out all chown calls)16:29
Chipacaogra_: using ld_preload?16:29
ogra_Chipaca, for what ?16:30
Chipacaogra_: for ripping out the chown calls16:30
Chipacawell, replacing them16:30
jdstrandnot for sqlite16:30
ogra_well, sqlite consiste of two binary files ... i just drop them in the right place with the copy plugin16:30
tyhicksChipaca: he patched out the calls to chown(2)16:31
ogra_usually i have some debs involved so i get the wrapper setup that gives me LD_PRELOAD anyway16:31
ogra_but having to patch the code means i need to build it separately from the snapcraft run16:31
jdstrandChipaca: the problem with the launcher is that $HOME of the real user is being used for the euid of 0 under sudo16:31
Chipacajdstrand: yes16:32
jdstrandChipaca: which is why we get root owned dirs in $HOME16:32
Chipacayes16:32
jdstrandthis isn't about fixing all that, though that is an incredibly annoying bug16:32
Chipacahttp.chipaca comes up against this a bit16:32
Chipacado you know the bug number offhand?16:32
jdstrandlet me find the bug16:32
Chipacaheh16:32
jdstrandI'll find it16:32
jdstrandhttps://bugs.launchpad.net/snappy/+bug/146623416:33
ubottuLaunchpad bug 1466234 in Snappy "Apparmor denial for access to SNAP_APP_USER_DATA_PATH as root" [Critical,Triaged]16:33
jdstrandChipaca: ^16:33
jdstrandtedg suggests in comment #2 that we shouldn't set SNAP_APP_USER_DATA_PATH for root16:34
Chipacayes, but agreed with setting it because there are valid reasons for using it16:35
tedgRoot isn't a user :-)16:35
jdstrandChipaca: that is one idea. someone told me to not include /root in the policy, which is why we don't now. another option is setting SNAP_APP_USER_DATA_PATH to SNAP_APP_DATA_PATH16:35
Chipacawrt stgraber, I suspect this was running as root, not with sudo16:36
jdstrandwhen running as root16:36
jdstrandChipaca: I think the lxc command is a binary in the lxd package16:37
* jdstrand checks16:37
jdstrandChipaca: yes, it is. so he used 'sudo lxc' at the time16:37
jdstrandplus, only the launcher does the mkdir, not the services (unless they wrote the service to do it)16:39
jdstrandwell, actually, scratch that16:39
jdstrandI forgot for a moment the luancher is used by the systemd service16:40
jdstrandso, from a traditional security perspective, it is probably wrong to have a process running under sudo to read files from the real user's home directory, so the current behavior of setting SNAP_APP_USER_DATA_PATH to the real user's home is wrong16:42
jdstrandtyhicks: what do you think? ^ (different topic from chown-- looking at 1466234 now)16:42
tyhicksjust a minute - I'm in a meeting16:44
jdstrandoh, I'm supposed to be in that meeting16:46
jdstrandactually, I'm optional there16:46
tyhicksjdstrand: it just ended16:47
tyhicksjdstrand: yes, I dislike the idea of root mucking with files in the user's home dir16:47
jdstrandme too16:47
jdstrandChipaca: ^16:47
tyhicksjdstrand: with chmod being allowed now, there's nothing stopping services from dropping setuid root binaries in a user's home dir :/16:48
jdstrandChipaca: I think the choice is to either not set SNAP_APP_USER_DATA_PATH when running as root or set it to SNAP_APP_DATA_PATH16:48
Chipacajdstrand: not to /root/apps/yadda/yadda?16:48
jdstrandwell, that is another choice, yes. I was explictly told *not* to allow that, and I asked in the bug for why, and got no response16:49
jdstrandasac: hey, do you recall why we didn't want to allow the launcher to create /root/apps/pkgname/version ?16:50
jdstrandmvo: do you? ^16:50
ogra_is /root writable ?16:51
jdstrandit is these days16:51
ogra_ah, i see16:51
jdstrandit is in /etc/system-image/writable-paths16:51
jdstrandit may not have always been16:52
ogra_yeah, i think it hasnt in the past16:52
jdstrandI don't know how this would fit into all snaps either16:52
elopioplars: can you check once again, please?17:06
plarselopio: http://paste.ubuntu.com/13602227/17:08
elopioplars: nice!17:09
elopionow I don't understand why the payload is still empty, but things are improving.17:09
elopioplars: can you please install the subunit package?17:11
plarselopio: done17:14
elopiothanks17:14
sergiusenselopio, can you check my last comment on https://github.com/ubuntu-core/snapcraft/pull/13017:21
elopiosergiusens: how could it be running on 80 if you are calling it with 8080}17:24
elopio?17:24
elopioI don't get a defunct service.17:25
sergiusenselopio, just run `sudo netstat -putan` (as a spanish speaker it is hard to forget those netstat switches :-P)17:30
sergiusenselopio, I might have messed up the flags, that is why I ask ;-)17:30
elopiosergiusens: it's not in 80, nor in 8080. But snappy service status says it's active and running.17:31
sergiusenselopio, snappy service is wrong; check `ps -ef|grep gopaste`17:32
kyrofasergiusens, pretty sure they're order independent :P17:32
elopiosergiusens: yes, defunct.17:32
elopiosergiusens: I agree that it's better than before, fwiw.17:32
jdstrandChipaca: ok, typos aside, I laid everything out in https://bugs.launchpad.net/snappy/+bug/1466234/comments/617:32
ubottuLaunchpad bug 1466234 in Snappy "Apparmor denial for access to SNAP_APP_USER_DATA_PATH as root" [Critical,Triaged]17:32
sergiusenselopio, let's merge this then and I'll think of how to get sqlite working17:33
sergiusenskyrofa, they are; but you know how it goes :-)17:33
jdstrandChipaca: I requested an architect comment, so hopefully we can get a direction on the bug17:33
jdstrandChipaca: one problem with /root/apps is upgrades/roolbacks17:34
Chipacajdstrand: why?17:34
sergiusenselopio, ok, I'm preparing the changelog again17:34
sergiusensChipaca, btw, if a service goes defunct, systemd/snappy still think it is in a good state17:35
sergiusenswe are probably missing a service toggle17:35
Chipacasergiusens: depends how it goes defunct17:35
Chipacasergiusens: as per the table in systemd.unit17:36
Chipacasorry, no, systemd.service17:36
sergiusensChipaca, oh, it sort of just gives up on living due to the jail apparmor and seccomp created for it17:36
Chipacasergiusens: man systemd.service, look for 'table 1'17:36
sergiusens:-)17:36
elopioour examples are depressed :(17:40
ogra_elopio, no worries, we'll give them free psycotherapy vouchers17:43
elopiowe need an eliza snap :)17:43
ogra_+1 !17:44
* ogra_ actually wants two and have them play nethack against each other ;)17:44
kgunnhey sergiusens, getting an error with snapcraft17:45
kgunnhttps://pastebin.canonical.com/145201/17:45
kgunnwhat's strange, is...i'm pretty sure this worked and i got a snap with no errors...17:46
kgunnso not really sure what happened..17:46
sergiusenskgunn, oh, 0.6 fixes that and I am releasing right now17:47
ogra_kgunn, i get that when i crate a file like "config.sh" with no content17:47
ogra_(i.e. no shebang)17:47
sergiusenskgunn, if in a hurry git clone https://github.com/ubuntu-core/snapcraft.git17:47
sergiusenskgunn, then $PATH_TO_SNAPCRAFT_GIT/bin/snapcraft17:48
sergiusenselopio, https://github.com/ubuntu-core/snapcraft/pull/12717:48
elopiosergiusens: go for it. And hopefully no 0.7 branch anytime soon.17:50
* Chipaca suddenly realises there's a hole in the cloud, and strikes it rich by offering "service as a service"18:03
Chipacasergiusens: btw, snapcraft is still broken wrt things not in the first plane18:03
Chipacasergiusens: because pyyaml is broken that way18:03
sergiusensChipaca, you talking about encoding I guess18:04
Chipacasergiusens: yes18:04
Chipacasergiusens: just saw you close bug 151815018:04
ubottubug 1518150 in Snapcraft "A Chinese character in the snapcraft.yaml crashes the snapcraft" [High,Fix committed] https://launchpad.net/bugs/151815018:04
sergiusensChipaca, shouldn't that work?18:04
Chipacasergiusens: here, try: 𠀡18:05
sergiusensChipaca, hah, xchat is also broken :P18:05
Chipacasergiusens: hit ctrl+shift+u, then type 20021, then hit enter or space18:06
sergiusensChipaca, the particular bug report is fixed though18:06
sergiusensChipaca, I just downloaded the snapcraft.yaml from librarian and tried18:06
Chipacasergiusens: ok. But there's a lot of cjk codepoints, not all of them in the first plane18:07
sergiusensChipaca, yeah, that is broken18:07
Chipacasergiusens: and pyyaml incorrectly reports things in higher planes as "special characters"18:07
sergiusensChipaca, yeah, seems like it18:07
sergiusensChipaca, I'll add a task for pyyaml on l18:08
sergiusenslp18:08
Chipacait's broken in some subtle way; the spec says some codepoints are not allowed, and these characters utf8 starts with the byte sequence that corresponds to those characters codepoints18:08
Chipacai.e. this character in utf8 is 0xF0 0xA0 0x80 0xA118:08
Chipaca*codepoint* 0xF0 is invalid18:09
Chipacano, wait, that's wrong also18:09
Chipaca"The allowed character range explicitly excludes the surrogate block #xD800-#xDFFF, DEL #x7F, the C0 control block #x0-#x1F (except for #x9, #xA, and #xD), the C1 control block #x80-#x9F, #xFFFE, and #xFFFF."18:10
ChipacaI have no idea why it thinks U+20021, 0xF0 0xA0 0x80 0xA1, is invalid18:10
sergiusensChipaca, care to add that to the bug?18:10
Chipacapoint me at the bug18:11
sergiusensChipaca, https://bugs.launchpad.net/ubuntu/+source/pyyaml/+bug/151815018:11
ubottuLaunchpad bug 1518150 in pyyaml (Ubuntu) "A Chinese character in the snapcraft.yaml crashes the snapcraft" [Undecided,New]18:11
Chipacaand i'll paste random facts at it until somebody fixes it :-p18:11
sergiusensChipaca, yeah, lets just bug barry ;-)18:11
* barry hides18:11
Chipacabarry: sergiusens: there, https://bugs.launchpad.net/snapcraft/+bug/1518150/comments/918:14
ubottuLaunchpad bug 1518150 in pyyaml (Ubuntu) "A Chinese character in the snapcraft.yaml crashes the snapcraft" [Undecided,New]18:14
barryChipaca: i'll look later.  i'm on a deep bug hunt right now18:14
Chipacabarry: yeh, i don't think this is urgent, but it'll byte somebody at some point :-)18:15
sergiusensbarry, you took too long, the bugs now hunt and HAUNT you ;-)18:15
barrysergiusens: bugception18:16
Chipacaok, stopping now. barry, if i write a patch for this thing, should it be against xenial, or upstream?18:17
barryChipaca: i don't know upstream too well, but if the xenial version is equiv to latest upstream, probably xenial will be fine (as quilt patch) and i'll submit it upstream18:19
jdstrandChipaca: re why> /home/$user/apps/... is (presumably) already handled by upgrades and rollbacks since it was always in the design. /root is not (currently) integrated into that, but would have to be if we supported /root/apps18:38
jdstrandChipaca: it isn't that it can't be done, it is whether or not it should be done18:38
elopiolook plars, we have output again! :) http://paste.ubuntu.com/13602944/18:49
elopiothanks, without you I would have never noticed it was because of the subunit binary.18:49
plarselopio: great! I'll add subunit to the list of dependencies to deploy19:09
jdstrandChipaca: fyi, I'll have the /root fix for apparmor in a little bits19:46
jdstrandbit*19:46
mvojdstrand: I don't know why /root/apps/pkgname/version is not allowed, sorry19:48
jdstrandmvo: that's fine. there are now actionable items in the bug report. I'm fixing the apparmor bits now. it sounds like Chipaca may be interested in the change to wrapper generation19:49
jdstrandmvo: are we doing anything special with upgrades and rollbacks with $HOME?19:49
jdstrandmvo: cause if so, will need to do it for /roots/apps too19:50
mvojdstrand: I need to look, I don't think so, but worth checking19:51
Chipacamvo: jdstrand: on upgrade we copy the home datadir over19:55
jdstrandmvo: uh oh19:56
Chipacamvo: jdstrand but it's easy to fix to also copy the root datadir19:56
jdstrandmvo: ImportError: No module named 'LibAppArmor'19:56
jdstrandmvo: when did we drop python3-apparmor?19:56
mvojdstrand: where do you see that19:56
mvojdstrand: on the image? I don't think we did drop that intentionally19:56
jdstrandmvo: sorry, python3-libapparmor19:56
jdstrandmvo: snappy-debug.security scanlog19:57
mvojdstrand: hm, I can have a look tomorrow why but I'm pretty sure this did not happen on purpose19:58
jdstrandmvo: this then gets into the question of how big do we want snappy-debug to be19:58
jdstrandmvo: maybe I should just add it back to the seed and then we can discuss at a later date?19:59
mvojdstrand: +119:59
sergiusenselopio, kyrofa I've created https://github.com/ubuntu-core/snapcraft/tree/2.x20:49
kyrofasergiusens, why not create the stable 1.0 and use master for 2.0 development?20:50
sergiusenskyrofa, I have thought about that; if we don't change documentation for the time being that could be good20:52
sergiusenskyrofa, btw, 1.x or 1.0?20:52
kyrofasergiusens, ah, right, 1.x. But good point about the documentation20:53
sergiusenskyrofa, they auto pull from master; I thought I'd work on 2.x for a bit and then do some swapping20:54
kyrofasergiusens, good call20:54
* sergiusens heads to aikido practice20:58
Chipacabarry: so, i've got a fix for pyyaml, but i'm not sure how to add tests to it -- the test suite is rather intractable for driveby fixes21:54
Chipacabarry: and 'tis tricky to give you a diff without this being in some kind of ... something :-)21:55
barryChipaca: i've never looked at the code.  maybe there's an upstream repo somewhere?21:55
Chipacabarry: subversion21:55
* barry has a sad21:56
Chipacai guess i'll file a bug and do the patch upstream21:56
Chipacaum, the trac seems to be readonly, or i've forgotten how to use it21:57
barryChipaca: yep.  we can always add the patch to quilt for ubuntu and/or debian21:57
barryyou might need to be logged in21:57
Chipacahttp://pyyaml.org/report21:57
Chipacasigh21:57
barryyeah, it sucks that all these tracs are silos21:58
Chipacabarry: are short unicode builds still a thing?22:04
barryChipaca: not in python3 anymore22:04
Chipacabarry: and 2.7?22:04
barryChipaca: yes in 2.722:05
barrybut who cares about 2.7? <wink>22:05
Chipacabarry: do you remember what maxunicode was in a short unicode build?22:06
barryChipaca: i don't remember.  i could probably figure it out if needed22:06
Chipacabarry: nm, got it22:06
barryah cool22:07
Chipaca6553522:07
barrymakes sense!22:07
Chipacaand that's the max unicode as far as pyyaml is concerned22:07
Chipacaanything higher than that and it barfs22:07
barryeven in a wide build?22:07
Chipacabarry: yes, it explicitly checks22:18
Chipacabarry: the spec says "the following are not allowed [list of codepoints]"22:18
Chipacabarry: pyyaml negated that list against a non-wide build, and uses that regexp to check22:18
barryChipaca: ah22:19
Chipacaanyway, got a quilt patch. now what :)22:20
barryChipaca: probably best to open a bug on the debian package and either attach the quilt patch to that, or attach a debdiff.  this is a DPMT maintained package, so i am happy to take a look and shepherd the patch through debian (which should autosync to xenial)22:23
Chipacabarry: to use debdiff i'd have to add a changelog and get a fresh .dsc, right?22:24
barryChipaca: yep, but i'm happy to do that work if you have a bug # and a quilt patch22:25
Chipacaok, i'll go with that :)22:25
Chipacabarry: bug 80682622:50
ubottubug 442941 in ubiquity (Ubuntu Oneiric) "duplicate for #806826 debconf failed to upgrade from 1.5.27ubuntu1 to 1.5.27ubuntu2: exit status 128 - Use of uninitialized value $reply in scalar chomp at /usr/share/perl5/Debconf/FrontEnd/Passthrough.pm line 66" [High,Fix released] https://launchpad.net/bugs/44294122:50
barryChipaca: got it22:50
Chipacaubottu: lulz22:50
barrydebian bug 80682622:50
ubottuDebian bug 806826 in src:pyyaml "pyyaml does not support literals in unicode over codepoint 0xffff" [Normal,Open] http://bugs.debian.org/80682622:50
barry:)22:50
Chipacabarry: and I managed not to mention U+1F4A9 in the whole report!22:53
ChipacaI must be growing up22:53
barry:)22:54
barryChipaca: i see what you mean about the test suite.  the patch would be better with tests, but if none of the existing tests regress and you vouch that it fixes your problem, that'll have to be good enough.23:11
ChipacaI tried adding to the tests, and something completely nonrelated failed, so i gave up23:12
Chipacathe test is easy: yaml.load("\N{PILE OF POO}")23:13
Chipacadoes it explode? y/n :)23:13
barryChipaca: cool.  at the very least i can add a dep-8 test for that23:14
Chipacabarry: dep-8?23:20
barryChipaca: autopkgtests23:20
Chipacacool23:20
Chipacabarry: well, yaml.dump() of the same should also give the same, fwiw23:21
barryChipaca: hmm, that doesn't fail for me with the unpatched pyyaml (neither does the yaml.load() with py2.7)23:23
barryChipaca: this does though: python3 -c "import yaml; yaml.load('\N{PILE OF POO}')"23:24
Chipacabarry: no, it doesn't fail, but it prints "\U0001F4A9"23:25
barryChipaca: what should it print?23:26
Chipacaprint(yaml.dump("\N{PILE OF POO}", allow_unicode="very yes"))23:26
Chipacabarry: ^ try that in one and t'other23:27
barryChipaca: ok.  gotta run out for a bit.  i'll either finish this tonight or tomorrow23:27
jdstrandChipaca: lucky you-- you get 'less' in the next stable update :) (I wanted to do the apparmro change for /root/apps there too so fixed your less issue :)23:42
Chipacahehe23:43

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!