[07:49] <dholbach> good morning
[08:26] <stevebiscuit> Chipaca, mvo: if I build the snapd binary, is there an easy way of getting it running on a snappy VM?
[08:29] <mvo> stevebiscuit: yes, for some definition of "easy". Chipaca has the details, it should be a matter of just building and scp to the snappy system $HOME and starting it here. However I have not done this yet, I can give it a quick try. you could even do the development on snappy itself from the classic environment. let me quickly try if my suggestion actually works
[08:35] <stevebiscuit> mvo: I've tried killing off the running snapd and then running the scp'd one but it gives me "error: daemon does not handle 0 listeners right now, just one" which is do with systemd from my reading of the code
[08:35] <mvo> stevebiscuit: ok, this is actually slightly more complicated, I get the same. here is what you need to do:
[08:36] <mvo> stevebiscuit: sudo systemctl stop ubuntu-snappy.snapd.service ubuntu-snappy.snapd.socket
[08:36] <mvo> stevebiscuit: sudo /lib/systemd/systemd-activate -l /run/snapd.socket ./snapd
[08:41] <stevebiscuit> mvo: that works a treat, cheers!
[08:46] <mvo> stevebiscuit: great! https://github.com/ubuntu-core/snappy/pull/291 so that this is easier to find in the future
[09:01] <liuxg> Chipaca, ping
[09:58] <asac> sergiusens: is it right that its odd that if i have a typo in snapcraft python code like referencing an non existing var
[09:58] <asac> that it just bails with the error but no indication of line number etc.?
[09:59] <asac> or is that normal for python?
[09:59]  * asac feels like being in debugging stoneage when hacking on snapcraft etc.
[10:00] <sergiusens> asac, it is not normal, just user friendly, for devel I recommend you change one thing
[10:01] <sergiusens> asac, either remove try/except here or change sys.exit to raise e
[10:02] <sergiusens> asac, I'll add an envvar there so you can devel easier
[10:25] <sergiusens> Chipaca, hey, mind doing a review?
[10:26] <Chipaca> sergiusens: REVIEWS FOR THE REVIEW GOD
[10:26] <Chipaca> sergiusens: but she's busy, so I guess I'll do them instead
[10:26] <sergiusens> Chipaca, https://github.com/ubuntu-core/snapcraft/pull/201
[10:27] <sergiusens> Chipaca, just the semantics for license keywords if you don't want to check code
[10:28] <Chipaca> sergiusens: reviewed
[10:29] <Chipaca> sergiusens: you dun good
[10:31] <asac> sergiusens: user friendly?
[10:36] <sergiusens> asac, end user; someone not introducing new code but using the tool
[10:36] <sergiusens> Chipaca, thanks
[10:40] <asac> sergiusens: ok, just couldnt see how not printing line number faults on error could help someone
[10:40] <asac> i would think best case someone using tool wont see errors
[10:40] <asac> but if he does he probably would want to have something useful to bug report
[10:41] <asac> but maybe i am missing some error cases where this would help
[10:52] <asac> sergiusens: anyway, env paired with entry in hacking.md etc. would be good (not that i read hacking.md, but at least i would have done that next i think)
[11:01] <asac> sergiusens: thoughts on https://gist.github.com/asac/d66ecdbb286e4a60269e
[11:01] <asac> thanks
[11:01] <asac> (note its not complete=
[11:02] <asac> not sure about run_raw... but i couldnt figure better way
[11:10] <asac> mvo: are there changes to snappy config protocol coming on master branch or do we stick to current approach for now?
[11:39] <brei> when I build a snap with snapcraft, not every file from the stage is copied to the snap folder. What are the filter criterias?
[11:58] <asac> kickinz1: so downloading i386 vs. amd64 docker-latest from https://docs.docker.com/engine/installation/binaries/ ... i see that the i386 is 7.X M and the amd64 is like 30M+ ?
[11:58] <asac> does that make sense?
[11:58] <kickinz1> asac: for me seems like i386 is only client side docker.
[11:58] <asac> kickinz1: like http://paste.ubuntu.com/14419442/
[11:59] <asac> 1.6 is 15M... latest is 30M
[11:59] <asac> even for 64
[11:59]  * asac scratches head
[12:00] <kickinz1> asac, there has been a lot of changes between 1.6 and 1.8-1.9 series (with arch support breakage).
[12:00] <asac> kickinz1: how do you check if a docker binary has just client?
[12:01] <kickinz1> You can try running it as a daemon, 'docker daemon' or 'docker -d' for prev 1.8
[12:01] <kickinz1> asac ^
[12:02] <sergiusens> asac, looks good, just not so sure about raw_config
[12:07] <asac> kickinz1: i moved to a different machien... do you have the link again :)?
[12:07] <asac> err
[12:07] <asac> sergiusens: ^
[12:07] <asac> sorry kickinz1
[12:07] <asac> thank kickinz1 ... seems that latest is busted i guess upstream
[12:09] <kickinz1> asac:  https://docs.docker.com/engine/installation/binaries/
[12:09] <asac> right wrong ping :)
[12:09] <kickinz1> asac: docker just support amd64 arch, so all other arches are client only iirc
[12:10] <asac> sergiusens: nevermind... found that github has history for me
[12:10] <asac> sergiusens: there is no raw_config ... just run_raw
[12:10] <asac> sergiusens: what i need to run is "yes" "" | make oldconfig
[12:10] <sergiusens> asac, good, because I have no idea what you were talking about wrt docker
[12:10] <kickinz1> asac, there are patches and an un-upstream way to build it that allow other arches to work.
[12:11] <asac> i couldnt figure how to do that with existing run methods
[12:11] <asac> sergiusens: ignore docker :)
[12:11] <asac> sergiusens: just the gist
[12:11] <sergiusens> asac, yeah, run_raw is what I meant, sorry
[12:12] <asac> sergiusens: yeah. so no ideas how to do it different?  e.g. good?
[12:13]  * asac finishes the patch and then thinks about tests
[12:16] <sergiusens> asac, I'm mostly sure you can implement run with what you have in run_raw
[12:16] <sergiusens> asac, btw https://gist.github.com/asac/d66ecdbb286e4a60269e#file-kbuild-diff-L56 your assert message is wrong here
[12:17] <sergiusens> asac, but I don't mind either way, this whole run thing is waiting for a refactor since forever
[12:18] <sergiusens> asac, the only thing missing in your diff are unit tests ;-)
[12:19] <asac> right. have to add one more featuure to allow starting with the kconfigfile instead of defconfig
[12:19] <asac> then will think about tests
[12:19] <asac> but i am very illeterate onthat,, so might need hand holding
[12:22] <asac> sergiusens: you think using an argument raw=False for run in __init__.py (avoiding a dupe in that file) would be good?
[12:22] <asac> oir should i rather just leave it to the refactor master?
[12:24] <kyrofa> Good morning
[12:25] <asac> hey kyrofa
[12:35] <kyrofa> Good morning again now that my internet is working
[12:40] <sergiusens> asac, I mean, the `exec $*` in regular run might as well be what you have in raw
[12:40] <sergiusens> kyrofa, ah, that explains the absence :-P
[12:40] <sergiusens> good morning
[12:41] <kyrofa> sergiusens, hey :P
[13:27] <kyrofa> Gahhh sergiusens the yaml tests sent me for a loop this morning
[13:27] <kyrofa> patcher = unittest.mock.patch('os.path.exists')
[13:27] <kyrofa>         mock_wrap_exe = patcher.start()
[13:27] <kyrofa>         mock_wrap_exe.return_value = True
[13:27] <kyrofa> ^^ didn't see that
[13:28] <kyrofa> sergiusens, I was starting to add debug prints to /usr/lib/python3.4/os.py :P
[13:50] <sergiusens> kyrofa, yeah, patching can do this sometimes
[14:02] <asac> sergiusens: right. i kind of sensed there is something in here... guess having exec nicely takes over the process and hides the parent shell hack for env
[14:02] <asac> sergiusens: on that, do you know why the initial code does not set the env through the python call_
[14:02] <asac> ?
[14:02] <asac> not complaining for my case, just wondered if python doesnt have easy way to just pump env to the command it runs
[14:04] <asac> bah... out of all the skills i have, i seem to be not smart enough to login to my local all snaps kvm using ssh -p XXXX ubuntu@localhost
[14:04] <sergiusens> asac, I inherited like this; I don't know
[14:05] <asac> let me reboot to latest .... maybe the dev 17 image is buggy on this, but i thought i managed to log in at some point
[14:05] <sergiusens> asac, all the env management still needs some work so actually use keys instead of a list
[14:07] <asac> sergiusens: does ssh to all snap kvm work for you?
[14:11] <asac> interestingly all host keys are 0 size... guess i know what might have happened ... i killed the first boot when it generated stuff
[14:12]  * asac redoes image
[14:13] <asac> do we crewate those keys using cloud-init still?
[14:13]  * asac wonders where to file bug in case this is the prob
[14:14] <sergiusens> asac, yes, still with cloud-init
[14:17] <asac> jdstrand: * adjust program to not use 'setgid' until per-snap user/groups are supported
[14:18] <asac> jdstrand: could we reference a tracking bug for that feature in the comment (though very nice guidance here)
[14:18] <asac> thats with snappy-debug.security scanlog
[14:19] <jdstrand> asac: sure
[14:23] <sergiusens> elopio, bumper https://github.com/ubuntu-core/snapcraft/pull/201
[14:24] <asac> jdstrand: do you remember why busybox did use setgid on everything?
[14:24] <enoch85> kyrofa, hey! shall we continue? :)
[14:24] <asac> i remember we looked at the code a while back
[14:24] <asac> for the hello-world.sh thingy
[14:24] <kyrofa> Hey enoch85 :) . I have a meeting here in a minute, let me ping you
[14:25] <enoch85> kyrofa, okay! :)
[14:28]  * asac tries to disable SUID
[14:32] <kyrofa> sergiusens, elopio sorry guys... internet is flaking
[14:33] <sergiusens> kyrofa, we can wait
[14:33] <asac> jdstrand: nevermind... i fixed a config in build system to not support suid of bb and now it works \o/
[14:33] <jdstrand> asac: I don't, however bash can be used for hello-world.sh these days
[14:34] <jdstrand> (just fyi)
[14:38] <asac> yeah
[14:38] <asac> seems bizbox has a feature called CONFIG_FEATURE_SUID... i disabled it and now its fun
[14:39] <asac> sergiusens: is stage a no-op these days? seems that stage happens for each part during build?
[14:59] <asac> sergiusens: is there a wildcard feature for binaries? i would like to add all that is in stage/bin/ as binaries without having to keep them in sync
[15:02] <sergiusens> asac, you can use directories in your 'snap' directive
[15:03] <sergiusens> asac, stage is more complex in 2.x; it takes into account the dependencies (1.x is broken in this regard)
[15:07] <sergiusens> jdstrand, hey; is there a reason why I can't have spaces in my start keyword?
[15:07] <sergiusens> jdstrand, without allowing that I have to resort to things like this https://github.com/ubuntu-core/snapcraft/blob/master/examples/shout/snapcraft.yaml#L9
[15:07] <sergiusens> jdstrand, with a wrapper script that only does https://github.com/ubuntu-core/snapcraft/blob/master/examples/shout/shout.sh
[15:09] <jdstrand> sergiusens: are you saying you want to put the contents of the shout.sh script in 'start' in the yaml?
[15:10] <sergiusens> jdstrand, I want to be able to write 'start: shout --home $SNAP_APP_DATA_PATH'
[15:10] <jdstrand> right
[15:10] <sergiusens> jdstrand, ah, '$' is also not allowed
[15:11] <jdstrand> so, 'start' is not used with the APP_ID since in this example the app id is shout_server_0.52.0
[15:12] <jdstrand> so, I think this is to make sure that the systemd file is sane
[15:12] <sergiusens> jdstrand, I don't follow
[15:13] <jdstrand> what is in 'start' ends up in ExecStart in the systemd service
[15:14] <sergiusens> jdstrand, correct, and I'm using SNAP_APP_DATA_PATH, no APP_ID
[15:14] <sergiusens> not*
[15:15] <jdstrand> we need to be careful with ExecStart, since a crafted 'start' could run commands outside of confinement
[15:16] <sergiusens> jdstrand, so basically if I need switches or envvars I need to resort to wrapper scripts?
[15:16] <jdstrand> if we make the review tools and snappy not allow ';', perhaps we could allow arguments to start
[15:17] <jdstrand> I wasn't talking about env vars
[15:17] <sergiusens> jdstrand, that sounds reasonable
[15:17] <jdstrand> I would assume that if the unit properly set SNAP_APP_DATA_PATH then whatever was in ExecStart would see it
[15:18] <sergiusens> jdstrand, just asking since it is not allowed, spaces either iirc
[15:19] <sergiusens> if it is just because of a ; then I hope we can just block offenders
[15:19] <sergiusens> jdstrand, no rush though
[15:19] <jdstrand> if someone were to expand this in the manner you would like, we would have to proceed very carefully, because that would be an avenue of attack for a malicious app developer
[15:19] <sergiusens> jdstrand, yeah, that's why I say no rush :-)
[15:20] <jdstrand> historically, I feel like 'start' was used as part of the app id. ie, if 'name' was not specified or something
[15:21] <jdstrand> if start is not part of the app id ever any more, then allowing arguments would seem to make sense (if done carefully)
[15:22] <sergiusens> jdstrand, so appids are now uuids, right?
[15:22] <sergiusens> Chipaca, ? ^
[15:22] <sergiusens> well """now""" (many air quotes)
[15:23] <jdstrand> sergiusens: no, not in my current understanding. the app id is today as always. what we are moving to is uuid_appname_revision
[15:24] <jdstrand> sergiusens: where appname the 'name' in the yaml
[15:24] <jdstrand> for a service/binary
[15:24] <jdstrand> so pkgname goes to uuid and version goes to revision
[15:24] <jdstrand> but appname stays the same
[15:25] <sergiusens> jdstrand, right, pkgname is snap-id basically which is a uuid
[15:26]  * jdstrand nods
[15:27] <sergiusens> jdstrand, another thought is that the only envvars that can be used in a service are those exported in the service itself and we control that (snappy)
[15:36] <jdstrand> sergiusens: I thought that is more or less what systemd is doing with Environment
[15:41] <elopio> ping stgraber: I was trying to get a xenial lxc in travis to run the snapcraft tests, but it doesn't have network access:
[15:41] <elopio> https://travis-ci.org/ubuntu-core/snapcraft/builds/100523000
[15:41] <elopio> can you help me here?
[15:46] <kyrofa> enoch85, today may not work very well... I seem to be having some internet connectivity issues
[15:46] <stgraber> elopio: it may just be a timing thing, that is, your container doesn't have an IP address yet
[15:46] <elopio> stgraber: mmm, I'll add a sleep to check.
[15:47] <kyrofa> enoch85, but I do plan on working on that .snap today
[15:47] <enoch85> kyrofa, ok, feel free to work on the apache snap whenever you want. It's above my skill set anyway, so I feel I can't contribute.
[15:47] <stgraber> elopio: I'm also very surprised that you're able to install and run lxd in what looks like Travis' docker based environment. I thought they restricted what package and repositories you could install to prevent getting their whole infrastructure to be taken over
[15:47] <enoch85> kyrofa, I'm available all day. Just ping me. :)
[15:47] <kyrofa> enoch85, very good. I'll make sure to walk you through wha tI've done :)
[15:47] <elopio> stgraber: this is not a docker container. This is a trusty machine in google compute.
[15:48] <stgraber> elopio: oh, interesting, I didn't know they supported that
[15:49] <elopio> stgraber: it's been working for a couple of months. It's nice, but now we need xenial.
[15:49] <enoch85> kyrofa, sounds good :)
[15:50] <kyrofa> enoch85, sorry about that. I just keep dropping today :(
[15:51] <enoch85> kyrofa, np mate. get a better line :) ;)
[15:51] <kyrofa> I wish
[15:51] <enoch85> kyrofa, anyway, no stress.
[15:52] <enoch85> kyrofa, need to be done by the 20:th, so there is plenty of time
[15:52] <kyrofa> enoch85, alright sounds good :)
[16:33] <elopio> stgraber: no luck. After the sleep it still can't ping anything.
[16:38] <stgraber> elopio: might be worth running ifconfig inside the container and outside of it, just to check if things look sane
[16:39] <elopio> stgraber: ok, it'll take some minutes. Travis is slow today...
[16:39] <sergiusens> kyrofa, elopio I'll bbl in case someone pings me
[16:40] <kyrofa> sergiusens, alrighty
[17:04] <kyrofa> elopio, mind taking a look at https://github.com/ubuntu-core/snapcraft/pull/204 ?
[17:23] <elopio> stgraber: the host shows lxcbr0. The container shows nothing:
[17:23] <elopio> http://pastebin.ubuntu.com/14422058/
[17:23] <elopio> kyrofa: let me see.
[17:24] <elopio> kyrofa: what about an example test for this change?
[17:28] <kyrofa> elopio, what does that mean in snapcraft? Something that exercises this in snapcraft/examples?
[17:30] <stgraber> elopio: what's your travis.yml?
[17:30] <elopio> kyrofa: yes. Like a real world example that uses the new code.
[17:30] <stgraber> elopio: the behavior you're getting would be consistent with having an older version of lxc, or lack of lxcfs or systemd being unhappy in the container
[17:31] <elopio> stgraber: https://github.com/elopio/snapcraft/blob/lxd/.travis.yml
[17:31] <elopio> stgraber: I'm getting it from the lxd-stable ppa.
[17:31] <kyrofa> elopio, alright I'll throw something together :)
[17:31] <elopio> kyrofa: thank you.
[17:32] <stgraber> elopio: ok, add a "sudo apt-get dist-upgrade -y" after the apt-get update and add "lxcfs" on the apt-get install line, just in case travis runs with recommends disabled by default.
[17:32]  * elopio tries.
[17:32] <stgraber> elopio: and add a "lxc exec xenial -- ps faux" too so we can see what's running in the container
[17:36] <kyrofa> elopio, do I also add the new example to TestSnapcraftExamples ?
[17:36] <elopio> kyrofa: to build it, yes. I'm guessing that it will require manual inspection to check that it works ok, am I right?
[17:37] <kyrofa> elopio, ah, yeah this is a runtime fix, not build
[17:37] <elopio> we might need to add a manual_tests dir, like in snappy.
[17:37] <kyrofa> elopio, and we're not running runtime tests, right?
[17:38] <elopio> kyrofa: we can from xenial, so just let me finish this PR and there will be automated runtime tests.
[17:39] <elopio> kyrofa: but I'm wondering is how to confirm that your example works ok automatically. Well, I better wait to see the example.
[17:40] <kyrofa> elopio, yeah, basically without this fix it fails to run since it can't link up with libgl. But it builds fine. With the fix it links and runs successfully
[17:41] <elopio> kyrofa: ok, so we can at least check that it doesn't return an error. For now leave it with the build test, and soon we'll start adding runtime tests for the examples.
[17:41] <kyrofa> elopio, good deal, adding now
[18:07] <kyrofa> elopio, alright, take another look at that PR for the example
[18:09] <elopio> stgraber: it pinged!
[18:09] <elopio> kyrofa: are you in a hurry, or can I go and get something to eat first?
[18:09] <stgraber> cool, so it was either an old pre-installed lxc or missing lxcfs
[18:09] <kyrofa> elopio, I'd like to get this merged today just so I can prepare the release, but my EOD is a few hours away yet
[18:10] <kyrofa> elopio, I'd never stand between you and food ;)
[18:11] <elopio> kyrofa: let me better review it first. Otherwise I'll keep thinking of you during lunch :p
[18:11] <kyrofa> elopio, hahaha
[18:12] <kyrofa> elopio, fortunately I don't think it could be simpler
[18:13] <elopio> yes, looks good. I like that we now have an opencv example!
[18:13] <kyrofa> elopio, awesome!
[18:54] <elopio> stgraber: is there a fancy way to wait for the ip? The sleep is making my nice script ugly.
[18:55] <stgraber> elopio: not a very nice one, but there is nicer than just a big sleep, yeah
[18:55] <stgraber> while ! lxc info tryit | grep -q eth0.*IPV4; do
[18:55] <stgraber>     sleep 5s
[18:55] <stgraber> done
[18:55] <stgraber> replace tryit by your container name
[18:56] <elopio> thanks!
[18:56] <elopio> -
[20:32] <kyrofa> sergiusens, is the fix to bug #1530995 to chmod before running, or run with sh?
[20:33] <kyrofa> sergiusens, which is the safer assumption-- we have permission to chmod, or it's actually using sh and not bash or something?
[20:51] <asac> sergiusens: by default all files from stage/ go to snap/ right?
[20:51]  * asac about to hit send on answer on ML
[20:57] <asac> damage done :=
[22:08] <sergiusens> asac, answer is yes