[00:14] <psanchez_> hmm, ok, not a good start for snappy and docker ... -1 ... I'll ask again tomorrow.
[07:16] <fgimenez> good morning
[07:28] <davidcalle> Morning o/
[07:35] <clobrano> good morning
[09:01] <JamesTait> Good morning all; happy Friday, and happy Respect Day! 😃
[09:07] <clobrano> A basic question: how do you test changes in snappy itself? how can I install it?
[09:08] <mvo> clobrano: if you modify snappy itself you can just copy in into our snappy system and run it via sudo ./snappy something
[09:11] <clobrano> mvo: doesn't snappy need any particular permission? Can I run it just like that?
[09:12] <ogra_>  sure
[09:12] <clobrano> ok, thank you
[09:13] <mvo> clobrano: one nice property is that its a single binary so for the most part this method of just copyign will work
[09:13] <clobrano> mvo: great :)
[09:14] <ogra_> if you want to test it deeper in the system you can also make the filesystem writable, copy it in place and switch back to readonly
[09:14] <ogra_> i.e. to test something that happens during boot or so
[09:15] <ogra_> the base system is still just a normal linux system ... just without traditional package manager
[09:15] <clobrano> ogra_: how do I make the fs writable?
[09:15] <ogra_> sudo mount -o remount,rw /
[09:15] <ogra_> sudo mount -o remount,ro /
[09:16] <clobrano> ogra_: oh sure, I was expecting something special :D
[09:16] <ogra_> heh
[09:17] <mvo> yeah, ogra_ made great suggestions. some stuff can only be tested by copying (snappy booted for example which is run by a startup job)
[09:17] <clobrano> perfect
[09:24] <davidcalle> mvo, ogra_, I need to do a small doc fix in the 15.04 branch, should I merge to snappy/trunk first, then snappy/15.04?
[09:25] <mvo> davidcalle: easiest is probably to just do it 15.04 and then we merge the 15.04 change to trunk
[09:25] <mvo> davidcalle: i.e. just do it on 15.04 and we will take care of the rest :)
[09:28] <davidcalle> mvo, sounds good :) https://code.launchpad.net/~davidc3/snappy/oem-docs-formatting/+merge/271624
[09:29] <davidcalle> mvo, ty
[10:04] <longsleep> Hey folks, i noticed that my ODROID-C image does not have most of the firmware files in /lib/firmware - is that expected when a device tarball with system/.. tree is used?
[10:28] <ogra_> longsleep, you need to ship it in the device tarball
[10:28]  * ogra_ has the same issue on rpi
[10:37] <ogra_> davidcalle, *next' release will bring official RPi support ...
[10:37] <ogra_> we'Re not there yet :)
[10:39]  * davidcalle adds a "don't open before christmas" label on the email
[10:40] <davidcalle> ogra_, alright then :)
[11:01] <longsleep> ogra_: sorry i got distracted, you say i have to ship all the firmware in the device tarball right?
[11:01] <ogra_> yeah ... just unpack linux-firmware in the tree when you create it
[11:02] <longsleep> ogra_: ok cool, thanks!
[11:42] <asac> o/
[11:42] <asac> my pi is on #5 :)
[11:42] <asac> feels faster
[11:43] <ogra_> heh
[11:43] <ogra_> i bet thats subjective
[11:43] <asac> guess thats just the excitement because i had to invest to get tehre
[11:43] <asac> :)
[11:43] <ogra_> yeah
[11:43] <asac> and snappy list bails very quickly
[11:43] <asac> https://bugs.launchpad.net/snappy/+bug/1497245
[11:44] <ogra_> you and your exotic packages :P
[11:45] <asac> well, vendor field wasnt mandatory no?
[11:45]  * ogra_ isnt sure
[11:45] <ogra_> probably it is now
[11:45] <asac> in any case, think for list and in particular remove we should be super robust against anything
[11:46] <asac> i think its mandatory for 16.04 ... but not for 15.04
[11:46] <ogra_> we backported trunk ...
[11:55] <sergiusens> asac, vendor was always mandatory
[11:56] <sergiusens> asac, the store at least booted you because of that, did you manually install?
[11:56] <asac> sure
[11:56] <asac> my beloved zigbee gateway :(
[11:57] <asac> lol
[11:57]  * asac goes and manually removes the app from /apps and the other prominent places
[11:57] <sergiusens> asac, why didn't you upload it to the store ? sounds like an interesing app ;-)
[11:58] <asac> yay what i love about snappy is that there is no cache/meta files
[11:58] <asac> sergiusens: like most of our apps they are customer related as they prep for launching big devices :P
[11:59] <asac> thats why in store we have like 2% of the existing snappy apps at best
[11:59] <sergiusens> I was just punning :-)
[11:59] <asac> i know
[11:59] <asac> sergiusens: good that you are awake again
[11:59] <asac> the call is cancelled...
[11:59] <asac> NOT :)
[11:59] <asac> lol
[11:59] <sergiusens> asac, well, meetings! ;-)
[11:59] <asac> will be there in 1 minute
[12:00] <ogra_> huh ?
[12:00] <ogra_> why the heck is the clinic a hangout
[12:00] <ogra_> thats nonsense
[12:01] <asac> ogra_: its a meeting
[12:01] <asac> you coming?
[12:02] <ogra_> asac, what for ?
[12:03] <ogra_> asac, if we want to do a clinic it should be on IRC ... i.e. a special time where we help with specific developer problems
[12:05] <zyga> ogra_: moar hangouts
[12:28] <dholbach> I think https://code.launchpad.net/~zyga/snapcraft/gitignore/+merge/271313 and https://code.launchpad.net/~dholbach/snapcraft/1496789/+merge/271444 can be landed, right? :)
[12:32] <ogra_> asac, mvo_, so i dont think there is any solution to the Pi update issue without actually releasing a stable #6 ...
[12:32] <ogra_> i guess we should just release-note it that you have to manually copy the kernel pieces
[12:33] <mvo_> ogra_: yeah, its really really hard :/ will the generic kernel not boot at all? or what will happen?
[12:34] <ogra_> it hangs hard at "Starting kernel ..."
[12:34] <mvo_> meh
[12:34] <ogra_> doesnt even unpack it completely
[12:35] <mvo_> yeah, that does not give us many options :(
[12:35] <ogra_> right
[12:35] <ogra_> we could hack something into the s-i process or some such ... but that would still require a #6 release
[12:37] <ogra_> (at least for armhf)
[12:37] <ogra_> we could indeed do an armhf only release ... at the cost of gettin the image numbers out of sync
[12:40] <dholbach> thanks sergiusens
[12:47] <mvo_> ogra_: yeah, I really have no good idea either, lets create the device thing on the server so that we are safe for the future, I talk to sil
[12:48] <ogra_> yup. i see that :)
[12:49] <ogra_> mvo_, the other question is ... why is there a kernel in /boot at all :) i thought we added code to remove it in livecd-rootfs
[12:50]  * ogra_ wonders if that only went into rolling
[12:52] <ogra_> ah, so it did
[12:55] <plars> ogra_: heya, quick rpi2 question for you if you have a moment - We're seeing that our ethernet hwaddress changes on every boot, I'm guessing because it has the serial as all 0s in proc cpuinfo. Is there any way you know of to have it configure a serial or come up with a static hwaddress?
[12:55] <ogra_> plars, seems you are using an outdated image
[12:55] <ogra_> that was fixed a while ago
[12:56] <plars> ogra_: well, I'm not booting to snappy yet, this is just an ubuntu image as a base, and we'll boot snappy from there. What was the fix?
[12:57] <ogra_> changing uboot and reading the binary blob data into the kernel cmdline
[12:57] <ogra_> the binary blob provides the actual MAC in a variable
[12:58] <plars> ogra_: which binary blob is it reading, and how does it get passed?
[13:00] <ogra_> the binary blob that boots your board
[13:00] <ogra_> i.e. the graphics driver that acts as bootloader :)
[13:02] <zyga> ogra_: we're all waiting for our nvidia overlords to boot our x86 pc
[13:02] <ogra_> plars, i dont remember exactly, iirc you need to crate a cmdline.txt next to the config.txt
[13:02] <ogra_> it needs to contain something ... then the args get handed over to the uboot env
[13:03] <ogra_> (RaspberryPi2)ubuntu@localhost:~$ cat /boot/uboot/cmdline.txt
[13:03] <ogra_> elevator=deadline
[13:03] <ogra_> (RaspberryPi2)ubuntu@localhost:~$ fw_printenv |grep ^args
[13:03] <ogra_> args=dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1824 bcm2708_fb.fbheight=984 bcm2709.boardrev=0xa01041 bcm2709.serial=0x8b04db1c smsc95xx.macaddr=B8:27:EB:04:DB:1C bcm2708_fb.fbswap=1 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000  elevator=deadline
[13:03] <ogra_> that is how my Pi expands the cmdline
[13:03]  * ogra_ forgot if there were additional steps required
[13:04] <plars> ogra_: weird, I actually copied the boot directory from snappy
[13:04] <ogra_> from image #4 ?
[13:04] <plars> ogra_: so I'm not sure what could be missing
[13:04] <ogra_> that only landed in the last image
[13:04] <plars> ogra_: from one I downloaded last night
[13:04] <plars> ogra_: how recent was that?
[13:05] <plars> ogra_: I just grabbed the prebuilt one
[13:05] <mvo_> ogra_: yeah, that puzzles me too, on armhf we really do not need that in /boot
[13:05] <ogra_> well, if it is the one from ~/platform thats the most recent one
[13:05] <ogra_> mvo_, i think we just didnt pull the change into the PPA
[13:05] <mvo_> ogra_: oh, ok
[13:05] <mvo_> ogra_: that makes sense
[13:05] <plars> ogra_: yes, that's the one
[13:05] <ogra_> i see the right code in live-build/ubuntu-core/hooks/500-move-kernel-to-device-tar.binary in wily
[13:06] <ogra_> plars, weird
[13:06] <asac> ogra_: its fine
[13:06] <ogra_> should definitely work then
[13:06] <asac> ogra_: can you reply to the release thread explaining the workaround?
[13:06] <ogra_> asac, will do
[13:06] <asac> or togetyher with your release announce when the pi2 is there
[13:06] <ogra_> there too
[13:06] <ogra_> cant mention it often enough ;)
[13:06] <asac> just say "becauyse pi2 is not officially consumed from channels you need to do these copies after snappy update
[13:06] <asac> and done
[13:07] <asac> yep
[13:11] <ogra_> sergiusens, i'm looking at package.yaml for pi and BBB oem snaps ... i dont see where it would define generic_armhf vs raspi2_armhf
[13:11] <asac> i think sergio forgot to add that optioon to the yaml
[13:11] <asac> was an oversight. we discussed a while back to have another field
[13:11] <ogra_> "architecture: armhf" is what i see in both currently
[13:11] <asac> beyond architecture
[13:11] <asac> channel:
[13:11] <asac> or somethign
[13:11] <asac> generic
[13:11] <asac> pi2
[13:11] <asac> etc.
[13:11] <ogra_> i see platform ... but that only defines the default dtb
[13:12] <asac> does the azure have an oem snap?
[13:12] <asac> if so it might have that info
[13:12] <asac> but guess it still is created with manual
[13:12] <ogra_> if azure does it isnt in http://bazaar.launchpad.net/~snappy-dev/snappy-hub/snappy-systems/files
[13:12] <asac> that much i know
[13:12] <asac> but doesnt mean there isnt one
[13:13] <ogra_> right
[13:13] <asac> anyway, i know it was a missing field before 15.04 release and most likely we just didnt add it
[13:13] <plars> ogra_: weird, I don't see it in the uboot environment, but I do see usbethaddr there, but I don't see it get used
[13:13] <ogra_> plars, usbethaddr is wrong
[13:13] <asac> but adding a new field with default "generic" shouldnt be a problem
[13:13] <ogra_>  smsc95xx.macaddr is what you need
[13:13] <asac> just remember to add cards etc.
[13:13] <plars> ogra_: it's the one I ended up with when I booted snappy
[13:14] <plars> ogra_: but not when I boot something else - then it's random even though I copied all the bootloader files and uboot from the snappy image
[13:14] <ogra_> do you have uboot.env ?
[13:15] <sergiusens> asac, ogra_ since we don't want to expose s-i in snappy so high up; it was never there; you can build with u-d-f passing --device with the channel (like for azure); but it might only work for azure (as we don't want to make it easy to go down a path we can't support)
[13:15] <plars> ogra_: I do
[13:17] <asac> sergiusens: platform: ?
[13:17] <asac> that doesnt feel like surfacing something from si if we use that
[13:17] <ogra_> plars, and is it used ?
[13:17] <plars> ogra_: yes
[13:17] <sergiusens> ogra_, http://bazaar.launchpad.net/~phablet-team/goget-ubuntu-touch/trunk/view/head:/ubuntu-device-flash/snappy.go#L123
[13:17] <ogra_> (do you see it pload properly in the serial log)
[13:17] <asac> architecture: + platform: (optional, default: generic)
[13:17] <sergiusens> asac, we had platform; but in the end this would be in the kernel snap
[13:18] <plars> reading uboot.env
[13:18] <ogra_> sergiusens, deviceChannel ?
[13:18] <plars> ogra_: yep :)
[13:18] <sergiusens> ogra_, --device
[13:18] <asac> sergiusens: wait, in OEM snap specifying what platform this is for makes sense
[13:18] <asac> hmm
[13:18] <ogra_> ah, s.device ... right
[13:18] <asac> oor we just say kernel: pi2
[13:18]  * ogra_ looked to far down
[13:18] <asac> default: generic
[13:18] <asac> doesbnt matter for OEM snap... we will revisit those fields for 16.04 anyway then
[13:18] <sergiusens> asac, ok; makes sense
[13:21] <plars> ogra_: surely it's pulling the serial from the board somehow and not from the image though, right?
[13:21] <ogra_> plars, yeah, same thing
[13:21] <ogra_> bcm2709.serial=0x8b04db1c
[13:21] <ogra_> cmdline option
[13:21] <ogra_> seeded from the binary blob
[13:22] <ogra_> plars, if you intercept the boot, do you see "args" ?
[13:22] <ogra_> in printenv
[13:23] <plars> ogra_: no
[13:23] <plars> ogra_: was there something in the kernel needed for this?
[13:23] <ogra_> no
[13:23] <ogra_> thats pre-kernel
[13:23] <ogra_> but ...
[13:23] <ogra_> mmcargs=setenv bootargs "${args} console=tty0 console=ttyAMA0 root=${mmcroot}"
[13:24] <ogra_> and snappy_boot calls "run mmcargs" on first boot
[13:25] <ogra_> plars, ah, and:
[13:25] <ogra_> loadfdt=fdt addr 0x100; fdt get value args /chosen bootargs
[13:25] <ogra_> try running that line
[13:25] <ogra_> and then see if args= exists
[13:25] <ogra_> if so, add it to your non snappy boot
[13:26]  * ogra_ forgot you have to pull te dtb from ram in to have the values
[13:26] <plars> ogra_: just "args"? no still not set
[13:26] <ogra_> that was the bit :)
[13:26] <plars> ah, I see
[13:26] <plars> get value args /chosen bootargs
[13:26] <plars> hang on let me try that
[13:26] <ogra_> fdt addr 0x100
[13:27] <ogra_> you need that first
[13:27] <plars> ogra_: running loadfdt did it, thanks!
[13:27] <ogra_> cool
[13:27] <mvo_> ogra_: we have a raspi2_armhf platform now!
[13:27] <ogra_> mvo_, yeah, i see that !!
[13:28] <ogra_> mvo_, i'll triger an edge build to see of it imports fine
[13:28] <asac> on rolling? nice ;)
[13:28] <ogra_> no
[13:28] <ogra_> https://system-image.ubuntu.com/ubuntu-core/15.04/edge/
[13:28] <ogra_> on 15.04 edge :)
[13:29] <ogra_> build running
[13:30] <mvo_> I added it to rolling and 15.04 so we should be fine
[13:30] <mvo_> now I guess we just update reference image and the upgrade problem is solved for good
[13:30]  * mvo_ is happy
[13:33] <ogra_> mvo_, the importer will now run 5min longer
[13:33] <ogra_> or so
[13:33] <ogra_> new arch means moar diffing
[13:34] <mvo_> meh
[13:35] <clobrano> I saw an unexpected (for me) behavior in hw-assign: each call to hw-assign overwrite the udev rule for the <snap>, while the list of write_path in <snap>.apparmor.json.additional is just updated
[13:36] <clobrano> so that udev rule only contains 1 device node at a time
[13:37] <asac> hmmm
[13:37] <asac> http://paste.ubuntu.com/12448771/
[13:37] <asac> Chipaca: mvo_: the above looks pretty garbled formatting wise
[13:38] <asac> is it possible to improve the way we present multi-line yaml?
[13:38] <asac> like the content: one for the network interface looks much nicer
[13:39] <asac> jdstrand: see clobrano's questiona bout hw-assign... feels like a bug
[13:39] <asac> 6 lines above
[13:41] <jdstrand> yes, it does
[13:42] <clobrano> http://paste.ubuntu.com/12448814/
[13:42] <mvo_> asac: I think the problem is the \t it can only represented in the "" form. looks at the startup above, this is what it should look like
[13:42] <jdstrand> clobrano: can you file a bug at https://bugs.launchpad.net/snappy/+filebug ? you might add a comment that it is (perhaps) hindering on your progress on the other bug
[13:43] <clobrano> sure
[13:43] <clobrano> well, it is not really hindering, I just saw that I had to change device name at each try :D
[13:46] <dholbach> fgimenez, I'm not sure this helps a whole lot, but I thought I'd share what I was playing around with: https://code.launchpad.net/~dholbach/snapcraft/examples-tests/+merge/271649
[13:46] <dholbach> fgimenez, great work on adding the examples test
[13:46] <dholbach> tedg, the qmldemo example is still broken on wily - do you know how to fix it?
[13:46] <fgimenez> dholbach, i'm looking into it right now, thanks a lot :)
[13:47] <dholbach> <3
[13:48] <sergiusens> pitti, you once told me, iirc, that we can enable autopackage tests on PPAs?
[13:48] <pitti> sergiusens: that's the plan, yes
[13:49] <pitti> not implemented yet because too many diversions and kernel testing having higher prio
[13:49] <pitti> sergiusens: but I definitely want this too; might still be a few weeks, though
[13:49] <sergiusens> pitti, oh, still on paper :-) ah, but sooner than I thought :-)
[13:50]  * sergiusens puts a 1 month away reminder
[13:50] <pitti> sergiusens: is this something important/critical for snappy development?
[13:50] <tedg> dholbach: I think so, I just need to get a wily VM setup
[13:51] <sergiusens> pitti, we just added lots of tests; we run them before merging but would be nice to run on package build as well
[13:51] <sergiusens> pitti, so important, but not critical
[13:51] <pitti> sergiusens: i. e. next week I need to work on prototyping the "run deb chroot on snappy" thingy for the snappy sprint the following week; if this is important, I'm happy to work on that on the sprint and have you test it :)
[13:51] <sergiusens> pitti, I will defer to elopio on that ;-)
[13:51] <sergiusens> pitti, sounds good!
[13:52] <pitti> sergiusens: the mere act of running them is actually fairly simple; it's mostly how to trigger them and what to do with the results which is the interesting bit there
[13:54] <tedg> jdstrand: Is there a permission to read/write the user's home directory?
[13:54] <sergiusens> yeah, I hear you, integrating it all together is what mostly blocks things from getting done
[13:55] <sergiusens> tedg, wrt question, I was thinking that the wrapper for binaries should cd to $SNAP_APP_DATA_PATH (the one in $HOME)
[13:55] <clobrano> jdstrand: here it is https://bugs.launchpad.net/snappy/+bug/1497299
[14:04] <tedg> sergiusens: Why?
[14:06] <sergiusens> tedg, there was a docker bug that affected that; but yeah; I guess it will break badly in most occasions
[14:06] <sergiusens> tedg, although no binary can read outside its containment either
[14:06] <tedg> sergiusens: I think we should probably have a conversation about how we expect UAL to work with core launcher.
[14:07] <tedg> sergiusens: I'm curious if we want to seperate out "run as a user" and "run as root"
[14:07] <tedg> sergiusens: For instance, setting the XDG* variables so that most things will "just work" with that being their only writable directory
[14:11] <kickinz1> tedg, sergiusens docker is not allowed by apparmor to read data outside its own $HOME/apps/docker/version directory.
[14:13] <tedg> sergiusens: mvo_: Do we have an idea how to finish off this branch? https://code.launchpad.net/~mvo/snapcraft/more-python-apt/+merge/270831
[14:15] <sergiusens> tedg, I like it already, we just need to trim manifest.txt
[14:24] <sergiusens> tedg, my brain will explode with this one https://code.launchpad.net/~ted/snapcraft/inception/+merge/271656 :-P
[14:24] <tedg> sergiusens: Yeah, it's kinda fun. We should run all our tests in the snappy CI service :-)
[14:25] <sergiusens> tedg, hah, this is rather useless though as we depend on build tools still but rather interesting in itself
[14:25] <sergiusens> tedg, if we get rid of build tools we can certainly cater for this
[14:26] <tedg> sergiusens: Yeah, it feels like something we should do. If we want other people to include a snapcraft.yaml in their repos.
[14:33] <elopio> ev: can we talk about jenkins?
[14:33] <ev> elopio: sure!
[14:34] <elopio> ev: I played with the svn config sync plugin and it needs to push to git. For that we need to put the ssh public key of jenkins on launchpad.
[14:34] <elopio> ev: is that ok for you?
[14:41] <ev> elopio: I'm confused; what's the security risk you're eluding to?
[14:42] <elopio> ev: no security risk. Just to push the config to the git branch in launchpad we need the key.
[14:43] <ev> elopio: and while we're talking about Jenkins, can I ask that you guys focus on getting your Canonistack Jenkins driving daily testing of Snappy through SPI before all the time gets eaten up transitioning to Jenkins as a service?
[14:43] <ev> I didn't realise this was going to require so much work for you guys, otherwise I wouldn't have suggested Jenkins as a service until we saw a steady stream of results in SPI.
[14:44] <ev> elopio: can't you have a separate ssh key that you generate for the purpose?
[14:44] <ev> something like http://superuser.com/a/232406
[14:45] <ev> oh, I guess when we're dealing with public keys it really doesn't matter
[14:45] <ev> so yeah, I think it's reasonable for you to put the public key in LP
[14:45] <ev> +1 :)
[14:46] <elopio> ev: I'm not sure if I can tell the plugin to use a different key.
[14:46] <ev> yup, that's fine
[14:46] <elopio> ev: ok, let me ask for it on an RT.
[14:46] <ev> cool
[14:46] <ev> please do CC me
[14:46] <elopio> ev: what's taking long is that we had a release this week, and we have another next week.
[14:46] <elopio> so we actually didn't touch ci at all this week.
[14:47] <ev> elopio: is there anything I can do to help you guys get your tests running daily on SPI?
[14:48] <elopio> ev: I think not, this was the last question I had. Let me give it another try after the snapcraft release.
[14:48]  * ev nods
[14:48] <Chipaca> asac: mvo_: tbh the file-contents-as-value-of-key thing is rather nasty, and i'd change it if we're going to do that long term
[14:49] <Chipaca> asac: mvo_: ie treat values that are files differently, e.g. yaml points to /1.0/packages/{pacakge}/config/{uuid}, and you GET/POST to .../uuid to operate on the file
[14:52] <tedg> pitti: Have you talked to the libertine container folks about running deb containers?
[14:52] <pitti> tedg: libertine? no, I didn't (I don't even know what that is)
[14:53] <pitti> tedg: running it as an actual container is fine; but AFAIUI sabdfl suggested running them in a chroot with shared processes
[14:53] <mvo_> Chipaca: I don't follow, sorry. but I'm in a meeting rightnow so maybe I just need to re-parse this
[14:53] <pitti> so that from within that you can "admin" snappy
[14:53] <tedg> pitti: It does both, depending on kernel version
[14:53] <pitti> this comes at quite a high cost and loss of reliability, though
[14:53] <tedg> pitti: Basically it is the same feature for phones
[14:53] <tedg> pitti: It would be nice if we had one set of this code :-)
[14:53] <asac> Chipaca: yeah sounds reasonable
[14:54] <asac> Chipaca: like a resource:// thingy
[14:54] <asac> is there any yamnl extension that is standard for such?
[14:55] <pitti> tedg: ah, https://launchpad.net/libertine ? thanks for the pointer, I'll look at that
[14:55] <tedg> pitti: You should chat with ChrisTownsend about it some
[14:55] <tedg> pitti: I think they're basically the same goals, it'd be nice if we converged on one solution for phone/snappy
[14:56] <ChrisTownsend> pitti: tedg: Yeah, I'm here.  bregma too
[14:56] <pitti> tedg: not sure -- this sounds like you actually want to containerize and isolate this
[14:56] <pitti> tedg: but either way, I'll look at this and talk to ChrisTownsend, thanks
[14:57] <tedg> pitti: It started more with containers, but because of the old kernels on the phones it grew chroot. So the description might not match the implementation in that regard.
[14:58] <ChrisTownsend> tedg: Libertine supports both chroot and lxc.
[15:00] <Chipaca> dunno
[15:03] <ChrisTownsend> pitti: FYI, I'll be around today, but off next week.  bregma can definitely help too.
[15:14] <pitti> ChrisTownsend: does this do anything like discovering sysv init and systemd jobs in the chroot and start them from the host, but properly chrooted?
[15:15] <pitti> ChrisTownsend: that seems to be the most brittle/heuristic aspect of all
[15:16] <pitti> ChrisTownsend: for what Michael and Mark have in mind, it's probably better to actually start the deb env as a proper container with all services and its own init, but then provide a mode to just chroot into it if you want to do snappy admin/devel
[15:16] <pitti> that's the model which seems most robust to me and still reasonably fulfill the requirements
[15:16] <ChrisTownsend> pitti: Wouldn't we want to use LXC instead of a chroot?
[15:17] <ChrisTownsend> pitti: The chroot is only intended for kernels that don't support unprivileged LXC's.
[15:17] <pitti> ChrisTownsend: with LXC you can't see/admin the host processes and bits; AFAIK this was the raison d'être for this
[15:17] <pitti> ChrisTownsend: oh, and it's certainly a privileged container
[15:17] <dholbach> fgimenez, I could imagine that the problem has to do with the fact that the examples directory is not mentioned in setup.py anywhere, but I'm not 100% sure yet
[15:18] <tedg> pitti: I was wondering if we could setup snappy-remote to make it so that it would know it is in an embedded situation.
[15:18] <ChrisTownsend> pitti: Hmm, ok.  I really don't have any sense of your requirements, so I may be off base in my replies:)
[15:18] <tedg> pitti: So then you could manage your snappy system, but it'd actually be using REST.
[15:18] <pitti> ChrisTownsend: the idea of that was not to treat some debs as an "app", but to provide a "classic ubuntu" like environment for development and admin'ing snappy
[15:18] <pitti> mvo_: ^ right?
[15:18] <tedg> pitti: I think that sabdfl's concern was being able to manage the core system from the container.
[15:18] <pitti> ChrisTownsend: well, neither have I, I'll mostly come to this as a consultant :)
[15:19] <mvo_> pitti: yes, right, its a full environment to "sh" into
[15:19] <ChrisTownsend> pitti: lol, ok
[15:19] <pitti> tedg: right; so it can't be a container when you get a shell into it
[15:19] <tedg> pitti: It could if the snappy tool used the REST interface transparently.
[15:19] <pitti> tedg: that doesn't preclude actually booting it as a container, so that you have the .deb services running properly
[15:19] <dholbach> fgimenez, or integration-tests/manage.py maybe
[15:19] <mvo_> pitti: and one use case if e.g. install and run tcpdump on snappy
[15:19] <mvo_> pitti: or install strace/gdb etc
[15:19] <pitti> right
[15:20] <tedg> Hmm, strace is trickey
[15:20] <pitti> so for that mode you'd chroot into the root fs; you would see the snappy pids, but can't do /etc/init.d/apache2 restart (you need a "container shell" for that)
[15:20] <tedg> mvo_: So it would have to be completely unconfined
[15:20] <pitti> so it's always a bit weird
[15:20] <pitti> yes, this isn't a security thingy -- you want full access
[15:21] <tedg> mvo_: Is the concern that you'd need gdb/strace for things already installed as snaps, or for building debugging them?
[15:22] <pitti> both I think
[15:22] <ChrisTownsend> I'm thinking the way we make libertine containers may be too confining for this purpose.
[15:23] <pitti> yeah, it's two rather different use cases
[15:23] <ChrisTownsend> We make an unprivileged chroot and only bind mount a few necessary host dirs.
[15:23] <mvo_> tedg: worth clarifying in budapest, but my understanding was that it is intended to troubleshoot a existing system
[15:23] <pitti> ChrisTownsend: libertine basically is a snap for a bunch of debs, treated as an "app", right?
[15:24] <tedg> pitti: Well the idea is that it maintains a container long term, so you end up apt-get'ing more over time.
[15:24] <tedg> pitti: So it's not a one-shot type thing.
[15:24] <pitti> well, I'll take a look at it either way
[15:25] <pitti> I don't feel that this "deb env on snap" thing will be a lot of code, it's just getting the concept right
[15:25] <pitti> thanks for your input! /me needs to leave now
[15:34] <tedg> 'night pitti !
[15:34] <elopio> sergiusens: tedg: could there be a way for a snapcraft.yaml to include another snap?
[15:34] <tedg> Man I hate my wifi
[15:35] <tedg> Oh, oh! An inception2 branch!
[15:35] <sergiusens> elopio, crazy talk
[15:35] <sergiusens> elopio, just use reusable parts ;)
[15:35] <sergiusens> elopio, the snap as a binary or the snap layed out as an overlay?
[15:36] <tedg> I mean we could decompress it and just copy the same files into the new snap.
[15:36] <tedg> The potential use I could see is that someone wants to include a particular version of a framework.
[15:37] <elopio> sergiusens, tedg: I don't know. I'm just thinking. Lets say I want to test hello-world.snap. Maybe I don't want to put the test binaries in the production snap because they import testtools and a lot of other stuff.
[15:37] <elopio> so, maybe I could write a test snap that includes the production snap.
[15:37] <elopio> then I install hello-world-test.snap. And it has both the production and test binaries.
[15:38] <tedg> I was thinking that perhaps we could have a plugin you could add to your snapcraft that would do some testing things. Like run the bins under gdbserver.
[15:38] <tedg> That's kinda how QtCreator does things. It builds a new click that has the gdbserver connection setup.
[15:38] <tedg> Then it connects to that.
[15:39] <elopio> that sounds cool. But not what I'm after here. I want a test that runs the binary and checks the output. And a test that starts the service and makes requests to it.
[15:41] <tedg> Hmm, would you want that to be an external package then?
[15:41] <tedg> foo.snap and foo-tests.snap
[15:42] <elopio> tedg: maybe. For small tests that don't have a lot of dependencies, I can just add a binary foo-test to foo.snap.
[15:43] <elopio> but for complex things, like a full suite on the docker snap. We probably want to import a testing framework, and testing helpers. For that I would like a docker-tests.snap.
[15:43] <tedg> I wonder if we just make it super easy to make that foo-tests.snap and then you don't care how small/big it is.
[15:44] <elopio> but that docker-tests.snap should have access to everything docker.snap has. And we don't have dependencies to say docker-test.snap depends on docker.snap, so I'm thinking we should bundle docker.snap inside docker-tests.snap, somehow.
[15:45] <elopio> tedg: that foo-tests.snap it's super easy to make with snapcraft already, just a python project gives us everything we can possibly need.
[15:45] <elopio> but how to call foo from foo-tests.snap?
[15:46] <tedg> foo-tests.run
[15:46] <tedg> It would have to export its own binary
[15:46] <fgimenez> dholbach, mmm i think it has to do with the examples directory not being available from plainbox's ${PLAINBOX_PROVIDER_DATA}, the symlink at http://bazaar.launchpad.net/~fgimenez/snapcraft/build-examples-test/view/head:/integration-tests/runtests.sh#L40 is not properly created
[15:48] <elopio> tedg: right, how to call foo.run from foo-tests.run if foo.snap is not installed.
[15:48] <elopio> oh wait, I'm sorry. You are working on the release. I forgot I said we will discuss about this next week.
[15:48] <elopio> tedg: sergiusens: I'll add a meeting to the calendar.
[15:48] <tedg> elopio: Wait, why don't you install foo.snap ?
[15:48] <dholbach> fgimenez, my last hypothesis was that the examples directory was missing because it wasn't mentioned in neither setup.py nor integration-tests/manage.py, but that might be my lack of understanding of plainbox
[15:49]  * tedg wants to install all the foo's!
[15:49] <dholbach> I need to go now though, dinner with the whole family
[15:49] <dholbach> have a great weekend everyone!
[15:50] <elopio> tedg: the runner would have to know that if you install foo-test.snap, you also need foo.snap installed. That maybe is not a big deal, but sounds like old world dependencies.
[15:50] <elopio> tedg: and one thing I don't know is if foo-test.snap would be able to access the data files of foo.snap. And would it have permission to call foo.run?
[15:53] <tedg> elopio: It wouldn't by default, but I think that foo-test.snap could be unconfined.
[15:54] <tedg> elopio: We really don't need confinement there.
[15:55] <elopio> tedg: we have used unconfined tests in nice ways on the phone. I like that.
[15:55] <elopio> on the other hand, if you are confined to the same things that your test subject is, that will give you a view closer to the consumer of that subject. And you could push the tests to the store.
[15:57] <tedg> Sure, I guess I feel the most important part is that the foo.snap stays in its natural confinment. Even if we peak in, we want it to run as closely to the real world as possible.
[15:57] <tedg> As soon as we adjust its snap in anyway, we're getting away from that.
[15:59] <elopio> that's true.
[16:00] <elopio> and bundling it into another snap doesn't leave it untouched at all.
[16:00] <sergiusens> elopio, tedg I would make snap testing at this stage more like blackbox testing in any case; check snappy [config|service|...]
[16:02] <elopio> sergiusens: I want this to be blackbox, always. But how would you install those blackbox tests into the snappy board?
[16:04] <elopio> ev: we don't know where the public ssh key is. Can you please give us a hand?
[16:07] <sergiusens> elopio, composing the snap is not a good idea; you will most likely need to change apparmor rules and such
[16:07] <sergiusens> elopio, drive it externally; use ciaas?
[16:07] <elopio> sergiusens: yes, tedg made good points about that. I don't like it now.
[16:08] <elopio> sergiusens: do you mean like we do for snappy? Use something like adt-run that will copy the tests into the testbed?
[16:09] <elopio> or do you mean to write the tests using only snappy-remote and ssh?
[16:09] <tedg> barry: So we want to be able to set "sys.executable" before running setup.py. Is there a good way to do that?
[16:09] <tedg> barry: I was hoping I could use python3 -c, but that doesn't seem to get the order of operations right.
[16:10] <tedg> I probably should use "right" since this is kinda a hack :-)
[16:16] <plars> ogra_: https://developer.ubuntu.com/en/snappy/start/#snappy-raspi2 is still correct on building rpi2 images? the oem and device bits especially?
[16:16] <ogra_> plars, well, ther versions are outdated, but the rest is fine
[16:16] <ogra_> http://people.canonical.com/~platform/snappy/raspberrypi2/ has the bits
[16:17] <plars> ogra_: do you know when those will be in the store?
[16:17] <ogra_> this will stay valid for another week or two until we start building automatically
[16:17] <ogra_> the pi2 snap is in the store already
[16:17] <ogra_> you can use that one as well (but it is the same)
[16:17] <plars> ogra_: so you don't need to download them separately now?
[16:18] <ogra_> the snap
[16:18] <ogra_> you still need the device tarball ... we commited the first bits to system-image today for that bit
[16:18] <plars> ogra_: ah, so that will still be needed for a while I guess? :(
[16:19] <ogra_> well, next stable release will be fully integrated
[16:19] <plars> cool
[16:21] <barry> tedg: sys.executable gets set from argv0.  on linux i don't believe there's an easy way to override that before the runtime starts up (there is on osx)
[16:22] <tedg> barry: Can I do it in some sort of wrapper script?
[16:22] <tedg> barry: I'm looking at the import stuff and it seems to be all for importing modules.
[16:22] <tedg> barry: Where I just want to import a script.
[16:23] <barry> tedg: right, importlib is all about importing modules.  i think you could do it in a wrapper that calls exec with the intended argv0
[16:23] <barry> tedg: but, why do you want to set sys.executable?  usually that just comes from the shebang line or the prompt
[16:24] <tedg> barry: The problem is that setuptools is pulling it when we're building the snap, which then is the path to the build version and we want to to be the path of the installed version when on the final system.
[16:25] <barry> tedg: did you see that setuptools supports --executable (iirc) which lets you set that explicitly?
[16:26] <tedg> barry: In theory, but I couldn't get it to work really. It seems to be building the cmdline tools at install time instead of build time, and the parameter only works on build.
[16:26] <tedg> barry: I don't get anything in usr/bin of the build directory
[16:27] <barry> tedg: ah.  well, that's less than helpful ;)
[16:28]  * barry thinks
[16:29] <tedg> Hmm, doesn't seem subprocess allows setting arg0 to a different value.
[16:29] <barry> tedg: right, i think you need to use os.exec*()
[16:30]  * tedg takes the kid gloves off
[16:31]  * barry digs through setuptools
[16:38] <barry> tedg: http://stackoverflow.com/questions/28575431/setuptools-entry-points-console-scripts-have-specific-python-version-in-shebang
[16:38] <barry> i haven't tried it but that's a new one for me
[16:39] <sergiusens> barry, fwiw, I saw the sys.executable trick in a debian bug
[16:40] <barry> sergiusens: "sys.executable trick"?  means setting that attribute early enough?
[16:41] <sergiusens> barry, yeah, import sys\nimport setuptools\nsys.executable = 'path_to_exec'\nsetup(...)
[16:41] <sergiusens> barry, here it is http://stackoverflow.com/questions/17237878/changing-console-script-entry-point-interpreter-for-packaging
[16:42] <barry> sergiusens: oh that's cute
[16:43] <sergiusens> tedg, btw, seem pyversions -r would solve the python3.4/3.5 issue
[16:43] <barry> py3versions
[16:43] <barry> but that doesn't take a -r
[16:44] <sergiusens> -i I guess then
[16:44] <barry> would you want the default python3 version? or will you allow people to select 3.4/3.5?  (note that wily can have both installed, and 3.4 is going away for X)
[16:44] <tedg> Cool, good. That was next on my list of things to fix.
[16:45] <tedg> barry: If they grab the default python3 plugin we'll grab what ever is default.
[16:45] <barry> tedg: ack
[16:45] <tedg> barry: But we allow people to grab any package they want, so eh, whatevs
[16:47] <tedg> I really don't want to be writing a shell script to call a python script to call python.
[16:47] <barry> tedg: i think writing/amending the setup.py is probably the better way
[16:47] <barry> not great, but better
[16:48] <barry> tedg: this seems like a possible upstream related bug: the
[16:48] <barry> https://bitbucket.org/pypa/setuptools/issues/204/support-for-interpreter-argument-in
[16:48] <tedg> Yeah, that seems related.
[16:49] <tedg> console scripts is slightly out of the fold.
[16:49] <asac> sergiusens: around?
[16:51] <barry> tedg: https://bitbucket.org/pypa/setuptools/issues/272/generated-script-shebang-line
[16:52] <tedg> Cool, thanks barry
[16:54] <barry> tedg: jaraco is a local python dev and we've got a hackathon next week.  i'll ping him personally about it (if i can remember :)
[16:59] <tedg> barry: We should be landing most of the python snapcraft bits today/early next week. It'd be interesting to get some feedback at the hackfest about it.
[16:59] <tedg> barry: Those are the people we want to delight :-)
[17:00] <barry> tedg: awesome!  i'd love to take a closer look.  will you be sending more details to the mailing lists, or is there a wiki i can look at?
[17:00] <tedg> barry: We're working on docs. I think when those get real we'll spam the world.
[17:00] <tedg> barry: As an example you can kinda see things with the examples
[17:00] <barry> tedg: sounds great.  i will help spread the word
[17:01] <tedg> barry: Heh, that was bad English
[17:01] <tedg> barry: We have examples :-)
[17:01] <barry> :)
[17:04] <tedg> barry: I think the other thing that will be interesting for Python folks is that we're going to allow builds in LP, but that can pull deps from non-archive sources. So using pip or whatever.
[17:05] <tedg> That's in progress, but a big change that folks have wanted.
[17:07] <barry> tedg: that's *very* interesting
[17:08] <tedg> Yeah, I'm super excited about it. But really cjwatson did all the work. I'm just being a fan boy.
[17:09] <barry> we're all cjwatson fanboys here :)
[17:09] <fgimenez> nice weekend everyone o/
[17:16] <elopio> sergiusens: do you mean run clean twice in integration tests?
[17:16] <elopio> or in unit tests?
[17:17] <sergiusens> elopio, either or both ;-)
[17:17] <sergiusens> elopio, just add snapcraft clean to the make-clean test
[17:17] <elopio> sergiusens: I don't like this many mocks, so I'll do it in integration.
[17:17] <elopio> sergiusens: yes, on it.
[17:21] <elopio> sergiusens: pushed. Thanks for the suggestion.
[17:24] <elopio> ¿dónde están hablando de eso? No veo ninguna discución en IRC.
[17:24] <elopio> *discusión
[17:25]  * beuno blinks
[17:25] <elopio> wrong channel.
[17:25] <beuno> :)
[17:25] <sergiusens> elopio, who was that for :-P
[17:26] <elopio> sergiusens: balloons. He's learning spanish.
[17:26] <sergiusens> elopio, ah :-)
[17:28] <tedg> We should have a snappy-es channel. I really need an excuse to have better Spanish.
[17:29] <beuno> tedg, I think the plan of record is to slowly just start speaking in spanish and make english look out of place
[17:29] <tedg> beuno: si, me gusta snappy
[17:30] <tedg> Is their something similar to "our Spanish snappy overlords"? ;-)
[17:31] <tedg> Never thought about it, but are their different memes for non-English languages? There must be.
[17:32] <elopio> https://translations.launchpad.net/snappy
[17:32] <elopio> galician is winning
[17:33] <elopio> tedg: this is a good one for you https://s-media-cache-ak0.pinimg.com/736x/1c/ed/4c/1ced4cd96aef0fe01a4a240f9edeb606.jpg
[17:33] <tedg> Heh, I always find most interesting what doesn't get translated. Like "2fa"
[17:35] <tedg> elopio: I'm not sure what that woman has to do with the mob.
[17:35]  * tedg is probably missing some context
[17:35] <sergiusens> tedg, our standups have mostly spanish speaking people in them anyways ;-)
[17:36] <sergiusens> we can switch from german to spanish every other day
[17:36] <tedg> Ha, that won't be confusing.
[17:37] <tedg> If there's one thing I can say, it is that my Spanish is better than my German.
[17:37] <elopio> tedg: "that woman" https://en.wikipedia.org/wiki/List_of_El_Chavo_characters#Do.C3.B1a_Florinda
[17:40] <tedg> Interesting, I had no idea.
[17:43] <ev> elopio: the system public ssh key? You'll want to file an RT to get that
[17:43] <elopio> ev: yes, look at the rt CCed to you. They don't know where it is either.
[17:44] <ev> ha, having a look
[17:53] <elopio> sergiusens: is this python config part going to be in the wiki?
[17:55] <sergiusens> elopio, if it can, yes
[18:01] <elopio> sergiusens: tedg: am I doing something stupid here? http://paste.ubuntu.com/12451892/
[18:02] <elopio> I'm getting FileNotFoundError: [Errno 2] No such file or directory: '/tmp/socat/snap/./bin/snapd-socat.wrappe│
[18:02] <elopio> r'
[18:12] <elopio> got it, I was missing my binary in the snap:
[18:15] <elopio> now, what kind of apparmor should I put in this to work?
[18:30] <sergiusens> elopio, that is an iteration ;-)
[18:31] <sergiusens> tedg, you don't need the filename stuff for python2/3 plugins themselves?
[18:34] <sergiusens> elopio, do you mind rechecking this one https://code.launchpad.net/~fgimenez/snapcraft/build-examples-test/+merge/270798 ?
[18:50] <elopio> sergiusens: that one is not ready.
[19:05] <tedg> sergiusens: ?
[19:07] <sergiusens> tedg, I saw your MP; is it possible for pip installed things to provide scripts and us not taking care of that?
[19:08] <tedg> sergiusens: They won't be binaries, right?
[19:08] <tedg> sergiusens: It's only the first binary executed that needs it.
[19:09] <sergiusens> tedg, ah, I can have my snapcraft.yaml be an integrator one that just has a requirements.txt and it installs a binary and and and
[19:10] <sergiusens> tedg, it is a harder problem to solve though
[19:10] <tedg> sergiusens: But then you don't have setup.py redoing the shebang
[19:10] <sergiusens> well, at least not with this mechanism
[19:10] <sergiusens> tedg, so when using pip the shebang thing is clean?
[19:12] <tedg> sergiusens: Seems to be
[19:13] <sergiusens> tedg, ok then; finally some good news :-)
[19:13] <sergiusens> with python
[19:13] <sergiusens> ;-)
[19:13] <tedg> At least for the ones in nova, it seems most of the modules just don't have a shebang.
[19:14] <sergiusens> btw, if you want to look https://code.launchpad.net/~sergiusens/snapcraft/organize/+merge/271702
[19:14] <sergiusens> elopio, maybe too ^
[19:15] <tedg> K, should it delete the copy plugin?
[19:17] <sergiusens> tedg, not sure we want to yet, do we?
[19:18] <sergiusens> tedg, if we remove the copy plugin we need to drive everything through a makefile
[19:18] <sergiusens> tedg, or of what type would the project be?
[19:18] <tedg> Hmm, yeah... not sure.
[19:19] <sergiusens> this was the question that I accidentally skipped from my notes earlier today
[19:19] <tedg> Was thinking the non-specified type, but we kinda play tricks with that right now.
[19:19] <sergiusens> tedg, yeah, no type means something different now
[19:19] <sergiusens> tedg, we can have the 'nothing' type ;-)
[19:19] <tedg> 'notnull'
[19:20] <sergiusens> hmm, 'nop'
[19:20] <sergiusens> :-)
[19:20]  * tedg gets with the program 'nada'
[19:21] <sergiusens> nada is good :-)
[19:23] <sergiusens> tedg, hmm, moot to remove the copy plugin though; as we move, not copy