=== retrack is now known as Guest2661 [06:26] good morning [07:05] good morning [07:17] hey [07:18] mvo, howdy, question for you ... do you know why the docker group is added to the ubuntu-core image? is docker needed for some snappy features to work? or it's just to enable the docker snap to be installed since groups can't be added later? [07:18] context is snappy personal/desktop and knowing if that group is needed there [07:40] seb128: yes, its needed for docker and we can not yet add groups later [07:40] seb128: it won't be needed on desktop [07:40] mvo, k, why is docker special cased? Just because it's a popular demo? [07:41] or is it needed for snappy features to work? [07:41] seb128: because it was the only way to get docker working :) and its a important package for us [07:42] mvo, but it's not on the personnal desktop image? [07:42] just making sure :-) [07:42] well, let's start without it [07:42] we can do similar hack later if needed [07:43] seb128: yeah, just go without it for now [07:43] but I guess we are going to need a solution to this groups problem [07:43] seb128: we sort of have, we have libnss-extrasusers on the image, we just have no mechanism that can write to it [07:43] ah, I see [07:43] mvo, thanks :-) [07:44] and yeah, its a known issue, we need to solve that and add adduser support for extrasusers plus a snappy extension that can add additional user === davmor2_hols is now known as davmor2 [08:27] ppisati, do you have an image anywhere with the working 1G RAM on the RPi2 ? i have tried all combos of uboot and kernel that i can find and cant ever get more than 256M [08:27] ogra_: yes i do [08:28] ogra_: https://github.com/piso77/ubuntu-embedded [08:28] ogra_: sudo ./make_img.sh -b raspy2 -d 14.04 [08:28] all i found was the early version on p.u.c [08:28] ogra_: or i can build one for yo [08:28] ogra_: and put it somewhere [08:28] oh, i meant a binary, no i can build it myself ... is that buildin an actual snappy image ? [08:28] ogra_: nope, vanilla ubuntu [08:29] ogra_: but i'm using uboot, ppa kernel, etc [08:29] ok [08:30] seems to me like the kernel does not actually get to change any memory settings with the current setup i have ... it always only shows whatever the start.elf initialized [08:31] ogra_: do you have a dmesg of your kernel? [08:31] ogra_: or a log of the entire boot? [08:31] several, yes, my screen is set to auto-logging :) [08:31] ogra_: but try that img, has 1G of ram [08:32] i will === dholbach_ is now known as dholbach === dholbach_ is now known as dholbach [08:57] ppisati, (sorry, was dragged into a meeting) here is the screen log of various boots http://people.canonical.com/~ogra/scree-rpi.log [08:58] ogra_: do we know what this is https://www.raspberrypi.org/downloads/snappy/ [08:59] ogra_: my colleague installed it and it's some version of vivid [08:59] (the third last is your old 3.18 image from people.u.c, which was the only one i had 1G with yet, all other boots have either 128 or 256M depending on which start.elf i use) [08:59] zyga, it should be some old snappy version of vivid [08:59] i'm just trying to get an updated image together (but struggle with it a little) [09:00] ogra_: for rpi2? [09:00] yes [09:00] ogra_: thanks, I have kissiel here who will gladly use one when available [09:01] zyga, I missed some of your conversation, are you talking about the image? [09:01] i'll send a mail announcement once it is done ... [09:01] kissiel: yes [09:01] zyga, cool [09:01] kissiel: it seems that there is no updated image yet, just wait for ogra_'s announcement [09:01] ogra_: thanks, I owe you way too many beers, perhaps at fosdem :) [09:01] zyga, I'll build it from scratch then [09:02] heh :) [09:10] ogra_: what you see if you do a cat /proc/meminfo? [09:11] (RaspberryPi2)ubuntu@localhost:~$ cat /proc/meminfo |grep Total [09:11] MemTotal: 119468 kB [09:11] SwapTotal: 0 kB [09:11] VmallocTotal: 1941504 kB [09:11] CmaTotal: 8192 kB [09:11] with the most recent boot ... [09:11] ubuntu@raspy2:~$ uname -a [09:11] Linux raspy2 3.19.1-11-generic-bcm2709 #11-Ubuntu SMP PREEMPT Tue May 26 10:34:36 UTC 2015 armv7l armv7l armv7l GNU/Linux [09:11] (do you need all of it ?) [09:11] ubuntu@raspy2:~$ cat /proc/meminfo [09:11] MemTotal: 947020 kB [09:11] ... [09:11] ogra_: i can put the img somewhere if you want [09:11] (RaspberryPi2)ubuntu@localhost:~$ uname -a [09:11] Linux localhost.localdomain 3.19.1-11-generic-bcm2709 #11-Ubuntu SMP PREEMPT Tue May 26 10:34:36 UTC 2015 armv7l armv7l armv7l GNU/Linux [09:11] same kernel [09:12] it's definitely the binary blob or uboot [09:12] do you have anything special on the cmdline ? [09:12] wait [09:12] ubuntu@raspy2:~$ cat /proc/cmdline [09:12] earlyprintk console=tty0 console=ttyAMA0 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait noinitrd [09:12] ogra_: do you? [09:12] console=tty0 console=ttyAMA0 root=/dev/disk/by-label/system-a init=/lib/systemd/systemd ro panic=-1 fixrtc [09:13] nothing :) [09:14] * ogra_ grumbles about /var/log/dmesg being empty [09:14] ogra_: i'm uploadinf that img [09:14] ppisati, thanks, i'll try mine with your bootloader [09:14] i wonder if it is the uboot.env that loic created [09:15] ogra_: all the bootloader binaries i'm using [09:15] ogra_: are here, btw [09:15] ogra_: https://github.com/piso77/ubuntu-embedded/tree/master/boards/raspy2/bootloaders [09:15] well, i tried them [09:16] didnt make any difference [09:16] you dont load an initrd ... [09:16] perhaps our load address is wrong and we trash something when loading it [09:16] ogra_: but the initrd is discarded by the kernel [09:17] ogra_: one thing that i noticed in your bootlog [09:17] ogra_: is that the kernel can actually see the entire mem [09:18] ogra_: bullshit [09:18] where do you see that ? [09:18] ogra_: i read it wrong [09:18] [ 0.000000] Memory: 95036K/131072K available (6873K kernel code, 437K rwdata, 1972K rodata, 448K init, 782K bss, 27844K reserved, 8192K cma-reserved) [09:18] yeah [09:18] [ 0.000000] Memory: 938380K/966656K available (6873K kernel code, 437K rwdata, 1972K rodata, 448K init, 782K bss, 20084K reserved, 8192) [09:18] that's my img [09:18] yup+ [09:19] ogra_: you should try to load the initrd with my img [09:19] will do [09:19] ogra_: http://people.canonical.com/~ppisati/disk-2015-06-04-14.04-raspy2.img.gz [09:21] jdstrand: hi, for later when you are online - now that we have a private /tmp via the ubuntu-core-launcher I proposed to simplify our /tmp handling (bug https://bugs.launchpad.net/snappy/+bug/1462916). apparmor would need to be updated and /tmp would have to be allowed. do you have any concerns about this approach? i.e. shifting the security from apparmor to the launcher (I put some info into the description) [09:21] Ubuntu bug 1462916 in Snappy 15.04 "Simplify TMPDIR handling" [Undecided,New] [09:23] jdstrand: would love to hear your opinion on this :) [09:26] Hi folks, is there a way to solve "Failed to start Load/Save Random Seed." error on boot? [09:27] * ogra_ has been wondering that for ages :) [09:28] * longsleep will investigate [09:29] Good morning all; happy Monday, and happy Upsy Daisy Day! 😃 [09:33] hi - can anyone tell me which devices are officially supported by Snappy Ubuntu and which of those have stable images? [09:40] So the problem on the systemd-random-seed service is that it tries to write to /var/lib/systemd/random-seed which is read only. That is very bad as essentially all snappy devices boot up with little to no entropy, at least if the Kernel does not have http://lkml.iu.edu/hypermail/linux/kernel/1312.1/04772.html [09:41] longsleep, can you file a bug ? [09:41] we should definitely shipo that dir writable [09:42] well random-seed is a file - i am not sure if the whole /var/lib/systemd should be writeable [09:42] mvo, ^^^ i guess you agree [09:42] not sure we can make the whole dir writable ... but surely that file [09:42] ok - i file a bug [09:42] * mvo nods [09:43] (for the dir we'd need to ask the security team about possible impact) [09:44] * longsleep thinks the security team should think about entropy and randomness [09:44] Snappy bugs go here: https://bugs.launchpad.net/snappy ? [09:44] yep [09:51] What Kernel version is the default for Snappy images if no device tree is provided? [09:59] (RaspberryPi2)ubuntu@localhost:~$ free [09:59] total used free shared buffers cached [09:59] Mem: 947012 120888 826124 12768 6684 81884 [09:59] ppisati, ^^^ [09:59] \o/ [10:00] yo! [10:00] what was that? [10:00] but i blindly copied your whole dir into my image [10:00] ogra_: so it was the bootloader [10:00] i'll now start removing file by file and see when it drops back to 128M [10:00] ogra_: cool, nice find [10:01] well, we dont ship any of the fixup files ... or any extra config.txt or cmdline.txt [10:01] i suspect it is related to that [10:02] Random seed service bug report here: https://bugs.launchpad.net/snappy/+bug/1462954 [10:02] Ubuntu bug 1462954 in Snappy " systemd-random-seed fails to start because /var/lib/systemd/random-seed is read only" [Undecided,New] [10:03] longsleep, thanks [10:03] jdstrand, can we get some security team opinion on bug 1462954 if we can make the whole dir writable or not [10:04] ogra_: I think a real solution would be to make the random-seed file available to both system-a and -b (it should be shared). [10:05] yeah, that should happen automatically ... [10:05] the writable partition is shared between them ... [10:06] ogra_: ah i see - thats cool then [10:06] (RaspberryPi2)ubuntu@localhost:~$ sudo snappy install webdm [10:06] Installing webdm [10:06] .... [10:06] operation not supported [10:06] webdm failed to install: unpack /tmp/webdm683385157 to /apps/webdm/0.8 failed with exit status 1 [10:06] (RaspberryPi2)ubuntu@localhost:~$ [10:07] grmpf ... [10:07] Jun 8 10:06:31 localhost snappy[857]: main.go:62: DEBUG: [snappy internal-unpack /tmp/webdm683385157 /apps/webdm/0.8 /] failed: operation not supported [10:07] Jun 8 10:06:31 localhost snappy[824]: main.go:62: DEBUG: [snappy install webdm] failed: webdm failed to install: unpack /tmp/webdm683385157 to /apps/webdm/0.8 failed with exit status 1 [10:07] hmm, not so informative ... [10:10] dang ... and i get that for every snap it seems [10:10] ppisati, are we sure the kernel has all necessary options set ? [10:11] (apparmor and systemd etc ...) [10:11] So, now that i have a base image. How can i modify the image after build? Are there some post hooks or something? [10:11] ogra_: it's the same kernel + config options since... [10:11] ogra_: months? [10:11] ogra_: wait [10:11] i used your latest deb for this image [10:12] ah, i forgot to push it [10:22] ogra_: http://kernel.ubuntu.com/git/ppisati/ubuntu-vivid.git/log/?h=snappy_raspi2 [10:46] mvo: really, TMPDIR is cleared in setuid? [10:46] daaarn [10:47] * Chipaca has one of those "how did this ever work then" moments [10:56] * kissiel is away: lunch [11:02] Chipaca: yep - but all under control we just need to decide what approach to take [11:02] Chipaca: its only cleared for users, i.e. when running as root its fine [11:26] wow, so removing all new files but fixup.dat gets me a completely messed up boot [11:55] mvo, sergiusens, Chipaca, so i have a working edge RPi2 image now ... but i cant install any snaps ... all info i get in syslog is the line above (operation not supported) ... any hint how i can debug that a bit more detailed than from syslog [11:57] ogra_: is that edge? or 15.04 ? [11:58] sudo ubuntu-device-flash core --channel=edge --oem pi2_0.12_all.snap --developer-mode --device-part=device-pi2-0.12.tar.xz -o pi2.img rolling [11:58] ogra_: is this a wily snappy? [11:58] thats what i used to build it [11:58] ogra_: if so it's build with go 1.4 [11:58] yeah, wily [11:58] ogra_: and something is busted there [11:58] oh, so it is also broken on BBB ? [11:58] ogra_: in wily, yes [11:58] * mvo looks [11:58] * ogra_ has no recent BB image around, else i would have cross checked [11:58] ogra_: and amd [11:58] ah, phew [11:59] * ogra_ goes and builds a 15.04 image then [12:00] ogra_: that is the 'rolling' parameter to your ubuntu-device-flash command from above? [12:01] it picks the rollin channel from http://system-image.ubuntu.com/ubuntu-core/ [12:01] +g [12:01] sudo ubuntu-device-flash core --channel=stable --oem pi2_0.12_all.snap --developer-mode --device-part=device-pi2-0.12.tar.xz -o pi2.img 15.04 [12:01] thats builds from the release channel ... [12:01] -s [12:02] i see thanks - still a lot i dont know about :) [12:02] well, system-image will be going away [12:02] images will be fully built from snaps [12:03] * longsleep is trying to build rolling edge image for ODROID C1 now [12:03] ogra_: that sounds awesome [12:07] Channel: edge, Version: 63 - does that sound good? [12:08] yes, but it will be broken as you can see above :) [12:08] wont allow you to install snaps [12:08] ogra_: thats what i wanted to confirm [12:08] ah [12:08] to verify that the image i build behaves the same as yours [12:08] the release image works fine :) [12:09] yes here too [12:12] * ogra_ installs chatroom to test [12:13] GRRR [12:13] ERROR: chatroom.ogra failed to install: unpack /tmp/snaps/webdm/0.8/tmp/chatroom275084212 to /apps/chatroom.ogra/0.1-8 failed with exit status 1 [12:14] so this is with 15.04 now [12:14] tryin to install chatroom from webdm [12:14] mhm it was not yesterday [12:16] * ogra_ tries from cmdline [12:16] ogra_: ok ok edge rolling it fails for me too [12:18] ha, cmdline works fine [12:18] seems to be a bu in webdm [12:18] *bug [12:19] mhm i just tried from cmdline [12:19] http://paste.ubuntu.com/11647904/ [12:19] worked fine for me [12:19] and i can reach the chatroom at port 6565 in chromium [12:20] * ogra_ tires another package from webdm [12:20] http://paste.ubuntu.com/11647909 [12:20] (thats on edge though) [12:20] yeah, there it seems to be expected [12:21] so cmdline works in 15.04 but not in webdm? [12:21] * longsleep tries that [12:22] only for the chatroom package it seems ... go-example-webserver installed just fine [12:22] * ogra_ tries pastebinit [12:22] hmm, weird, works too [12:23] i will be trying to build a snappy release https://github.com/strukturag/spreed-webrtc next weekend so i get familiar with these things too. [12:23] * ogra_ uninstalls chatroom and tries again [12:23] ah, nice [12:23] chatroom in a more complex way :) [12:24] yeah - i didnt know about chatroom since yesterday [12:24] well, its just a hacked together example to test node-snapper :) [12:25] webrtc stuff makes a good show if your snappy install is in the local LAN when you hold talks ... you can involve the audience easily ;) [12:26] hey folks, is this a good place to be asking stupid questions about making snaps? [12:26] mattyw, only if you promise they are really stupid :) [12:26] mattyw: if you start ask stupid questions too i do not feel so alone :) [12:27] awesome, I'll do some typing and we'll see where we end up [12:27] ha [12:27] woah ! [12:27] Jun 8 12:24:32 localhost ubuntu-core-launcher[962]: open /apps/chatroom.ogra/0.1-8/amd64/node_modules/express/lib/middleware/init.js: permission denied [12:27] Jun 8 12:24:32 localhost ubuntu-core-launcher[962]: 2015-06-08 12:24:32 ERROR snappy logger.go:199 chatroom.ogra failed to install: unpack /tmp/snaps/webdm/0.8/tmp/chatroom031296173 to /apps/chatroom.ogra/0.1-8 failed with exit status 1 [12:28] WTF ! [12:28] so - I'm trying to make a tmux snap - because why not. After it's installed I run tmux.tmux and I get told ERROR: profile "tmux_tmux_1.9.6" does not exist. [12:28] my next question is - tmux requires libevent - so I need to install that somewhere, what should I be doing with SOs? [12:28] mattyw, did you check for apparmor denials in syslog ? [12:28] mattyw: not sure if that qualifies for a stupid question - but if you want a stupid answer - i guess its related to apparmor profiles :) [12:29] libs should go into your snap with LD_LIBRABRY_PATH pointing to their dir [12:31] ogra_: Chipaca is debugging that on wily - or do you get the above error on 15.04? [12:31] mvo, yes ... note the "amd64" in that error [12:31] (i'm obviously on armhf) [12:31] longsleep, ogra_ awesome, just as an experiment can we talk about apparmor profiles and LD_LIBRARY_PATH as if I don't know anything about those things? ;) [12:32] mvo, it looks like an ownership error with the file to me ... though installing from commandline with snappy install doesnt show that error [12:32] ogra_: chatroom installs just fine here: http://paste.ubuntu.com/11648126/ [12:33] longsleep, yeah, for me too if i use the commandline [12:33] just not from webdm [12:33] ok let me try [12:33] webdm generates the permission denied error above in syslog [12:34] bah, now it also fails from cmdline :( [12:34] open /apps/chatroom.ogra/0.1-8/amd64/node_modules/express/node_modules/cookie-signature/.npmignore: permission denied [12:34] another permission error ... different file though [12:35] and i also get: [12:35] 2015/06/08 12:33:52 Warning: failed to remove /apps/chatroom.ogra/0.1-8: %!s() [12:35] chatroom.ogra failed to install: unpack /tmp/chatroom360659173 to /apps/chatroom.ogra/0.1-8 failed with exit status 1 [12:35] hmm [12:35] (RaspberryPi2)ubuntu@localhost:~$ ls /apps/ [12:35] bin chatroom.ogra go-example-webserver.canonical pastebinit.mvo webd [12:35] and in fact i have an empty chatroom.ogra dir under apps [12:36] snappy remove chatroom.ogra doesnt remove the dir [12:36] ogra_: if that is still wily, then I think Chipaca found the issue, setgid does no longer work with go 1.4 [12:36] mvo, no, thats still 15.04 [12:37] ogra_: failed for me too with webdm [12:37] plenty of errors in syslog [12:37] mvo: hey, any reason we update-grub on updates? [12:37] mvo, i gave up on rolling for now, need to get that Pi image to work [12:37] ogra_: *ick* do you get that on bbb/kvm? [12:37] surely not on kvm [12:38] i developed the new chatroom package in kvm, there it always worked fine [12:38] sergiusens: well, thats what updates grub.cfg? or am I misunderstanding the question? [12:38] and one out of three install attempts on the 15.04 RPi image worked too [12:38] mvo: yes and it causes a nice bug [12:38] (resulting in a working chatroom) [12:39] sergiusens: *autsch* which one? the one you reported friday? [12:39] ogra_: My syslog when installing chatroom though webdm on 15.04 http://paste.ubuntu.com/11648298/ [12:39] mvo: yes [12:39] Jun 08 12:36:28 odroid systemd[1]: [/lib/systemd/system/snappy-workaround-apparmor.service:5] Unknown lvalue 'WantedBy' in section 'Unit' [12:39] wow [12:39] ogra_: meh [12:40] longsleep, are you sure you have all ubuntu apparmor patches in your kernel ? [12:40] ogra_, ah, I just have to do LD_LIBRARY_PATH=my_snap_lib_dir ./myapp, that's what you mean right? there's no magic other than that? [12:40] longsleep: (I like you nick!) - he "'WantedBy'" is my bug, sorry for that [12:41] ogra_: no i have not all the patches - currently running unmodifed hardkernel tree [12:41] longsleep, ah, then your issue is different [12:41] mvo, http://paste.ubuntu.com/11648369/ [12:42] mvo: https://bugs.launchpad.net/snappy/+bug/1462501 [12:42] Ubuntu bug 1462501 in Snappy trunk "Updates on grub systems lose track of panic=-1" [Critical,New] [12:42] mvo: no worries - i am new to snappy and trying to figure out the ecosystem [12:42] mvo: removing update-grub solves the azure problem [12:42] longsleep, the systemd kernel config options and apparmor are both essential [12:42] ogra_: ok i have "look at kernel patches" on my list :) [12:43] longsleep, ppisati can most likely help you [12:44] ogra_: problem is that i am on Kernel 3.10 + messy patches. I hope the patches required are small :) [12:44] in fact it is the whole of apparmor you need to replace [12:44] mvo: Do you want me to file a bug for the "'WantedBy'" problem? [12:45] and there is a bunch of cgroups options that snappy uses [12:45] for systemd [12:45] some work for tonight then :) [12:47] longsleep: feel free, but I think I know the reason and work on a fix [12:48] longsleep: https://wiki.ubuntu.com/SecurityTeam/AppArmorForPhabletKernels for easy apparmor instructions [12:48] sergiusens: thanks! [12:50] ogra_: are you using 15.04/edge? [12:51] guess that only stable will get you 15.04/stable [12:51] rsalveti, edge is obviously completely broken [12:51] rolling you mean? [12:51] which made me switch to 15.04 stable 1h ago [12:51] well, edge/rolling [12:51] 15.04/edge should be good [12:51] rolling/edge is known to be broken [12:52] 15.04 stable is broken for me too [12:52] let me try 15.04 edge then [12:52] right, 15.04/edge is what we're trying to release :-) [12:52] but currently blocked by the grub issue [12:53] ogra_: did you find out the reason why you can now access the entire 1gb ram on rpi2? [12:53] rsalveti, bootloader [12:53] took me 4 days but i finally found the working combo of files :) [12:54] oh, great then [12:54] using kernel7.img without config.txt makes the bootloader ninary fall back to some weird bits [12:55] *binary [12:55] my oem snap now uses uboot.bin again with a confi.txt pointing to it [12:55] and you need the fixup.dat file apparently [12:56] once i found a working combo that actually doesnt break when installing snaps i'll push that to people.u.c [12:58] great [12:58] well, not so great that on all images i tried yet the snap installation has issues [12:59] * ogra_ boots 15.04 edge and crosses fingers [13:00] wonders if the 'WantedBy' is also in 15.04/edge [13:00] no, we were both tryin stable [13:00] +g [13:00] ogra_: it's g+ [13:01] :P [13:01] ogra_: no i was trying 15.04/edge [13:01] oh [13:01] i was not aware about the distinction [13:01] sergiusens: meh [13:01] i am now :) [13:01] sergiusens: I look at the bug now, it looks a bit like its not going to be fun [13:02] * mvo makes a extra big cup of tea [13:02] ok, webdm installed [13:02] * ogra_ tries installing chatroom via UI again [13:03] ogra_: that even works for me without the kernel patches [13:03] right, but your apps wont be confined [13:03] saying that i just wonder what the difference is regarding installing from webdm and commandline one fails because of missing kernel patches while the other works fine [13:04] yay [13:04] chatroom installed fine [13:04] nice [13:04] mvo: guess we'd also need to upload ubuntu-core-config to wily [13:04] just saw your change [13:04] mvo: rsalveti well I have the workaround fix [13:05] rsalveti: yes, also lp:~chipaca/snappy/muppets needs to land to unblock will [13:05] mvo: but we are indeed crippled [13:05] rsalveti, ok, looks like this image works, i'll push it to my people.u.c account and mail a call for testing [13:05] sergiusens: oh, the workaround to use the core config? [13:06] mvo: yeah, as mentioned in the bug https://code.launchpad.net/~sergiusens/ubuntu/wily/ubuntu-core-config/panic/+merge/261378 [13:06] neat [13:06] mvo: but I also extended u-d-f to be able to add a grub.cfg in the oem snappy package, it would work fine if update-grub was not part of a snappy update [13:07] mvo: it also breaks the semantics for when we truly decouple kernel and os [13:07] snake works too ! [13:07] ogra_: great [13:07] sergiusens: yeah, I think we need a static grub.cfg just like uboot [13:07] :q [13:07] I'll be triggering a new rootfs in a few (once ubuntu-core-config) gets published [13:07] mvo: I have this azure one alreay :-) [13:07] rsalveti, oh, then i'll wait for that [13:07] then wait for the grub fix [13:07] sergiusens: post it! [13:07] mvo: I'm just looking for blessings to remove update-grub [13:07] * ogra_ needs to mow the lawn before it rains anyway [13:08] rsalveti: https://code.launchpad.net/~sergiusens/ubuntu/wily/ubuntu-core-config/panic/+merge/261378 [13:08] * rsalveti wondering what would be the side effect of removing update-grub [13:08] sergiusens: +1 from me for this, we "just" need to expand handleAssets() for grub [13:08] * ogra_ afk [13:08] mvo: expand or reduce? :-) [13:09] sergiusens: well, expand as its empty right now ;) and then remove 20_snappy from grub config [13:09] sergiusens: let me know if you want help with that, I look forward to simplify this area of the code [13:09] sergiusens: I cherry pick you azure fix in the meantime for vivid, ok? [13:10] mvo: sure, but this doesn't fix azure [13:10] sergiusens: it does not? [13:10] mvo: only the fact that we lose panic=-1 on snappy updates [13:11] mvo: if I remove update-grub from snappy update's code path there is no work in handleAssets (immediate at least) as grub.cfg sits in the common area [13:11] which makes the fact that we are running update-grub slightly worse ;-) [13:11] as rollbacks are endangered [13:11] indeed [13:12] sergiusens: I'm confused, but let me try to work it out after I cherry-picked your workaround fix [13:13] mvo: update-grub creates a grub.cfg in /boot/grub/grub.cfg based out of what is in system-a or system-b [13:14] sergiusens: fix for vivid uploaded [13:17] sergiusens: I guess we mean the same. so if we would stop calling update-grub today, what would write /boot/grub/grub.cfg? [13:17] mvo: u-d-f on device provisioning [13:17] mvo: or ... (one sec) [13:17] * sergiusens commits and pushes [13:17] sergiusens: and if the kernel is updated to a new ABI version? [13:18] sergiusens: oh, are you suggesting to use /vmlinuz, /initrd.img for this? [13:19] mvo: so this may need more thought [13:19] links should help yeah [13:19] mvo: but I would standarize the names [13:19] as long we're covering the update and rollback with them [13:19] meh, yeah, the rollback is a problem with that, I think we need handleAssets for this [13:20] sergiusens: +100 for the idea though, I think that will make things much easier [13:21] mvo: ok, I'll start with this [13:21] and here https://code.launchpad.net/~sergiusens/snappy/noUpdateGrub/+merge/261381 [13:21] * sergiusens needs to run some errands [13:23] sergiusens: looks like a good start :) will you continue once you are back from errands or shall I look at what is needed for handleAssets() ? [14:25] tyhicks: would you mind commenting on the TMPDIR conversation ^ (this is also bug https://bugs.launchpad.net/bugs/1462916) [14:25] Ubuntu bug 1462916 in Snappy trunk "Simplify TMPDIR handling" [Undecided,New] [14:29] jdstrand: sure - give me a bit to finish reading email and then read backscroll in here [14:29] tyhicks: I think they are proposing what we understood to be the plan [14:32] mvo: I can continue [14:34] sergiusens: cool, please merge the bits I couldn't resist removing [14:37] mvo: what do you think about unforking si client 3.0? [14:38] barry: I meant to file a bug (but haven't yet), there is one tiny regression compared to the forked version. if there is a error during the apply step it seems 3.0 does not pass that error to the machine readable progress [14:39] mvo: ah, yes please do file a bug. i will fix that asap and get it into wily. i'd really like to unfork snappy. [14:39] barry: does that sound possible? I have not looked further and pushed it on my todo instead but I remember something like "update failed, see syslog for details". for snappy and the progress fd I would love to get the actual backtrace for log and display [14:39] barry: ok, let me file a bug [14:40] mvo: is this still the right snappy fork branch: bzr+ssh://bazaar.launchpad.net/+branch/ubuntu/system-image-snappy/ [14:41] (if so, i see the divergence - should be an easy fix) [14:43] jdstrand, did you see my ping from a few hours ago ? [14:44] jdstrand, bug 1462954 - would be nice to get a security team opinion if we could just make the whole dir writable [14:44] bug 1462954 in Snappy trunk " systemd-random-seed fails to start because /var/lib/systemd/random-seed is read only" [High,Triaged] https://launchpad.net/bugs/1462954 [14:44] ogra_: honestly, I think that is a question better suited for pitti [14:44] oh, ok [14:45] jdstrand, it was more a "ship an empty writable file vs make the whole systemd dir writable" [14:46] i guess pitti will just say "do it, so we get working random seeds" :) [14:46] barry: bug #1463061 [14:46] Bug #1463061: Please report errors via --progress as well [14:46] ogra_, jdstrand: hmm -- I sent patches for the two failing units to mvo in Austin [14:46] bug 1463061 in Ubuntu system image "Please report errors via --progress as well" [Undecided,New] https://launchpad.net/bugs/1463061 [14:46] mvo: so I suppose they didn't land yet? can we land them now? [14:46] +1 [14:46] pitti: meh, I was sure I landed them, one sec [14:46] mvo: ack [14:46] ogra_: well, random_seed is only one thing that is in that directorty. eg, 'catalog' is in there. the other stuff I don't really know about. random-seed should be writable. I would think we would solve for the lack of entropy how we do with cloud-- I think cloud-init may do something [14:50] sergiusens, ping [14:51] jdstrand: /var/lib/systemd/random-seed is just the place where the seed is saved on shutdown and restored on boot; it's not a magic thing to initialize the seed on first boot [14:52] pitti: this was added https://launchpad.net/ubuntu/+source/ubuntu-core-config/0.6.12 [14:52] pitti: is this maybe used before the writable paths are ready? [14:52] kyrofa: hey [14:52] mvo: ah, possibly -- I thought these got initialized in initramfs [14:53] but I remember now that this isn't the case actually [14:53] mvo: systemd-random-seed.service should order itself after mounting /var/lib/systemd/random-seed, though [14:53] good morning [14:53] oh wait -- these aren't in fstab, are they? so systemd can't detect that [14:54] mvo: but either way, it waits for systemd-remount-fs.service [14:54] pitti, writable paths should be in fstab [14:54] sergiusens, hey! So I'm in the planning stage for adding more details to the snappy scope. Initially I just figured I'd wait until webdm/the snappy service implemented it, but I've been asked to move forward with it [14:54] pitti: I think they are in fstab [14:55] sergiusens, so I wanted to talk to you about your vision for how that should be done. Can I do it in such a way that you can make use of it? [14:55] ogra_, mvo: ok, then I need to look at a current snappy image [14:55] * pitti keeps the bug open [14:56] kyrofa: can you set something up? I guess we can work together, but I need to get this release behind me [14:56] jdstrand: I was kind of thinking the same thing -- connected Internet-of-Things things are almost necessarily connected to the internet :-) [15:00] rsalveti, when do you plan that image build ?= [15:00] ogra_: sorry, was waiting another package upload [15:00] ogra_: let me trigger that now [15:00] cool,. thanks [15:01] sergiusens, sure I can. My desire is you save you work-- if you feel like my doing it in a way that you can use would take more effort than it's worth, I totally understand! [15:03] sergiusens, also, this can wait a few days [15:03] kyrofa: we just need to sync [15:04] sergiusens, alright :) . When would be a good day/time? [15:05] kyrofa: later today, invite rsalveti too so he can juggle the tasks around :-P [15:07] sergiusens, sounds good [15:16] ogra_: don't hate me for being right :-p [15:16] no no, you are totally right :) [15:23] tedg: why would we need socket activation in upstart? /me no follow [15:23] Chipaca, For handing Mir sockets to a specific job for trusted prompt sessions. [15:23] tedg: we == snappy i mean :) [15:24] Chipaca, Here we is Ubuntu App Launch. [15:24] tedg: anyway, cool that we don't :) [15:24] tedg: ah! makes more sense [15:26] ogra_: what's needed for uboot on x86? [15:26] ogra_: and on amd64 [15:26] and uefi [15:26] and [15:27] * Chipaca feels sick already [15:27] Chipaca, lol, no idea, iirc there was a linaro project for x86 support [15:29] ogra_: once you have a working raspi image, throw it this way to give it a try. [15:29] * tbr knows of actual x86_64 embedded hardware being designed right now that will ship with uboot [15:30] tbr, yeah, i was actually joking in a meeting 20min ago ... i think this approach is rather to get your laptop use uboot :) [15:32] elopio, i forgot to mention you this bug https://bugs.launchpad.net/snappy/+bug/1461487 on friday, it's very rare, i've already pasted it to the doc [15:32] Ubuntu bug 1461487 in Snappy "snappy list error if executed during a snappy-autopilot update" [Undecided,New] [15:32] fgimenez: right, I saw it. I will try to reproduce. [15:33] elopio, yeah, i'm just waiting for the next rootfs [15:36] sergiusens, rsalveti I sent you both an invite for later this afternoon. Please let me know if that doesn't work-- I'm pretty open today [15:41] elopio, should we move the notes in the pad to the Snappy Image Tests doc? [15:42] elopio: fgimenez: ogra_: sergiusens: new image should be out [15:42] fgimenez: we should make a summary once we finish the testing. Collect all the important bits in bug reports, guides or docs. And send an email with that. [15:43] rsalveti, weird, i didnt see LP even start a build [15:43] mvo: sergiusens working on grub be like https://youtu.be/A52p9jc-gOo [15:43] ogra_: https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/vivid/ubuntu-core-system-image/ [15:43] fgimenez: I'm moving some things to the top, to make that easier. Please check it out so we don't forget anything. [15:43] * ogra_ wonders if his watcher script is buggy [15:43] rsalveti, ah, amd64 failed [15:43] it wont import [15:43] elopio, ok [15:44] hmm, no [15:44] * ogra_ curses arm64 for being so similar ! [15:45] oh, hm [15:45] rsalveti, ignore me [15:45] ogra_: yeah [15:45] that also annoys me [15:45] but the importer can nowadays take 1h [15:45] * ogra_ checks where system-image stands [15:45] jdstrand: sorry for naging, but do you see any risks with https://code.launchpad.net/~mvo/ubuntu-core-launcher/tmpdir-simplify-lp1462916/+merge/261419 (and the also required apparmor change)? [15:46] Chipaca: lol [15:46] yeah, nothing new in http://system-image.ubuntu.com/ubuntu-core/15.04/edge/ yet [15:46] I realized that for some reason snappy on my bbb does not have webdm installed and tried to install it, but it seems to fail. Is this a known breakage? [15:46] operation not supported [15:46] webdm failed to install: unpack /tmp/webdm954724051 to /apps/webdm/0.8 failed with exit status 1 [15:47] or maybe I just need to reinstall? [15:47] I did update ubuntu-core before trying this [15:47] I'm currently on ubuntu-core/rolling/edge rev 63 [15:47] ogra_: yeah, still importing [15:48] yeah, rolling/edge is still not yet in proper shape [15:48] rsalveti, yup, the new phones made it really slow ... will be fun once we have even more phones :) [15:48] rsalveti: ah, ok. So use 15.04/edge for now I guess? [15:48] plars, yeah, with the next image [15:49] Chipaca: pretty much :-) [15:49] (which should be done soon) [15:49] ogra_: you mean the next image in 15.04 or in rolling? [15:50] plars, 15.04/edge is currently building [15:50] ogra_: ack [15:50] thanks! [15:50] :) [15:52] jdstrand, mvo: I looked at the design described in bug #1462916 and like it [15:52] Bug #1462916: Simplify TMPDIR handling [15:52] bug 1462916 in Snappy trunk "Simplify TMPDIR handling" [Undecided,New] https://launchpad.net/bugs/1462916 [15:53] jdstrand, mvo: before reading that bug description, I don't believe that I ever heard the long term plans for /tmp... but I don't see anything wrong with that design [15:55] tyhicks: cool, that sonds promising. It seems ok to me as well, *but* you/your team are the security experts and I was wrong too many times to fully trust my judgement here. it certainly would make it easier for packaging apps as /tmp would just be availalbe like on "normal" systems [15:55] (expect for the isolation of course) [15:55] I agree [15:56] mvo: I can review the two MPs that you have proposed this afternoon (I am booked with meetings until then) [15:56] Chipaca: that youtube link started what seems to be a nice playlist ;-) [15:56] Chipaca, sergiusens: I leave in a bit to play hockey, if https://code.launchpad.net/~mvo/ubuntu-core-launcher/tmpdir-simplify-lp1462916/+merge/261419 and https://code.launchpad.net/~mvo/ubuntu-core-security/private-tmpdir-update/+merge/261396 gets a +1 it would be great if you could merge/upload them :) [15:57] (sorry for dumbing work on you!) [15:57] mvo: enjoy the game! [15:58] mvo: crack some skulls! or curse mildly under your breath. [15:58] or both ! [15:59] * Chipaca didn't say xor [15:59] sergiusens: you're welcome :) [16:01] aha ... there is the new image [16:01] nice, my ODROID told me that its going to update in 9 minutes [16:02] longsleep: to reboot? [16:02] err it just updated - seems the clock is wrong :) [16:02] yes [16:02] going down for reboot now [16:02] how does that work. how does it detect new images that quickly? [16:03] i didnt know it does ! [16:03] woah ! [16:03] some thing called snappy autopilot [16:03] Broadcast message from root@localhost.localdomain (Mon 2015-06-08 16:02:03 UTC): [16:03] snappy autopilot triggered a reboot to boot into an up to date system-- temprorarily disable the reboot by running 'sudo shutdown -c' [16:03] The system is going down for reboot at Mon 2015-06-08 16:12:03 UTC! [16:03] yeah :D [16:03] without even asking me !!! [16:03] longsleep: autopilot polls [16:03] longsleep: it's got a systemd timer unit [16:03] thats pretty rude ! [16:04] i see - can it be turned off? Privacy problem too :) [16:04] systemctl disable ? [16:04] i didnt have any chance to stop it doing that [16:04] longsleep: yes, i think you can disable it from u-d-f at least, but not entirely sure [16:04] i mean turn it off by default when building an image [16:04] longsleep: what's the privacy issue? [16:05] oh, it tells me that over and over [16:05] sergiusens: how did one disable autopilot? [16:05] whoever controls the server it polls from, knows about the snappy instance and the from address and whatever else is sent with the poll [16:05] Chipaca: with snappy config [16:05] ah, there ya go [16:05] ogra_: longsleep Chipaca https://gist.github.com/sergiusens/8d4d2b99b2e5d87c7a50 [16:06] sergiusens: great thanks! [16:06] sergiusens, lol, yeah ... here docs ... [16:06] so insane [16:06] argh, my first router went down [16:06] * ogra_ still doesnt get why we cnt just do: snappy config ubuntu-core autopilot=false [16:07] ogra_: patches welcome ;) [16:07] but instead have to create a here doc to pipe into the command [16:07] Chipaca, well, i heard it was wanted like that [16:08] i would prefer to have it disabled by default / can be controlled on image building [16:08] ogra_: actually i'd expect it to be something more like "snappy set ubuntu-core autopilot=false" [16:08] although to me snappy $pkg set $blah makes more sense than the way it is now, but still [16:08] Chipaca, yeah ! [16:09] longsleep: just set it in the oem snap [16:09] sergiusens: i do not know enougth of that stuff yet - my package.yaml is https://github.com/longsleep/snappy-odroidc/blob/master/oem/meta/package.yaml [16:09] sergiusens: where would i set it? [16:10] sergiusens: a i think i get it [16:11] config: -> ubuntu-core: -> autopilot: false [16:11] snappy config [16:11] yeah [16:11] longsleep: right under hostname [16:12] sergiusens: understood - thanks! [16:12] aha, my reboot just happened [16:12] my ODROID rebooted fine and is now on 80 [16:12] longsleep: just proposed the change [16:12] * ogra_ waits to see what the RPi comes up with [16:14] *sniff* ... 79 [16:15] and the reboot mmessage comes up right after reboot [16:16] * ogra_ runs snappy update manually to see if that changes anything [16:16] root=/dev/disk/by-label/system-a grmpf [16:17] ogra_: must be your u-boot [16:18] damn and now i'm in an autopilot reboot loop [16:18] sergiusens, is there a waqy to disable autopilot at image build time ? [16:18] this wont work for RPi users [16:19] oh superweird ! [16:20] (RaspberryPi2)ubuntu@localhost:~$ grep ^snappy_ab /boot/uboot/snappy-system.txt [16:20] snappy_ab=b [16:20] ogra_: why won't this work RPi users? It should if the U-Boot has all the features. [16:20] sergiusens: https://code.launchpad.net/~chipaca/snappy/mangle/+merge/260934 ready for a re-review [16:22] ogra_: check the snappy_mode value. If it is try, it can set snappy_ab to a before booting. [16:22] longsleep, yes, it can [16:22] longsleep, the issue is "why does it" [16:22] :) [16:22] * ogra_ guesses that needs another day of inspecting ... sigh [16:22] i thought i was done [16:23] because it finds the snappy_stamp file or your u-boot does not support test -e [16:23] i suspect the latter [16:23] but i wont dig into that today anymore [16:23] It was added quite recently [16:23] * ogra_ will just roll an image now [16:23] cmd_test: implement -e test for file existence (e5e897c01b1cd496187ca56a38ff5559d27f951c) [16:23] right and i cant use mainline on the RPi yet [16:24] I had the same issue with ODROID. Overwrote some of the commands from snappy-system.txt with commands from legacy U-Boot [16:25] See https://github.com/longsleep/snappy-odroidc/blob/master/oem/boot-assets/boot.ini#L31 - I use if fatexist instead of test -e [16:26] Check for this commit in your U-Boot http://git.denx.de/?p=u-boot.git;a=commit;h=e5e897c01b1cd496187ca56a38ff5559d27f951c [16:26] oh, even from stephen warren [16:26] and trivial too [16:27] yeah - but it might require another commit to the fat drivers, which are not trivial. [16:28] especially not if you do not have fs: add filesystem switch libary, implement ls and fsload commands (045fa1e1142552799ad3203e9e0bc22a11e866ea) [16:28] Unknown command 'fatexist' - try 'help' [16:28] gah [16:28] Thats implemented here: fat: implement exists() for FAT fs (b7b5f3195fa5a31ab1505e0c87054dc6dc71627b) [16:29] well, i uess i could just fall back to fatread [16:29] iirc we used that before [16:30] What U-Boot are they based on for the RPI2? Seems like its similar old than the one for ODROIDC. You might want to read my thread at http://forum.odroid.com/viewtopic.php?f=111&t=13924 where i list all the U-Boot features required to support snappy-system.txt [16:30] https://github.com/swarren/u-boot/tree/rpi_dev [16:30] not upstreamed yet [16:31] hmm, even though it spiled an error it switched me over [16:31] *spilled [16:31] anyway [16:31] thats for tomorrow ... [16:32] * ogra_ rolls an image for now ... still better than what we had before [16:32] i'll fix the rest tomorrow [16:32] i will be happy to learn how the snappy-system.txt can be improved to work better for old U-Boot's :) [16:33] longsleep, well, convince lool to not always use edge features of uboot :P [16:35] well, i had to port fatwrite which was total pain in the a.. - The ODROID u-boot is just hopeless. But the test -d change might not be in any U-Boot provided by hardware vendors as it is from 3 Feb 2014 and they tend to use much older stuff .. [16:35] but the test -e patch is tiny [16:35] on the other side i am not sure if a change from beginning of last year qualifies as edge feature [16:36] heh, for many boards it will :) [16:36] right, but most ARM U-Boots which are not upstreamed are based on 2011 U-Boot - which has none of these features. [16:37] yep [16:38] in any case, i suggest to add variables for the load save and exists commands to snappy-system.txt so the things can be customized seperately without having to overwrite the whole snappy_boot as i am currently doing. [16:47] longsleep, ogra_: hmm I dont remember using edge "features", more like "recent bugfixes"! :-) [16:47] fatwrite was broke on some MMCs impls IIRC [16:47] but outside of that, it just a bunch of CONFIGs to turn on [16:48] lool, and test -e is brandnew :) [16:48] ogra_: ah did I use that in snappy-system? [16:48] hmm, or not [16:48] yeah [16:48] iirc we used to just have a fatload there initially [16:48] hmmm [16:49] * ogra_ will have to dig up older images [16:49] anyway, thats for tomorrow ... i'm pushing the new image to my people.c.c now [17:00] ogra_: cool, guess you're using image 80 [17:00] since I noticed it is already published [17:02] rsalveti, right ... just finishing the upload [17:03] rsalveti, but see above, needs more work to make snappy update work (and to make you not end up in an autopilot initialized reboot loop) [17:08] lool: Yes fat driver in older U-Boots has some problems. But those can be fixed easily while forward porting all the "new" commands is a lot of work, if you have to start from a totally messed up U-Boot from 2011. See my changes at https://github.com/longsleep/snappy-odroidc/blob/master/oem/boot-assets/boot.ini#L25 what i had to change. [17:09] longsleep, well, it would be nice to make porting easy for people ... i.e. without having to patch their bootloader [17:11] ogra_: yeah - as i suggested above, put the commands into variables and make these easily overwritable. Otherwise i suggest to use fatload, fatwrite and fatexist by default. I also could not use ${bootpart}. [17:11] yeah, i had that issue as well [17:12] * ogra_ did the first odroid port ages ago [17:12] Also my u-boot requires uImage rather than vmlinuz which means the bootz needs to be a variable as well. [17:13] yep [17:13] ogra_: Why did you stop working with the odroid ? [17:13] because i had to stop my snappy work and go back to the phone [17:13] i recently changed teams and can now officially work on snappy :) [17:14] ogra_: i see :) - So maybe we see more and better ARM device support for snappy soon [17:14] hopefully :) [17:15] http://people.canonical.com/~ogra/snappy/odroidc/ that was my port from feb [17:15] (if you go one level up there are more) [17:15] * longsleep takes a look [17:16] they all come from a pre-oem-snap time though [17:16] longsleep: problem is how would we support an old u-boot [17:16] not much useful anymore [17:16] lool, we dont need to support it [17:16] longsleep: if you know of backwards compatible alternatives to the commands we used, perhaps we could use these [17:16] ogra_: i was about to ask where is the OEM is [17:16] we just need to offer alternatives for people using older ones [17:16] ... when they port themselves [17:17] so it becomes easy for them to hack around missing features [17:17] lool: well - i pretty much like the new commands. But as i suggested, put them in variables so they can be easily replaced. [17:18] ogra_: a you are using something called flashtool-assets to split up the bootcode. I read somewhere about this but was not able to figure it out - do you have a link on this? [17:18] longsleep, that died with the invention of the oem snap [17:19] it nowadays is boot-assets in the snap iirc [17:20] ogra_: i see - i am now splitting up the bootcode on oem build and added it as raw-files with the correct locations. Felt a bit strange but seems like the way to go. [17:26] sergiusens: you've got a conflict in lp:~sergiusens/snappy/noUpdateGrub [17:27] ogra_: Can you check on your RPi2 if any of the /var/lib/systemd mounts are writeable? mount |grep /var/lib/systemd [17:27] Chipaca: no worries, we won't land this just yet :-P I'll merge trunk in any case [17:27] awww [17:27] was getting all exited here [17:28] or excited [17:28] on of those [17:30] rsalveti: can you get https://code.launchpad.net/~sergiusens/ubuntu/wily/ubuntu-core-config/panic/+merge/261378 into wily? [17:30] rsalveti: it seems it's already in the ppa for vivid [17:30] GODDAMNED ! [17:30] sergiusens: sure [17:30] did anyone else notice that opening paste.ubuntu.com in firefox nowadays flushes your paste buffer ? [17:31] so if i want to middle-click-paste in it i always end up with ".ubuntu.com" [17:31] ogra_: no, I stopped using firefox ever since it started to randomly fail for me [17:31] heh [17:32] pitti: when you have a moment, could you take a look at https://code.launchpad.net/~mterry/snappy/systemd-restart/+merge/261266 ? [17:32] seems the URL bear claims focus and highlights automatically now [17:32] which makes my selected text unselected in the terminal window ... and my middle click pastes whats in the URL bar [17:32] so annoying [17:32] how do i answer to Ubuntu store feedback? Buy mail? Or can i do it here as well? I do not seem to find a way to do it in the web portal. [17:33] longsleep, http://paste.ubuntu.com/11652938/ [17:33] a bunch of systemd dirs are writable [17:33] ogra_: oh yours is actually there [17:33] sergiusens: what's the state of https://code.launchpad.net/~sergiusens/snappy/frameworkPath/+merge/260202 ? [17:33] longsleep, mine ? [17:33] ogra_: oh no you posted the content of writeable-pahts [17:34] ogra_: i too want a browser with a URL bear [17:34] ogra_: check which are actually mounted [17:34] longsleep, all of them i would hope :) [17:34] ogra_: none of them for me [17:34] Chipaca: if I merge that we have a point of no return and need to introduce the backward<->forward hooks [17:34] sergiusens: so, after .1 [17:35] longsleep, hmm, right and they are also not in fstab [17:35] might be an initrd bug,m not sure [17:35] ogra_: Ok - thanks so thats the problem for https://bugs.launchpad.net/snappy/+bug/1462954 [17:35] Ubuntu bug 1462954 in Snappy trunk " systemd-random-seed fails to start because /var/lib/systemd/random-seed is read only" [High,Triaged] [17:36] longsleep, well, i thinnk there was more to that bug as i understood from pitti and mvo today [17:36] like ordering of the unit etc [17:36] (if it runs before fstab gets processed it wont work ...) [17:37] right - i just wanted to confirm that none of the /var/lib/systemd stuff is there - because other writable mounts are there - eg /etc/hosts [17:40] Chipaca: yeah [17:51] * ogra_ calls it a day ... (will write the announcement in a bit though) [18:04] Chipaca: https://code.launchpad.net/~sergiusens/snappy/cmdline/+merge/261434 can you check if this seems reasonable from what we discussed? [18:08] or rsalveti ^ [18:14] so i guess the reason for the missing writable mounts is [ ! -e "$dstpath" ] && continue [18:14] essentially all writable mounts which not exist are skipped [18:14] yes [18:15] writable dirs/files are bind mounts ... so the mountpoint needs to exist [18:16] right - so this is a problem :) i do not see a quick workaround either [18:16] i am glad that i figured out how it works though ;-) [18:17] the fix is to ship an empty file :) [18:26] sergiusens: it looks fine, can you also propose that for 15.04? [18:26] if you didn't yet do it [18:27] would probably be nice to have a short explanation why we need rootdelay in there, with a fixme or a bug [18:27] not sure we got a bug for it, guess just the card [18:27] ogra_: right - so i hope this will be included with one of the next 15.04 releases. [18:27] sure [18:33] rsalveti: I don't until the MP is merged, image is built and tested, but I can [18:33] Where are the tools that build the OS snap? [18:33] * tedg is trying to crib ideas === j12t_ is now known as j12t [18:36] sergiusens: checking... [18:37] sergiusens: it does look sane. Want me to actually review? [18:41] Chipaca: sure [18:41] sergiusens: +1'ed with a suggestion [18:41] Chipaca: if you can review the soon-to-appear-15.04-branch would be nice as well :-) [18:41] rsalveti: NO >_< [18:42] i mean, sure, just give me a shout [18:42] :-) [18:42] typo [18:43] Chipaca: I thought [ ] is more obvious, I can change if you want [18:43] yeah, like the suggestion [18:43] since grep returns 0/1 [18:43] using with $() also opens another shell call just for it [18:44] sergiusens: spawning [ to spawn a shell to spawn grep seemed like a little too spawn-heavy for my test [18:44] taste* [18:44] Chipaca: there [18:44] * Chipaca looks for more nits to pick [18:45] Chipaca: I am not going to redo all of that script if that is your nit pick :-P [18:45] Chipaca: keep in mind it's rm'ed in that other MP ;-) [18:45] sergiusens: we should move to uboot instead [18:45] sergiusens: make it so [18:46] sergiusens: go go go :) [18:46] by which i mean: it's +1'ed [18:47] Chipaca: rsalveti should we test it works correctly in rolling first? [18:47] +1 [18:47] +1.33 [18:47] * Chipaca used 1/3 of his spare +1 [18:47] sergiusens: sure [18:48] I can trigger another image for rolling and we can see [18:48] would be nice to have a test for this guy [18:49] rsalveti: we can have a u-d-f based test that creates, grabs grub.cfg and checks; not sure where that would go though [18:50] right, can't we have this as part of the self tests? [18:53] sergiusens: wrt “// FIXME: we want to get rid of the current symlink”, i have no idea the context of that comment [18:53] sergiusens: sounds like crazy talk to me :) [18:53] Chipaca: it was relevant talk a while back :-) [18:53] Chipaca: just remove the fixme :-P [18:53] sergiusens: so i nuke it? [18:53] * Chipaca aligns the satellite's reticle [18:53] rsalveti: Chipaca as soon as this is in place https://code.launchpad.net/~snappy-dev/+recipe/snappy-daily-wily a new image build can be triggered [18:54] * sergiusens needs to walk the dogs [18:54] yeah, will take at least ~20 minutes [18:58] utlemming: mind taking a look at https://code.launchpad.net/~sergiusens/ubuntu/wily/ubuntu-core-config/azure-datasource/+merge/260201 ? [18:59] just noticed it's another pending mr we have [19:01] rsalveti: +1, looks good [19:01] thanks [19:02] sergiusens: let me know if you want me to land https://code.launchpad.net/~sergiusens/ubuntu/wily/ubuntu-core-config/azure-datasource/+merge/260201 into wily as well [19:02] Chipaca: lol, no one spotted the obious mistake :-) [19:03] sergiusens: obviously [19:04] sergiusens: whitespace around =? [19:04] Chipaca: yeah, and extra 's' in channels.ini [19:05] sergiusens: my apolologies [19:05] hmm [19:06] indeed :-) [19:06] sergiusens: merged and uploaded https://code.launchpad.net/~sergiusens/ubuntu/wily/ubuntu-core-config/panic/+merge/261378 [19:15] sergiusens: are you proposing another mr to fix these issues? [19:16] rsalveti: yeah [19:16] ogra_: hey, curious if the developer docs should be updated to point to your images? [19:16] Chipaca: no need to apologise :-) [19:17] Chipaca: rsalveti https://code.launchpad.net/~sergiusens/snappy/cmdline/+merge/261443 [19:17] I gave it a good whirl too on an azure and a regular amd64 image this time [19:17] jdstrand: for rpi2? yes [19:18] sergiusens: sorry for apologising [19:22] * rsalveti waits for it to be merged [19:23] rsalveti: yes, for rpi2 [19:24] glad to hear [19:24] jdstrand: he will be publishing the final 15.04.1 image for it tomorrow iirc [19:24] then we can update the docs [19:24] great :) [19:24] Chipaca: this is one of the reasons I don't like shell scripting... [19:25] sergiusens: i *love* shell scripting! [19:25] sergiusens: except for the part where it's cobbled together by drunk, horny monkeys [19:25] or was that perl [19:25] i forget [19:33] rsalveti: triggered another build [19:33] great [19:33] package build that is [19:34] sure [19:36] $ echo | snappy config ubuntu-core -- - [19:36] panic: reflect: call of reflect.Value.Type on zero Value [19:36] * Chipaca having fun [19:41] jdstrand: you around? [19:47] Chipaca: yes [19:47] jdstrand: have you been able to peek at mvo's branches? [19:47] jdstrand: or are you deferring to tyhicks on that? [19:48] for /tmp? [19:48] jdstrand: yaarp [19:48] I asked tyhicks to comment since he talked with you guys about all of it [19:48] I've already reviewed them [19:48] ok [19:48] well, I reviewed the complete fix [19:48] tyhicks: yep [19:48] i saw yours, just wanted to check whether jdstrand was aware [19:48] I don't see the benefit of the simple fix but let me know if I'm missing something [19:48] tyhicks: so, I should upload the change to /tmp for ubuntu-core-security and upload? [19:49] s/and upload$// [19:49] jdstrand: ah, I didn't look at the ubuntu-core-security MP - looking now [19:49] tyhicks: it gets confusing, this simplifies the logic, makes everybody use /tmp/ [19:50] tyhicks: that's all [19:50] tyhicks: it basically just says /tmp/** [19:50] tyhicks: I'm fine with the policy review, I just wanted to make sure you were ok with the concept [19:50] where apparmor isn't enforcing the app isolation in /tmp [19:51] jdstrand: I'm fine with it since the launcher will either successfully set up a private /tmp or the snap will not be launched [19:51] lool, does webcam-webui-snap have uploading capabilities? [19:51] tyhicks: ok, then I'll do the upload [19:51] rickspencer3: no [19:52] jdstrand: ack [19:52] lool, so, I want to upload a picture every hour or whatever period of time [19:52] would it make sense for me to just branch your project and add a binary that does that? [19:53] tyhicks: will you be around a bit longer? [19:53] tyhicks: i think i'll branch mvo's branch, fix the strdup check so we can land it [19:53] tyhicks: if you're going to be here :) [19:53] Chipaca: ack - I can do another review after you branch that [19:54] rickspencer3: sure! totally fine; I'm afraid I threw this together in a couple of hours some months ago and didn't touch it, but we should have a nicer and more complete example around this [20:02] tyhicks: https://code.launchpad.net/~chipaca/ubuntu-core-launcher/tmpdir-simplify-lp1462916/+merge/261446 [20:04] looking [20:04] ta [20:06] Chipaca: approved :) [20:06] huzzah, etc [20:07] tyhicks: by which i mean: thanks! :) [20:08] Chipaca: are you reading for me to upload ubuntu-core-security with the change in https://code.launchpad.net/~mvo/ubuntu-core-security/private-tmpdir-update/+merge/261396 to wily now? [20:09] Chipaca: https://code.launchpad.net/~sergiusens/snappy/backportAz/+merge/261449 [20:09] ready* [20:09] rsalveti: do we have a rolling image ready? [20:09] jdstrand: i think we need to land https://code.launchpad.net/~chipaca/ubuntu-core-launcher/tmpdir-simplify-lp1462916/+merge/261446 in wily at the same time [20:09] actually, there is no chance of regression [20:09] jdstrand: actually, yeah, land that; the launcher will land later, and all will be well [20:09] no, cause the rules are more open than before [20:09] jdstrand: yeah, nothing will break :) [20:09] right [20:09] :) [20:09] sergiusens: nops, was waiting the package to be published [20:09] which seems to be the case already [20:10] jdstrand: except for that pesky "security" thing [20:10] but, it's wily [20:10] let me build another image [20:10] * jdstrand nods [20:10] and check if the core-config migrated as well [20:10] nah, still in proposed [20:11] sergiusens: I'd assume we want to wait your core-config change as well [20:11] so we can test everything [20:13] rsalveti: which one, the one you set to needs fixing? [20:13] sergiusens: https://launchpad.net/ubuntu/+source/ubuntu-core-config/0.6.20 [20:14] that's not needed and not going to go into 15.04 [20:14] rsalveti: oh, yeah, that one is needed [20:14] rsalveti: to test different things though [20:14] right, guess I can just trigger an image now [20:14] and trigger another later [20:14] who knows how long it will take for this to migrate [20:14] rsalveti: yeah, let's get things rolling :-) [20:15] rsalveti: and that package is already in our vivid sources ;-) [20:15] alright, just started a build for wily [20:15] jdstrand: how does one _unload_ an apparmor profile? [20:16] Chipaca: apparmor_parser -R [20:16] jjohansen: and is there something that'll remove the .json files and unload it all at once? [20:17] or lower level echo -n "" > /sys/kernel/security/apparmor/.remove [20:17] and any other artifacts [20:17] ogra_: pi2 \o/ ... [20:17] ooh, almost RESTful :) [20:17] ogra_: does docker work well? [20:18] Chipaca: .json files I am not sure about, as for all at once? I am not sure what you mean, all profiles, the profile and its children? [20:19] jjohansen: i mean, the .json files, the file that is compiled to (json files are compiled to actual profiles, right?), and unloading from the kernel [20:19] i ask because we don't seem to be doing that, whatever taht is [20:19] maybe it needs doing one by one [20:19] Chipaca: ah right, yes the json files get compiled to actual profiles [20:21] afaik neither aa-clicktool nor aa-easyprof have options to clean out the json and profile files [20:21] jdstrand: is there a packaging hook, or something for this [20:24] asac, i havent tried docker [20:24] asac, i only installed random snaps to make sure it doesnt break [20:25] ogra_: can you check? [20:25] might need a roundtrip of kernel flags [20:26] asac, not tonight ... but surely tomorrow morning [20:26] just install docker + owncloud :) [20:26] cool [20:26] thanks [20:28] asac, it isnt done yet anyway, needs a bit more love (as i said) [20:28] you end up in an endless reboot loop once autopilot installed an upgrade ... [20:28] (until you manually intervene) [20:28] sorry, reading backscroll [20:29] Chipaca: there is nothing to remove the .json file except package uninstall [20:30] Chipaca: click handled all this for us in the past. ie, install a package, and click would drop a symlink in /var/lib/apparmor/clicks and aa-clickhook would handle loading the profile [20:31] Chipaca: on uninstall, click would remove the json file and aa-clickhook would remove the profile. uninstall would *not* unload the profile from teh kernel (it would be gone after the next reboot) [20:39] asac, FYI http://paste.ubuntu.com/11656307/ [20:40] jdstrand: is there a reason for not removing it from the kernel? [20:41] i guess we could add devicemapper support to quieten that message ... installs fine (no idea what to do to check it works) [20:41] sergiusens: thanks for +1'ing the launcher thing [20:42] asac, the log ... so yes, devicemapper stuff needed http://paste.ubuntu.com/11656370/ [20:43] * ogra_ goes back to watch the womens football world championship ... :) [20:44] sergiusens: waiting system-image importer [20:46] sergiusens: https://launchpad.net/ubuntu/+source/ubuntu-snappy/1.0.1-0ubuntu1build1 should hopefully unblock ubuntu-device-flash [20:48] Chipaca: because application lifecycle on touch couldn't guarantee that the app was stopped and not running when it was uninstalled [20:48] ah, ok [20:49] we can't guarantee that either, but we can guarantee it'll stop working :-p especially if we unload its apparmor [20:50] ogra_: ack. thanks [20:50] the idea is that if it continues running, all of a sudden the profile is removed and it is unconfined. yes, next start it wouldn't start up (touch guaranteed that too), but it was that window of it potentially continuing to run after uninstall [20:50] jdstrand: ah, no profile == unconfined? [20:51] jdstrand: i thought no profile == no access [20:51] yes [20:51] no [20:51] jdstrand: so we probably don't want to do that then [20:51] you can think of apparmor as targeted policy [20:51] untill we kill everything on uninstall [20:51] only what you define as confined is confined [20:52] a'ight [20:52] we could flip that around with full system confinement, but that is a very difficult prospect. maybe some day [20:53] jdstrand: there are already scenarios that make us want to have a way to kill the app [20:53] removing the cache file and the profile (and the json) but leaving the profile in the kernel is fine [20:53] so we probably want to look into that, and if we do go that route, unloading is an option again [20:53] meanwhile, just removing jsons and such should be enough [20:53] maybe, yes [20:53] we'd have to be super sure we did it right [20:53] jdstrand: at what point during boot do old profiles get loaded? [20:54] leaving it in the kernel isn't a big deal [20:54] Chipaca: the old profiles can't on reboot-- the cache and the profile is gone [20:54] so then the launcher will correctly fail to change profile and bail [20:54] jdstrand: at what point during boot do the .json files get converted to profiles? :) [20:55] and the json, yes [20:55] the json has to be gone too [20:56] but profiles get loaded before apps are allowed to start [20:56] right now, that is via a systemd dependency [20:56] there is more systemd/apparmor work to do that will change things a bit (they will load even earlier if the cache exists) [21:02] rsalveti: it says failed for ppc though :-P https://launchpad.net/ubuntu/+source/ubuntu-snappy/1.0.1-0ubuntu1build1 [21:03] sigh [21:03] rsalveti: I'll check when I get back tonight [21:03] *** Test killed with Quit: ran too long (10m0s). [21:03] yeah, still no good [21:04] jdstrand: the thing is i'm looking into regenerating the jsons on boot, for all packages (maybe not on every boot, but) [21:04] rsalveti: ooh, differnet error [21:04] let me retry the build once at least [21:05] Chipaca: why are you looking at that? that will all change once apparmor does something more like seccomp, no? [21:05] jdstrand: so i probably don't need to worry about unloading things at that point anyway [21:05] Chipaca: if you were going to fiddle with apparmor stuff, I'd suggest getting rid of the click compat bits [21:06] :) [21:06] jdstrand: because if we change the way snappy generates some of these artifacts, we need to regenerate them at least the first boot after snappy update [21:06] jdstrand: so i'm looking at regenerating all the artifacts, not just apparmor [21:07] jdstrand: trying very hard not to get side-tracked with big cleanups while doing so (not succeeding entirely) [21:07] Chipaca: you are going to want to be careful with regenerating the json files then. the mtime of the symlink in /var/lib/apparmor/clicks is examined by aa-clickhook to know if it should regenerate the profile [21:08] mhmm [21:08] Chipaca: I'm not sure what really needs to be regenerated here... the only thing that is there is a symlink to something installed by the snap [21:09] jdstrand: so, basically, i'm trying to make sure that deactivating and reactivating a package does everything it should, such that regeneration is no more than deactivating and reactivating everything (in the right order for frameworks and apps) [21:09] (this also all applies to aa-profilehook and the /var/lib/apparmor/snappy directory, which operates like /var/lib/apparmor/clicks, but for profiles) [21:09] what does 'deactivating a snap' mean? [21:10] jdstrand: in particular i'm aware of apparmor because due to a bug wily is not currently generating the .json files [21:10] jdstrand: and we didn't realise because we aren't cleaning them up [21:10] jdstrand: that it's not active :-p [21:10] so installed but not active? [21:10] jdstrand: where active means its bins is in /apps/bin, its services are running, and it's generally ready for use [21:10] I didn't think we supported that [21:10] jdstrand: not user-facing, no [21:11] jdstrand: but when you update, the old version is deactivated, the new one activated [21:11] jdstrand: if the activation fails, rollback is simply reactivating the old one [21:11] I fairly confused by what you want to change wrt apparmor as it exists today [21:12] that said, I was not aware of these wily issues [21:12] I thought we were already doing the things you mentioned [21:13] jdstrand: i'm not sure what i need to do either :) i need that everything that is installed has the apparmor (and other bits) as if the current version had installed it [21:13] so one thing to keep in mind, click-apparmor will remove a dangling symlink in these directories [21:13] ie, it cleans up when a snap is uninstalled [21:13] right [21:14] jdstrand: i think the best thing would be for me to write it down in an email [21:14] jdstrand: i'm not sure how much sense i'm making at this point [21:14] ok [21:14] well, it is probably me lacking context [21:15] actually, I'm not sure click-apparmor will remove the dangling symlink [21:15] jdstrand: possibly. If i were more awake i'd probably be able to muddle through :) [21:15] * jdstrand looks [21:15] jdstrand: but i'll write out the email in the morning [21:15] and we can chat after [21:16] tomorrow is going to be hard to catch me during your core hours [21:16] (meeting and a lot of preparations for it) [21:16] but yeah [21:16] jdstrand: my core hours remain spread wider than usual for people in my timezone :) [21:17] jdstrand: anyway, i'm not blocked (yet), so no worries [21:17] ack [21:25] sergiusens: failed same way even after a rebuild [21:26] slangasek: ubuntu-snappy is still unhappy on powerpc, but failing when executing the tests now: https://launchpadlibrarian.net/208607597/buildlog_ubuntu-wily-powerpc.ubuntu-snappy_1.0.1-0ubuntu1build1_BUILDING.txt.gz [21:27] rsalveti: ok. any reason to not just delete the powerpc binaries for now, given that this isn't a supported arch? [21:28] no specific reason, guess we could just disable the tests for it, that will at least let it building [21:29] Can someone please point me to the latest apparmor git tree? [21:29] rsalveti: I'd prefer deleting the binaries to disabling the tests :) [21:33] slangasek: fine by me [21:33] sergiusens: ^ [21:33] guess we'd need a package change to not build for it then [21:33] no [21:34] oh, just to avoid it not being promoted [21:34] just leave it FTBFS on powerpc for now [21:34] even better [21:34] forgot we had that option [21:34] that won't block it from being promoted, that's why we delete the old binaries [21:34] same for https://launchpad.net/ubuntu/+source/goget-ubuntu-touch then [21:34] since that's blocked waiting ubuntu-snappy [21:34] and if this is a toolchain bug, which seems likely, if and when the toolchain is fixed you get the package back for free [21:35] right [21:35] longsleep: I think you may want to talk to ogra_, but he is eod [21:37] longsleep: actually, I see https://developer.ubuntu.com/en/snappy/guides/porting/ references 'Kernel requirements for Snappy (ubuntu and vanilla/android)' [21:38] jdstrand: yeah but that seems outdated - http://kernel.ubuntu.com/git/jj/ubuntu-vivid.git/log/?h=apparmor is the latest i could find [21:38] but this here has some changes i do not see http://kernel.ubuntu.com/git/ppisati/ubuntu-vivid.git/log/?h=snappy_odroidc and i wonder where these come from [21:39] longsleep: are you wanting to work with a kernel version newer than 3.19? [21:39] yeah, I'm not sure [21:39] jdstrand: no - i have Kernel 3.10 [21:39] ogra_ has done this a few times. we might need to get https://wiki.ubuntu.com/SecurityTeam/AppArmorForSnappyKernels updated [21:40] right, i am tempted to just take the ppisati kernel and rebase it to latest from hardkernel [21:40] but i want to understand where the Apparmor comes from too [21:41] longsleep: in short, apparmor is in the upstream kernel, however Ubuntu has a (rather large) delta that we are in the process of upstreaming. the ubunta archive kernels and the phablet kernels on the page should all be up to date [21:41] i guess i just found http://kernel.ubuntu.com/git/jj/linux-apparmor-backports/ [21:42] I can't speak to the ppisati kernel [21:42] which looks promising [21:42] jjohansen: can you tell longsleep where to find the latest apparmor patches or recommend what tree to look at if he is enabling a 3.10 kernel? [21:46] longsleep: jjohansen may be able to make that recommendation, but waiting for ogra_ for how to port for snappy specifically is going to be the best bet [21:46] jdstrand: yeah i spoke to ogra_ earlier today already === devil is now known as Guest38897 [22:22] rsalveti, sergiusens: ok, looking more closely I see that the blocker is that goget-ubuntu-touch won't update, because it builds on powerpc but is uninstallable. So it seems the version of udf in wily doesn't have this dependency [22:24] so for /that/ problem, we don't want to just remove the powerpc binary of udf because the next rebuild will just bring it back. Options are: make the udf dependency on ubuntu-snappy-cli arch: dependent (omitting it on powerpc); make goget-ubuntu-touch build-depend on ubuntu-snappy-cli so that it will be dep-wait, then remove the old binary; make udf specify its architecture list and remove the old binar [22:24] y [22:43] longsleep: it is best to just pull them out of the ubuntu kernel trees atm, that patchset is maintained and gets patches on it, the upstream set is in development [22:44] longsleep: so git://kernel.ubuntu.com/ubuntu/ubuntu-vivid [22:44] just git log security/apparmor [22:45] longsleep: actually you will want s/ubuntu-vivid/ubuntu-willy/ [23:09] slangasek: yeah, makes sense, thanks for looking into that [23:39] rsalveti: slangasek if I go with dep-wait