[00:03] kyrofa: there is now https://github.com/ubuntu-core/snapcraft/pull/266/files === chihchun_afk is now known as chihchun [07:56] So here's probably an annoying question. What's the latest snappy image for a Raspberry Pi 2? [08:05] I'm going to sort of assume it's https://people.canonical.com/~mvo/all-snaps/rpi2-all-snap.img.xz. [08:07] good morning [08:09] nhaines: thats the very latest 16.04 image, its a bit rocky at this point because we are adding lots of new features [08:09] elopio, hey, still awake!? :) take a look at http://10.55.32.97:8080/log/all [08:09] nhaines: if you prefer something stable to work with, the 15.04 rpi2 image is also available [08:09] mvo: I just got an RPi2 with a PyGlow module, and was hoping to play around with it. :) [08:09] nhaines: http://releases.ubuntu.com/vivid/ubuntu-15.04-snappy-armhf-raspi2.img.xz [08:09] elopio, "Checking comment on PR #379 for job github-snappy-integration-tests-cloud" [08:10] mvo: is there no 15.10-based image? [08:10] fgimenez: yes. Everything seems correct, except that m-o can't connect :( === lu is now known as Guest98847 [08:10] elopio, http://stackoverflow.com/questions/8378235/javax-net-ssl-sslexception-java-security-invalidalgorithmparameterexception-th maybe we are missing an ssl store === l is now known as Guest57199 [08:11] mvo: I'll skip the "why" and ask where I can sort of figure out where to look for up-to-date information. [08:11] fgimenez: I installed the ca-certificates-java. [08:11] still no luck. [08:11] elopio, ok i'll take over from here, thx! and get some rest [08:17] nhaines: here is a bit of background info on the 16.04 image state https://lists.ubuntu.com/archives/snappy-devel/2016-January/001400.html [08:17] nhaines: we only have 15.04 and 16.04 currently [08:18] fgimenez: ok, I give up now anyway. The old jenkins doesn't have the java certs installed anyway and it works. [08:19] elopio, maybe something is missing in trusty, i'll investigate. the current images have been built with the new keys, right? [08:20] fgimenez: the rolling edge all snaps, yes. [08:20] fgimenez: to generate the 15.04 we need snappy-cloud-image not to pass os, kernel and gadget parameters to udf [08:20] elopio, ok, i'll rebuild that one and create a new floating ip and a testing project in github [08:20] I guess we can make that the default is not to pass the parameters. [08:21] elopio, that's already done iirc, let me check [08:21] elopio, https://github.com/ubuntu-core/snappy-cloud-image/blob/master/pkg/image/image.go#L94 [08:23] fgimenez: for some reason that didn't work: http://162.213.35.179:8080/job/create-cloud-image-vivid/1/console [08:24] elopio, ok i'll check that too [08:35] mvo: gotcha. Will I be able to upgrade from 15.04 to 16.04, or will that require a reflash? [08:40] nhaines: it will require a reflash. if you want to avoid that the 16.04 image is probably better but it under heavy development :) [08:46] mvo: finished flashing the 16.04 image, ran 'snappy enable-classic', then 'snappy shell classic', ran 'sudo apt install nethack-console'. Worked but game not available. Sad. :) [08:50] Anyway, reflashing isn't currently a problem, so I'll manage either way. Thanks for the help. :) [09:44] Good morning all! Happy Wednesday, and happy Chocolate Cake Day! 🙌 🎂 [09:45] JamesTait: hey, good morning! [09:45] JamesTait: how is the snap.yaml stuff going? I would love to upload the first snaps ;) [09:48] Things are moving now I've managed to nail down the requirements. I expect to have something to land and test on staging today. [10:05] Now you made me want chocolate cake :( [11:27] hmm, ubuntu_geek is gone [11:33] kyrofa: sergiusens: wdyt of http://paste.ubuntu.com/14678463/ (and then make plugins that support, honor those global flags) [11:33] utlemming: so I tested the vagrant stuff on two machines and the "stable" image loops on both, so is pretty unusable ... [11:34] utlemming: also both machines are very different (one intel i7 one amd opteron 6272 etc.) ... [11:34] asac, i dont like "+j_count = 0" [11:34] why? [11:34] utlemming: the "edge" image seems to work though, so using that for now ... [11:34] <=0 -> use auto counrt [11:34] count :) [11:34] asac, it should check /proc/cpuinfo and default to the number of cores instead [11:35] ogra_: did you read the code? [11:35] meh [11:35] :) [11:35] didnt read to the end [11:35] search for it... its doing that [11:35] hehe [11:35] yeah [11:35] my head feels like wrapped in a 2m thick layer of cotton wool [11:35] without that its pretty sleep inducing to build a kernel :) [11:35] (hard to get through doors this way :P ) [11:36] yeah, sorry [11:36] hehe [11:36] you went to the doc? [11:36] :) [11:36] or just bad day/headaches? [11:36] ogra_: give it one meter more and you will know my current state :) [11:36] just got up after a 12h flight :) [11:36] did you guys pick up the ubuflu? [11:37] at least i only cracked my back and didnt catch a cold or ubuflu as usual [11:37] I did /o\ [11:37] dholbach: yeah, a little bit [11:37] pfft... 12h is easy :P [11:37] seb128 as well [11:37] ogra is getting old :) [11:37] lol [11:37] well, get some rest then [11:37] asac, well, over all it was a 17h tour [11:38] but yeah, getting old [11:38] better if you are available strong for less time its better than weak for long time :) [11:38] and i cant take much rest, so much to do [11:39] ... generic initrd ... all-snaps in livec-rootfs ... dragonboard images ... aall in the next 10 days [11:40] mvo, so for the initrd, when do you want to merge the two pieces ... during snap creation or earlier ? [11:40] (i assume we want to sign the file inside the snap ) [11:41] s/ sign the file inside/ a signed file inside/ [11:41] * dholbach hugs didrocks, ogra_ and seb128 [11:42] hope you're all better soon again [11:42] * ogra_ hugs dholbach didrocks and seb128 [11:42] * didrocks hugs dholbach, ogra_ and seb128 as well, (sharing germs!) [11:42] same for you :) [11:42] * ogra_ puts up a virus scanner to catch the worse bits [11:42] this is the infirmary :) [11:50] big ubuflu it seems :) [12:02] asac, set verbose seems very specific to the system (it is different on others) [12:02] asac, if you create a bug, I'm happy to look into it [12:03] sergiusens: not sure what you mean [12:04] lots of build systems have verbose flags to see the real command line [12:04] like V=1 etc. [12:05] sergiusens: look at this for an example with scons.py [12:05] sergiusens: https://github.com/asac/snapcraft/tree/multithread-and-verbose-infra [12:05] dholbach, didrocks seb128 get better! And maybe ogra_ too [12:05] * Sweet5hark stacks up on phone desinfectors ... [12:06] didrocks, I have thought of a probably better way to do pkg-config without changing pkg-config, but by wrapping the call. I'll experiment and propose [12:07] thanks sergiusens [12:07] thanks sergiusens! [12:08] thanks sergiusens :) [12:08] sergiusens: ah, not uninteresting, yeah, wrapping would be less instrusive [12:08] asac, I'm a bit skeptical about this, but we can explore [12:08] didrocks, wow, get some rest, thoes double negatives are hard to follow ;-) [12:09] sergiusens: ahah, yeah, tireness is twisting my mind! :-) [12:09] sergiusens: what makes you think? [12:09] sergiusens: a) it surely needs to be a global CLI option, no? [12:09] asac, are we philosophical today ? [12:10] b) bc, you want to be able to toggle this on and off from CLI and not hack on yaml to change behaviour [12:11] asac, lets start with `snapcraft snap`, does it work with how you did it? [12:12] sergiusens: oi if you want to talk about how to implement the global option, then yeah lets talk [12:12] asac, verbose makes sense to be global, the -j could be a yaml thing though (just because of what I mention before); if not you need to expose these globally [12:12] sergiusens, hi there... I was told you had some thoughts on the vendoring approach taken here? https://github.com/ubuntu-core/snapcraft/pull/197 [12:12] sergiusens: so thats interesting question. think right now you can only change this if you run build explicitely [12:12] which could be fine if you think about it, but i would prefer if the option woudl be everywhere [12:12] sergiusens: no -j is not yaml [12:12] its cli [12:12] you want to toggle on/off for debugging etc. [12:13] at best all are global [12:13] felt bad to put it for top level options, but we could do that [12:13] thought maybe you know a trick how a command would still parse the options [12:14] pindonga, I'm really not comfortable with vendoring; especially for things that the team is not upstream for [12:14] sergiusens: anyway, thanks for checking. what i want are those common.XXX functions that have the behaviour as its now for build, just better so it works for snap etc. [12:14] and default target [12:14] sergiusens, so what do you suggest we do? here's the problems I see [12:15] i can go and make plugisn that support these do it [12:15] 1. ssoclient is only provided as a package on xenial, thus depending on this will mean snapcraft will only work in xenial once this change lands [12:15] pindonga, can't this land in xenial? [12:15] pindonga, that is fine [12:15] pindonga, xenial is the only target for snapcraft 2.0 [12:15] 2.x [12:15] 2. the upload also requires the storeapi code, which if not vendored will also require us to package/land click-toolbelt on xenial (same caveat) [12:16] also, packaging and getting this into ubuntu seems to be taking quite some time [12:16] and I fear we'll miss feature freeze [12:16] pindonga, landing in ubuntu is part of the workflow though [12:16] pindonga, maybe ask dholbach for help with packaging, I'm sure he is more than glad to help [12:17] sergiusens, ok [12:17] beuno, fyi... looks like we'll have to go the packaging route in the end [12:17] pindonga: what do you need to get into the distro? [12:17] dholbach, click-toolbelt [12:18] * pindonga checks what the latest status of that was [12:18] r57? [12:18] sergiusens, oh, no LTS backports planned for snapcraft ? [12:18] there's no new changelog entry in debian/changelog in lp:click-toolbelt? [12:18] dholbach, yes, version 0.5.1 [12:19] is this normally uploaded through auto-landing? [12:19] dholbach, this is a new package, hasn't reached the archive yet afaict [12:19] pindonga: so r57 == 0.5.1? [12:19] so no idea what to answer there [12:19] yes [12:19] ok ; isee [12:19] let me review it [12:20] ogra_, yes, 16.04 LTS [12:20] pindonga, sergiusens, I don't understand why it needs to be pacakged? [12:20] ogra_, if it were written in go I'd be more confortable [12:20] also, I wouldn't use the name click-toolbelt if we do [12:20] it [12:21] beuno, we have no guarantees of maintainership; and it is bad practice to vendor in the archive [12:21] beuno, in that case we'll have to split the storeapi code into a diff project and package that [12:21] sergiusens, it [12:21] even more work [12:21] Hi, is it possible to use *.so files that are compiled on ubuntu lts 14.04 on a raspberry pi to make a snap and use them as a framework? [12:21] beuno, if it is a fork; then it is missing tests which was said was not necessary [12:21] beuno, this is just proper ubuntu work, nothing really out of the ordinary [12:21] I don't understand, upload is a core part of snapcraft [12:22] why would we want to have it separate? [12:22] beuno, yeah, the libraries used though, that's different [12:22] sergiusens, hmm, that creates a gap ... most people will only upgrade once 16.04.1 comes out [12:22] beuno, well, the conversation started with the fact that there are no tests as it is tested upstream, but if it indeed is a fork, then it is not vendored, but a fork [12:22] and should have all the required tests [12:23] beuno, sergiusens what's vendored is the store api code (just so we don't have to duplicate the code in both places) [12:23] I could as easily do the latter removing one thing to vendor [12:23] sergiusens, the old code is being deprecated [12:23] the other (ssoclient) would still be needed via dep (which is already in xenial) [12:23] we're keeping it around so the phone can still upload clicks for a little while [12:24] we're merging in that code into snapcraft, and killing off the other tool as soon as it's sensible [12:24] beuno, as long as you maintain this part of the code [12:24] Good morning [12:24] sergiusens, by you, you mean all of us? [12:24] I mean, this was suppose to be part of snapcraft [12:24] kyrofa hi i have an other question about snappy can you help me out a bit? [12:25] and pindonga is pitching in so more things can be done on other bits [12:25] NoiZeR, sure :) [12:25] if there are concerns about code quality, we can surely split it up and land it in chunks [12:25] beuno, if we are going to be upstream... https://github.com/ubuntu-core/snapcraft/pull/197/files#diff-175727a8f2ba0ea9e34a671983b5ad2dR36 [12:25] beuno, that is indeed wrong, why does it say `vendor`? [12:26] kyrofa so i uses a Raspberry Pi and i have some libs compiled on a Raspberry Pi with ubuntu 14.04 LTS but is it possible to make a snap of it en use it as a framework? [12:26] sergiusens, because it hasn't been clear how to pull these things together [12:26] kyrofa it is an *.so file [12:26] kyrofa thx in advance xD [12:27] sergiusens, dude, you're leading snapcraft. There was the store upload feature on the roadmap, and I volunteered to get someone to work on that code since we have some domain experts [12:27] NoiZeR, potentially. Libraries still often need other libraries, which would also need to be included in the .snap [12:27] I don't get why you'd want to try keep that split as a separate thing [12:27] NoiZeR, why are you wanting to make a framework instead of a normal .snap, by the way? [12:27] sergiusens, if you'd rather develop the feature yourself, pindonga has plenty of other things to work on [12:27] beuno, I just said, remove `vendor` add tests and it is fine [12:27] cool [12:27] ok [12:28] beuno, I am worried that when snap-ids come all this code will be useless, that's why I am worried [12:28] kyrofa im making a system where users can make some application for my application. But they need the Library that is installed on the Raspberry Pi [12:29] kyrofa so the developers can make third party modules for the device [12:29] sergiusens, I understand, we'll need to change quite a few tools around it [12:29] sergiusens, nessita is currently planning that change [12:29] sergiusens, I don't think the core of this will change (uploading) [12:29] NoiZeR, alright. Note that frameworks are more for sharing some sort of service, not sharing libraries. Not to mention frameworks won't exist in 16.04 (they're being replaced with something else) [12:29] sergiusens: beuno: i feel that some coupling of store and snapcraft is unavoidable with the consequence that there will be changes that need invest on both sides... to ensure things move smoothly, couldnt you couple store and snapcraft through integration tests that run on landings on both sides. in this way if store wants to land changes that break snapcraft they will have to facilitate that the snapcraft change also happens? [12:30] but there are plenty of things to add: assertions, name reservations, etc [12:30] and vv [12:30] NoiZeR, as i understand there are no framework snaps anymore ... but you can create library snaps that your app can consume (but nobody elses but your snaps will be able to use them, they are bound to teh developer) [12:30] we have autopackage tests enabled; but the store does not land on ubuntu so it is up to the store to implement those testing hooks [12:30] asac, ^ [12:31] NoiZeR, your users might be better served by your making some sort of snapcraft plugin to integrate your libs into their .snaps [12:31] asac, I don't think we'd be breaking each other in this case, the server might change, and if it does the client has to regardless of where it lives [12:31] NoiZeR, or just make a build system that makes it easy [12:31] sergiusens: if snapcraft already tests store interactions through autopackage tests on landings thats fine [12:31] yeah, what kyrofa said [12:31] kyrofa, a build system what do you mean with that? [12:32] kyrofa can you use snaps into snaps? [12:32] beuno: sure, just saying that snapcraft maintainer would feel less reluctant to land more code that couples store to snapcraft if i knew there are tests for the integration running in store landing process too... [12:32] NoiZeR, is your library built by a build system? e.g. cmake, autotools, etc.? [12:32] cmake [12:32] (just my way to say, i wouldnt hold back with more coupling just because something will change; we ahve to do it anyway so ratehr think how to make it efficient and not how can we wait longer) [12:32] kyrofa cmake and an other lib by qibuild but [12:32] NoiZeR, cmake is good-- snapcraft supports cmake with a plugin [12:33] kyrofa but cmake needed qibuild so i need to install the python library aswel i think [12:34] kyrofa but how thats works i don't know. Just i know its needed to be installed with pip install qibuild [12:34] asac, sure thing [12:35] pindonga: AFAICS the package is good to go, minor nits would be: the package description could be a bit longer, copyright in d/copyright could be extended to be 2013-2016, and there's no manpage for click-toolbelt - I didn't quite follow the discussion you had above about having a separate package or something.... do you know if everyone's happy with the name click-toolbelt and the placing of all bits in the code? [12:36] from a mere packaging POV I'm happy to upload if everything else is settled [12:36] dholbach, thanks but it looks like we won't need it after all [12:36] we're moving that code into snapcraft itself [12:36] ok... [12:36] let me know if I can be of any help [12:36] NoiZeR, okay I'm not familiar with that... hold on [12:36] NoiZeR, let me do some research into qibuild real quick :) [12:36] kyrofa np [12:37] kyrofa its with the NAO robot related [12:37] NoiZeR, but my point is this-- your job would be easier if you focused on building your snap with snapcraft rather than bundling the .snap yourself [12:37] NoiZeR, since .snaps must include their dependencies [12:39] kyrofa ok wait a sec :p So the i need to just add the library to my snaps and then it will work correctly? And can the other snapps uses this library? [12:39] NoiZeR, each .snap will need to include your lib and its dependencies [12:40] NoiZeR, this allows your lib to update without potentially breaking the user's snaps before they're ready for it, etc. [12:41] kyrofa but every snapp does need to get this library into there snap? [12:41] kyrofa or is it possible to share a coulple of library [12:42] NoiZeR, what exactly will your users do ? [12:42] do you expect them to build a snap themselves to make use of your project ? [12:43] (with an emphasis on *build* ) [12:43] ogra_ yes So they need to make some snaps and the other snap need to call them [12:43] cut that sentencs at "and" :) [12:44] *sentence [12:44] _ogra so you can it sea like android is my snap and they will install some snapps and that are the apps [12:44] NoiZeR, they need to build something ... why not focus on that then insteasd of binary usage [12:44] provide them the easiest way to do that build step and you win [12:45] ogra_ what do you mean exactly with that? [12:45] make that build step bundle what they need ... [12:45] into their snap [12:45] dholbach: https://developer.ubuntu.com/en/snappy/ snappy log is broken here [12:46] what is broken? [12:46] ogra_ so that they always include the needed libraries? [12:47] NoiZeR, exactly [12:49] ogra_ ok but thats one solution but is there a way in the future to share libraries just like in a normal linux? [12:49] "a normal linux" ? [12:50] just lik i install the C++ library on my Ubuntu 14.04 and i can include the files (the library is en *.so file) [12:51] and some python stuff [12:51] you can make shared library snaps today ... but only shared for *your* snaps .... i.e. they are bound to your account so you cant break our security model ... random dependencies like in other package managers wont be possible [12:51] asac: what is broken on thage page? [12:51] ogra_, and you can't really do that today-- you'll be able to in 16.04 [12:51] *that page? [12:52] NoiZeR, so for 15.04, the real answer to your question is no. Snappy is a bit different from other packaging systems-- you can't share dependencies like that [12:52] kyrofa, heh, right, my brain is still in a different TZ from my body :) [12:52] ogra_, ha! You home now? [12:52] yep [12:52] ogra_, might take you a week man, sorry :P [12:52] but brain is still over the atlantic somewhere ... following up at snail speed [12:53] yeah [12:53] ogra_ so when i install all the apps as the same user it will work correctly [12:53] ? [12:53] with shared libaries? [12:54] right, so that you can use the sname libs from a single snap ... [12:54] but others wont be able to consume them [12:54] dholbach: the snappy logo doesnt load here [12:54] right on top right [12:54] its broken image [12:55] dholbach: do you see it? [12:55] you're right -I can see the problem too [12:55] ogra_ the account is that on the snap store that you mean or just a user on the device? [12:55] NoiZeR, snap store [12:55] dholbach: good :) [12:55] asac: it wasn't clear to me that "snappy log" was supposed to be "snappy logo" :) [12:55] yeah, typos ftw [12:56] NoiZeR, if any random person would be able to put libs there that any other random person can use, you would completely break the security model of app isolation ... [12:56] asac: https://assets.ubuntu.com/sites/ubuntu/latest/u/img/cloud/tools/snappy/snappy.png was dropped it seems [12:56] davidcalle: ^ better not to rely on assets [12:56] NoiZeR, and are security wise back to deb/rpm [12:56] kyrofa, morning btw [12:56] sergiusens, hey there :) [12:56] dholbach: hmm. will you check it out? [12:57] yes [12:57] NoiZeR, meaning to give *every* person in the app store root on your device ... [12:57] ok let me know. wanted to download it for a rpesentation [12:57] :") [12:59] asac: fixed [13:00] neat [13:01] NoiZeR, i.e. in the deb/rpm world *every* package maintainer has full root access to your system ... in snappy every store account has root inside their area of the system, but not in the system itself or in other accounts ... if lib snaps would allow cross account usage that woulod mean the provider of the lib has full root access to all snaps that consume it [13:02] ok I understand that [13:02] dholbach: there also exists an orange snappy logo somewhere [13:02] that has the word snappy printed [13:02] can you browse assets etc. to see if they are there? [13:03] NoiZeR, so in the case where your users have to build something from source anyway, just focus on making that step the best experience by providing a snapcraft plugin that automatically takes care for everything [13:04] ogra_ kyrofa thx for the help when i have some questions will ask them here [13:04] so the users can concentrate on their code [13:04] great, that is why we are here :) [13:05] NoiZeR, sounds good :) [13:09] ogra_, when will the dragonboard be here: https://developer.ubuntu.com/en/snappy/start/ ? [13:10] kyrofa, within the next 10 days i hope [13:10] ogra_, awesome! I just received mine yesterday. Do you happen to have a writeup I can follow? [13:10] we need auto-builds first ... [13:11] kyrofa, http://people.canonical.com/~ogra/snappy/dragonboard/README [13:12] if you are on xenial the latest in-archive u-d-f shoudl work btw ... else you need to download the one from that dir [13:13] ogra_, excellent, thank you! [13:20] ogra_: hey, did you have to patch snapdragon u-boot for things to work with snappy? [13:20] lool, http://bazaar.launchpad.net/~ogra/+junk/dragonboard/files [13:20] only the config [13:20] there is also a README about all i had to do [13:20] ogra_: ok, CONFIG_SUPPORT_RAW_INITRD is what I suspected [13:21] ogra_: ah I didn't see the u-boot part in your original dragonboard README hence the Q [13:21] yeah, and support for our uboot.env [13:21] so where is the snappy command docs? I see there is a "snappy man" command than dumps a raw manpage ... but "man" doesnt seem to be available or installable on snappy? [13:21] lool, yep, that work pre-dates snappy work [13:21] lool, afaik even linaro now uses my boot setup ;) [13:21] for normal linux [13:22] ogra_: cool [13:22] urgh CONFIG_SYS_REDUNDAND_ENVIRONMENT [13:22] yeah [13:22] redundan*d* [13:22] hehe [13:23] I wonder if it's a germanism [13:23] no idea who originally created the option [13:23] asac, can you check my comment https://github.com/ubuntu-core/snapcraft/pull/257/files ? [13:24] BBB uses it which is why all other boards inherited it for having a working fw_setenv/printenv (and uboot-go) we use in snappy [13:25] ogra_: uboot-go? [13:25] (the option just adds a 4byte header to uboot.env ... not even sure why thats "redundant" in any way) [13:25] ogra_, mvo asac lool so initrd and multiple kernel snaps; mvo is thinking we should create a full initrd on kernel snap build time; that seems hard to do for snaps that we don't control though (and becomes hard to maintain) [13:26] it's also a problem for lvm, crypto and stuff [13:26] sergiusens, well, i suppose we might want to sign the initrc file inside the snap [13:26] *initrd [13:26] besides, isn't there core snappy logic setting up writable bind mounts from initrd? [13:26] so you cant really assemble at snap install time [13:26] ogra_, so if we ever update the OS, we will need to rebuild all the kernel snaps (or someone) [13:26] sergiusens: I'm not sure I'm thinking this, this is what I'm doing currently, we discussed cat'ing multiple initirds [13:26] lool, we are at the step before running the initrd :) [13:27] sergiusens, ? [13:27] sergiusens, do you plan to ship any parts of that in the os snap ? [13:27] * ogra_ wouldnt [13:27] mvo, oh, sorry, this is based on a quick SCaLE conversation with ogra_ [13:27] ogra_: btw, did you had a chance to test http://people.canonical.com/~mvo/all-snaps/dragon-all-snap.img.xz ? [13:28] ogra_, so when talking to mvo half of the initrd would be in the OS snap; the other half in the kernel [13:28] my vision would be that we have a generic and a modules part ... and that we merge them at kernel snap creation time into one file we can sign [13:28] sergiusens: no worries [13:28] mvo, not yet ... will do so beofre EOD [13:28] ogra_: on a grub system we don't extract anything, we load directly from the squashfs [13:28] ogra_: thanks! [13:29] ogra_: so extracting and combining would be a step backward :( [13:29] sergiusens, mvo, yeah, i dont think that will fly nicely ... both parts should live in the kernel snap [13:30] the other option would be to merge them from the bootloader on the fly .... but that will likely result in very slow boots and probably also require quite some bootloader hackery [13:30] maybe I'm missing something, but why would both parts live in the kernel snap? it seems if one is in the OS snap, that is ok? as long as it has no modules or other kernel dependant stuff ? [13:30] mvo, how would you boot that ? [13:30] it seems like grub supports loading multiple initirds? [13:30] does uboot too ? [13:30] i never tried that [13:31] ogra_: grub has a concept of multiple initirds, so grub would load both, never tried it either, I'm not saying it works, just that that looks like a really good option :) [13:31] ogra_: uboot is a bit of a problem :( [13:31] that might result in horrid math stuff when you create a gadget to get all the load addresses right [13:31] ogra_: but on the upside, on uboot we need to extract the files, so we can do merging with uboot [13:31] mvo: so cat-ing multiple initrds would work technically; is this the solution we want to implement for 16.04? we could [13:31] ogra_: you mean horrible math for uboot? [13:32] yeah [13:32] lool: at least on grub it sounds like the best option we have [13:32] mvo: we tested the multiple initrds yesterday, it works [13:32] lool: \o/ [13:32] lool: you rock [13:32] where to load the bits so they dont overlaop and can be used as one file from ram etc etc [13:32] mvo: for u-boot, it's technically possible, but I personally would prefer if we did u-boot -> grub -> multiple initrds [13:32] eeek ! [13:32] ogra_: so for uboot we can extract+conact so that the bootloader does not have to do terrible math [13:32] or we just cat the initds together on disk since u-boot systems are handled specially anyway [13:33] lool: yeah, I think the later is the better option [13:33] the good thing about u-boot -> grub is that it gives us squashfs again [13:33] lool: ohhhh [13:33] lool, well, that gets us issues with signed initrds [13:33] lool: thats very interessting [13:33] lool, why would you do uboot->grup and not just grub ? [13:33] it's a recent patchset from Linaro (end of Dec) that enables the EFI interface in u-boot for grub-efi to use [13:33] ogra_: right, so if we have a signed kernel we can not concat, can we? because it has to be signed ? :( [13:33] ogra_: it's too much work for us [13:33] having two bootloaders sounds super awful [13:34] people could also do efi/tianocore instead of u-boot, but this is not our battle [13:34] lool, rhight, which is why i'D stay with uboot alone until we can do a full switch to grub [13:34] ogra_: there are always tons of firmware before us [13:34] the question is whether we want to support both u-boot and grub or only grub everywhere in some way [13:34] if we get grubs multiple-initird + direct loading from squashfs blobs, that would be a big plus. even if it means two bootloaders [13:34] I prefer the latter (grub only interface for snappy) [13:34] kyrofa, sergiusens: are we going to add a new milestone for snappy? [13:35] err, snapcraft [13:35] it will be slow and error prone to chainload [13:35] not to mention the maintenance burden [13:36] supporting uboot in snappy seems like the smaller amount of work in the end [13:36] ogra_: error prone in what way? I expect that nothing would ever update uboot, it would just be like a mbr. what kinds of errors will happen? or is it the grub part that is the concern? (sorry for these silly questions, trying to understand the concern better) [13:36] dholbach, yeah, but given the semantic thing, not sure it should be 2.1 or 2.0.2 depending on what we have :-) [13:36] I haven't actually tested the grub-efi stuff though [13:36] dholbach, I'll sort it out today [13:36] lool: heh :) [13:36] sergiusens: cool [13:38] ogra_: nevermind, maybe we should have a quick call at some point to discuss the various pros/cons [13:38] mvo, it is simply one layer more than needed ... it will at least be a lot slower to boot ... error prone just in the way of having to make sure the configs keep working together when something changes [13:39] ogra_: *nod* [13:39] ogra_: don't get me wrong, I'm not advocating it, just trying to explore the options and the various pluses/minuses [13:39] yeah, i get that [13:40] preferably we'd just have a postinst step in the os and kernel snaps that creates a merged file when either of them gets updated ... [13:41] finally found the thread again http://lists.denx.de/pipermail/u-boot/2015-December/239054.html [13:41] but if we sign the file at build time and require signed initrsds that will not work [13:41] from a snapcraft PoV we can do either; a kernel plugin which downloads the generic initrd from the archive which we use, but that brings in update issues if the generic thing changes under us [13:42] well, its all a matter of signed or not signed ... if we dont require the initrd we boot to be signed we can easily merge at any point [13:42] ogra_: exactly, this is why I like the idea about loading multiple-initirds via the bootloader. but I do get it that its horrid with uboot [13:43] mvo, well, its just a lot more work to do the math ... and it is board specific so porting gets a lot harder [13:44] once you have all the numbers it wont be much harder [13:44] but often porting to a new arm board even finding the right address and RAM sizes for booting with one initrd can be really hard [13:44] * mvo nods [13:45] so we double up the hardest bit :) [13:46] kyrofa an other question ps im NoiZeR from before todauy [13:46] *today [13:47] noizer_, sure [13:48] kyrofa can i mount the snappy sd card in an ubuntu system and then place the *.so files on it. Can the snaps then use the lib? [13:51] sergiusens: pushed updaetd rev with comment [13:51] sergiusens: not sure what mvos comment means [13:51] mvo: you say we should produce the complete initrd in kernel snap? [13:54] * asac reads more backlog [13:54] mvo: so current plan is to make mmodules/firmware only initrd in kernel snap and general one in OS snap and concat them preferable at bootloader [13:54] this would be exactly waht we would i fiigure [13:55] you are hesitant to that approach? [13:55] maybe having a call would help [13:55] :) [13:55] asac, that approach means that porting is a lot harder [13:56] but in the light of signed initrds that might be the only workable one [13:56] if we dont sign or dont require them signed there are more options (and less harder ones) [13:56] bootloader requirement yet [13:56] i think we coudl still merge them on disk [13:57] for bootloaders that cant concat load [13:57] not if they need to be signed [13:57] sure not [13:57] but not all platforms will need signing for hardware assertions [13:57] so if we ever want to enforce them signed, merging at load time is the only option [13:57] surely i dont want to ship core logic like our general boot code etc. in the kernel snap [13:57] thats even worse imo :) [13:58] right. think we want merging at load time [13:58] thats what mvo and lool propose [13:58] what do they propose? [13:58] merge at load time? yes +1 on that [13:58] asac, it will just make porting to new arm HW a horror trip for most porters [13:58] new arm HW probably gets a recent uboot which should be ok... i am more worried about old hardware [13:59] usually upstreams dont even give you a load address for a single initrd [13:59] it is hard enough to find the right bits and pieces for one [13:59] sure [13:59] i am mostly interested if and when we would get the load concat feature for pi2, beagle and dragon [13:59] now you need to figure out two of them and your uboot version needs to be recent enough too [13:59] if thats soonish, then i am bullish going down that oath [14:00] that would rule aout about half of the boards we have community suported today [14:00] yes, as i said i would love to also see a on-disk on update merge option [14:00] not every board is mainline or even uses a uboot version from the last 12 months [14:00] but its tricky due to how kernel and OS can move independently [14:00] no, thats not the prob [14:01] sure it is... you need to regen the combination of both [14:01] so you can fall back [14:01] both can have postinst magic to merge if either changes [14:01] at least its not trivial [14:01] yes, but you have to keep the combination of before the update [14:01] but you break the file signature for signed initrd [14:01] so you cannot just use the kernel or OS path [14:01] but have to mix them [14:01] sure [14:02] thats just a feature problem, but i think mvo is resistant for the reason i mention [14:02] you have one initrd per installed snap [14:02] (that is ... two kernel snaps, two OS snaps... i.e. 4 initrds) [14:03] there is still the option i wanted to do initially :) [14:03] and simply not merge them at all [14:03] but loop mount and link the contents [14:03] all from inside the generic initrd [14:08] kyrofa: thanks for the reviews [14:09] as far as I could I fixed everything up [14:10] dholbach, awesome! And no problem :) . I'm going back through it again real quick [14:18] dholbach, almost there on the variables [14:18] dholbach, I know it's tough to hit a moving target-- thanks for your efforts! [14:23] kyrofa: no worries - it's great to work with you guys as a team :) [14:23] dholbach, agreed :) [14:23] hmm [14:23] so is lib/firmware in a versioned subdir or not? [14:23] i see both things on my desktop [14:23] but apparently only old stuff in /lib/firmware/VERSION [14:23] lib/firmware isnt [14:24] it comes from linux-image-extra [14:24] but there is a versioned subdir with the kernel specific FW [14:24] but why do i have stuff in unverseioned /lib/firmware? [14:24] hmm [14:24] thats plain FW [14:24] ok so what the kernel installs goes into a versioned dir? [14:24] not bound to the kernel version [14:24] right [14:25] lib/firmware/$kvers [14:25] not really [14:25] that has device trees and kernel specific FW files [14:25] so upstream firmeare_install creawtes stuff [14:25] without version [14:25] it does for modules [14:25] so i am confused what goes were [14:25] hmm, thats beyond me then ... afaik generic firmware comes from linux-generic-extra [14:26] i have something in my kernel firmware_install that is in /lib/firmware [14:26] and specific FW should end up in the versioned dir [14:26] ask the kernel team then [14:26] apw: where do we put firmware? what goes in unversioned dir and what goes in versioned subdir of /lib/modules? [14:26] when using install_firmware kernel target it installs it withyout any version dir [14:26] and part of the stuff it puts there is in our unversioned dir e.g. atmsar11.fw [14:27] in lib/modules the only bits you should have in the toplevel are the dependency files from depmod [14:27] asac, firmware which is in a linux-firmware is in an unversioned directory, stuff that comes out of the kernel source goes in a version specific directory [14:27] ok, do we carry a patch or something against upstream? [14:27] asac, we supply the path we want it installed to which we put the version number in [14:27] or is it user space tool outside kernel that knows that location and loads stuff? [14:27] asac, the debian package version number [14:28] * ogra_ wonders what asac is after here [14:28] ok. well, think for our smnap its fine to have it not versioned as its really independent anyway on FS [14:28] lets see what happens :) [14:29] asac, well, it really depends on the search paths in the different tools [14:29] just stick to whatever firmware_install spits out feels safe [14:29] right. i was wondering waht does the firmware searching [14:29] if its part of the kernel then its fine [14:29] otherwise have to check out [14:29] why would you change what we have ? [14:29] i am not changing anything [14:29] * ogra_ tries to understand your target [14:29] i am building upstream kernel from source [14:29] into a snap [14:30] well, then just use make install with adjusted target dir and be done [14:30] so what loads the firmware :)? [14:30] i can do that [14:30] but only if it works :) ... [14:30] need to conf that whoever knows about search paths [14:30] heh, why wouldnt it [14:30] is in the OS [14:30] well, uif the kernel itself looks for it [14:31] then it would probably prefer to have it where it expects it \ [14:31] e.g. maybe we patched the kernel or use special config when building the in archive kernels to adjust the search path [14:31] but make install should cover this [14:32] and afaik you can still run any ubuntu with a home built mainline kernel (if your config is fine indeed) [14:32] so userspace shouldnt matter much [14:33] ok so the kernel itself looks for the firmware in the path? [14:33] then its all fine without doing anything [14:33] * asac keeps it :) [14:33] well, not the kernel but the respective drivers i think [14:34] elopio, standup? [14:34] just boot it, i'm sure your drivers will complain if they cant find firmware :) [14:34] booting is for the weak :P [14:34] lol [14:34] knowing is better [14:34] i cant boot it as we still have to figure the initd story above [14:34] i am just getting ready to whatever there [14:35] well i can boot, but not simply by using udf etc. [14:35] you can always use the system-image way :) [14:35] elopio, you up yet? [14:36] my initrd doesnt even have an init right now :) [14:36] guess showing how to include atmsar11.fw in the initrd is as good as any :) [14:36] for what is that ? [14:36] no idea :) [14:37] doesnt sound disk specific [14:37] just to show how you can use your snapcraft.yaml to select firmeware that goes into your initrd [14:37] of course not [14:37] and thats all you want from the initrd [14:37] if you have a better example out of what pops out of x86_64_defconfig let me know [14:37] let me give you the list to pick one from [14:37] * asac waits for build to finish [14:38] * asac thinks he would prefer to have 32 cores [14:38] ogra_: snapcraft-master/examples/kernel$ find parts/kernel/install/lib/firmware/ | pastebinit [14:38] http://paste.ubuntu.com/14679351/ [14:38] tell me which fw is better to pick as an example :) [14:39] maybe the e100 stuff? :) [14:42] anyway, good for now [14:42] lool: i got firmware feature now too. whats next? [14:43] ogra_: if you want to play, check out: https://github.com/asac/snapcraft/blob/kbuild-kernel-wip/examples/kernel/snapcraft.yaml [14:43] but no init in initrd so dont hope too much :) [14:44] asac: I'm working on a snapcraft snippet for a partner right now; I want to test the grub-efi patchset on dragonboard next to confirm feasability of multi initrd [14:44] gotcha [14:44] but worst case, we just cat the initrds together on arm [14:44] right [14:44] but mvo hates that idea [14:44] cat-ing? [14:44] but guess thats what he has to get over with [14:44] it's not different to the copy we're doing today anyway [14:44] hate is such a strong word :) [14:44] since we dont have squashfs loading on arm anyway [14:44] if thats the way, thats the way [14:45] lool: its because of rollback and keeping the rightg initrd combos [14:45] with independent kernel and OS snap [14:45] we need to keep all the previous cats around though so that we can rollback [14:45] but that should be ok [14:45] its a bit tricky, but guess nothing mvo cant solve [14:45] its just more book keeping [14:45] generally we need to agree on what happens to A/B with kernel and OS updates separate [14:45] I guess OS was never considered part of it [14:45] guess cat /path/to/os/version1/initrd /path/to/kernel/version2/initrd > boot/initrd-version1-version2 [14:45] dang :) [14:45] but now that it contributes an initrd snippet, it should trigger an A/B switch, shouldn't it? [14:46] asac: exactly [14:46] yeah we need to track latest OS and latest kernel and previous combo [14:46] for the bootloader now [14:46] long term i think snappy should have a tx log so you can stepwise go back and see exactly what combo of packages was installed [14:46] :P [14:46] asac: indeed [14:46] but thats nerdy i figure without having more advanced usecases [14:46] but still would be great to know how snappy system evolved to its current state [14:47] asac: I guess we agree that we explore grub multiple initrd and for uboot concat even though that means no signed initrds on uboot(?) [14:48] mvo: i would like to start with concat on arm yes [14:48] the option lool explores would be still nice [14:48] but as an option [14:48] and as a requirement if you want secure boot of course [14:48] fgimenez: did you have luck with m-o? [14:48] asac, any reason to not just glob lib/* ? [14:48] for those that cant do it we can validate signatures in our concat code of both pieces and put the merge sig next to it [14:49] ogra_: sure, the idea is that you can explicitely pick what you want for the initrd ... into the main snap all that gets installed with firmware_install gets in [14:49] e.g. after initrd the whole list you have in paste is avail [14:49] lool, mvo, asac, why not loop mount an img file with modules and fw and create the right links instead of merging two files [14:49] well, we want to leverage the ability to use drivers from initrd [14:49] to mount the FS [14:49] i think [14:49] yes [14:49] if we dont want that we can avoid all of this alltogether [14:50] if you loop mount the toher initrd you need to have the driver for that place [14:50] which you might not have [14:50] at that stage [14:50] * ogra_ would create a loop mountable file for lib/modules|firmware ... and simply have *all* modules available from the start ... then move mount that during boot to the rootfs lib/ [14:51] where do you put that mountable file though? [14:51] point of initrd is taht it gest shovelled into mem magically [14:51] without drivers [14:51] and simply make sure the kernel has vfat and loop mount support (plus whatever fs is used inside the loop file) [14:51] asac, next to the initrd [14:51] right. if we want to put that requirement up (e.g.t hat you can mount boot without modules) [14:51] into /boot [14:51] which is guaranteed to be vfat [14:51] then we can just not do an initrd in kernel snap [14:51] which we always support in our kernel [14:51] but we said for now we want to explore if we can do it right [14:52] sergiusens, elopio, PR updated --> https://github.com/ubuntu-core/snapcraft/pull/197 [14:52] *shrug* [14:52] i find that more "right" than merging two initrd halves :) [14:52] two initrd halves sounds right :P [14:52] hehe [14:52] you coudl get both from nfs [14:52] not to me :) ... but i'm not tied to the loop mount idea [14:52] nicely in uboot [14:53] or from the cloud of course [14:53] sure [14:53] i was hoping we woudl do somethign like that [14:53] but you have to do that on bootloader level anyway [14:53] or from pxe [14:53] but i couldnt get clear enough buy in so i think for now its best to just do it [14:53] with modules in initrd [14:53] and concat [14:54] and optimize later [14:54] then you can as well just provide a tool that repackst the initrd on the fly [14:54] that would be way less complex and effectively the same as merging the files [14:55] not so sure... i cant figure what that would mean right now, but we have understood the merge in bootloader approach as well as merge on disk [14:56] so lets unblock and do it :) [14:56] and its only one mvo patch away [14:56] and of course we have to produce an indep initrd on builders somewhat [14:56] but i think lool/mvo know what to do there [14:56] elopio, yes, it seem to be working now, after update-ca-certificates -f [14:57] elopio, i'm finishing some tests and then i'll put this into the ubuntucore/jenkins-ubuntu container, so that the next deploy will include it [14:59] elopio: fgimenez: did you guys think about having good tests for our kernel auto rolblack feature? [14:59] think would be key to have a strong stoory there [14:59] like having a bunch of instrucmented kernel modules taht make stuff crash or loop infinite in certain ways at certain times of boot process and ensure that our auto rollback never breaks [14:59] asac: we have tests for this in the integration suite that break the kernel snap [15:00] asac: but more cases of course is always better [15:00] mvo: with reboots? [15:00] asac: yes [15:00] i mean does that run on real hardware etc.? [15:00] asac, i can do the generic initrd within the next 2h ... thats just copy paste from code we use elsewhere [15:00] VM? [15:00] asac: on kvm, but elopio was working on tests on the BBB too for the kernel/os (or was it rpi2?) === opmodoro is now known as lcapra [15:00] but i think having to define "modules and firmware that go into the kernel" vs "just having all of it available at any time" is the wrong approach [15:00] mvo: thats cool if we have... how do we break the kernel? i think we need 3 cases: kernel does not exist at all, kernel panics early but does not reboot, kernel panics but ddoes reboot, kernel just gets stuck forever never reaching health check [15:01] *that go into the initrd [15:01] ok 4:) [15:01] ... sorry [15:01] ogra_: we dont define what goes into kernel [15:01] we define what goes into kernel at initrd stage [15:01] after that all is available of course [15:01] just in case thats not understood [15:01] asac, yes, i corrected myself above [15:01] i think thats wrong [15:01] kk [15:02] asac: yeah, I'm not sure which of those, I have a look now [15:02] all modules should always be available ... my loop mount way would provide that [15:02] you would need the kernel snap on th vfat [15:02] and just mount it [15:02] i dont think mvo wants that [15:02] the modules img [15:03] well, its close to the same [15:03] not the kernel snap :) [15:03] well, I see a gap here when it comes to e.g. PXE booting [15:03] can just be the kernel snap i figure [15:03] right [15:03] mvo, why ? [15:03] to support that our kernel would have to have all network drivers build-in [15:03] its just amnnother file you have to add to the PXE config [15:04] PXE already has to load initrd and kernel binaries [15:04] so you add a third file it loads to give you the modules img [15:04] the rest happens from the initrd [15:04] and where will that file be stored, i.e.. how does the kernel/initrd access it? [15:05] in ram ? [15:05] on the computer [15:05] I get that [15:05] 8sure [15:05] I mean, how does the kernel/initird access it? how is it availalbe to it? [15:05] how is the initrd available to the kernel in that setup ? [15:06] (you load it to ram and uncompress from there ... not much different to loop mount it from there if your kernel has the necessary bits included (which is has)) [15:07] ogra_: I know how inird works, but I do not know how you can load an image file into ram and tell the kernel to loop mount it from ram, that part is what I'm missing [15:08] me too, but i shouldnt be hard to figure out [15:08] PXE surely has the mechanics to get us that [15:08] (the load into ram part) [15:08] ogra_: ok, it sounds a lot like a second initird to me :) [15:08] mvo, except that it is not ... because it is the same modules.img your booted system uses [15:09] ogra_: the load to ram part I'm not worried about, sure, pxe can do that. the "how does the kernel get it from ram and mount it part" is what I'm not sure about. I'm sure its possible, I wonder it its straightforward [15:09] a second initrd is essentially what we do today (selection of modules included) just chopped into two parts [15:09] ogra_: right, the model you propose is different, but conceptually it sounds a lot like a initird to me [15:09] what i'm proposing is to drop the duplication of modules altogether [15:10] and re-use the modules.img on the booted system [15:10] ogra_: I think we agree on that [15:10] ogra_: oh, sorry, now I see what you mean [15:10] so you dont have to define firmware or modules in the snapcraft.yaml like asac currently does [15:12] this model is easy to implement in the vfat case ... but yeah, it will need some research and coding for PXE ... in the end though, PXE should give us the img content *somewhere* [15:12] mvo: can you tell me what to do in the end? [15:12] ogra_: its a very interessting idea. for pxe there are some more things to consider, the modules dir is ~200mb big, so a netboot would have to download that and put it into ram. [15:13] i think what ogra is proposing needs more thinking [15:13] but thats just me [15:13] if you feel comfortable to do that just tell me what the kernel snap should spit out/include\ [15:13] and i can replace initrd with that [15:13] initrd-modules-only that is [15:14] mvo, yeah, one would hope that someone had thought about it by today, given i promote that setup since the last orlando UDS :P [15:14] asac: well, the most simple way forward is that we require the kernel snap to be able to boot without loading modules. this way we can have the os snap provide the initird and we are unblocked and can research more options [15:14] if we do it we shouldnt spit out a separate modules.img [15:14] but just loop mount the kernel snap though [15:14] mvo: yes, but note that this is not true for the ubuntu packages [15:14] * ogra_ wanted that setup for normal ubuntu installs since ages :P [15:14] so we deviate from ubuntu [15:14] for the snapcraft built ones [15:14] ogra_: you mean the pxe case? or the how-to-get-from-ram-inito-a-mounted-dir thing? [15:14] that was another reason i didint like to do that in this step [15:15] mvo, the modules.img [15:15] in general [15:15] i had a long BOF session at the last orlando UDS where iheavily promoted it ... sadly it never happpened [15:15] asac: right, I think this only works if we do the same to our kernel, i.e. move squashfs in instead of having it as a module [15:15] ogra_: was anyone there against it? [15:15] ogra_: or did it just not happen [15:15] ? [15:16] * ogra_ has a hard time to type ... the cats are so happy i'm back that i barely am allowed to use the kbd today [15:16] mvo, the latter ... [15:16] ok [15:16] even the kernel team was interested back then [15:17] mvo: i dont think our kernel te4am will put that requirement up [15:17] after a year or so i stopped pushing for it though ... and never got to it myself ... the idea is ages old though and could have been researched by today ... including PXE [15:17] they want nfs boot and friends [15:17] hence my hesitation [15:17] nfs using crazy wifi modules [15:17] etc. [15:17] it would mean to include almost all network and disk drivers in the kernel statically [15:17] e.g. no-fly [15:18] asac, well [15:18] asac: right, multiple initird is my best answer then [15:18] basically all that is in current initrd [15:18] asac, today we include almost all network modules in the initrd [15:18] yes [15:18] all those would then go into the kernel statically [15:18] so its huge [15:18] you only move it around [15:18] no [15:18] if its module they dont get loade [15:18] d [15:18] sure [15:18] if its static its in the kernel all time [15:19] it gets loaded into ram with the initrd.img ... it wont be used later indeed [15:19] e.g. only what is needed vs all in ram [15:19] yep [15:19] but if you load a 19M kernel.img or a 19M initrd.img doesnt make much difference [15:19] both is bad [15:19] sergiusens, this was our previous standup: "So do you know anything about that?" "No, we'll have to ask sergio." "Okay, have a good day!" [15:19] pindonga, I'll look now [15:20] kyrofa, lol [15:20] * sergiusens reads the full backlog first [15:20] ogra_: the initird is freeed again once the boot is complete, no? [15:20] yes [15:20] mvo, yes, indeed ... [15:20] sergiusens, so it's a good thing ;) [15:20] but the loading will be the same, no matter where your modules live [15:21] s/modules/drivers/ [15:21] sure, but the mem waste will be something i am sure some folks in kernel/foundations/server team will not take time to even consider to consider [15:21] asac: on our suite we have a more or less generic way to break every snap. It would be great if you help us extend it with common ways to break a kernel update. [15:21] asac, there is no mem waste ... thats what i'm saying [15:21] elopio: right, see the four classes of issues i mentioned avbove [15:22] at most there is a move of the existing waste [15:22] asac: and it currently works on rpi and kvm. And it will work on bbb as soon as its kernel works again. [15:22] if we have one instrumentation for each of that i would feel pretty good to start with [15:22] * elopio reads back. [15:22] elopio: think it was somewhere close to where i highlighted you :) [15:23] * asac realizes lots of lines were said here after [15:23] asac: I think we have break systemd and break kernel, probably more [15:23] asac: fgimenez and elopio did a really good job here, we even found kernel bugs this way :) [15:23] right, kernel has subclasses that i mentioned [15:23] asac: yes, sorry for just glossing over this [15:26] there is a prob when the kernl doesnt actually panic though [15:26] and just hangs [15:26] yep [15:27] * ogra_ had that during dragonboard porting [15:27] at least in this scenaro we get back to a good state once you power cycle [15:27] asac: https://github.com/ubuntu-core/snappy/tree/master/integration-tests/tests Look at the ones starting with "failover". We have loops, crashes and empty files. [15:27] Sadly all the ones that touch the kernel have open bugs :) [15:27] yeah [15:28] elopio: ok so we have all the cases i metioned? great! [15:28] elopio: now i would like to use those to make a demo out oof it (e.g. so folks can experience this themselves)) [15:28] is that easy? [15:28] ok calls now [15:28] ttyl [15:29] asac: it is. go run ./integration-tests/main.go -filter failoverSuite [15:31] asac, if you have a running vm, "go run ./integration-tests/main.go -ip -port -filter failoverSuite" makes clearer the failover working === l__ is now known as lcapra1 [16:10] sergiusens, thanks for the review... sorry I missed those vendor bits there, just pushed the fixes [16:16] pindonga, no worries; I'm doing live testing now for the final bits [16:30] elopio, help with figuring this out would be appreciated btw https://travis-ci.org/ubuntu-core/snapcraft/jobs/105199856 :-) [16:32] sergiusens: merge with this one: https://github.com/ubuntu-core/snapcraft/pull/266 [16:43] sooo, how do I strace in snappy? [16:43] by using the snappy-debug snap i guess [16:44] ogra_: how? [16:45] it should ship an strace binary [16:45] Are there any snappy images that take the new package format? [16:45] * ogra_ doesnt know "how" though ... in the past i used th hack up the wrapper script for such stuff ... but that isnt possible in the new world [16:46] qengho, yeah http://people.canonical.com/~mvo/all-snaps/ [16:46] ogra_: sudo snappy install snappy-debug && strace yields comman not found [16:48] and 'find /apps/ -name strace' doesnt enlight me either [16:49] well, it would be snappy-debug.strace ... but yeah, thats not there [16:50] so ignore me then (i dont use snappy-debug much anyway, someone who does might be better to answer this ) [16:53] ogra_, yeah, snappy-debug only includes the security tool right now [16:54] yeah [16:56] ogra_, I would argue that one would enter the classic dimension and attach to a pid [16:56] if more than attaching is required, I am not sure how to do this then :-) [16:56] except by using the binary directly [16:57] since snappy-debug does not provide strace itself [16:58] Sweet5hark, any chance you're running an all-snaps image? [16:58] qengho, can you define "new"? i.e. squashfs-based? [16:59] Hi how can i deploy my snap that i made on Ubuntu 14.04 LTS deploy on my snappy OS [16:59] sergiusens, thats a pretty good idea .... we should have documentation for such stuff :) [16:59] kyrofa: indeed. [16:59] qengho, yes, but it's under heavy development right now. If you're okay with that, check out http://people.canonical.com/~mvo/all-snaps/ [16:59] NoiZeR_, i usually scp it over and then use sudo snappy install --allow-unauthenticated /path/to/snap [17:00] scp it to /tmp or /home/ubuntu [17:00] NoiZeR_, or use snappy-remote [17:00] yeah, if that still works [17:00] ogra_, I don't recommend scping to /tmp as it's tmpfs right now. You need that space to verify signatures :P [17:00] ogra_, unless you're on rolling/all-snaps I guess [17:01] i try to scp it [17:01] kyrofa: please elaborate? I run the edge image in vagrant on amd64 .... [17:02] kyrofa: (because the "stable" image was looping on two machines for me) [17:02] Sweet5hark, if you're willing to be even more cutting-edge, use the amd64 image out of http://people.canonical.com/~mvo/all-snaps/ and you can use the "Classic Dimension." Are you familiar with that at all? [17:02] Sweet5hark, then you can do as sergiusens recommended [17:02] with strace [17:02] kyrofa: nope, im not familiar with any of this ;) [17:02] Sweet5hark, good, no problem! [17:03] kyrofa, ah, indeed, depends how big your snap is :) (i usually use ~/ anyway) [17:03] Sweet5hark, if you use one of those images (which will become rolling eventually), you can use the classic dimension with snappy enable-classic, which allows you to use .debs and stuff. Good for prototyping [17:03] ogra_, well https://github.com/ubuntu-core/snapcraft/blob/master/docs/debug.md [17:03] thank dholbach [17:04] ogra_, yeah, true enough [17:04] Wtf where can i drop files on snappy [17:04] NoiZeR_, you mean with scp? [17:04] i tried in /temp but he sais read only file system [17:04] yes [17:05] NoiZeR_, try the ubuntu user's home directory [17:06] ok i pushed it on /home/ubuntu [17:06] thx [17:06] kyrofa: how do i best deploy that all-snaps image in vagrant/virtualbox? [17:07] Sweet5hark, for vbox I suggest just downloading that amd64 image and converting it into a vdi [17:07] * dholbach hugs sergiusens and kyrofa... and jdstrand! :) [17:07] * ogra_ thanks dholbach [17:07] I just extracted and updated the information from the the appdev manual :) [17:07] thanks for the reviews! [17:08] dholbach, which is a magical mythical document [17:08] Nice to get that stuff out there [17:08] I hope it will very soon be just part of the realm of magic, myths and fantasies :-) [17:08] Sweet5hark, however, I'm not familiar enough with vagrant to help you much there, sorry :( [17:09] dholbach, ha! [17:09] all right my friends - I call it a day - see you all tomorrow! :) [17:09] Seeya dholbach :) [17:10] Sweet5hark, does what I suggest for vbox make sense though? [17:12] oh vagrant [17:12] mvo, we might want to update utlemming on how to get the latest rolling stuff [17:14] kyrofa: I hope so? I created a libreoffice snap with snapcraft, set up an snappy env with vagrant/edge, installed the snap and with some intermediate LD_LIBRARY_PATH hacking foo got it to at least spit out an answer to "libreoffice --help" ... [17:14] Sweet5hark, ooo! [17:15] kyrofa, ooo feels like open office, this is libre office being discussed :-P [17:15] * kyrofa rolls his eyes [17:16] kyrofa: everything beyond that ("libreoffice --cat" "libreoffice --convert-to pdf" for now) though returns RC 77 suggesting that libreoffice thinks it needs to restart and setup a profile in ~/.config/libreoffice or somesuch ... [17:17] Sweet5hark, are you getting apparmor denials? [17:17] kyrofa: and now I want to know about the nature of "somesuch" -- so strace and tools like that to see what libreoffice is doing would be helpful ... [17:18] kyrofa: oh, ahem *cough* [17:18] Sweet5hark, approximately a million of them, perhaps? [17:18] a minor detail: Im sudoing/running as root for now because .... #YOLO [17:19] kyrofa: were should I check for apparmor denials [17:19] ? [17:19] Sweet5hark, you might want to tell libO to search for the profile under SNAP_APP_USER_DATA_PATH instead [17:19] Sweet5hark, those are logged to syslog, but try running snappy-debug. First install it with `snappy install snappy-debug` [17:20] Then run it with `snappy-debug.security scanlog` (use sudo if you're not su'd) [17:20] Sweet5hark, it should spit all sorts of recommendations if you're getting denials. Look through there for the config files [17:21] Sweet5hark, well, you really dont want to sudo an app ... it will try to write stuff to /root most likely [17:22] ogra_, not right now-- sudo still gets $HOME->/home/ubuntu [17:22] instead you want the right app wrappers to adjust paths and such for a normal user [17:22] your app will actually run as root anyway [17:22] kyrofa, weven after the command wrapper fired up as root and dumped all of env ? [17:22] kyrofa, fyi I did this instead of the manual m https://github.com/ubuntu-core/snapcraft/pull/268 [17:23] ogra_, indeed, $HOME is real uid, not effective [17:23] ah, cool [17:23] ogra_, so if you su, then what you say applies [17:23] right [17:23] ogra_, don't spread old var names :-) [17:23] ogra_: Im happy when libreoffice runs as root for a start. once it does that, Ill care about the rest. [17:23] sergiusens, shhh [17:24] sergiusens, i simply dont have the new ones in my head yet :P [17:24] kyrofa so i made a snap with this yaml file name: firstsnap [17:24] version: 0.1 [17:24] # The vendor for the snap (replace 'Vendor ') [17:24] vendor: Vendor [17:24] summary: test [17:24] description: test [17:24] icon: icon.jpg [17:24] parts: [17:24] cam: [17:24] plugin: python2 [17:24] source: run.py [17:24] eek ! [17:24] but how can i run it? [17:24] NoiZeR_, use paste.ubuntu.com please [17:24] NoiZeR_, ugh, please pastbin [17:24] * ogra_ points to paste.ubuntu.com [17:24] kyrofa ok xD [17:24] NoiZeR_, not at all ... [17:24] you would have to define an "app" entry for that [17:24] to either have it run as a service or as executable app [17:25] this is not the first time I see someone trying to put a file under source, not sure how to make it clear it is not the intended way [17:25] ogra_, I think it's still safe to assume 15.04 here, no? [17:25] we need a better error message there [17:25] kyrofa, well, it is probably safer to ask nowadays :) [17:25] ogra_, touche [17:25] ogra_ ok but what does i need to change then? [17:25] rolling is actually starting to get semi usable now :) [17:25] http://pastebin.com/ycB5RGGP [17:26] kyrofa: sooo, I ran "sudo snappy-debug.security scanlog" in one shell, which provides no output. then I ran my command in the other shell -- still no output. [17:26] ogra_ kyrofa here is the pastbin [17:26] NoiZeR_, look at the snapcraft-examples ... there should also be apps among them [17:26] Sweet5hark, huh, no denials eh? Check syslog just for sanity's sake real quick? [17:27] kyrofa: nothing screaming at me from syslog -- last line is a cronjob note [17:27] kyrofa, if Sweet5hark is running unconfined, that may be the issue [17:27] sergiusens, oh, good point. Sweet5hark are you unconfined? [17:28] * sergiusens reboots and will lose all backlogs; yay irc with no backend to support the backend [17:28] kyrofa: elaborate? [17:28] sergiusens, you need a bouncer [17:28] kyrofa, that is the backend to support the backend ;-) [17:28] sergiusens, :P [17:29] Sweet5hark, have you done anything special regarding security policies? [17:29] kyrofa: nope [17:29] Sweet5hark, okay, then you're just not getting denials. That's okay, but I do agree then-- you need to resort to strace [17:30] Sweet5hark, in which case I suggest you download that amd64 all snaps image [17:30] Sweet5hark, do you know how to get that working in vbox? [17:33] why not kvm ? [17:33] ogra_, he asked for vbox [17:33] (saves you from converting the img) [17:36] NoiZeR_, look at the snapcraft.yaml of the godd example for an executable app ... i think thats the most simple thing you can do [17:37] for a service look at the snapcraft.yaml of the shout example [17:37] you want the "apps:" entries [17:39] * kyrofa thinks cmake printing 100% more than once is an oxymoronic feature designed specifically to tease [18:01] sergiusens, pushed fixes... didn't do anything about upload progress bar yet... I think that could be added later in a separate PR (as an improvement on top of this) [18:01] pindonga, sounds good [18:01] ooooh ! [18:01] http://www.cnx-software.com/2016/01/27/orange-pi-one-quad-core-arm-linux-development-board-launched-for-9-99/ [18:02] someone needs to do a snappy image for that ! [18:05] sergiusens, does snapcraft 2.0 no longer have a force? [18:06] kyrofa, we removed it to do it properly, remember? [18:06] kyrofa, i think sergiusens just forgot to build it with the --luke option :P [18:06] kyrofa, as in, we still need to get together and design it [18:06] ogra_, groan [18:06] hah [18:07] sergiusens, indeed, just double-checking. I was just taking a look at bug #1477904 [18:07] bug 1477904 in Snapcraft "snapcraft.yaml needs to make all stages dirty (was: the can't add file without recreating entire package)" [High,Triaged] https://launchpad.net/bugs/1477904 [18:09] kyrofa, ah, you want to tackle that one? It is almost like auto-force :-P [18:10] sergiusens, just marking steps dirty via modification times is easy... but is that the best way? If they just add one little unrelated bit to this copy plugin part down there, now I have to re-pull and build the linux kernel part I have at the top? [18:11] sergiusens, can we be smarter about this? [18:12] kyrofa, do you want to look at this one today? :-) [18:12] if so, hangout time [18:13] sergiusens, there are others I can look at as well-- I imagine you're a bit swamped [18:15] kyrofa, I feel swamped, not sure that I am swamped :-P Yeah, better tomorrow as it is quite complex; I've been thinking about splitting into three parts; but better do it tomorrow, yeah [18:15] sergiusens, yeah, sounds good :) [18:16] sergiusens, we should start having planning meetings every once in a while [18:17] sergiusens, this issue, --force, plugin api improvements, etc [18:17] kyrofa, yeah, together with our bug triaging meetings ;-) [18:17] heh [18:17] let me set that up [18:18] sergiusens, yeah, though I feel like we've gotten better about that [18:20] kyrofa, so friday it is [18:21] sergiusens, excellent, thank you :) [19:05] sergiusens, thanks so much for approving that PR! [19:05] np, looked good, did what it said it would do; so thanks! [19:14] sergiusens: yes, I was hoping to have ud-f- in the archive [19:14] sergiusens: codereview *cough* ;) [19:32] mvo, didn't I review already? In any case, I think simplestreams is on trusty [20:19] elopio, ideas http://paste.ubuntu.com/14682191/ ? [20:28] sergiusens: is that local? [20:29] xenial? === chihchun is now known as chihchun_afk [20:34] sergiusens: I had a similar problem, it didn't find fixtures. I worked it around with PYTHONPATH=/usr/lib/python3/dist-packages ./runtests.sh unit [20:34] to me it seemed like python3.5 crazyness. [20:42] elopio, k, yeah local on xenial [20:55] elopio, kyrofa aside from requesting an integration tests, comments appreciated https://github.com/ubuntu-core/snapcraft/pull/270/files [20:56] sergiusens, awesome! [21:51] kyrofa, elopio and now we have an integration test