[01:15] how do you switch to the alpha channel? or maybe install a gui? [01:15] also why is man not included and why is the root fs image ext4 instead of squashfs? === c74d is now known as Guest7878 === c74d3 is now known as c74d [07:00] good morning [07:21] Good morning all [07:22] hey dholbach and davidcalle, good morning [07:22] mvo, hello, I don't seem to be able to catch rcj on IRC, any idea at what time he is around? [07:25] hi mvo [07:30] davidcalle: a good question, he is US timezone AFAIK [07:33] mvo, thanks, I've noticed, but still no luck in my european evenings. Nothing urgent though, I'll drop him an email. [07:36] * mvo nods [08:21] Morning folks, I've been asked a question on G+ which I'd like to check with you [08:22] willcooke: what is the question? [08:22] Will it be possible to install Snaps on the U7/.deb based at some point in the future? [08:22] Either for developers to test their packages [08:23] or as an alternative to .deb based application installation [08:28] willcooke: I think so, yes. not sure when, but conceptually I see no reason why that would not work. you can have snap/deb in parallel just fine [08:28] cool, thx mvo [08:35] Good morning all; happy Senior Health & Fitness Day! 😃 === erkules_ is now known as erkules === sabdfl_ is now known as sabdfl [11:41] asac, https://code.launchpad.net/~rick-rickspencer3/+junk/go-template [11:41] Chipaca, ^ since you were there last week ;) [11:50] mvo, where do I file a bug against snappy build? [11:52] rickspencer3, see topic ;) [11:52] oops, it was beneath the fold in xchat-gnome and I didn't see it :) [11:56] thanks ogra_ [11:56] np :) [11:59] pitti, do you happen to know what handles /etc/mtab in the boot process (pre and post systemd) ? [12:00] for pre-systemd i see /etc/init.d/checkroot.sh but i'm not sure that is used at all [12:15] ogra_: how do you mean "handle"? [12:15] /etc/mtab is not the thing you are looking for :) [12:16] ogra_: post-systemd> entirely ignored; the Debian package turns it into a symlink to /proc/mounts for legacy stuff that parses it, but systemd, util-linux, our boot scripts etc. all don't care [12:16] ogra_: I'm not entirely sure about mountall, though [12:17] ogra_: we never ran /etc/init.d/checkroot.sh, both upstart and systemd override it [12:19] hmm, k [12:20] pitti, we have some code on the system-image server that unpacks/repacks the rootfs tarball to enforce it being a symlink ... i'm not really sure why it is there [12:20] ogra_: it's only there because some stone old software might look at it [12:21] ogra_: which is hopefully not the case on touch, and most certainly not on snappy, so if we just want to remove it there, please let's [12:21] yes, and we dont use any such software on the phone and snappy images [12:21] yet stgraber once added code for it [12:21] this is sorta like the x86 A20 gate -- everyone hates it, it's broken, but it's sticky as glue :) [12:22] ogra_: I think we shoudl start dropping it from snappy; we can test the core, snapps should not ever care [12:22] and drop it from touch as well if we ever aren't in a "deep freeze/product release round the corner" situation [12:22] well, we use systemd, so i guess on snappy it is already gone [12:22] touch not so much ... and i cant reqally find out why [12:23] ogra_: as I said, our systemd package creates an /etc/mtab -> /proc/mounts symlink for legacy stuff compat [12:23] initrds /init makes it a symlink on start [12:23] but i dont find any code that turns it back into a file after run-init [12:23] yet, on my desktop installs it is a file [12:24] ogra_: oh - we had a bug about that the other day [12:24] ogra_: I don't remember which package any more, it was something like lxcfs or so [12:24] somethinng removes the link and makes it a "proper" mtab again [12:24] which accidentally turned an mtab symlink into a file [12:25] but I added a postinst to fix that, so maybe that got broken again [12:25] ogra@styx:~/Devel/tmp/meizu$ ls -l /etc/mtab [12:25] -rw-r--r-- 1 root root 1096 Mai 22 12:59 /etc/mtab [12:25] thats my utopic laptop [12:25] rickspencer3: what problems did you bump into? (bug#?) [12:25] ogra_: ah, utopic [12:25] ogra_: yeah, we didn't have any transition code under upstart [12:25] Chipaca: https://bugs.launchpad.net/snappy/+bug/1459212 [12:25] Ubuntu bug 1459212 in Snappy "snappy build does odd things with poorly formed /meta/package.yaml" [Medium,Confirmed] [12:25] pitti, well, i doubt it is any different in upstarted vivid installs [12:25] which touch is [12:25] * genii sips and ponders "odd things" [12:26] mvo: maybe we should state “snappy build is a SISO pipe” [12:26] ogra_: right [12:26] ogra_: sorry, you said "desktop", and I implied "vivid/wily" [12:26] but grepping recursively for mtab in /etc/init* doesnt really reveal anything ... [12:26] Chipaca, Chipaca https://bugs.launchpad.net/snappy/+bug/1459212 [12:26] rickspencer3: yep, mvo pointed me at it [12:26] probably mountall :/ [12:26] * ogra_ goes to get the code [12:26] ogra_: right, transition happens by /lib/systemd/system/debian-fixup.service calling /lib/systemd/debian-fixup [12:27] rickspencer3: at least, an easy fix :) [12:27] ogra_: so what are you looking for now? [12:27] pitti, a way to make mtab not being created in the vivid touch images [12:27] and just keep the symlink around that we get from the initrd [12:27] ogra_: ah, right; TBH I have no idea what creates it in the first place [12:28] i want to drop the link creation from the system-iage server so we dont need to re-pack the rootfs anymore [12:28] this thing is like "slap, go away", and it comes back [12:28] yeah [12:28] i remember discussion turning it into a symlink around dapper time :) [12:29] *discussing [12:29] * pitti grep -w mtab * in /var/lib/dpkg/info [12:29] * pitti sees bug 906293 and closes it [12:29] bug 906293 in util-linux (Ubuntu) "Get rid of /etc/mtab and store userspace options in /run" [Low,Triaged] https://launchpad.net/bugs/906293 [12:30] ogra_: hm, neither package maintscripts nor debootstrap [12:31] live-build? livecd-rootfs? [12:31] pitti, yeah, it must be something in the boot process ... else the symlink the initrd created would just stay around [12:31] Übereinstimmungen in Binärdatei /sbin/mountall. [12:31] * pitti RTFC [12:31] sigh, i knew it [12:31] yeah [12:32] so i will do then ... [12:32] ./src/mountall.c:written_mtab = TRUE; [12:32] ./src/mountall.c:if (written_mtab) [12:32] fairly sure we have a "winner" [12:32] yep [13:12] mterry: hey, so made my snaps, but then when i ran clocks, it gave me [13:12] https://pastebin.canonical.com/132060/ [13:12] did you ever hit that? [13:12] weird RAOF hit it last night as well when he tried... [13:12] kgunn, oh right! [13:13] kgunn, export LC_ALL=C.UTF-8 [13:13] kgunn, sorry, should have been part of my instructions [13:13] np! [13:13] kgunn, any specific reason to use a secret pastebin in the public channels ? [13:13] :) [13:13] nope sorry [13:19] mterry: success!....now matters of curiosity, why does it herald "bad system call" on running clocks [13:19] probably seccomp blocking it ? [13:20] kgunn, interesting, not sure. I'd have to look at a strace -- didn't remember seeing that [13:20] kgunn: dmesg should tell you what syscall was blocked [13:20] kgunn: then you can use "scmp_sys_resolver $nr" to figure out what the number means [13:27] thanks mvo [13:28] dirty dog....seems it no longer appears when i launch clocks, like a one time failure ? [13:28] rcj, ping [13:29] davidcalle, hi, what's up? [13:30] rcj, hey, do you mind having a look at this Amazon ec2 doc bug? https://bugs.launchpad.net/developer-ubuntu-com/+bug/1456390? [13:30] Ubuntu bug 1456390 in Ubuntu Developer Portal "Snappy ec2 getting started - various issues" [Undecided,New] [13:31] davidcalle, I can in a few minutes. My mouse pointer has disappeared so I need to restart before I can do precision tasks like clicking on links. I'll be back on in 5m. [13:31] rcj, :) [13:39] davidcalle, I've added our team to the bug and I'll get us started on this [13:41] rcj, many thanks! I'll take care of updating the doc, just ping me or add changes in a comment on the report. [13:53] davidcalle, I provided answers and assigned it back to you. [13:54] rcj, great, thanks :) [14:03] good morning. [14:55] bug 529993 [14:56] bug 529993 in mountall (Ubuntu) "mountall produces unharmful error messages when /etc/mtab is symlink to /proc/mounts" [Low,Fix released] https://launchpad.net/bugs/529993 [15:37] sergiusens: what's the plan with the cmd directory? [15:39] elopio: we need to rewrite what is inside the code; the external interface (cli) should behave the same [15:40] elopio: but basically moving to client/server; so the client could be easily mocked when this happens too ;-) [15:42] sergiusens: good. I was thinking that the cli conformance tests should be unit tests, leaving just a few functional tests. But I started digging how to fake the replies and that was not easy. [15:48] @reviewlist [15:48] https://code.launchpad.net/~stephen-stewart/webdm/remote-and-local-collections-with-sorting-and-filtering/+merge/260305 | No reviews (less than a day old) [15:48] https://code.launchpad.net/~sergiusens/webdm/godepsForBuilding/+merge/260156 | No reviews (less than a day old) [15:48] https://code.launchpad.net/~stephen-stewart/webdm/repent-harlequin-said-the-ticktockman/+merge/260120 | No reviews (less than a day old) [15:48] https://code.launchpad.net/~stephen-stewart/webdm/the-sun-is-the-same-in-a-relative-way/+merge/259933 | No reviews (4 days old) [15:48] https://code.launchpad.net/~stephen-stewart/webdm/icons-icons-icons/+merge/259800 | No reviews (5 days old) [15:48] elopio: no, not at all; well you can set functions to var in as in var snappyInstall = snappy.Install and then in the tests set snappyInstall to your mock [15:49] elopio: if you aspire to 100% coverage, this is the only way I think, unless Chipaca has a more clever idea [15:50] elopio: the go stdlib is full of exported functions calling the internal function and testing just the internal functions [15:50] beowulf: feeling better? [15:50] sergiusens: much [15:50] sergiusens: migraine [15:51] mvo: if you want some reviews ^^ i can start you by saying i haven't enough tests [15:52] sergiusens: hum, Let me try that. I'm not looking for 100% coverage (yet), I just want to improve tests like 01_test_info, which just check that the reply conforms to the spec. [15:52] sergiusens: i added a card to trello to expose what would be the results of a GET of installed snaps to the GO templates, i want to bootstrap the app so it doesn't have to make an api call first time [15:52] beowulf: heh :) dinner time here, probably needs to wait until tomorrow (unless someone lese beats me to it) [15:52] elopio: the what I propose will get you going [15:52] mvo: no problem , dinner here too [15:53] sergiusens: I'll play with that. But better wait and do it with the refactor of cmd, I think. [15:53] beowulf: oh nice, if we do this, can we just do updates with websockets for all else? [15:53] elopio: that will happen next month the soonest I guess [15:55] beowulf: I'll do your reviews, and let me set something up for tomorrow to discuss websockets, that template you speak of and all else [15:55] sergiusens: in the mean time we can work on tests for the functional part. Install a snap -> check the list, and things like that. Which are better without fakes and will survive the refactor. [15:55] sergiusens: yeah, but in the first instance i'm going to rely on a local (to the browser) list initially populated from the Go template, then sync the list in the background when the user won't notice [15:56] elopio: agreed [15:56] sergiusens: also, sad face that we don't have https, is that correct? [15:56] beowulf: works for me; initially we were going to go with a go template for most things so I'm not against it [15:56] beowulf: no https as no cert, but there is a plan [15:57] no https would rule out a couple of browser apis that webdm could use [15:57] sergiusens: elopio: I have zero clever ideas. [16:02] Chipaca: oh you [16:03] beowulf: well, it's coming === rickspencer3_ is now known as rickspencer3 [16:48] bzoltan, tedg: want to join a quick hangout to talk about IDEs & snapcraft? [16:48] mterry: tedg: I can do a quick chat [16:48] isnt that just a plugin thing ? [16:49] ogra_, well that's what I'm trying to determine, but it may influence how buildenvs are stored etc. fact finding [16:50] ah, well, i thought we would have pluggable build envs too :) [16:50] mterry: ogra_: I am fine with wjatever plugin :) as long the rootfs and the tollchain of the target is directly accessible from the IDE :) [16:50] so it would just be a matter of picking the ones the IDE wants [16:51] bzoltan, right, but other parts might have different reqs ... so i would envision we have pluggable backends as well as pluggable frontends ... and the frontend just defines what it wants as backend env [16:51] ogra_: mterry: so any sort of wrapping and confining of the toolchain+rootfs is a showstopper for the IDE [16:51] there might be parts that require a clean bootstrap ... others might want to use a VM ... you want to use a chroot etc [16:52] ogra_: I am fine with it... as long after plugging I can access the toolchain and the rootfs :) [16:52] sure, your backend plugin would have to do that then :) [16:52] ogra_, why would there be so many options? you just want a .snap sitting there, why require a VM? [16:52] ogra_: Noooooouuuuu :) I do not want chroot... I want sysroot + native toolchain_S_ [16:52] mterry, no idea, i was just making up examples :) [16:53] * bzoltan things that G+ hangouts are overrated :D [16:53] mterry, i think snapcraft should be flexible here [16:53] s/things/thinks/ [16:53] bzoltan, +1 .... excludes the community from the discussion :) [16:53] ogra_, the more flexible we are, the less magic we can be. We wanted to be very magic, ideally user never has to think about buildenvs [16:53] mterry: I second that [16:53] mterry, not a user ... a provider for a certain setup [16:54] bzoltan, talk me through the different options. So VM doesn't make sense, sure. chroot is bad because your plugin would need to be root to get in? let me look at what sysroot is, /me is dumb about it [16:54] pypi will have different reqs than node-snapper ... [16:54] C++ will have another set of different reqs [16:54] or java ... [16:55] ogra_, provider for a certain setup? oh, the pypi plugin just gets pointed at the buildenv and uses it. Why does it care what sort of chroot thing it is? [16:55] mterry: Ok... I explain... [16:55] bzoltan, ok, so sysroot is a gcc option? [16:55] I thought a VM was fine as long as the build directory is in the local FS. [16:56] i.e. we just need to be able to pull the directory through some magic into the VM. [16:56] That way the IDE can get full source introspection. [16:56] tedg, you also want direct interaction from your Ui etc ... [16:56] mterry: In our present app dev SDK we use the chroot model.. it means that whatever Ubuntu you are on... even on LTS ... and you develop for 15.04 phone ...teh chroot will be an x86 15.04 chroot with the armhf libs and headers. And we use the x86 toolchain (make-compiler-linker) to build the armhf binaries [16:57] Ah, so it needs some routing of STDIN/STDOUT [16:57] yeah, i guess so [16:57] mterry: that is a major hackaround ... but a huge saving because we do not need target specific toolchain... we simple use the target system's native one [16:57] but bzoltan experimented with VMs and can surely give you more details about the probs [16:58] bzoltan, hold up [16:58] bzoltan, you say "target specific toolchain" and "target system's native one" [16:58] mterry: yes [16:58] bzoltan, can you please dumb that down a tad? "target specific" is x86 toolchain that knows how to build armhf? [16:59] And "target system's native one" is actual armhf binary toolchain? [16:59] mterry: so when you compile a phone app you do not use the LTS edition of the toolchain, but the one from the chroot [16:59] I think he's also saying that it's the version of the target as well. [16:59] tedg: precisely! [16:59] So the wily would default to gcc 5.2 but trusty would be a gcc 4.x [17:00] Ok... So "target specific" is same-arch-diff-version, and "target native" is same-arch-same-version? [17:00] bzoltan, So then do you run the test suite using a QEMU hack? It seems like you still need to be able to exec armhf binaries in that chroot somehow. [17:00] tedg: mterry: some bed time reading -> https://developer.ubuntu.com/en/blog/2015/03/18/everything-you-always-wanted-know-about-kits-were-afraid-ask/ Kits here are the chroots [17:00] mterry: a sysroot is nothing else than a rootfs, plain simple. The catch here is that you have a specially built toolchain that can use that rootfs as the source to get the headers and libraries. So instead of /usr/include, it usues /path/to/sysroot/usr/include. Same for libs [17:00] And then the toolchain can also cross compile? [17:01] well, irt kind of does a native build [17:01] mterry: the toolchain itself is a cross toolchain running directly on the host. Its the standard way for most cross build SDKS, including android and Qt [17:01] but using the host toolchain inside the chroot for speed [17:01] mterry: thats why all lots of tools support it by default [17:01] in a cross way [17:02] tedg: for running cross build tests you need a emulator of course [17:02] it is the same ugly thing that scratchbox and the SuSE OBS use ... [17:02] zbenjamin, yup. Just trying to think of non-c++ stories too [17:02] So then do we need kits to be more generic than QtCreator? We need "snapcraft Kits" ? [17:02] mterry: does a nodejs for example need to be installed systemwide? can't you use a nodejs installation from everywere to build your app? [17:03] mterry, as long as it is gcc you can use that model for all langs gcc compiles [17:03] mterry: ogra_: We as the SDK team are lobbying for target specific toolchain + sysroot modell for years... the chroot solution is an acceptable ugly solution we can live with.. but hat is not troublefree either. VM is pointless. [17:03] zbenjamin, I'm sure. And Python can set PYTHONHOME and all that [17:03] zbenjamin, but I'm not sure about other compiled language cross compile stories. I'm sure they have something, but I've never done it in Go for example. I know it can, just don't know what it needs [17:03] mterry: go as well can be used from everywhere afaik [17:03] zbenjamin, take a look at node-snapper ... [17:04] bzoltan, zbenjamin: (A) you're IRC nicks are driving me crazy [17:04] i envision we'll use something similar for node in the end ... the prob with node is that you need to compile the modules on the fly ... [17:04] mterry: lols :D [17:05] bzoltan, zbenjamin: (B) it sounds like you guys just want a directory sitting on disk. Nothing special to it. It will just contain its own headers and build env setup. Right? [17:05] ogra_: mterry: also i was wondering how to handle snaps that combine languages? For example C++ and go and nodejs .... [17:05] mterry, I think they want the default paths changed. So, really, it's kinda snap-ifying the toolchain. [17:05] zbenjamin, good question ... on a snap level you just throw the binaries into the right places [17:05] zbenjamin, I suspect each language plugin would know how to set up its own parts [17:05] zbenjamin, for building that gets interesting :) [17:05] mterry: almost yes, the qt tools need to be built for cross building but when we have the base sysroot + toolchain thats easy [17:06] there we will need the magic that mterry mentioned earlier ;) [17:06] tedg, default paths? I think they just want to set sysroot at build time to point at the buildenv, not by default [17:06] mterry: ogra_: so if we think about sysroots and chroots. We totally could use a chroot as sysroot, all we need is the toolchain and the cross build tools (in our case Qt) in a extra location [17:07] or copied in place ... or linked/bind-mounted/overlayfs-mounted [17:07] ogra_: don't say the bad word "overlayfs mounted" :D [17:07] might be our future :) [17:07] zbenjamin, but you want to ideally be handed a directory that has usr/include usr/lib etc inside it, as well as usr/bin/gcc and friends. [17:08] mterry, quite a bit more ... [17:08] we're talking about gigs of build deps [17:08] mterry: zbenjamin: yes, the sysroot is really not much else then the directory structure of the target system + dev files ... PLUS we need the toolchain what knows the whereabout of that diretcory [17:08] ogra_, yeah, the whole shebang. everything needed to build [17:08] (the whole phone framework) [17:09] ogra_: as i said we could use the libraries from the chroots. We just need a external toolchain [17:09] bzoltan, the toolchain which sits outside that directory? why wouldn't it be the toolchain inside that dir? for speed reasons in case we are cross compiling? [17:09] ogra_: are you sure it is gigs? [17:09] mterry, yeah [17:09] mterry: thats a gcc requirement [17:09] mterry: afaik a relocatable toolchain can not be inside the sysroot [17:09] mterry: the location of the toolchain is not relevan, as long it is not emulated but native [17:09] mterry, using the native armhf compiler in a qemu-user-static chroot works but is slower and less reliable ... [17:10] OK, well that's fine [17:10] so you use the host one in cross setup [17:10] Host toolchain is fine [17:10] and copy/link/mount it into the chroot [17:10] But libs and everything inside the sysroot directory [17:10] ogra_, wait, I thought it didn't need to be inside? [17:10] ogra_: mterry: catch is we can not move the toolchain around. Gcc does not like this. Except we compile it as relocatable. [17:10] yeah, they need to match the target [17:10] zbenjamin, why is that a catch? [17:11] zbenjamin, gcc outside, all libs inside, right? [17:11] yeah [17:11] mterry: i mean we can not use the default toolchain from the archive [17:11] * tedg needs to run, but will catch up on the backlog when back (much better than hangouts) :-) [17:11] mterry: at least not how its compiled atm afaik [17:11] * mterry *likes* hangouts [17:11] zbenjamin, oh wait weird. I didn't know we were using non-archive gcc [17:11] mterry, especially the transcripts that you can read from logs next week ? [17:11] :P [17:12] zbenjamin, so there's some SDK-specific gcc we need to use? [17:12] ogra_, I get it! I just like your pretty faces is all [17:12] haha, we like yours too :) [17:12] mterry: either that, or we need to make sure the archive toolchain supports what we need [17:12] mterry: we probably need to ask a toolchain dev what is required here [17:12] zbenjamin, (why hasn't that happened? was it rejected?) [17:12] zbenjamin, what exactly is it that we need that archive doens't do? [17:13] mterry: it needs to support the --sysroot switch. [17:13] mterry: and i think you need to compile that in [17:13] zbenjamin, oh weird, that's not standard? curious [17:13] mterry: yes, it was rejected with the explanation that they have no time and hands to maintain more than one toolchain [17:13] mterry: basically we need a toolchain that can select the sysroot its compiling against [17:13] So this is not a solution we could build today [17:13] Or on 14.04 [17:13] bzoltan: mterry: yeah but back then we didn't come up with the idea that the default toolchain could do that maybe [17:14] zbenjamin: mterry: we did not know [17:14] yeah I get that. I'm just trying to figure out how to move forward [17:14] Sounds like this won't be sufficient for now [17:14] mterry: correct.. on LTS we are busted either way ... because we can not replace the toolchain there [17:14] mterry: yes so that would be only possible when we SRU that toolchain back [17:14] that's not happening :) [17:14] mterry, get doko really really drunk :) [17:14] :D [17:15] ogra_: add a "really" there :D [17:15] (step 1) [17:15] ogra_: you can not get doko _THAT_ drunk [17:15] zbenjamin, bzoltan: OK... so this sysroot solution is a future idea. For next LTS [17:15] yeah, it is also very hard to get him drunk :) [17:15] zbenjamin, bzoltan: what do you need on 14.04? You need a chroot? [17:15] mterry: sysroot + fixed toolchain on new releases and chroot on old releases [17:15] If I give you a sysroot directory on 14.04, you can't compile using it? [17:16] bzoltan: mterry: since we switching to distribute the SDK from an installer rather the archive we could ship a new toolchain with it (maybe) [17:16] mterry: on LTS we already have the chroot based solution... kind of okey [17:16] bzoltan, ok... so on 14.04 give you a sysroot directory with toolchain *inside* [17:16] zbenjamin: +1 [17:16] hmm, ok... [17:16] mterry: yes, and we chroot to that sysroot [17:17] mterry: our problem is we are hitting a wall with backporting the SDK because of the new Qt that is incompatible [17:17] you dont really want to diverge from the archive toolchain [17:17] bzoltan, zbenjamin: OK, so only difference is if toolchain is inside or outside sysroot [17:17] ogra_: true :/ [17:17] we need to find a way to utilize it from the archive [17:17] ogra_, yeah... but I'm ok having a copy of it inside a directory [17:17] mterry: yes.. and that with the present solution we chroot [17:17] ogra_: how about using exactly the same sources but enabling the sysroot caps? [17:17] ogra_: would that break ABI? [17:18] mterry, sure, but that looks more like "we need to recompile it first" [17:18] zbenjamin, bzoltan: is there a bug for this yet? [17:18] (to enable --sysroot) [17:18] zbenjamin, it could even get you different binary behavior ... [17:18] mterry: no i kind of just came up with it :D [17:18] ogra_: crap [17:18] zbenjamin, file one then from the SDK point of view, I'll subscribe and watch :) [17:18] ogra_: yeah thats the problem , we probably need to ask someone who knows the toolchains by heart [17:18] right [17:19] either doko or infinity [17:20] zbenjamin, bzoltan: so even if I gave you a chroot, that would still work in the --sysroot future, right? You just wouldn't use the toolchain sitting inside of it [17:20] mterry: yes, it would work [17:20] mterry: you can even setup the very same chroot to do, lets say nodejs builds in it [17:20] mterry, how do you handle the "cross" part in that ? [17:20] ogra_: same as we do with the click chroots? [17:21] how do we handle it there ? qemu-user-static ? [17:21] which is just build once per chroot, right? [17:21] ogra_: cross toolchains [17:21] ogra_: native cross toolchains [17:21] oh do you still have one chroot that builds more than one arch output? [17:21] thats enough for QT ? [17:21] mterry: ogra_: the present chroot solution in single run builds are pretty functional... the problem with it is that 1) takes time to set up 2) unstable for long run 3) needs root to set up [17:22] mterry: ogra_: 4) slows down the IDE horribly 5) leaks mounts [17:22] bzoltan, zbenjamin: I'm not saying I love chroots. By all means, get rid of them [17:22] 6) restores the leaked mounts after boot [17:22] bzoltan, zbenjamin: but for now, they're the only game in town :) [17:22] mterry, that might work for just Qt ... as soon as anything in the build tries to exec something inside the chroot you will need something like qemu-user-static again though [17:22] mterry: getting rid of chroot is good... but not by migrating to th even more messy VM [17:23] bzoltan, I'm not advocating that. I like the sysroot thing just fine, I think. But it's just not available today is what I'm saying [17:23] bzoltan, go get it added to our toolchain and then we can talk [17:23] and it is limited to the toolchain ... [17:23] ogra_, yeah we may need one-chroot-per-arch [17:24] mterry: the funny things is that pretty much all otherplatform's sdk is using that sysroot + toolchain modell ... nobody is using chroot or builder VM [17:24] mterry, and something clever that notices if some build script tries to exec something inside the foreign chroot [17:24] this needs to be wrapped in qemu-user-static calls [17:25] ogra_, or just run the build under qemu in the first place [17:25] and be slow [17:25] no [17:25] +1 ogra_ [17:25] bzoltan, that's fine! I can't control other platforms' sdks :) [17:25] then your RPi will be faster :) [17:25] mterry: that is called scratchbox :) NOOOOOO [17:25] bzoltan, but nor can I flip a switch and fix our toolchain. File a bug! [17:26] ogra_, bzoltan, zbenjamin: OK, OK :) No qemu-by-default... [17:26] mterry: +1 will be done [17:26] mterry, so your first focus shoilld be to develop such a switch then :) [17:26] *should [17:26] ogra_, oh that's just for the toolchain-in-or-out-of-sysroot problem. That doesn't solve the qemu problem, does it? [17:26] ogra_: atm all projects that are compiled for the phone need to be cross build capable. And so all tools since we have no qemu there. But thats just the phone story of course [17:27] ogra_, yeah maybe if a snapcraft language plugin needs qemu, it will know that and do what's necessary. but most won't need qemu, one hopes... [17:28] mterry, qemu-user-static (just syscall translation) is reasonably fast ... but also slightly unstable (not all syscalls can be translated) ... what i would do is to run such a syscall translated chroot and then replace the armhf toolchain with a cross x86 one that skips any syscall translation [17:28] mterry: ogra_: bzoltan: sorry guys i need to run, GF is already looking angry at me :D [17:28] zbenjamin, thanks! [17:29] i will read teh backlog... man always the interesting conversations when i need to go D [17:29] zbenjamin: do you have GF? Duuude, where is your comitment? :D [17:29] bzoltan: lol , what was it with wife and kids you just mentioned? [17:29] mterry, that way your compiler runs at host speed ... and in cases where tools inside the chroot needs to exec some native binaries they can use the syscall translation and run natively [17:29] zbenjamin: that is after life [17:30] ogra_, you're saying the x86 one inside the chroot would be able to skip the qemu translation being done on it? qemu is fancy enough to not translate native code? [17:30] mterry, right [17:30] neat [17:31] qemu-user-static uses the binfmt kernel module ... that checks the binary header of a file you try to exec [17:31] if it is foreign it wraps it [17:31] if not, it doesnt :) [17:31] OK. So sounds like the way forward is provide chroot directories to the SDK. They'll have toolchains inside. Which we may swap in and out for native toolchain as speed demands dictate. We'll have one chroot per arch [17:32] sounds good [17:32] That's not very controversial [17:32] mterry: ogra_: sounds smart enough to me [17:32] And in future, maybe we can skip stuffing the toolchain inside, if we get future --sysroot smarts [17:32] and at some point in the future we'll have --sysroot support [17:32] *snap* [17:32] :) [17:33] (...py) [17:33] * bzoltan lays back and opens the can [17:33] you havent yet ? tsk :) [17:33] bzoltan, and that's basically the same as current SDK uses, right? [17:33] This solution needs root though, which is a bummer. I'd like that --sysroot option too [17:33] https://www.youtube.com/watch?v=3YmMNpbFjp0 [17:33] mterry, fakechroot :) [17:34] * ogra_ hides in a hole [17:34] :D :D [17:34] mterry: we already get used to the chroot what requires root access... it is not great but keeping it is not a regression :) [17:35] ogra_, what is wrong with fakechroot? I know I rarely use it for anything. Does it just have holes in it's wall? [17:35] it is pretty similar to deb2snap :) [17:35] Yeah, it uses preload too [17:35] ogra_, but deb2snap is so great! Surely fakechroot is too [17:35] mangling all sorts of vars and preloading [17:36] haha [17:36] well, it is surely not worse [17:36] Hrm... [17:36] we should try it our [17:36] *out [17:36] Someone hasn't learned to love deb2snap yet [17:36] i must admit i havent used it yet :) [17:37] ogra_, eh, it's what you'd expect [17:37] * ogra_ is still stuck with nodejs mostly :) [17:37] mterry: so what is the catch with that fakecroot? [17:37] and my shiny projects are usually shell ... foor that i just copy a busybox static binary in place ;) [17:37] (and snap that up) [17:38] bzoltan, you can run it as non-root [17:38] ogra_, tsk tsk :) [17:38] bzoltan: our chroots do only need root for creation not for ru [17:38] ning tools [17:38] zbenjamin: I know that :) [17:38] Crappy.phone keyboard :) [17:39] yeah, somone needs to port ubuntu to the neo900 [17:39] zbenjamin, why not during a run? I would think you'd still need to enter the chroot there? [17:39] zbenjamin: I am sure your GF is delighted that you irc on the phone [17:39] haha [17:39] mterry: thr [17:39] zbenjamin, yeah get out of here and kiss your SO [17:40] Schroot does not need root for running [17:40] mterry: it is true... we do not need root to enter the schroot [17:40] bla [17:40] bzoltan: she loves it [17:41] bzoltan, zbenjamin: Huh. setuid then? [17:41] I guess so [17:43] But schroot is not optimal .. We should do better even with a chroot solution [17:50] mterry: I do not know how schroot does it [17:51] zbenjamin, bzoltan: oh interesting -- what's wrong with schroot? Seems like rest of Ubuntu uses it happily [17:53] mterry: there are lots of problems 1) bootstraping it takes ages and it is a fragile process 2) it leaks mount points... after some time you mind find 10k mount points, because schroot re-mounts sessions what are dead 3) it does require root to set up 4) it is still an extra step between the rootfs and the IDE, so it is slow [17:53] mterry: worst thing, it leaks mounts. And then restores them all on boot. We once had a dev with 16k mountpoints [17:53] mterry: keep in mind that we hand this weapon over to entry level app developers... schroot is safe as long hard core gurus are using it [17:54] zbenjamin: dude if you jusr write redundant stuff then you are really better off kissimg your GF :) [17:55] mterry: every call to schroot.adds 3 seconds when executin something in it. Now imagine Qtc querys the qmake tool 5 times per kit when starting. also cmake. that adds.up.to a loong startup time [17:56] bzoltan: pff just because you.type faster on your fancy keyboard [17:57] zbenjamin, bzoltan: so alright. this is all solved by future --sysroot, I guess. Is there some other chroot tool I should be using today until --sysroot happens? [17:57] zbenjamin: I can switch over to the nexus10 just out of solidarity :D But my wife is out of the house... so no reason [17:58] bzoltan: lol [17:58] mterry: I think if you can be happy for the time being with schroot then we are on the safe track ... the VM idea we should forget as soon as possible :) that was a scary one [17:59] bzoltan: as long as it works on Windows and Mac, the solutions are fine [18:00] sergiusens: well sysroots.do work [18:00] sergiusens: yeps... sysroot with toolchains do work [18:00] sergiusens: it is the chroot what pulled off he cable from the portable SDK [18:00] bzoltan: redundancy dude.. Redundancy [18:01] jdstrand: btw https://code.launchpad.net/~sergiusens/snappy/frameworkPath/+merge/260202 [18:01] zbenjamin: LOL [18:01] sergiusens: yeah, I saw that-- haven't had a chance to look at it yet [18:03] jdstrand: ack, just in case [18:05] sergiusens: The VM idea for the above listed reasons is a big no for the app developer... even on OSX and Windows. Application development without IDE is not an application developer. And IDE without code completions, syntax highligh, context sensitive help, source code debuging and quick builds is not an IDE. [18:06] sergiusens: the VM might be good for some sort of system emulation and for packaging non compiled non UI projects.. for everything else VM does not deliver [18:15] * tedg back [18:16] I don't understand how a chroot works on Win/Mac? [18:16] Seems like you need at least Linux kernel headers… I guess… but you'd really want more than that. [18:28] tedg: on Win it does not, at all.. on Mac it might, but still doubt that it is trivil [18:29] So it seems on those we'd need VMs, eh? [18:30] tedg: VM is a dead end .. if no chroot then the only viable solution is the sysroot + toolchain [18:31] Ah, I see. So you're basically rolling the chroot into the toolchain. [18:31] tedg: few years back I have asked if we want a portable SDK. I have asked if we want to port the app development story to OSX and Windows. I was told on many occasions that I better focus on providing an Ubuntu based SDK and if sombeody from teh community picks up the porting stuff then fine [18:32] Seems there's no way to get tests to work in that case though. Even one same arch. Without a VM. [18:32] tedg: I am not sure if I understand what you mean [18:33] I mean if you build the test, and it links against let's say a kernel syscall, you couldn't run that test on Win32. [18:33] Even if they were both x86 [18:34] tedg: when we talk about running a test we talk about runtime use case... tests are runtime actions. For that case we need the emulator or a VM. [18:34] sergiusens: regarding your comment about reviewing code, mvo expressed an interest there :) [18:34] tedg: but that is runtime and not editing-compiling-linking-packaging [18:34] Don't you really want it to be edit-compile-link-test-package ? [18:35] beowulf: but mvo also mentioned he wasn't an expert in js, that is what I think we should have [18:35] For things like unit tests, not integration. [18:35] tedg: no.. testing for me means running. Different game [18:35] tedg: true... unit tests brings an extra hook [18:36] bzoltan, So if I'm correct, your issue with VMs is that you can't get a directory of source code out for highlighting, etc? [18:36] tedg: in the present app sdk we do not have much story for unit tests... functional tests we support, but that is runtime [18:37] bzoltan, Why couldn't we mount a dir from the host system into the VM, and use the edits in the host system? [18:37] tedg: that is just one ... code completion, source code debugging, context sensitive help... al features what makes the IDE [18:38] tedg: it still would not give us access to the qmake in the VM [18:38] bzoltan, Sure, but if we told the IDE that your sysroot is /foo/bar and to compile you run "in-vm-gcc" as long as that gcc could mount the VM, mount /foo/bar and run on it. Everyone is happy. [18:38] bzoltan, I'm pretty sure I can do that :-) [18:39] i.e. you run vm-qmake and perhaps that ssh's into the VM and runs qmake [18:39] tedg: and you give me 10 developers for 2 years and tell the Qt upstream that we have just forked their IDE :) [18:39] It seems all you need is access to the files and the STDIN/STDOUT of the tools. [18:40] How they do their job isn't important to the IDE. [18:40] tedg: horrible slow ... already the chroot puts 3-5 seconds on each call.. with ssh and VM it would be horror [18:40] So if I can make it fast, you'd be happy with a solution like that though? [18:41] tedg: VMs just simple do not blend with IDEs [18:41] I think we could do tricks with things like LXC to make it not be performance bound. [18:41] tedg: but why? [18:41] Well, we can use the toolchain we have for one :-) [18:42] tedg: I doubt you can even fix any VM to get even close to direct access... [18:42] But once we abstract at that level the archiecture thing starts to fade away. [18:42] Basically all the path/directory issues get handled by the container. [18:42] tedg: the real solutiuon is what we talked with mterry, ogra_ and zbenjamin [18:44] That solution seems pretty limited, and requires some base changes in how our toolchain works. Which seems expensive. [18:45] * mterry is happy with whatever you folks come up with [18:45] * mterry is tired of thinking about toolkits [18:45] Perhaps I'm not seeing it, but a relocatable toolchain really only solves the problem of building it. But not really the rest. [18:47] tedg: I give you a scenario... I am an app developer.. I have set up my project so it targets three devices ... one phone, one desktop and one legacy tablet. Very generic scenario ... One app different targets. I am editing my code .. each target have different API set... I push tab-tab to complete a property of a component... the IDE looks up for the name .. I change teh target and hit tab-tab ... I canhe again tab-tab ... do you want to pre-deploy 3 VMs and [18:47] access 3 times ssh/mounting for that simple property? Or do you start the VM when I hit tab-tab? It just does not work that way with IDEs... you want direct access to the rootfs and to the toolchain [18:47] I only want to start the VM when you hit compile. [18:47] So instead of gcc it calls lxc-gcc [18:48] So all the code is available directly to the IDE in the host system. [18:48] tedg: compliation is just one thing.. code completion, syntax highligt, context sensitive help, etc [18:48] Sure, and you get all of those with local access to the files. [18:48] tedg: developers want code completion before compilation [18:49] tedg: how do you mount the rootfs + the qmake from a non deployed VM? [18:50] I don't do it that way. The VM deployment mounts from the host fs. [18:50] The VM's image is tiny, only enough to mount to host's exported filesystem. [18:50] So all the files remain in the host. [18:50] tedg: so you expect the SDK to deploy _all_ VMs what the developer configures as target? [18:51] Only for building one target, one VM right? [18:51] tedg: and even if you mount the file system... how do you expose the qmake? [18:51] Correct [18:51] Expose qmake via SSH [18:51] mterry: tedg: no .. building is the last step [18:51] So you call lxc-qmake [18:51] tedg: not possible to expose the qmake via ssh [18:51] It will start the container if need be, and give you STDIN/STDOUT of Qmake in the container. [18:52] tedg: start the container on-demand? Wait for how long for a code completion? [18:52] We're expecting to run containers for all of your legacy apps as well. They'll have to be fast :-) [18:53] No, code completion is done on the host file system. [18:53] You don't need the container to access the files. [18:53] tedg: but I need to run the container at least once to export the files [18:53] I'm not sure what you mean by "export the files" [18:54] tedg: Simple... the IDE needs instant, random access to the files of the target and native access to the toolchain. that is it... [18:54] I think that getting the files from the host FS to the container FS will just be a bind mount. [18:55] tedg: we can not and will not fork the IDE :) to work around our VM hacks... Qt upstream already rolling eyeballs seeing our chroot hacks [18:55] tedg: opposite direction... the IDE on the host needs to access the target file system [18:56] I guess I'm not asking you to. I'm saying that we have a compiler kit that is "lxc-gcc" and "lxc-g++" and those would be used in that case. [18:56] The IDE doesn't need to know more about containers. [18:56] tedg: I need to cross-check it... qmake is one level higher than gcc/g++ [18:57] I imagine we won't need to fork to make qmake configurable :-) [18:57] tedg: and as I said, the IDE needs editing time access to the files and qmake not only build time [18:58] tedg: well... that is exactly what we made with Meego :) back in stone age [18:58] bzoltan, how does eclipse solve that problem? [18:59] tvoss: does it? [18:59] bzoltan, I used to use it in a somewhat vanilla configuration quite some time ago for embedded development and it works surprisingly well [18:59] tvoss: does eclipse support Qt development with VMs? [19:00] tvoss: with local sysroot and toolchains yes .. that is not a challange for QtCreator either [19:00] bzoltan, that's not the question, the question is: cross-compiling/building for a target that is not the host [19:00] tvoss: the questio is where the IDE pulls the toolchain and the target fs [19:00] bzoltan, sure, just saying that eclipse has way more exposure to embedded dev environments than qt creator has ... likely to have some good ideas over there [19:00] tvoss: if it is local then it is a nobrainer [19:01] bzoltan, sure, the interesting questiion is non-local (vm/container/device) [19:01] whatever [19:01] bzoltan: tedg: how is android solving this? [19:01] tvoss: no, it is not eclipse vs. qtc issue ... it is local sysroot versus container issue [19:01] rsalveti: with sysroot [19:01] I think a native cross compiler + sysroot would generally fix the issue [19:01] right [19:01] as long we ship both, we're good [19:01] rsalveti: precisely [19:01] we don't need anything else [19:01] I don't want to rely on our toolchain at all [19:01] So then Android apps don't have unit tests? [19:02] we could consume it from the archive, but I prefer having a pre-built (static only) toolchain that we ship as part of the sdk or something [19:02] tvoss: rsalveti: both eclipse and qtc were primarily made for dealing with toolchains and sysroots [19:02] same thing as android does [19:02] rsalveti, I don't think it would harm to investigate into other ides [19:02] in the sense of gaining inspiration [19:02] anything besides sysroot (vms/chroot and so on) is just super complicated when you need to support multiple os [19:03] tvoss: just do not make the impression that :) this issue is about a shortcoming of the QtC :) code completion and syntaxt highlight, API help are the same in all IDEs [19:03] bzoltan: right, and why are we not doing that *today* already with touch? [19:03] I don't get this idea of relying on backporting a toolchain and such [19:03] rsalveti: I am asking that question since day zero :) [19:03] well, and what is the answer that you got? :-) [19:04] there must be at least one [19:04] bzoltan, I don't say that it's a shortcoming, just saying we should look around :) [19:04] rsalveti: we went thru this with ogra_ and zbenjamin with mterry. We need to check the possibility of enabling --sysroot in out toolchain and make it relocatable. [19:04] bzoltan: right, my question is why can't we do that? [19:05] you said above that nobody wanted to maintain 2 toolchains in the archive [19:05] tvoss: Of course... what android sdk does better is that they use sysroot [19:05] what I'm saying is that we don't necessarily even need to maintain this *in* the archive [19:05] tvoss: the overall solution across IDEs is providing a native cross toolchain + sysroot [19:05] rsalveti: my hope is that this 2nd toolchain would be the same as the 1st... with some extra build options [19:06] bzoltan: right, and that is something that your team could maintain, right? [19:06] simple, create a snap for your toolchain ;-) [19:06] rsalveti: sysroot support and relocation of the toolchain are what we need [19:06] rsalveti: Yeps, i will investigate this option [19:06] bzoltan: right, then just fix it :-) [19:07] rsalveti: duuuude [19:07] on our side all we need to do is offer a way for a sysroot to be used/consumed [19:07] the plugin can do whatever [19:08] rsalveti: my agenda here is to raise the red flag that the VM path is just a big hack and would cost us more then doing the right thing (tm) [19:08] We're gonna need some sort of container solution if we expect to be able to install debs as dependencies anyway. (i.e. for nodejs core) [19:08] Even if that ends up in a sysroot [19:08] right [19:09] tedg: +1 for that .. chrootable sysroots are fine [19:09] bzoltan: it's fine, from the snapcraft perspective we could support sysroot, chroot and also vms [19:09] rsalveti: \o/ [19:09] guess that's the basic requirement you were looking for [19:10] rsalveti: do we need apt there? I thought we will have devpacks for that [19:10] I think that, at least for the short term, we'll need apt to get started. [19:11] rsalveti: tedg: what we agreed with the boys before was that as an intermediate solution we keep the schroot alive and do not cause feature regression. As next step we investigate the toolchain issues and possible fix it and offer a sysroot + toolchain solution. For testing and runtime and all other non compiled non UI projects VM is just fine. [19:11] yeah, I see we moving from apt -> devpacks [19:11] rsalveti: +1 for the android way but keepn in mind we need.a binary compatible tc to.our.libs [19:11] since then we could have people providing devpacks as well [19:11] like upstream and so on [19:12] zbenjamin: sure [19:15] * bzoltan EOD ... 10pm here [19:15] i wonder if its possible to let apt run against a sysroot/rootfs... something like apt-get --rootfs /somepath install somepackage [19:16] bzoltan: you are going to sleep? Now i need to wonder about your commitment :D [19:16] that's kind of tricky indeed [19:16] what we did on linaro was always creating the sysroot from scratch [19:16] either building it (when using OE), or debootstraping it [19:16] since you generally don't want to update the sysroot so often [19:16] tedg: rsalveti: mterry: zbenjamin: ogra_: Thanks for this discussion.. let's itterate it :) I really enjoy that we move this ship forward [19:16] I think for apt you probably want to just use LXC there. Then it thinks that the sysroot is / [19:17] bzoltan: +1 [19:17] It is also "root" and can change all the files it wants [19:17] * tedg watches out for icebergs [19:17] ;-) [19:17] tedg: right, that's a way, more to the chroot direction [19:17] tedg: yeah sure, i just know that other package managers _can_ do that :D [19:17] for sysroots I think devpacks will be very handy [19:18] rsalveti: +1 , devpacks sounds perfect for sysroots [19:18] rsalveti, Do you imaging a devpack is a just an app snap with known paths that we mount into the sysroot? [19:19] rsalveti: and since we are doing the tool from scratch it should be able to use a sysroot... snapcraft --sysroot ~/device1/sysroot install-devpack touch-sdk [19:19] i.e. /usr/include/foo for the foo devpack [19:19] tedg: yeah, something like that [19:20] we could have a small sysroot + a huge qt devpack for example (painful but to show how it could work) [19:20] for personal it's easier since Qt is part of the core image [19:20] so we could have a big sysroot including our supported libs by default [19:20] *supported sdk libs [19:21] So then my package definition would be something like: devpacks: [a, b, c], npm: [d, e, f], go: [x, y, x] [19:21] And that builds up the dependencies for the environment. [19:21] right [19:22] the main advantage, I see, for devpacks is that we don't necessarily need to consume the dependencies from the archive [19:22] of course we could have devpacks produced out of the archive, initially [19:22] Perhaps someone could sell devpacks in the store ;-) [19:23] maybe :-) [19:24] rsalveti: tedg: regarding snaps. Will it be possible for us as the SDK ship devtools for the device (gdbserver , custom app launcher) as a snap so we can pull it from the store for a device? [19:24] rsalveti: tedg: so we do not need to include them in the core image but can painless install them when required [19:24] right, don't see why not [19:24] zbenjamin, I think we need something like that, not sure exactly how that'll work quite yet. [19:25] That way you could have stable/devel builds for the device as well. [19:25] Using the example of needed gcc 5.2 for wily vs. gcc 4.9 for vivid [19:25] on the "other" Qt device they pull packages from the archives when developer mode is enabled [19:27] Hmm, that's crazy :-) [19:27] :D [19:27] Clearly they need more sunlight. [19:28] better than the current situation on our phone, where a developer manually needs to make the rootfs writeable to install gdbserver ;) [19:28] I think we really want you to be able to build for a specific device without having that device available. For things like buildservers for instance. [19:28] yes definately [19:29] right, either something we make available via the store [19:29] or something that the sdk could sideload [19:29] rsalveti: sideloading from the SDK is problematic as soon as we have binaries [19:29] zbenjamin: why? [19:29] rsalveti: gdbserver needs to have the right architecture for the device. [19:30] zbenjamin, https://code.launchpad.net/~ted/ubuntu-seeds/gdbserver/+merge/257872 [19:30] right, but we could simply sideload the right snap [19:30] where do we get the correct package from is the better question [19:30] tedg: wow finally :D [19:31] zbenjamin: right, depends on who is the owner of it :-) we could always consume what is available in the archive [19:31] rsalveti: we can hardly ship all possible snaps for the sdk, if we can provide a central place to download them thats fine too [19:31] and produce something snappy compatible [19:31] right, we can have something :-) [19:33] rsalveti: so you worked for linaro? Then you know exactly how to build a binary compatible static toolchain for our devices right? :D [19:34] its horrible to find informations about that topic [19:34] zbenjamin: I was managing the team doing that [19:34] but in a smaller scope [19:35] we used sysroots for 2 things [19:35] one was using open embedded to create the sysroot and the cross compiler in order to allow people building stuff for arm64 (when we didn't have any major os supporting it) [19:36] and the other was to help bootstraping new architectures for debian/ubuntu, mostly using multi-arch [19:36] getting the toolchain right is the painful thing [19:37] which is why linaro was producing native toolchains based on their own gcc versions [19:37] i figured, we considered shippig a custom static tc for the click SDK but i was not sure i could make a compatible toolchain and could not find enough informations about how to test that [19:37] to help people that were relying on codesourcery [19:38] yeah, it's a bit painful, but I think we had that documented somwhere [19:47] rsalveti: i hope you still have that somewhere :D [20:10] zbenjamin: I think I do, need to dig a bit :-) [20:10] sergiusens: hey, need help, trying out queryPackageNames branch and http://webdm.local:4200/api/v2/packages/?q=xkcd isn't returning what i'd expect, is the url i'm using correct? [20:10] rsalveti: \o/ thats good [20:34] beowulf: it would exactly the same thing a "snappy search" from the cli would return [20:43] beowulf: which would be similar to curl -H "X-Ubuntu-Release: rolling-core" https://search.apps.ubuntu.com/api/v1/search?q=xkcd |python -m json.tool [20:44] sergiusens: then i'm doing something wrong as http://webdm.local:4200/api/v2/packages/?q=xkcd returns 22 results, as does a search for 'foo' [20:44] beowulf: are you running the version compiled from the MP? [20:44] sergiusens: yes [20:45] sergiusens: 0.6 [20:45] current is 0.6.1 [20:45] (as in, trunk is 0.6.1, i don't mean /apps/webdm/current is 0.6.1) [20:45] beowulf: remember, just in case, that it case to be in $GOPATH/src/launchpad.net/webdm and not some random branch [20:46] sorry for typos, ssh on a slow connection [20:46] sergiusens: *facepalm* [20:48] beowulf: I thought it would be somethin like that [20:54] mterry, I added a section "Snap from scratch" at the bottom of the doc. Thoughts? [20:55] mterry, I think it distills a bit of what we've discussed today, with the idea of targets providing their own toolchain. [20:55] * mterry looks [20:59] I imagine we could build the first dev packs with chroot and deb2snap or something like that. [20:59] And then in time we can make them smarter. [21:06] sergiusens: +1, \o/ [21:17] tedg, you and your snappy-native thinking :) [21:17] tedg, I'm still in desktop land [21:17] Ha, I've been upset at dpkg for a long time :-) [21:18] The reason I like that idea is that it basically inverts the cross-build problem. [21:18] And, I think, then makes it simpler. [21:18] tedg, so this is vearing toward an idea I was thinking about. There is a lot of similarity between devpacks and our language plugins [21:19] Hmm, that is true [21:19] tedg, I don't think I'm ready to say one eclipses the other. But maybe they could [21:20] tedg, and more importantly, that would solve the problem of what to call the languages: key ;) [21:20] Perhaps they build on each other. It seems like you'll want a base target and then others. [21:20] For instance, the base would have to provide the C compiler for Python modules. [21:20] tedg, you're thinking about the gcc toolchain? [21:21] tedg, I guess. But then you'd just say "devpacks: python3, c++" (except in yaml) [21:21] It's already a list field [21:21] Or maybe the python3 devpack would dep on the c++ one? (on the expectation that compiled modules are common) [21:22] Yeah, that could provide more power to dev packs that are perhaps, non-standard in that they'd run other things. [21:22] tedg, but for example, the codespell app doesn't need c++ [21:22] It would just need a python3 plugin (or devpack or whatever we call it) [21:23] Hmm, it would still need to know the target platform though. [21:23] i.e. whether python3 was installed there [21:24] tedg, maybe. I think it's reasonable for python3 to just always install python3 into the snap [21:24] tedg, but you're talking about the version of Core that is targeted, right? [21:24] Yeah, I can see that. Let deduplication solve the problem. [21:24] Yes [21:24] tedg, (also, some apps may want a specific version of python3) [21:25] tedg, version of Core is a whole separate problem though I think. early versions of snappy.yaml spec had (have?) a target: key [21:25] where you specify core or whatever [21:25] That seems reasonable to define in yaml vs build command [21:25] slash devpack [21:26] Hmm, I was thinking that you'd have multiple though, eh? [21:26] tedg, multiple what? [21:26] Are there fat snap packages? [21:26] core-armhf, core-arm64, core-i386 [21:26] tedg, target multiple versions of coree? [21:26] tedg, ah. well that's not quite version. that's arch [21:26] core-mips ;-) [21:26] we're talking two different things I realize [21:26] I was talking 15.10 vs 16.04 [21:27] tedg, ok, regarding arch... let me reread :) [21:27] Sure, I think they're related though. The version is a number and an architecture, no? === devil is now known as Guest2761 === Guest2761 is now known as devil_ [21:28] tedg, well I was approaching from version may mean whether python3 is available in Core or not. Whereas that's not an issue across arches [21:28] so sure... python3 devpack is available for multiple arches. no problem, right? [21:29] tedg, ^ [21:29] I think so yes. But I think it has to then depend on a "core devpack" that is arch specific. [21:29] For example, to build modules. [21:30] tedg, well in each buildenv, the devpack would the right version of python in [21:30] *put [21:30] The right version of python, but what about the modules it downloads and installs? [21:31] Seems also the Python devpack would then need to have a version of python to put into the environment for every target arch. [21:31] you could have a base devpack for the interpreter [21:31] tedg, it's the intermediary for that... so it can make sure [21:31] and just use pypi for the other stuff [21:31] tedg, sure [21:31] rsalveti, right -- I assume pip is getting every dep we care about [21:32] I mean, there are also compiled deps [21:32] Yeah, so if I'm on x86 building an arm package. The python devpack would have a compiled armhf version of the interpreter that'd show up in the install directory. [21:32] right [21:33] you could as well have fat devpacks (multi-arch) [21:33] and just install for the target snap arch you're creating [21:33] Not sure if the python devpack needs to be target arch based. i.e. python-armhf-devpack, because we probably don't want to have everything in one. [21:33] Does fat here mean multiple host arch or multiple target arch? [21:34] depending on the case it could mean btoh [21:34] since it could provide the cross compiler + target libs [21:35] I think we're gonna have to support some minority architectures, so it might not be useful to be really fat. [21:35] right [21:35] We might need to kinda have a mixed situation there. [21:35] yeah, it's just a convenience [21:36] When snappy queries the store it'll look for the host architecture. [21:36] So besides building, we can assume that's correct. [21:36] right [21:36] So it seems that a devpack needs the target architecture in the name? [21:36] Or is there some way we can be more clever there? [21:37] http://bazaar.launchpad.net/~lool/snappy/snappy.devpacks-definition/view/head:/devpacks.md [21:37] what is a bit complicated is that this interferes directly with the snap naming [21:37] same for apps, frameworks and such [21:37] that same problem of starting an 'app' that is armhf only [21:37] Ah, I didn't realize there was docs already for this. [21:37] and then wanting to do the x86 version for it as well [21:38] would you rename that? would you go with fat by default? [21:38] fat might be too fat though [21:38] when querying the store is a bit easier [21:39] since you give the arch [21:39] Oh, that doc describes a lot of how snapcraft should work... [21:39] tedg: yeah, check the links part of that doc [21:40] more how it could work at this stage [21:43] So it might be that we can just support querying the devpack yaml directly to align all the architectures. [21:43] tedg, we've been able to avoid dealing with a lot of what devpacks might entail by avoiding compiled languages in our first pass [21:44] yeah [21:44] solving the case for pure python/nodejs apps is already a big win [21:45] Not sure that the dependencies don't kill us there before we get to the win. Seems like a lot of modules have a compiled part underneath [21:46] tedg, medium win then :) [21:46] that could as well be true, but could still be a second step thing [21:46] Heh [21:46] Can we please bikeshed the amount of win it would be? [21:46] :-) [21:46] tedg, there's a lot of bootstrapping to get to even one win :) [21:46] I'll take a baby win from the ground up [21:46] tedg, let me open a google doc for that [21:48] mterry, So could a "language plugin" really be a devpack that provided a "snapcraft" command? [21:48] tedg, I'm not sure they are perfectly overlappig [21:48] tedg, it was just a quick thought [21:49] tedg, it might be safer to continue with language plugins for now [21:49] I think they overlap from the lifcycle perspective, which is really nice. [22:06] how do you switch from 15.04 stable to rolling/alpha?