[01:25] PR snapcraft#1340 closed: state: save all the build packages as global [01:28] PR snapcraft#1343 closed: go plugin: Cross compile with CGo [02:10] PR snapcraft#1358 opened: release changelog for 2.31 [02:12] Is it possible to install snap on Arch Linux without pacman? [02:12] Such as through a .tar.gz file? === chihchun_afk is now known as chihchun [06:04] mvo: would be awesome if you can look at https://github.com/snapcore/snapd/pull/3365 again this morning and if you're ok, merge it now that niemeyer approved it [06:04] PR snapd#3365: tests,packaging: add package build support for Fedora for our spread setup [06:04] CI is already green [07:07] good morning [07:19] PR snapd#3443 opened: Unity7 interface grows gtk2/3 settings and user's specific ini [07:27] fgimenez: 2.26.4 is in -proposed now everywhere, could you please kick of the SRU verification? [07:29] mvo: sure! on it, the automated tests were actually triggered a few days ago but there was a problem in the logic that determines the release branch to execute the tests from https://travis-ci.org/snapcore/spread-cron/builds/238479940#L300 i'll fix that and retrigger [07:31] fgimenez: thank you [07:32] mvo: np i'll begin too with the changelog review, will keep you posted [07:36] morphis_: do you happen to know if the "proxy" helper is installed on fedora/suse by default? [07:38] mvo: btw i see that the deb was detected in -proposed about 6 days ago, is that correct? has it been updated since then? (just to make sure that we are detecting changes in proposed correctly) [07:41] fgimenez: updates arrived yesterday [07:41] fgimenez: 6 days looks not quite right [07:42] mvo: ok thx i'll take a look [07:42] fgimenez: https://launchpad.net/ubuntu/+source/snapd has the exact times, ~21h ago [07:43] flexiondotorg: cholcombe and I have already done some poking around on https://github.com/rust-lang-nursery/rustup.rs/issues/1144 [07:47] mvo: ok thanks, in spread-cron we are currently watching http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html (interestingly there's no snapd in that page now, and it seems that it was there 6 days ago..), i'll update it to look into https://launchpad.net/ubuntu/+source/snapd [07:55] mvo: proxy helper? [07:55] morphis_: its a binary from libproxy called "proxy" [07:55] ah, let me check [07:55] morphis_: is that there by any chance? [07:56] not installed by default [07:56] but libproxy-bin provides it [07:56] let me try [07:56] morphis_: if its not there by default, thats all I need to know. that is on fedora or suse? [07:56] mvo: fedora 25 [07:56] mvo: let me try suse [07:57] morphis_: thank you [07:57] mvo: if you're adding a dependency can you add it for suse and fedora in packging too? [07:58] morphis_: I will explore it a bit, maybe dlopen(libproxy) is fine [07:59] mvo: is that for adding proxy support system wide? [07:59] mvo: libproxy-tools it is on suse and not installed by default [08:01] interestingly proxy itself doesn't look at the usual env vars [08:01] morphis_: yeah [08:01] Chipaca: oh? that sounds very wrong [08:01] ¯\_(ツ)_/¯ a matter of looking at the env and falling back to libproxy i guess? [08:02] morphis_: I talked to Son_Goku yesterday and he mentioned that fedora is not setting the systemwide proxy via environment but instead uses network-manager and the best option to talk to it seems to be libproxy [08:02] Chipaca: indeed [08:02] mvo: yeah libproxy looks really good for this [08:02] Chipaca: it will also mean we need to (potentially) create a new http.Client{} for each request as the proxy is per-url in this model [08:03] http.Transport [08:03] but yes [08:03] morphis_: the cheapest option is the exec.Command("proxy", url).Output() but adding a dependencies feels slightly ugly [08:03] Chipaca: yeah, sorry, a transport into the client of course [08:03] it's terrible that after decades of broad deployments proxy support in linux is not in libc [08:04] and everyone has to muck around with their special little snowflake [08:04] mvo: implementing our own libproxy isn't much better, isn't it? [08:04] (contrasting other platforms that do this right) [08:04] zyga: yeah, bionic has this AFAIK [08:05] morphis_: of course not :) sorry, what I mean is adding a hard dependency, I was curious to see if we could just dlopen(libproxy) and just ignore if its not there [08:05] mvo: however, how would applications get the proxy configuration? [08:05] mvo: ah [08:05] why not [08:07] morphis_: looking again it is probably ok nowdays, it used to be heavy on further dependencies but it seems like all of those are now pulled in via plugns. to support "pac" proxy configuration a JS interpreter is needed iirc [08:07] mvo: the biggest problem i see with all this is the "here's a list of proxies, try them in order" idea [08:08] Chipaca: hm, we could simply not support that (initially at least)? [08:08] mvo: also note proxy can return something that looks like socks://.... [08:08] lel [08:08] Chipaca, morphis_: thinking about it some more I guess we need to call an external helper or we pull in cgo again into the deamon [08:08] Chipaca: yeah [08:09] * mvo scratches head [08:12] this sounds like *fun*! [08:17] mvo, zyga: I would like to get https://github.com/snapcore/snapd/pull/3365 merged now [08:17] PR snapd#3365: tests,packaging: add package build support for Fedora for our spread setup [08:19] * zyga looks [08:22] mvo: one more fun thing about using libproxy [08:23] it brings lin libstdc++ [08:23] hmm [08:23] ogra@sabrelite:~$ snap info core |grep refreshed [08:23] refreshed: 2017-06-08 04:24:21 +0000 UTC [08:23] ogra@bbb:~$ snap info core|grep refreshed [08:23] refreshed: 2017-06-07 04:23:44 +0000 UTC [08:24] why is that different ? isnt the refreshed value coming from the server ? [08:24] no, that's refreshed as in the device refreshed [08:24] I think [08:24] that doesnt match uptime [08:25] 4:20 UTC is actually the time the cron job runs for armhf [08:25] so that should be generation time [08:26] oh ... but the bbb didnt refresh ... it is the generation time of the currently installed snap [08:28] morphis_: merged [08:28] Chipaca: yeah, ugly but we also have that I think [08:28] zyga: you're my hero! [08:29] PR snapd#3365 closed: tests,packaging: add package build support for Fedora for our spread setup [08:29] mvo: have that where? [08:29] zyga: lets wait until the suse one finishes then we can merge that one too [08:31] Chipaca: in the core snap [08:31] Chipaca: libstdc++ I mean [08:31] mvo: ah! i meant into the snapd binary itself [08:31] mvo: ok to land the --base branch? [08:32] Chipaca: yeah, I think we don't want this directly, probably via helper [08:33] zyga: which one? 3439? or the older 3317 ? [08:33] 3439 [08:34] zyga: it needs reviews first, no? I will do the first review next [08:34] icey Excellent! [08:34] mvo: yes, I meant that I wanted you to review it :) [08:34] zyga: aha, yes, will do [08:34] why does build.snapcraft.io needs access to organizations? [08:34] iliv, to put webhooks into the tree [08:34] iliv: for hooks [08:35] (well, endpoints( [08:35] well.. [08:35] my account has access to a number of organizations that have nothing to do with my repos that I'd like to use with build.snapcraft.io [08:36] iliv: I see, I think it would make sense to allow partial access to just the things that are needed [08:37] I kinda trust you people but most of those organizations will not be happy seeing build.snapcraft.io having access to their accounts [08:38] it is only needed to put the hook in place once though ... [08:38] theyx could turn it off afterwards (i know that doesnt help much in most cases) [08:40] so, what exactly will be done to those organization accounts? [08:40] you say a webook will be created.. but in which repo (if there are more than one)? [08:41] * ogra doesnt know the tech details, perhaps cjwatson can help [08:49] iliv: the github access page lets you control this on a per-org basis; you don't have to grant access to all your orgs. build.snapcraft.io will only install a webhook in repositories that you specifically enable there. [08:49] mvo: zyga Chipaca can we land snapd#3348? it has two +1 and is almost green (just an autopkgtest failure due to the reexec flakiness), it will fix two nightly suites and unblock the spread-cron branches for executing the core-revert test automatically [08:49] PR snapd#3348: tests: add core revert test [08:51] PR snapd#3348 closed: tests: add core revert test [08:51] and in fact the org access is only needed so that BSI can list org-owned repos through the github API; there's a separate permission for installing webhooks [08:51] mvo: thanks! :) [08:51] cjwatson, I can't control it, though. All I see is a green check mark against organization name and it is not interactive. [08:51] (we ask for admin:repo_hook read:org, in the github scopes language) [08:52] iliv: you can control it on https://github.com/settings IIRC (I would check more specifically but my internet connection is being unusually terrible today) [08:53] fgimenez: looking [08:54] fgimenez: that PR had my upublished review [08:54] darn github [08:54] iliv: oh, I think if it's just a green tick then that's because the org owner has allowed it [08:54] iliv: we don't get to control any of this stuff, it's all up to github [08:55] fgimenez: can you please look at the PR again? [08:55] https://help.github.com/articles/about-oauth-app-access-restrictions/ has some more details [08:57] zyga: sure, thanks! i'll prepare a follow addressing your comments [08:57] fgimenez: thank you, I'm sorry I didn't realize I never submitted this review [08:58] the snap-confine-from-core and the other re-exec tests keep failing randomly [08:58] I don't remember any branch that didn't fail on this in the past week [08:59] I'll try to figure out WTF is going on there :/ [08:59] PR snapd#3370 closed: many: derive implicit slots from interface meta-data [08:59] thank you zyga, i think sergio is also lookign into it [09:00] anyway, as far as I can tell we ask for the absolute bare minimum that we need to install webhooks and see any org-owned repos at all [09:02] does anyone know of a vim extension that would add a tab to an exsiting vim whenever vim is invoked from command line but another process exists somewhere? [09:03] cjwatson, yes, it seems to work like you said. kinda weird... [09:06] mvo, zyga: approval and a merge of https://github.com/snapcore/snapd/pull/3406 would be nice :-) [09:06] PR snapd#3406: tests,packaging: add package build support for openSUSE [09:10] morphis_: looking [09:10] PR snapd#3444 opened: interfaces: compose the base declaration from interfaces [09:10] morphis_, mvo: can you please look at 3444 [09:11] zyga: sure [09:11] morphis_: can you merge master into 3406? [09:11] zyga: I did this 45min ago [09:12] morphis_: merging it now would shorten the diff [09:12] sure [09:12] morphis_: correct me if I'm wrong but I still see the fedora changes there [09:12] I merged before the merge of the fedora branch [09:13] isn't travis doing an automatic merge with master when it runs? [09:13] zyga: but merge now [09:13] morphis_: it's optional [09:14] morphis_: and we don't do it [09:16] zyga: shouldn't travis fail when a merge with master breaks any tests? [09:18] zyga: base snap branch looks good [09:18] morphis_: well if the run is old you don't know because master has moved [09:19] pedronis: right but at the point where travis starts, shouldn't it start testing from a combination of master + branch? otherwise test results will always be a lie if master has evolved [09:20] it does afaik [09:20] ok [09:20] but we might still break master because the check is not done on the merging [09:23] mvo: thanks! [09:23] Chipaca: can you do a 2nd review so that mvo can iterate on base snaps? [09:24] morphis_: github has a setting where you can enable a check that disables merging if master has moved [09:24] morphis_: bue we're not using it [09:24] morphis_: as it requires a rebase-based workflow [09:24] * zyga hugs rebase [09:24] zyga: reviewed #3444 [09:24] zyga: I see [09:26] zyga: on which pr? [09:26] Chipaca: on 3439 [09:29] PR snapd#3439 closed: cmd/snap-confine: add support for --base snap [09:29] morphis_: reviewed [09:33] PR snapd#3445 opened: tests: add linode-sru backend [09:34] morphis_: updated [09:34] zyga: ok [09:45] morphis_: sorry for being so picky about those comments but I think it is essential [09:45] morphis_: in a few weeks nobody will remember why fedora or suse is not included in a given test [09:47] zyga: commented on all and fixed most [09:47] zyga: np, but basically we have all test cases disabled right now [09:47] that is why we switched from a blacklist to a whitelist approach [09:47] morphis_: thanks, ping me and I'll re-read it [09:48] a followup PR will clean this up and enable most tests [09:48] zyga: ping :-) [09:48] aha :D [09:48] reading then [09:49] thanks [09:53] zyga: approved #3444, thanks for doing the cahnges! [09:53] morphis_: thank you! [10:03] morphis_: re-reviewed [10:03] morphis_: fix one typo and +1 [10:07] morphis_: some heavy conflicts on 3411 [10:10] zyga, hmm [10:11] "On core systems we don't pivot_root and that [10:11] has to change before base snaps are fully operational" [10:11] zyga, can we make sure to get proper ram usage statistics before implementing such a thing (or do i mis-read that ... you want to exec every snap on a core install in pivot_root ?) [10:13] PR snapd#3446 opened: errtracker: Include /etc/apparmor.d/usr.lib.snap-confine md5sum in err reports [10:16] ogra: yes, this is required for base snaps [10:16] ogra: core and classic will behave (finally) exactly the same [10:16] zyga, well, why do we need base snaps on core installs ... i though they are a lib layer on top of core [10:17] ogra: no, they are not [10:18] ogra: base snaps are needed so that we can run snaps that use base snaps (which will soon be most snaps, I think) [10:18] zyga, in any case we are supporting embedded boards and i suspect pivot_root means there will be more copies of libs and env in ram that we should really measure before [10:18] ogra: base snaps, roughly, will just use snaps other than core as the root filesystem [10:18] right, so whats the use case on a core install for that ? [10:18] ogra: note that we will have less things in the core snap this way [10:18] ogra: *every* snap will use a base snap [10:19] ogra: have you been following the base snap discussions on the forum/ [10:19] not closely the last days [10:19] ogra: core will become a snap with just snapd in it [10:19] but that sounds really awkward for low end [10:19] ogra: and what is currently in core will become ubuntu-16 base snap [10:19] ok [10:19] ogra: it is required for 16-18 migration [10:19] that is what i grokked before [10:20] ogra: and to allow running snaps with different bases on one system [10:20] but that you plan to pivot_root each and every snap is new to me [10:20] ogra: how would those work otherwise? [10:20] or that you want to support different base snaps on some 256M 200MHz IoT board installs [10:20] ogra: yes, we don't want to have 2nd class devices that don't support an essential feature [10:20] by simply merging the rootfs on boot [10:21] what do you mean by merging? [10:21] and keeping the snaps as they are without duplicating the world in ram [10:21] why do you think we'd be duplicating something in ram? [10:21] PR snapd#3447 opened: debian: unify built_using between the 14.04 and 16.04 packaging branch [10:21] merge core and base on booot ... execute natively [10:21] like we do today ... just with an additional split of the snapd bits into another snap [10:22] fwiw, libraries are mapped into ram based on inode number, if that stays the same nothing will be duplicated (someone needs to double check that) [10:22] if you want to go that route that really should see some measurements [10:22] ogra: I'm open to suggestions but IMO there is no other route [10:22] (double check that the inode number stays the same, the mmap mechanism is well understood) [10:22] * zyga checks [10:22] mvo, thats what i'm saying ... not saying it is wrong or anything but on that low end we shouldnt operate without ameasurments and data [10:23] *measurements [10:23] i'm sure pivot_root will have overheard, the question is if that is bytes or megabytes [10:23] *overhead [10:23] inode number is the same [10:24] ogra: why would pivot_root have overhead? [10:24] because it wraps around something, it is an additional layer [10:24] I don't think that's true [10:24] it's a chroot on steroids [10:24] yes [10:25] (kind of) [10:25] ogra: the only overhead is that as core and bases diverge we will see extra copy of libc used by the few services in the core snap, for everything else they will operate exactly the same as today [10:27] zyga, well, still, it would be good to measure the differences between with and without pivot_root regarding system resources ... we're in embedded territory here ... [10:27] ogra: that I agree with [10:27] ogra: it'd be good to come up with a benchmark we can repeat [10:28] well, just measure rem and cpu usage for the same app snap in both cases [10:28] *ram [10:28] ogra: specific snap (revision), specific board, specific script [10:28] sure [10:28] so that we can just run this via spread and stay aware [10:29] well, not needed to do it regulary ..… thats simply stuff you should research before coming up with such an implementation ... once you know if there is a difference and how big it is thats enough [10:29] it wont change over time so doesnt need to be a regular test [10:33] ppisati_, do you have a sabrelite ? [10:36] ppisati_, i'm seeing weird console distrotion when i boot with hdmi and console=tty0 with the current linux-generic from xenial [10:36] ogra: note that we came up with pivot_root last year, this is nothing new [10:37] ogra: given what we need to do there's no other way to implement base snaps [10:37] ogra: I suspect we actually use less memory now as before persistent mount namespaces *each process* would independently pivot_root [10:38] zyga, well, we get along without base snaps today ... and merging core and base on boot with keeping the native execution would work too ... core specific indeed [10:38] zyga: fixe [10:38] ogra: no it would not work because the whole idea of base snaps is to have more than one [10:39] zyga, note that i'm not saying anything is wrong though ... but we should have numbers before jumping ahead with implementation [10:39] ogra: so you cannot use your current rootfs as the rootfs for snaps [10:39] zyga: either we way we likely need a root filesystem, there still the question what really goes into core vs base [10:39] pedronis: I agree, I think this will clear up as we come closer to the sprint [10:40] zyga, we do that today ... and why would you want to support multiple base snaps on super-low-end devices ? [10:40] i always understood that this is not wanted for ubuntu core [10:40] ogra: we cannot do that once we have more than one base snap [10:40] ogra: and we want to have more than one base snap so that we can transition to 18 series [10:40] sure we can ... we can restrict what can be installed on a core installation [10:40] ogra: and to allow other distributions to equally participate in the snap distribution [10:41] ogra: if we restrict we will have to have a flag day for 18 transition and we don't want that [10:42] ogra: nope, i don't have one [10:42] ogra: what is that? [10:42] imagine you have something like a beaglebone, your ram is limited, your cpu is limited ... do you really want to allow people that install three snaps to have three oses running on that setup because the three snaps use three different bases ? [10:42] zyga: notice also that because of the configure hook the core snap will need a base snap too [10:42] ogra: yes, if that's what those people want to run I don't want to stand in their way [10:42] ppisati_, they renamed it to nitrogen6 ... its an IMX board [10:42] (at least the configure hook is a likely reason to need one) [10:42] pedronis: configure hook for which snap? [10:42] for teh core snap [10:42] the core snap has a hook [10:43] nowadays [10:43] pedronis: right, I was just checking [10:43] zyga, well, then we can simply give up on IoT boards will just come to hang and crawl with that [10:43] pedronis: yes, it is likely that the core snap will be special cased to be its own base snap [10:43] ogra: I doubt that [10:43] zyga: well that start to be weird [10:44] I fear duplication then [10:44] duplication of? [10:44] stuff that needs to be in the core snap and the base snap [10:44] like python or something [10:45] pedronis: the good thing that base snaps will allow us to do is to almost freely shape the core snap [10:45] not really [10:45] pedronis: as it will no longer affect how anything runs [10:45] if the core snap isn't really minimal [10:45] zyga, well, if i install the mysql snap from that fedora guy, along with the webserver snap from this arch guy on top of my ubuntu core board, i end up with three OSes on disk and in ram [10:45] pedronis: the initial ubuntu-16 will fork from the current core (snaps snapd) [10:45] zyga, thats definitely not feasible [10:45] ogra: yes but this is what you _wanted_ [10:45] ogra: if you don't want to do that, don't do that [10:45] zyga, how did i know ? [10:45] ogra: or get the stack from one vendor [10:46] ogra: snap info will tell you that [10:46] zyga, you are saying that a user of our SW should know the source in and out ? [10:46] ogra: anyway, I think this discussion is moot now [10:46] and why would i look at snap info [10:46] ogra: same could be said for base-16 and base-18 [10:46] all i wanted was the webserver to act as frontend to a mysql db ... [10:47] on my $mini-board [10:47] so i just picked the packages ... done [10:47] and then notice that the whole thing eats my disk and ram [10:47] zyga: anyway there's a problem that likely some core snap config should be part of a base snap configure, but that concept doesn't make sense [10:48] pedronis: can you give me an example? [10:48] we could surely make that an optional switch to allow additional base snaps ..… but thats definitely nothing we should have OOTB [10:48] pedronis: I suspect that base snaps won't have much configuration at first [10:48] where would sshd live? [10:48] pedronis: in the base snap [10:48] see [10:48] er [10:48] core snap [10:48] unless we start to allow apps and services on base [10:49] see [10:49] which is something we never discussed [10:49] base snaps won't have any configuration intially because they won't run any services [10:49] where else would system services live ? [10:49] in the core snap it seems [10:49] err [10:49] if we choose to move them to a blessed base snap [10:49] but I think we also had a different vision [10:49] we could do that [10:49] where the core snap was mostly just snapd [10:49] and its deps [10:50] <_28Kb> services must be snaps i guess [10:50] ogra: core will become a snap with just snapd in it [10:50] pedronis: yes, but that is the road ahead, for now we can start with just a copy of the current core as core and base and then trim them as we go [10:50] so the system services must be in the base snap [10:50] ogra: _will become_ :) [10:50] (in that setup at least) [10:50] well or we have a services snap [10:50] ogra, mvo could we move forward https://github.com/snapcore/core-build/pull/13 ? [10:50] PR core-build#13: Androidboot support [10:51] I think it is all doable but we need to discuss one more thing at the sprint: if we are going to get rid of bootable bits from the core snap [10:51] well [10:51] we cant get rid of them ... we can probably move them around though [10:51] it those go to a new helper snap (ubuntu-core maybe) then this will resolve other questions [10:51] that's what I meant [10:51] then on classic systems you'd need base snaps only [10:52] not even core or ubuntu-core [10:52] on systems that re-exec you'd pull in core and base snaps [10:52] and on core systems we'd pull in everything [10:52] * ogra isnt so worried about the resulting rootfs and how we assemble it, thats all easily solvable [10:52] but the "multiple base snaps allowed" is really bothering on core [10:53] ogra: do you have a plan for 18 transition? [10:53] thats fine if you have endless ressources like on a PC [10:53] zyga, allow exactly one base snap [10:53] ogra: that doesn't involve running both 16 and 18 snaps at the same time? [10:53] or allow two ... still better than "any" [10:53] ogra: that's one way to make sure nobody can transition until *everything* has a 18 snap as well [10:53] ogra: if we allow two we might as well allow any [10:54] ogra: until we measure and show this is terrible (doubtful) I think this is not a thing worth discussing now [10:54] what i really dont like is that you end up with the fedora, arch, suse base snaps being pulled in silently when installing some app snap [10:54] (on core that is) [10:54]