=== chihchun_afk is now known as chihchun === JanC_ is now known as JanC === cpaelzer_ is now known as cpaelzer [07:01] Chipaca: good morning [07:01] zyga: morning [07:03] Chipaca: I'm seeing this error all the time: [07:03] src/github.com/snapcore/snapd/vendor/golang.org/x/crypto/ssh/test/test_unix_test.go:25:2: cannot find package "golang.org/x/crypto/ssh/testdata" in any of: [07:03] does it ring any bells? anything changed in how we vendor stuff? [07:04] zyga: i have never seen it before [07:04] zyga: I've seen this once in a fedora build where we missed to copy testdata into place [07:05] but actually that was for one of our packages not the ssh one [07:05] Chipaca: ok, separate topic, I asked mvo how we might go ahead with storing more data into sequence but he didn't reply; I was thinking about storing a 2nd sequence like object [07:05] hey morphis :) [07:05] zyga: hey! [07:05] Chipaca: so that we can get away without patching anything in the state [07:05] Chipaca: so sequence-ex could store more stuff (flags, path, side info) [07:05] Chipaca: wdyt? [07:06] zyga: type SuperInfo struct { *SideInfo } [07:06] zyga: do you have time for another review on https://github.com/snapcore/snapd/pull/3222 ? [07:06] PR snapd#3222: many: fix test cases to work with different DistroLibExecDir [07:06] morphis: yes, I'll focus on reviews today (again) [07:06] morphis: I was off yesterday (despite being here a little) [07:07] Chipaca: super info I like [07:07] zyga: and make Sequence be []SuperInfo (or []*SuperInfo) [07:07] Chipaca: hmm [07:07] Chipaca: though [07:07] maybe not sure, we risk clashing fields [07:07] Chipaca: why not just sequence-ex with dedicated fields? [07:07] (we can clean up when patches are live) [07:08] zyga: because then you have to keep to slices in sync? [07:08] yes [07:08] zyga: and decide how to handle drift [07:08] I planned to already [07:08] hmm [07:09] in case of rollbacks? [07:09] zyga: thanks [07:09] well, this is already equally hard, there is no required data currently, we'd have to add the data only to the new sequence [07:09] and this is only for snap try, it is still broken today in master (from this pov) so it'd not be a regression [07:09] zyga: i mean, what's the advantage of having a second sequence instead of adding to the existing? [07:10] zyga: you're seeing something there that I am not [07:10] Chipaca: the advantage is that the second sequence would not embed a go type so it cannot clash on field names there; [07:10] zyga: we could even add SnapPath to SideInfo [07:12] ogra_: ping? [07:12] Chipaca: hmm hmm [07:12] Chipaca: the problem is that we may need more than just snap path [07:12] Chipaca: what I think we need to capture in this particular case is the fact that it was on encrypted directory [07:12] Chipaca: more so than the fact than the actual path (which I would capture anyway) [07:13] zyga: it being on an encrypted directory is something that should be detected at runtime, fwiw [07:14] Chipaca: no, because it may not even be mounted :/ [07:14] ok, wait [07:14] if it's not mounted, why do you need to know if it's encrypted? [07:14] Chipaca: have a look at the discussion here https://github.com/snapcore/snapd/pull/2837#discussion_r114855843 [07:14] PR snapd#2837: interfaces/apparmor: allow reading from ecryptfs [07:14] Chipaca: because it was encrypted when the user is logged in [07:14] Chipaca: then she logs out [07:14] Chipaca: and snapd restarts [07:14] Chipaca: and needs to write the apparmor policy [07:15] Chipaca: :/ [07:15] zyga: brb [07:15] no worries, I wanted to let you know about this to get you thinking [07:15] I'll try to fix it today but I want to agree on direction [07:16] * zyga -> breakfast [07:20] zyga: if the directory of a snap try goes away, the snap is removed [07:28] zyga: several rough edges here though [07:29] zyga: one is: if you snap install foo, and then snap try ./foo, you end up with a foo from the store and a foo from try. We don't handle this case well today at all (the whole snap is in "try mode") [07:29] zyga: and the question is, if the directory goes away in this case, what should happen? should it fall back to the non-try one? [07:30] i'd suggest we solve this by not allowing mixing try and non-try, at least for now [07:31] (any mixing of try and non-try can be made explicit with flags, later, where we don't mix them in state but copy config over or whatever) [07:31] this avoids all the uncomfortable corner cases of mixing the two modes [07:32] and makes the current approach of having 'trymode' on the snap (instead of the sequence) fine [07:32] and gives us an easy approach to getting the snap path, whichi is to add it next to trymode [07:32] the 'patch' code can look in changes for the snapsetup, or at the mount unit [07:33] (or we could look at the mount unit in runtime, but that feels 15.04ish :-) ) [07:40] zyga: i'm creating a forum topic about this [07:50] zyga: https://forum.snapcraft.io/t/disallow-mixing-try-and-non-try-snap-installs/484 [08:02] Chipaca, yes ? [08:02] ogra_: two questions for you sah [08:02] ogra_: we nuke locale support from core, yes? [08:02] or have we added it back recently [08:03] ogra_: (good morning! how're you doing today?) [08:03] Chipaca, nope, actually locale support was nuked everywhere now (ubuntu-base, cloud images etc) [08:04] ogra_: ok. 2nd question, what is the locale-control interface for, do you know? [08:05] i guess nothing ... but since we never shipped locales thats fine i guess [08:05] (thank you i'm fine :) ) [08:05] ogra_: i mean, we even have a snap to test it works [08:05] not on core [08:05] Chipaca: thank you! [08:06] we never did to my knowledge [08:06] so /etc/default/locale is from "outside", on classic? [08:06] thats a zyga question, not sure if snap-confine mounts it [08:07] zyga: ^ :-) [08:08] ogra_: by the way, it looks like core is currently shipping the locales for e2fsprogs [08:08] ogra_: i assume that's a bug [08:09] hmm, i wonder how they get there [08:09] yes [08:10] Chipaca: replied on the forum [08:10] seems fwupdate too [08:10] Chipaca: mmm, /etc/default/locale is from classic [08:10] zyga: ok, good [08:10] ogra_: and is /etc/default/locale writable in core? [08:10] Chipaca: but also (this is an ogra topic :) on core we do some writable path magic I'm not fully versed in [08:10] * ogra_ hopes /usr/share/locale too [08:11] Chipaca, i dont think so ... it should be hardcoded to C.UTF-8 [08:11] * ogra_ checks [08:11] ogra@pi3:~$ mount |grep locale [08:11] ogra@pi3:~$ cat /etc/default/locale [08:11] LANG="C.UTF-8" [08:11] yep [08:11] if the idea of locale-control is to not work at all on core, we should probably check for that [08:11] ogra_: and non-writable? [08:11] see the mount :) [08:11] (yes, non-writable) [08:12] ah :-) [08:12] ok [08:12] doesnt mean that wont change in case we get langpack snapsd or some such indeed [08:12] *snaps [08:13] I'm pretty sure locale-control needs fixing to work on !debianish [08:13] morphis ^ [08:14] i'm pretty sure they dont have /etc/default on non-debian [08:15] Chipaca: I think the whole /etc design is broken today [08:15] Chipaca: we want to entirely stop sharing /etc [08:15] Chipaca: and share (or mirror) selective elements [08:15] all these locale questions are because of https://askubuntu.com/q/910514/711 btw [08:17] Chipaca: locale-control happy isn't "broken" on any !debian system today as we don't have confinement anywhere .. [08:17] morphis: it's broken in that it doesn't do what's intended [08:17] i.e. it doesn't change the default locale of the system [08:17] Chipaca: doesn't it just allow access to /etc/default/locale via AppArmor? [08:18] morphis: yes [08:18] exactly [08:18] that is not a thing in !debian, afaik [08:18] sure [08:18] but we don't have any AppARmor on anything else than Ubuntu today [08:18] ah, now i gotcha [08:18] ok [08:18] if we get AppArmor anywhere else I think we can easily add the right paths to the AppARmor snippet [08:19] you can ship translations inside your snap and use them (not sure if the content interface would work here but i guess with the correct env set in your wrapper it should) [08:20] there are env variables for this [08:37] Hello, is there a work around available for the problems with apt-get update on an arm board when running in classic snap on Ubuntu core? [08:39] Chipaca: so quick question about try, is that the current design that if the directory goes away try mode snap should uninstall itself? [08:39] AdamH_: hey, do you have some more details about the issue? [08:40] zyga: This is the bug I am experiencing https://bugs.launchpad.net/snappy/+bug/1670475 [08:40] Bug #1670475: apt-secure complaints with classic snap on arm [08:42] ogra_: ^ I had a look at the bug but I don't know anything about it [08:42] ogra_: do you know what the root cause is? [08:42] ogra_: ports vs non-ports archive? [08:43] zyga, something with gnupg, i'm still not sure what ... has nothing to do with any archive servers, thats a local issue with the classic snap [08:43] ogra_: any idea why it is specific to arm? [08:44] ogra_: could it be some version of gpg that we've seeded into the images due to snappy? [08:44] ogra_: gnupg 1 vs 2 [08:45] zyga, that would be odd, i rather thing we remove to much when wrapping up the tarball for core [08:45] *think [08:47] AdamH_: so there you go, I don't know anything more [08:47] * zyga reviews https://github.com/snapcore/snapd/pull/3222/files [08:47] PR snapd#3222: many: fix test cases to work with different DistroLibExecDir [08:48] zyga: Thank you [08:49] AdamH_, the "workaround" is described in the bug though ... (turn off gpg checking) [08:50] * ogra_ glares at the first line of https://forum.snapcraft.io/t/failed-to-flush-stdout-permission-denied/485 and tries to wrap his head around that command [08:50] wondering if he made a typo there or if there is actually a snap that calls apt install [08:50] zyga: I thought it was, but it isn't [08:51] zyga: (the current behaviour for snapd to remove a try that vanished) [08:51] it goes into broken state [08:52] zyga: Yes I tried sudo apt-get --allow-unauthenticated upgrade but still get the same error? Is there a better way to disable the check? [08:53] AdamH_, upgrade ? you mean update, right ? [08:53] ogra: Yes sorry I mean update [08:53] heh, k :) [08:54] likewise you should be fine just typing Y on the question about unauthenticated packages [08:59] yes kyrofa ! I'm looking at this: https://github.com/nextcloud/nextcloud-snap/issues/60 and was trying to compile the nextcloud-snap for my rbpi3, after hitting the out of RAM issue when compiling mysql I found out your post (https://kyrofa.com/posts/building-your-snap-on-device-there-s-a-better-way) but I the 3 builds I tried all failed: [08:59] https://launchpadlibrarian.net/318427985/buildlog_snap_ubuntu_xenial_armhf_nextcloud-snap-pachulo_BUILDING.txt.gz [08:59] ogra: I don't get a prompt when trying to update, only when I am installing a package [08:59] AdamH_, yeah, thats what i mean [09:00] it should install afterwards [09:00] ogra: Yes it does [09:14] pachulo: I think it's about 2am for kyrofa; try in a few more hours [09:17] zyga: where does the decision to include or not include (say) unity7 in the core snap come from? [09:19] jdstrand: when you're around, i'd like to look at the issue of _filedir [09:23] Chipaca: in snap/implicit.go [09:24] aha, no wonder i couldn't find it in interfaces :-) [09:24] zyga: I think locale-control needs to move to implicitClassicSlots [09:24] thanks Chipaca ! [09:24] zyga: ogra_: right? [09:25] PR snapd#3274 opened: cmd/snap,tests/main: add force-devmode switch instead of spread system blacklisting [09:25] Chipaca: I think some commercial products use that slot but if you guys say it does nothing on core we need to re-visit thi [09:26] Chipaca: if you move it somebody should query the store to see if there are any snaps which use that interface already [09:29] morphis: zyga: I can confirm that if a snap does use that interface, attempting to actually modify the locale fails [09:29] in core [09:31] Chipaca: yeah, however we need to ensure we don't break existing snaps which are installed on a core system today and use that interface [09:32] morphis: yep [09:32] Chipaca: if snapd handles that already nicely, then I am fine [09:32] sadly answering that question is harder than it should [09:38] cannot create lock directory /run/snapd/lock: Permission denied [09:38] pstolowski: ^ halp? [09:38] Chipaca, yeah, i guess locale-control being classic only is fine [09:39] pstolowski: suspect version mismash but i don't know of what :-) [09:40] zyga: you got "cannot find package "golang.org/x/crypto/ssh/testdata"" fixed? [09:40] seeing that on travis now [09:41] Chipaca, where do you see that? [09:41] pstolowski: when trying to run any app [09:41] pstolowski: but i might have snap-confine from master and snapd from elsewhere [09:41] uh [09:42] so it's probably me (but please advise) [09:42] morphis: no [09:42] morphis: not sure what's going on really [09:43] Chipaca, how about checking the versions? dpkg -l|grep snap [09:43] Chipaca: snap-confine misalinged with profile [09:43] Chipaca: seems exactly the same what morphis ran into yesterday [09:43] pstolowski: i'm not running any of the things from the package at this point [09:43] Chipaca: are you playing around with development versions of snap-conifne? [09:43] snap-confine is from yesterday's master [09:44] snapd is snapd/2.24+git605.g2eda802.dirty [09:44] Chipaca: ok [09:44] Chipaca: how did you install it? [09:44] snap-confine? make hack [09:44] Chipaca: in general you need make hack + several bind mounts :/ [09:44] Chipaca: one for the binary into core (or disable reexec) [09:44] Chipaca: one for the apparmor profile into core [09:44] snapd, built it by hand and run-snapd-srv [09:45] ok, i'm glad zyga jumped in, because i'd never have guessed :} [09:45] ¯\_(ツ)_/¯ [09:45] Chipaca: the issue is that snap runs snap-confine from core and then the apparmor profile is in /etc/apparmor.d/snap.core.$number.usr.lib.snapd.snap-confine.real [09:45] Chipaca: and then that thing must match the one that was made by make hack [09:45] zyga: snap only runs snap-confine from core if it reexecs though [09:45] Chipaca: except for the paths (revision in core0 [09:45] so maybe my snap is from package [09:45] Chipaca: maybe bug? check if you have snap [09:45] that might explain that [09:45] Chipaca: (built from source) [09:46] Chipaca: anyway, a dumb solution is just to bind mount the apparmor profile over that in core and restart snapd [09:48] zyga: pstolowski: updated snap (manually) and snap-confine (make hack) fixed it [09:48] no bind mounts needed [09:48] morphis: https://github.com/snapcore/snapd/pull/3222#pullrequestreview-36469425 [09:48] PR snapd#3222: many: fix test cases to work with different DistroLibExecDir [09:48] morphis: reading rest [09:49] Chipaca: yep, this means that snap is honring reexec correctly! [09:52] PR snapd#3275 opened: snap: move locale-control to only be present on classic [10:12] * zyga finds code review in gitk better/faster than on github [10:12] :/ [10:15] trying to debug that apt issue i get the feeling it is related to how we handle /tmp [10:26] ogra_: oh? [10:26] ogra_: how do we handle /tmp on classic? [10:26] it's just the regular tmp, right? [10:26] (regular in snap execution environment) [10:26] we dont do anything at all [10:27] what i see in the logs is http://paste.ubuntu.com/24516298/ [10:27] see how it tiresd to use /var/snap/classic/common/classic/tmp/fileutl.message.wopbff [10:28] wow [10:28] what an insane profile! [10:28] ah wait [10:28] name="/var/snap/classic/common/classic/tmp/fileutl.message.wopbff" [10:28] so /tmp is special! [10:28] it is --devmode [10:28] it's a persistent /tmp in $SNAP_COMMON [10:29] (classic)ogra@pi3:~$ touch /tmp/foo [10:29] (classic)ogra@pi3:~$ [10:29] (in one terminal) [10:29] ogra@pi3:~$ ls -l /var/snap/classic/common/classic/tmp/ [10:29] total 0 [10:29] -rw-r--r-- 1 ogra ogra 0 May 5 10:28 foo [10:29] (in the other) [10:29] so it actually writes there [10:29] permissions and all seem fine too [10:30] yet, apt prints exactly that line every time it spills the gpg error [10:31] ogra_: that? [10:31] ? [10:32] ogra_: "that line", which line is that? [10:32] the one from the paste above [10:35] ogra_: what is ="/var/snap/classic/common/classic/tmp/fileutl.message.wopbff" [10:35] is that a socket? [10:35] mknod being the thing that is not normally allowed [10:35] maybe we can now allow it and use seccomp filtering [10:35] well, --devmode ... so it is aloowed [10:37] sigh .... i wish that log output would be somehow readable [10:37] ogra_: yeah [10:37] * zyga would like to understand what that profile name means [10:37] jjohansen: ^ if you can help me out [10:37] these lines are way to long to even grasp them [10:37] to quote so that you don't have to hunt pastebins: [10:37] May 5 10:12:22 pi3 kernel: [ 7741.891667] audit: type=1400 audit(1493979142.467:148626): apparmor="ALLOWED" operation="mknod" [10:37] profile="snap.classic.classic//null-/usr/bin/systemd-run//null-/usr/sbin/chroot//null-/var/snap/classic/common/classic/bin/dash//null-/var/snap/classic/common/classic/usr/bin/script//null-/var/snap/classic/common/classic/bin/bash//null-/var/snap/classic/common/classic/usr/bin/sudo//null-/var/snap/classic/common/classic/bin/bash//null-/var/snap/classic/common/classic/usr/bin/sudo//null-/var/snap/classic/common [10:38] /classic/usr/bin/apt-get" name="/var/snap/classic/common/classic/tmp/fileutl.message.wopbff" pid=12088 comm="apt-get" requested_mask="c" denied_mask="c" fsuid=0 ouid=0 [10:38] well, due to the ALLOWED it should technically work [10:40] yes [10:40] it does work [10:40] whatever happens it is something else that's broekn [10:40] broken* [10:41] well, nothing in classic changed when it started to happen [10:41] it must be releated to the surrounding env [10:42] i see that /etc/apt/trusted.gpg is intact and contains the right keys ... i can also list them with apt-key list ... [10:43] gnupg (1) is also installed fine and working otherwise === chrisccoulson_ is now known as chrisccoulson [10:44] (i can also add/remove keys with apt-key just fine) [10:46] * zyga has a crazy idea [10:46] to have a snapd that lives in an execution environment by itself [10:46] so snapd would run in a snapd namespace [10:46] that is consistent across distros more than the helper snapd outside [10:48] morphis: one more question on that PR [10:48] morphis: at the bottom [10:48] PR snapd#3276 opened: tests: additional setup in docker test for core systems [10:50] Chipaca: hey [10:50] Chipaca: back to that sequence topic [10:51] Chipaca: do you think this is something worth tackling today or better next week when gustavo can voice his opinion? [10:52] zyga: well, there's a bunch of work to be done. Is there any you can tackle without his input? [10:52] zyga: e.g. the code to grab SnapPath from existing state, for the migration [10:53] zyga: the code to determine whether a directory is part of an ecrypt-encrypted volume [10:53] not volume. Tree? path? w/e [10:54] zyga: i'd start there, and that's probably fiddly enough to take you over to monday [10:54] Chipaca: that is actually easy (a simple extension of what I wrote yesterday) but I think it is stuck on a design decision: of how snaps behave when a volume is not mounted [10:54] zyga: currently, they go to 'broken' state [10:55] Chipaca: yeah but what do we _want_ to happen? [10:56] zyga: yeah, that's design -> involves gustavo [10:56] yeah [10:56] let's do it next week [10:56] ok next branch [10:56] (worse, it's design i don't have an opinion on myself :-) ) [10:57] haha [10:58] zyga, are you working on a fix for https://forum.snapcraft.io/t/hook-timeout-mechanism-not-working-in-2-24/464/4 ? [10:59] pstolowski: no, you were [11:00] Chipaca, I intended to, but then zyga commented in the topic saying he would make it happen ;) [11:00] pstolowski: (i mean, that's what you said yesterday in the standup? or did i misunderstand?) [11:00] ahh [11:00] see, ordering is important :-p [11:00] ok [11:00] pstolowski: maybe his intention was to make it happen by bossing you around [11:01] :D [11:03] pstolowski: not at present, I'm doing code reviews again [11:04] hahaha [11:04] pstolowski: I think I commented on that earlier but perhaps I'm mistaken :) [11:04] pstolowski: feel free to take it [11:07] zyga: With unity 8 going away, does adding online-accounts interface make sense? [11:07] zyga: should we instead nuke all the unity8 support interfaces? [11:07] as unity8-in-a-snap is prooobably not going to happen [11:08] Chipaca: I'd not nuke anything, people are free to hack on those [11:08] zyga, ok [11:09] zyga: but if nothing is using them (and nothing is afaik) then they bitrot [11:09] Chipaca, yeah, we talked about this some time ago on the standup.. unity8 may be developed by community, there is some interest in continuing the project [11:11] fair enough [11:11] brb (rebooting) [11:14] * zyga thinks that bitroot is an interesting problem [11:38] zyga: its a learning/complain mode profile. The name starts with the profile that caused the exec chain [11:38] snap.classic.classic [11:38] and then each //null- is a new layer of exec, so what ever was confined by snap.classic.classic did an exec of [11:38] /usr/bin/systemd-run which called [11:38] /usr/sbin/chroot which called [11:38] /var/snap/classic/common/classic/bin/dash which called [11:38] /var/snap/classic/common/classic/usr/bin/script which called [11:38] /var/snap/classic/common/classic/bin/bash which called [11:38] /var/snap/classic/common/classic/usr/bin/sudo which called [11:38] /var/snap/classic/common/classic/bin/bash which called [11:38] /var/snap/classic/common/classic/usr/bin/sudo which called [11:38] /var/snap/classic/common/classic/usr/bin/apt-get [11:38] that final descendant caused the log message [11:39] zyga: What is the status on OpenGL with Intel drivers? Should it work? I'm having trouble getting Qt an OpenGL context. [11:39] that is quite a call chain, under older versions of apparmor you would have only see null-XXX where XXX is a unique number instead of the actual exec names [11:40] which was far less useful but did result in smaller names [11:45] jjohansen: thank you for the description! [11:46] dragly: there's no magic in snapd for opengl on intel, we don't bring any drivers form any other place, whatever the snap ships will run (or not) [11:46] dragly: all openGL situation nees love [11:47] GL surely works on clasic with intel for me [11:48] ogra_: sure but my point is that it depends on hardware *and* snap [11:48] ogra_: snapd should use opengl indirection helpers and load the right driver [11:48] well, i only ever try witrh the krita snap [11:48] ogra_: but nobody implemented that yet [11:48] ogra_: and snaps should not ship drivers [11:48] ogra_: (except for driver snaps) [11:48] ogra_: but now none of that is true [11:48] i know that makes havy use of GL ... so if you can install and run that one there is *some GL* [11:49] (used to work also on my nvidia desktop) [11:49] ogra_: as I said, there is whatever the snap author bundles [11:49] ogra_: snapd doesn't help [11:49] yeah [11:50] yay ... ok, one issue found with the gpg stuff [11:51] (we dont bundle /var/lib/apt/keyrings in the tarball ...) [11:51] zyga: Then I don't understand how it works to build on Travis (in a Docker container) and my own machines and run on Nvidia machines afterwards, but not Intel. [11:51] but even putting the right thing there doesnt fix apt :/ [11:51] dragly: we have some special handling for nvidia [11:51] dragly: but it's neither generic nor particularly good [11:52] Oh, that explains. Thanks! [11:52] Are there any examples that I could look at (Krita?) that solve this manually either in the wrapper or by shipping GL drivers? [11:52] dragly: no idea really, I can tell you what we do [11:52] I don't mind doing it manually, I just don't know what to look for :) [11:53] dragly: but ideally someone should use libglvnd all the way [11:53] dragly: a snap that is linked so that it doens't link to particular things but to libglvnd [11:53] dragly: and contains all the drivers (as a hack / demo) [11:53] dragly: and can find the right one using libglvnd [11:53] dragly: later on the drivers can be removed from the demo snap and moved elsewhere (TBD) [11:53] dragly: and if connected via content should still work [11:54] dragly: (as in, give you the right driver while the snap with the application code has none) [11:55] so libglvnd is a wrapper for whichever GL lib is installed? [11:56] dragly: libglvnd is an emerging way to do GL in a sane way on linux [11:56] dragly: everything else is terrible [11:56] dragly: (and deployed) ;-) [11:56] dragly: but there are many parts missing [11:57] dragly: improvements in snapd for the automatic driver installation, kernel parts, mouting the driver for the snap to see, working with libglvd to do the right thing, etc [11:57] I see [11:57] dragly: also build system improvements [11:58] dragly: so when people build whatever snap they don't have to hand-hold it to do the right thing (not ship drivers but ship glvnd) [11:58] we really want to use snaps to package our applications, but it sounds like its going to be a lot of work at the moment. Maybe its better to consider alternatives and wait for an "official" solution? [11:58]