[02:07] Hey everyone, if there's anyone interested in trying out snappy on Fedora, it'd be great if you could test out snapd [02:07] more details here: https://forum.snapcraft.io/t/call-for-testing-snapd-2-23-on-fedora/127/6 [03:47] does anybody know if there has been any other progress on this snap download connection reset bug? https://bugs.launchpad.net/software-center-agent/+bug/1617765 [03:47] Bug #1617765: Connections reset when downloading snaps from CDN [03:49] my experience is similar to the comments from others on that bug, it seems to affect snapcraft being run inside containers in particular [03:50] have tried both building inside lxd and docker containers and the core snap fails to download the vast majority of the time [05:04] mwhudson: ping === morphis_ is now known as morphis [05:04] PR snapd#3151 opened: Small adjustments for debian deb builds and packaging symlink for Debian Sid [05:15] PR snapd#3151 closed: Small adjustments for debian deb builds and packaging symlink for Debian Sid === chihchun_afk is now known as chihchun [06:43] o/ [06:44] mvo: I will take a small detour to check something that gustavo requested (build system tweaks) [06:44] mvo: meanwhile can you do a review on updates to https://github.com/snapcore/snapd/pull/3145 [06:44] PR snapd#3145: overlord/ifacestate: fix auto-connect during core snap transition [06:44] mvo: and let's plan how to change the plug names on core [06:44] mvo: (i.e. how to test that) [06:45] mvo: so that we don't cauese any regressions with this move [06:45] mvo: after the build system detour I will look at ensuring plug/slot name uniqueness validation [06:45] mvo: we have code for that but apparently it's not used or working [06:45] mvo: (I suspect the way implicit slots are added side-steps that) [06:46] PR snapd#3152 opened: store: make hash error message more accurate [06:46] zyga: ok [06:57] mcphail, we will figure out security in wayland [06:57] we need to do that anyway to get snaps to be amazing on a GNOME desktop [06:58] but we will keep Mir for IoT projects where security and simplicity are the main requirement [06:58] about wayland, I think we could use a test on fedora 25 + wayland to see how the interface shoud look like, I think it will be relatively straightforward [06:59] and try the same on ubuntu with apparmor to see what's to do with confinement [06:59] morphis was recently working on better X11 auth support for snaps [06:59] and that feels in line with similar work for wayland [07:05] mvo: ok, experiment done [07:06] now looking at plut/slot uniqueness again [07:06] thanks zyga [07:07] fgimenez: could you please have a look at 3148? it needs a second review and is pretty simple (I hope :) [07:07] Hello, I'm trying to update the snapd PKGBUILD on archlinux form 2.21 to 2.23.6. I removed the biuld patches that move the /snap dir to /var/lib/snapd/snap and updated the verison number. Everytime I would try to build snapd, I would get a linking error I cannot resolve. https://pastebin.com/YY3Pa3U8 [07:07] mvo: sure on it [07:08] fgimenez: btw, is the re-exec spread-cron happy now? with the new snapd now being build? [07:08] Aimilius: hello! welcome :-) looking at the diff [07:08] er, log :) [07:09] Aimilius: aha, I think I know what's wrong [07:09] That was quick :) [07:09] Aimilius: can you do a master snapshot pleaes [07:09] Aimilius: I landed a PR that makes the build system more flexible [07:09] Aimilius: look at cmd/autogen.sh [07:10] mvo: nope it seems to have the same errors https://travis-ci.org/snapcore/spread-cron/builds/219549331 [07:10] Aimilius: the problem is related to the set of libraries that are linked into snap-confine statically [07:10] Aimilius: in master each library (that can) has a knob to control if it is linked dynamically or statically [07:10] Aimilius: and the defaults are all dynamically as that is most easy to compile on all distributions [07:10] Aimilius: in 2.23.6 that wasn't the case [07:11] Aimilius: we are working on 2.24 now (not sure if it goes out today as we found some old bugs last night and want to fix them) but if you take master you will have a better chance at building it [07:11] zyga: I'm trying to rebuild now [07:11] Aimilius: excellent! [07:12] mvo: snapd 2.23.6+201704061805.git.996da23~ubuntu16.04.1 in that execution, is that ok? [07:13] fgimenez: that sounds good [07:15] Master builds fine, though some checks fail [07:17] I believe they are about apparmor confinement (test_restrictions test_complain_missed and test_unrestriced_missed, so I didn't excpect that would work [07:17] zyga: one quick question in your 3145 PR - if correct I can quickly push a branch for core [07:18] mvo: looking [07:18] Oh, they're about the linux-grsec restrictions [07:18] So everything works fine [07:18] mvo: yes, that's correct, also replied there [07:18] Aimilius: those are about seccomp I think [07:18] Aimilius: aha [07:19] Aimilius: interesting, feel free to report those on the forum, I'm sure the security team will be interested in them [07:19] zyga: I've got linux-grsec installed, that blocks unpriviliged dmesg access [07:19] aha [07:19] zyga: The test-suites try to access dmesg, so that doesn't work [07:19] zyga: ta, I will push a branch for core with that fix [07:19] mvo: OK [07:20] zyga: I think on plain linux (without grsecurity) it'll work fine [07:21] mvo: branch with plug/slot uniqueness enforcement on AddPlug/AddSlot is almost ready [07:28] zyga: it looks like snapcraft.yaml does not allow to use: "plugs: network-bind-hook: interface: network-bind" :/ [07:29] zyga: the valications allows only flat strings arrays [07:29] zyga: whichi is very stange [07:29] jdstrand I have had an "review pending" on my snap for five or six days. Is this normal, should I change something? I just wanted to try and upload an beta of mdt for testing.. it's namespace is mdt.kristbaum. [07:30] zyga: hm, looks like its just a bit of a strange syntax [07:31] mvo: no, you need to spell that differently [07:31] mvo: like this: [07:31] zyga: yeah, silly me [07:31] mvo: plugs:\n network-bind-plug:\n interface: network-bind [07:33] zyga: After a build on clean linux (without-grsecurity) the tests still fail. https://pastebin.com/WH1JBmMe [07:34] PR core#32 opened: Rename core-support,network-bind plugs to make them unique [07:35] mvo: have a look at https://github.com/snapcore/snapd/pull/3153 please [07:35] PR snapd#3153: interfaces: validate plug/slot uniqueness [07:36] PR snapd#3153 opened: interfaces: validate plug/slot uniqueness [07:36] Aimilius: hmm, can you please open a topic for that on https://forum.snapcraft.io/ - make sure to report the kernel version used [07:36] Aimilius: we may carry some seccomp fixes [07:36] Aimilius: not sure yet, we'll check it out [07:36] Ok, I'll open a topic [07:36] morphis: ^^ have a look please [07:36] zyga: uname -a Linux glorfindel 4.9.20-1-lts #1 SMP Fri Mar 31 15:23:31 CEST 2017 x86_64 GNU/Linux [07:37] mvo: note that this branch may be a flag day for core revert [07:37] mvo: as previous core won't validate and because the plugs are explicitly defined and slots are implicitly added later, may be dysfunctional (no network-bind slot, no core-support slot) [07:38] mvo: I can do a separate patch that fixes this by detecting it and changing the snap via implicit.go [07:38] mvo: let me know what you think [07:39] mvo: I also thing we could use a `snap --validate ./path/to/file.snap` command [07:39] (^ internal hidden thing maybe) [07:40] zyga: hm, if we need a flag day we need to think about this more carefully, maybe a HO in a bit for some brainstorm? [07:41] mvo: yes [07:41] mvo: I think we can avoid it [07:41] mvo: if we teach snapd to just rename those two plugs [07:41] mvo: (core:network-bind -> core:network-bind-plug) (same for core-support) [07:41] mvo: then no flag day needed [07:41] mvo: do you want to jump into a HO to discuss? [07:43] mvo: I'm in the standup HO if you want to [07:44] zyga: yes, one minute, just addressing some things [07:44] mvo: though sligthly complex, depending on where we do this exactly [07:44] PR core#32 closed: Rename core-support,network-bind plugs to make them unique [07:52] Aimilius, zyga: looking [07:54] I posted a topic on the forum here: https://forum.snapcraft.io/t/checks-fail-on-arch-linux/200/1 [07:58] Aimilius: thanks, will respond there once I find something [07:58] morphis: thanks for your effort [07:58] np [08:00] Aimilius: you're not building on a kernel with AppArmor support, right? [08:01] zgrep APPARMOR /proc/config.gz # CONFIG_SECURITY_APPARMOR is not set [08:01] morphis: though I do use apparmor [08:01] Aimilius: ok, then you should compile snap-confine with --disable-apparmor [08:01] Aimilius: how when the kernel side support is disabled? [08:02] I normally run linux-grsecurity with apparmor compiled [08:02] But the tests didn't like the dmesg restrictions so I booted my normal kernel without grsecurity and apparmor [08:03] Aimilius: ok, when you don't have AppArmor in the kernel you should build with --disable-apparmor --disable-seccomp [08:03] morphis: I'm running this command to build snap-confine ./autogen.sh --prefix=/usr --disable-apparmor --enable-merged-usr [08:03] we don't have runtime detection (yet) [08:03] please add --disable-seccomp too [08:03] morphis: without apparmor the seccomp tests wont run either, I think the logic there assumes both [08:04] morphis: does snap-confine depend on out-of-tree seccomp patches [08:04] morphis: because the arch kernel does support seccomp [08:04] morphis; and I have libseccomp installed [08:04] Aimilius: the problem is that only using seccomp will unhide a few problems nobody didn't look into yet [08:04] safest is to disable both right now [08:05] Ok, rebuilding === chihchun is now known as chihchun_afk [08:06] morphis: if I add --disable-seccomp, the checks still fail [08:06] Aimilius: ok [08:06] Aimilius: can you paste the log output for that? [08:06] morphis: sure [08:07] sabdfl: thanks. Sorry Plan A didn't work out. Loved what you were trying to do [08:07] morphis: https://pastebin.com/v2sh2f7Z [08:10] Aimilius: will look in a bit [08:11] morphis: I'll have to go leave soon [08:11] Aimilius: ok [08:11] But I'll look at the forum this afternoon [08:11] Aimilius: I will see that I get an Arch setup and try to reproduce [08:12] morphis: Shall I post my PKGBUILD? [08:12] Aimilius: please! [08:13] morphis: https://pastebin.com/qRRNSYmz [08:13] I use a ugly hack to install the systemd unit files [08:13] Aimilius: ok, we have something better for this in 2.24 [08:13] morphis: But with the --nocheck option, it appears to install [08:13] morphis: great to hear [08:14] Aimilius: something like make -C data/systemd DESTDIR=... install will do the job then [08:14] morphis: that was what I did [08:14] ah [08:14] morphis: with SYSTEMDUNITDIR=/usr/lib/systemd/system, because of the usr merge [08:14] ok, that is then what you should do [08:15] But it felt like I shouldn't use a Makefile there === chihchun_afk is now known as chihchun [08:18] mvo: have a look a tthis please https://github.com/snapcore/snapd/compare/master...zyga:rename-clashing-core-plugs?expand=1 [08:18] mvo: that's what I was thinking about (tests WIP) [08:18] mvo: essentially this is the essence: https://github.com/snapcore/snapd/commit/59ae873d82883f8887ac7c55bead62e60ab749f6 [08:18] (the last two patches are new in this brancH) [08:34] Is somebody here that is responsible for manual reviews in the snap store? [08:39] Kristbaum__: I think so [08:40] Kristbaum__: I think you may have better luck on forum.snapcraft.io, using the "store" category [08:40] Kristbaum__: as they are in different timezones [08:40] Kristbaum__: and you may get more organized replies there [08:43] zygra okay will try that [08:45] Kristbaum__: thanks! [09:03] mvo: I'm adding tests to RenamePlug [09:09] mvo: adding tests to ifacestate [09:11] actually running snapd spread tests in QEMU doesn't look much reliable these days .. [09:12] morphis: why? what are you getting? [09:12] morphis: I sometimes get startup errors but if it runs, it is quite reliable [09:12] timeouts for apt-get [09:12] ala "running late" [09:12] sometimes network connectivity dies [09:12] but could be that I have a debian container I am trying to run the tests in [09:12] morphis: late is normal [09:12] morphis: aha, I dind't try it that way [09:13] zyga: should the lxd backend work these days? [09:13] morphis— apt update/upgrade the base image to keep the time in check [09:13] otherwise each test run does that, and it's hella slow :-) [09:14] Chipaca: yeah doing that right now [09:14] (that's a technical term) [09:14] mvo, #3070 failed again, the failures seem to be unrelated to the branch, i.e. snap confine test failures, gccgo failures [09:14] morphis— niemeyer has a cron job for this, but not sure if it's reusable for qemu [09:14] Chipaca: let me check [09:15] morphis: I don't know [09:15] morphis: we don't exercise it I think [09:18] zyga: ok, worth a try later [09:18] zyga: but I must say using the debian openstack images is working like a charm so far [09:24] morphis: nice! [09:24] morphis: can you add instructions on how to get them to spread-qemu-images please? [09:24] zyga: had to modify the adt-* script a bit but will send a PR for your spread images repo later today or on monday [09:27] zyga: you had a chance to test snapd on fedora yesterday again? [09:29] Son_Goku: any more testing from your side happend? I've added +1 to all three bodhi requests now [09:30] morphis: nice [09:30] morphis: no, I didn't we found a new set of bugs that delayed 2.24 release [09:30] morphis: I'm still working on fixing them [09:30] ok [09:30] zyga: what kind of bugs? [09:32] morphis: https://forum.snapcraft.io/t/duplicate-plug-slot-names-inside-the-core-snap/184/7 [09:34] mvo: ok, I'm adding one more correction, I will migrate connection state as well [09:34] mvo: that is done, just adding tests [09:36] zyga: I see [09:38] PR snapd#3148 closed: errtracker: never send errtracker reports when running under SNAPPY_TESTING [09:41] mvo: ok ready [09:42] mvo: let me open a PR [09:48] pstolowski: some of the test failures in 3070 look real [09:48] zyga: ok [09:50] PR snapd#3154 opened: many: rename two core plugs that clash with slot names [09:50] Chipaca: hey, mvo is off now [09:50] Chipaca: I need a hand with critical 2.24 reviews [09:51] zyga— shoot [09:51] zyga— or i can look myself :-) [09:51] on it [09:52] Chipaca: https://github.com/snapcore/snapd/milestone/6 [09:52] Chipaca: there are two approaches used [09:52] Chipaca: (and both should land) [09:52] Chipaca: I added one more branch to 2.24 [09:54] Chipaca: - rename everything when starting so even old core snap is correct [09:54] why is our test suite brittle all of a sudden :-( [09:54] Chipaca: this is done in https://github.com/snapcore/snapd/pull/3154 [09:54] PR snapd#3154: many: rename two core plugs that clash with slot names [09:55] Chipaca: - validate everything so it doesn't happen again [09:55] Chipaca: https://github.com/snapcore/snapd/pull/3153 [09:55] PR snapd#3153: interfaces: validate plug/slot uniqueness [09:55] Chipaca: and lastly because we found this by fixing a bug in core transition [09:55] snapd#3145 needs niemeyer to re-review i guess [09:55] PR snapd#3145: overlord/ifacestate: fix auto-connect during core snap transition [09:56] Chipaca: this branch fixes the damage done there https://github.com/snapcore/snapd/pull/3145/files [09:56] (as he marked it as Changes Needed) [09:57] yep [09:57] that's fine [09:57] I want to go through the first round on the two new PRs [10:03] zyga— wouldn't it make sense, in snapd#3154, for you to also test RenamePlug for a case where a plug and slot have the same name? [10:03] PR snapd#3154: many: rename two core plugs that clash with slot names [10:03] pedronis: hey there, I was looking into replacing the exec'ing of ssh-keygen with using openssl via CGO as in: http://pastebin.ubuntu.com/24333148/ [10:03] Chipaca: sure, can do [10:03] * zyga does [10:03] pedronis: do you have any guidelines on how to use CGo? Like place actual implementation into .c file or some such? [10:06] tvoss: look at... [10:06] tvoss: https://github.com/snapcore/snapd/pull/3095 [10:06] PR snapd#3095: cmd/snap-update-ns: add C preamble for setns [10:06] Chipaca: ha, I cannot [10:06] Chipaca: it's validated now [10:07] Chipaca: http://pastebin.ubuntu.com/24333233/ [10:09] zyga: ack [10:09] zyga— state twiddling direclty? [10:10] zyga— or make a for-testing way of overriding the validation [10:10] because clearly you can be in that state [10:10] so we need to test it [10:11] Chipaca: I'll think of something [10:11] Chipaca: good point about being able to be in that state [10:11] zyga— I'm now wanting to check it even more, to make sure we don't break people that are in that state already [10:12] Chipaca: look at the other branches and suggest ideas [10:13] Chipaca: note that it is not strictly speaking "hurtful" to have clashing names [10:13] Chipaca: but we want to get away from that [10:13] I think it's a matter of order of things in the other branch fixing this [10:13] but we need to be able to load the state to fix it [10:13] zyga— idea: make space probes look futuristic to trick aliens into thinking we're more advanced than we are [10:13] pedronis: state is not validated there (just connections are changed) [10:14] pedronis: and interfaces.ConnRef has no validation [10:14] zyga— or did you mean ideas that were relevant to the branch [10:14] don't we do a big add all the snaps thing [10:14] Chipaca: re brittle> https://forum.snapcraft.io/t/hashsum-failures-during-tests/198 is what I see in a bunch of them [10:14] zyga: initializes does addSnaps, no? [10:15] pedronis: aha, good point [10:16] I don't see the aha there but yes, bit of a chicken and egg problem [10:16] Chipaca: ok, I have something for that now [10:16] pedronis: well, if we load core snap and validate it [10:16] mvo— D: [10:16] pedronis: it won't reach that point [10:16] so it explodes [10:16] is that aha? [10:17] * pedronis has probably no sense of humur today [10:17] Chipaca: have a look, just pushed [10:17] * Chipaca hugs pedronis [10:17] anyway I agree with Gustavo's yesterday point that we need to tread carefully through these fixes [10:17] pedronis: yes, I totally agree [10:18] will one of our spread tests exercise thiss? like the upgrade one? [10:18] pedronis: spread tests do exercise this already [10:19] pedronis: well, not directly this [10:19] * pedronis anyway, lunch [10:19] pedronis: but the branch that adds validation is all read [10:19] red* [10:19] pedronis: because now core doesn't validate anymore [10:20] zyga— dude, stop with the Not(IsNIl) :-) [10:21] any way to create env variables or signals that can be accessed by two different processes inside a snap env? [10:21] Chipaca: how should I do it? [10:21] zyga— NotNil [10:22] zyga— compare the failing output and you'll see what i mean [10:22] Chipaca: oooh, didn't know about it :) [10:22] Chipaca: ack, will do [10:22] my snap exposes a noejs app which spawns a sh script which fires a banary [10:22] binary [10:23] pshod— one easy way is to use the filesystem [10:23] pshod— that includes, for example, a unix socket [10:23] i am notifying the event by writing into a file [10:23] and then if the file has a particular content [10:24] i close my watch in the node app [10:24] chipaca : the sh script should write unto a socket and the node app listens on the socket? [10:25] zyga: help help help! [10:25] pshod— it depends on what you need to do, etc etc [10:25] pshod— please tell us more [10:25] ill try to be concise [10:26] pshod: so [10:26] pshod: if you set environment from a running process [10:26] pstolowski— “- Download snap "core" (1577) from channel "stable" (unexpected EOF)” <- just got that :-( [10:26] snap exposes a node app -> spawns a sh script that fires a binary->binary reads data from a sensor and writes into a file in SNAP_COMMON->node app reads the file and posts to a REST server [10:26] pshod: you cannot just "see" that easily in other processes without looking at /proc [10:27] pshod: not sure what you meant by a signal [10:27] pshod: tell me what you want to achieve [10:27] pshod— with you so far [10:27] pshod: (not how) [10:27] pshod: and we'll recommend osmething [10:27] Chipaca, is this with current code, or latest release? [10:27] pstolowski— https://travis-ci.org/snapcore/snapd/builds/219327815 [10:28] no the node app is configurable..we pass the number of tmes the binary writes into the fuke [10:28] file* [10:28] pstolowski— upgrade/basic failed with that [10:28] now what I have done is, i modifies the sensor binary so that after writing the said "n" observations into the file [10:28] Chipaca: https://github.com/snapcore/snapd/pull/3154#discussion_r110359632 (updated and added tests) [10:28] PR snapd#3154: many: rename two core plugs that clash with slot names [10:28] overwriting* [10:28] it writes "Closing Session" [10:29] so when i read the file from SNAP_COMMON [10:29] and see it's contents as "Clossing Session" I remove the watch from SNAP_COMMON and my node app exits gracefully [10:29] Chipaca, uhm. not good. would be great to find out how to provoke 'unexpected eof' in the test. what I did recently was a test with closing client connections, and that triggers normal EOF [10:30] people dont like this design and I am not pretty confident about it too [10:30] :| [10:30] pstolowski— send a RST [10:30] pshod: today I'm focusing on 2.24 release and I don't have much time to help [10:30] pshod— ah, so this is what you're doing now and you need to improve it? [10:30] pshod: maybe ask on the forum and we'll get to that [10:31] pshod— yeah, maybe it's better as a forum question, rather long-winded for here [10:31] zyga : hey thank you! [10:31] pshod— nothing wrong with the file-based approach as long as you're using locking appropriately [10:31] pshod— but there are other ways, sure [10:31] zyga: have gotten much help from you in the past :D [10:32] pshod: I'll gladly help next week but I need to focus on fixing something we just found out that is affecting core [10:32] pshod: namely https://forum.snapcraft.io/t/duplicate-plug-slot-names-inside-the-core-snap/184/6 [10:32] pstolowski— no i don't know *how* to send a RST, i know a RST is the usual cause for that unexpected eof [10:32] pshod: we won't release today (Friday = bad idea) but I want it to be ready [10:32] chipaca : locking is done properly, I suggested we just update a variable which gets updated when API for posting is called [10:32] pstolowski: network throttling things sent RSTs to kill connections [10:32] Chipaca, exactly... i'll look into that later, thanks. in the meantime - E_TOOMANYISSUES_ATM [10:32] but that was rejected too. [10:33] zyga : duplicate plug names [10:33] pshod— ok, open a forum thread, write everything out there please [10:33] zyga: sounds too much of fun :P [10:33] chipaca : on it [10:34] pshod— what you want to achieve, what you are doing now, why it was rejected/requested to be improved, what other things were proposed but also rejected [10:34] pedronis, Chipaca, pstolowski, mvo: I'm looking for proposals on how to fix the chicken/egg problem on core's snap.yaml: we need to load an invalid file to fix it [10:34] pshod— the 'what you want to achieve' there is missing in what you've written so far, afaict [10:34] my naive idea is to add a way to load snap.Info that's invalid [10:34] use that on the core snap only [10:34] and apply the fix right there [10:34] (the fix is currently in interface manager but it should move to snap manager) [10:35] zyga: how about a forum.snapcraft.io topic? to have an archived discussion? [10:35] mvo: +1 [10:35] chipaca : will do it like that [10:36] PR core#32 opened: Rename core-support,network-bind plugs to make them unique [10:38] commented on the forum about it [10:38] https://forum.snapcraft.io/t/duplicate-plug-slot-names-inside-the-core-snap/184/9?u=zyga [10:39] chipaca : wont i be able to register using my company id? [10:40] pshod— I don't know. It's not (yet) tied into login.ubuntu.com, if that is what you mean [10:50]