[00:11] wgrant: should http://cpisnap.ols.internal/api/v1/ still work? [00:11] ah [00:11] d'oh [00:11] .internal [01:05] hello, I'm trying to make a snap for openmw (re-implementation of the morrowind game engine) and when trying to launch my snap, I'm getting the error "libGL error: unable to load driver: i965_dri.so" I think I might be missing a plug. Does anyone happen to know which plug would make that driver available? I've tried adding x11, opengl, home, and pulseaudio. [01:14] jorian: This may be a red herring, but my question is, is there such a file inside your snap? [01:20] qengho: it seems to be in 3 places inside the snap directory [01:20] ./parts/openmw/install/usr/lib/x86_64-linux-gnu/dri/i965_dri.so [01:20] plus in stage and prime [02:07] I'm looking at the guide for creating a snappy interface on zygoon.pl. The method involves editing the snapd source and the recompiling the whole project. Is there currently a way to add interfaces externally / by installing a snap similar to how the core snap works? [02:11] I need to create an interface to allow writes to /var/run but I don't really want to have to manage a custom version of snapd [03:09] Thank you wgrant, I appreciate the help! [03:10] olympionex, the only reason the core snap can make any interface available is because the interfaces it supports are supported by snapd [03:10] It's not doing anything special [03:11] kyrofa: thanks -- I see that now looking at the source [03:11] kyrofa: all the interfaces are in snapd itself [03:11] qengho, if jorian comes back, the gl libs are required in stage-packages. Specifically libgl1-mesa-dri [03:12] olympionex, indeed: interface SUPPORT is in snapd, but then the producer (slot) side of interfaces must be supplied by a snap, and the consumer (plug) side must be consumed by a snap [03:12] The interface being USED by the plug/slot, however, must still be one that's supported by snapd [03:14] Oh wait... qengho nevermind, looks like that lib made it into prime. Just missing the right LD_LIBRARY_PATH then, it seems [03:14] I see - so the content plugin lets a snap provide access to some internal path, but if I want to access some other system path, I will have to add it to snapd [03:14] olympionex, yes. Although I'm curious about your use-case, do you mind sharing? [03:15] kyrofa: yeah, I just have a binary that I don't have the source to and it creates some socket files in /var/run [03:15] olympionex, yep that would do it :P . However, have you investigated using LD_PRELOAD to get around it? [03:16] kyrofa: i was using classic mode, and probably still can, but it looks like it changed a bit in 2.22 maybe [03:16] hmm, I'm have a look at that [03:16] 2.22 is a bit behind, 2.25 is out now [03:17] olympionex, there's a preload part out there already if you want to check it out. Here's an example of how to use it: https://github.com/nextcloud/client_theming/blob/master/linux/snap/snapcraft.yaml#L50 [03:17] Check out line 34 as well for where it's pulled in [03:18] I actually meant snapcraft 2.22. The new version doesn't add the library paths to the command wrappers it generates. I can of course just make my own script that gets called by the wrapper, but I need to research all the consequences. I was just checking again about running it in strict mode [03:19] olympionex, in that case it's even older: 2.29 is out :P [03:19] olympionex, yeah, strict is _always_ best if you can swing it [03:19] olympionex, classic snaps have trouble running on other distros, and they don't run on ubuntu core either [03:20] olympionex, definitely worth investigating that preload part, especially considering it's an extra line and a half [03:20] kyrofa: We've identified the two bugs in dashboard.snapcraft.io that combined to cause this issue. It's possible you'll run into similar issues again with that particular snap until we fix the root cause properly, but we know what's going on now so that should be quick. [03:21] wgrant, it's something particular to the snap?! [03:21] What did I break? [03:22] kyrofa: A review race caused one of the revisions to get stuck in a bad state, and that will possibly confuse further automatic refuses until we fix that old revision. [03:22] ... [03:22] s/refuses/reviews/ [03:22] Hopefully not refusals :) [03:22] wgrant, ah ha, I wondered about that weird rev :P [03:22] They shouldn't fail, just get stuck [03:22] wgrant, if I reject it, would that fix things? [03:23] kyrofa: Quite possibly. We already know we need to manually fix that revision, so rejecting it can't really make it worse. [03:23] So might be worth a try. [03:23] * kyrofa tries [03:24] kyrofa: fortunately i have a limited set of systems I have to run on. I use snapcraft to simply our internal deployment of some software. Still, i prefer to get things working in the most supported way to avoid suprises later. [03:25] olympionex, understood [03:26] kyrofa: Looks like that's worked. [03:26] So we just need to manually fix up 1418 later. [03:27] wgrant, no need to worry about that one [03:28] kyrofa: thanks for the preload info, i was looking at some ELF hacking but this may be cleaner [03:28] olympionex, yeah that would not be fun [03:28] And of course, any time :) [05:56] PR snapd#3393 opened: Add retry mechanism for snap install command to fix test === koza|away is now known as koza === JanC_ is now known as JanC === elfgoh_ is now known as elfgoh [07:11] pedronis, mvo mind unblocking re-reviewing https://github.com/snapcore/snapd/pull/3353 ? [07:11] PR snapd#3353: Add support for reboot parameter [07:18] good morning [07:19] PR snapcraft#1333 opened: state: search for the dependencies in the archive [07:21] morning zyga! [07:27] o/ [07:32] zyga: hey! [07:32] zyga: time for a review on https://github.com/snapcore/snapd/pull/3371 ? [07:32] PR snapd#3371: packaging: import packaging bits for opensuse [07:35] morphis: looking [07:39] zyga: CI hates me, another store timeout [07:52] zyga: but thanks for approving [07:52] zyga: and a review on https://github.com/snapcore/snapd/pull/3365 would be nice too [07:52] PR snapd#3365: tests,packaging: add package build support for Fedora for our spread setup [08:03] morphis: what does it mean https://github.com/snapcore/snapd/pull/3365/files#diff-3c11e56e5f7f82b0f352d0fe1851a107R268 ? [08:03] PR snapd#3365: tests,packaging: add package build support for Fedora for our spread setup [08:04] didn't the pkgdb helper land already [08:04] ah [08:04] sorry, I see [08:04] zyga: you can't use it at this place [08:04] ok :-) [08:05] ooh, listen to the wind! [08:05] it sounds like "reeeeviewwsss" [08:05] good morning peeps [08:09] morphis: done [08:10] hey Chipaca [08:10] * zyga feels miserable [08:11] zyga: :-( [08:11] zyga: did you see the issues ryebot was having yesterday? [08:13] Chipaca: yes but I didn't manage to reply [08:13] zyga: 's ok; i just don't want to have to remember it myself [08:14] if you're aware, i can forget [08:14] thanks :-) [08:41] Chipaca: I nitpicked a bit on one of your PRs [08:41] pedronis: good morning to you too :-) [08:42] zyga: ok, updated the PR [08:43] pedronis: good points you make [08:43] http://paste.ubuntu.com/24642129/ [08:43] pedronis: i'm currently working on a bug where we don't distinguish "already at the latest revision on that channel" from "that channel does not exist" [08:43] any wording suggestions before I do the full set? [08:44] pedronis: (you can try it! snap refresh core --channel potato) [08:44] pedronis: (then check 'snap info') [08:44] (ideally the output would be more consistent) [08:44] and not sure what to do with "allows" [08:44] maybe just skip it [08:44] 's fun [08:45] fun fact: we have 91 interfaces [08:45] I'll add a 100th anniversary interface ;-) [08:47] Chipaca: can we do it without store help? [08:47] pedronis: the store is telling us [08:47] (more or less) [08:48] pedronis: i'm fixing it alreayd [08:48] zyga: I don't want to be a party pooper but https://forum.snapcraft.io/t/snapd-2-25-blocked-because-of-possible-revert-race-condition/722 is currently higher on the priority list than anniversaries ;) [08:48] pedronis: basically ListRefresh is fine for multi-updates, but we need a special one for single updates that doesn't throw away this info [08:48] pedronis: pretty much exactly the same except if the store returns nothing, raise an error [08:48] Chipaca: just telling me why you won't fixing my nitpicks straight away :) [08:49] pedronis: yes, that's exactly what i'm doing [08:49] you're very perceptive today [08:49] but inpolite because I didn't say good morning yet [08:50] pedronis: what's a good name for something that does what store.ListRefresh does, but for a single refresh candidate? [08:50] mvo: you are completely right [08:50] mvo: I'll finish this PR and send it for review in a moment [08:51] mvo: but before I do that, let's chat here briefly [08:51] mvo: if the new syntax is a blocker (I mean |N) then the plan falls apart [08:52] zyga: yeah, its rather annoying [08:52] mvo: your plan to compile seccomp profiles in snapd is possible [08:52] mvo: but, well, needs discussion [08:52] mvo: we could make a new internal helper [08:52] zyga: we would have to generate old-style files for a while for backward compatiblity [08:52] mvo: why? I don't think so [08:52] zyga: old style with a frozen format so that also is slightly complicated [08:52] zyga: no? [08:52] mvo: (just whatever matches snap-confine) [08:52] zyga: aha, just leave the old ones around :) ? [08:52] mvo: yes [08:53] mvo: so we'd stop writing the old files entirely (but not remove them yet) [08:53] mvo: for revert specifically [08:53] zyga: hm, that would break in the case of "snap refresh core; snap install hello; snap revert core", then hello would no longer work. maybe not too terrible [08:53] mvo: aha [08:53] mvo: interesting, yes, it would work if snapd would start before "hello" but I see your point [08:54] zyga: aha, good point [08:54] mvo: but the bigger issue is that we'd have two seccomp profiles per security tag [08:54] mvo: one old and one new, with different semantics (because no |N) [08:54] zyga: yeah, that is the part that worries me, it feels fragile [08:54] mvo: but if we do that we can use the old text format [08:54] mvo: because we'd just write two files [08:54] mvo: one with |N and one without [08:54] heh [08:54] good point [08:54] * zyga greps for |N syntax [08:55] well, the seccomp bpf would make us future proof forever(tm) [08:55] weird [08:55] mvo: I agree on that [08:55] (needs some research but it looks like bpf is pretty stable as a binary format) [08:55] yes [08:55] zyga: what is weird? [08:56] mvo: I don't see any |N syntax with quick grep [08:56] Chipaca: LookupRefresh ? [08:56] are we even using this? [08:56] mvo: (also it would be faster for startup, which is nice) [08:57] zyga: let me double check the debdiff [08:57] mvo: can you grep for anything that uses this [08:57] thanks! [08:57] zyga: if we are not using it, that would be splendid [08:57] * zyga feels sick again [08:57] yes :) [08:57] (perfect timing, just before a conference) [08:57] zyga: uh, get some rest if you feel sick, I can work on this [08:58] zyga: seriously, get well, especially if you need to travel soon [08:58] mvo: no, I cannot do anything about it :/ [08:58] zyga: meh, ok [08:58] mvo: I'd rather finish the interface meta-data changes and feel better by doing that [08:58] mvo: sick as in food-sick, not flu-sick [08:59] zyga: autsch, ok. all right, I do the investigation on |N while you finish your interface work :) [08:59] OK [08:59] zyga, mind unblocking/re-reviewing https://github.com/snapcore/snapd/pull/3353 ? [08:59] PR snapd#3353: Add support for reboot parameter [09:08] zyga: we use the |S_IFREG in our default template now https://github.com/snapcore/snapd/blob/master/interfaces/seccomp/template.go [09:08] zyga: I will write a forum topic with some ideas, looks like it is not going to be super simple, but maybe not to hard either [09:20] mvo: zyga: one of you should probably review snapd#3350 if we want it in 2.27 [09:20] PR snapd#3350: interfaces,overlord/ifacestate: make sure installing slots after plugs works similarly to plugs after slots [09:21] anyway I suppose 2.27 is delayed by a fair bit because of 2.25 issues? [09:21] *fair bit now [09:21] pedronis: yeah, there is a slight crisis right now :/ [09:22] mvo: can't we just change the names of the seccomp profiles ? or start putting them into profiles/something [09:22] s/something/somehash/ [09:22] and teach 2.26.x snap-confine about new new place [09:23] s/new new/the new/ [09:24] pedronis: probably, there is still this problem that "snap refresh core; snap install daemon; snap revert core" may lead to the same issue we currently having. i.e. the new snapd does only write a new seccomp file but on reboot daemon starts so early that the old snapd does not generate a new seccomp file early enough. but still doing this would probably get us a long way forward [09:25] reverting core after installing new stuff is bound to create troubles anyway in some scenarios [09:26] * mvo nods [09:26] I mean can see some non security profiles one [09:26] mvo: do we use |S_IFREG or just S_IFREG? [09:26] pedronis: ack [09:27] has anyone used the snapcraft-preload project? I'm thinking there is a bug in the preload for the bind method, or possibly there is something i'm not doing. When creating an AF_UNIX socket, it would seem that it should redirect to a writeable path, but that doesn't seem to be the default behavior, at least for my case. I have manually patched it on my side and it seems to work, but just wanting to verify. [09:27] olympionex: guys who wrote that are in americas so I'd suggest opening a forum topic [09:30] zyga: the former |S_IFREG [09:32] mvo: aww, drat [09:32] mvo: well [09:32] mvo: plan B [09:32] mvo: revert that change [09:32] mvo: like we did before [09:33] mvo: if it is just one spot it is easier [09:33] ah, there are a few [09:33] all the mknod [09:34] hmm hmm hmm [09:34] zyga: yeah [09:37] mvo: so what is your plan? [09:38] zyga: can you merge https://github.com/snapcore/snapd/pull/3371 ? [09:38] PR snapd#3371: packaging: import packaging bits for opensuse [09:39] zyga: I'm still pondering, its tricky. worst case would be to revert the new syntax again, i.e. revert all mknod and keep quotactl via the secompSymbolTable branch. we need jdstrand input here [09:39] morphis: done [09:39] PR snapd#3371 closed: packaging: import packaging bits for opensuse [09:39] mvo: the question is, what do we lose if we do that [09:39] mvo: was this released in 2.25? [09:40] abeato: some feedback [09:40] zyga, ok, thanks [09:48] zyga, I've answered one of the comments [09:58] zyga: yeah, the mknod was released in 2.25, I think we need jamies input. but I will also try to find out more [09:59] mvo: can we replace |foo with a list of rules? [09:59] hmm, so using delta updates on the pi3, the delay is noticeable but not super awful [10:00] i fear that wont be usable on a bbb [10:00] PR snapd#3388 closed: osutil: add non-blocking flock [10:02] sorry I did this merge ^ but messed up a bit the commit message, because GH seems very partial on what it remembers or not when you move away and back to a merge in progress [10:22] PR core#35 closed: move xdg-open to proper paths [10:22] PR core#38 closed: Add another pi-config option [10:23] PR core#35 opened: move xdg-open to proper paths [10:23] PR core#38 opened: Add another pi-config option [10:23] mvo: fgimenez: do you guys have dev access to etcd? [10:24] actually, i just want to know a revision of etcd that isn't published [10:24] bah [10:24] never mind [10:24] Chipaca: i don't sorry [10:24] * Chipaca noticed the store's response [10:25] $ /tmp/snp install potato --revision 2 --channel whaat [10:25] error: snap "potato" not found (at least in channel "whaat/stable" at revision 2) [10:25] incredible how much work i had to do to get that (at least) there [10:27] PR snapd#3394 opened: interfaces/seccomp: add bind() syscall for forced-devmode systems [10:31] mvo, what is linux-armhf ? is that leftover cruft ? https://forum.snapcraft.io/t/issues-while-porting-ubuntu-core-16-image-for-olimex-lime2-platform-board/761 (snap info says its yours) [10:34] that's an old kernel [10:34] yes [10:34] from before we could have multiple architectures in the same snap [10:34] i just wonder why it is available [10:34] we did a big cleanup run a while ago [10:35] zyga: thanks! [10:39] gaah [10:39] ogra_: fwiw i can confirm it's in the store [10:39] at revision 9 [10:40] Chsure, i wouldnt have pinged if i had not checked it before ;) [10:40] yeah, in all channels [10:40] zyga: gah? [10:40] so, interface summary is very confusing to write [10:40] as plugs and slots have totally different purpose [10:40] what is a good summary of the "pulseaudio" interface? [10:41] it's both to be pulseaudio service and to talk to it as a clinent [10:41] zyga: "this interface lets you pulse your audio" [10:41] should we have two summary lines per interface? (up to) [10:42] zyga: are both those things available to you when you use it as a slot? [10:42] Chipaca: no, but this is not about plugs or slots [10:42] zyga: maybe it should be [10:42] Chipaca: this is about the new "interface" command and interface meta-data [10:42] Chipaca: what interfaces do you have? (not instances, types) [10:43] http://paste.ubuntu.com/24642921/ [10:43] Chipaca: my misery so far [10:43] Chipaca: maybe "allows operating as or using the pulseaudio service" [10:44] zyga: is the "allows" there something you've agreed to standardize on? [10:44] looks nasty af [10:44] no, this is still a prototype [10:44] :-) [10:44] af? [10:44] zyga: how about you drop the allows and verb the thing you have after it? [10:44] zyga: as fireworks [10:45] :-p [10:45] Chipaca: yeah, I'm open to suggestions, I'll finish this pass to just have something consistent for each and then propose the bits and pieces [10:45] zyga: i mean: "allows frobbing the plumus" -> "frob the plumbus" [10:45] this is just one patch among many and one that I suspect we'll bikeshed the most [10:46] Chipaca: try this on 5 different real interfaces or it doesn't count [10:46] zyga: i did :-) [10:46] Chipaca: how about network? [10:46] ogra_: hm, a very old kernel that I put there for experiments I think, I will unpublish it [10:46] zyga: "access the network" [10:47] zyga: pulsaudio would be, for example, "host or communicate with a pulsaudio server" [10:47] mvo, thanks [10:47] ogra_: done [10:48] zyga: unity8 would be "aw, bless" [10:48] Chipaca: kind of like the pulseaudio thing but this is still very confusing IMO, from users POV (no deep insight into plug-vs-slot semantics) there's a lot of difference here [10:48] zyga: remember these are more for orientation and not explanation [10:48] that is, details at the url [10:48] yes, that is true [10:49] the URL is already in the meta-data [10:49] zyga: a user looking at this list is either refreshing their memory, or wanting to know what to read to learn more [10:49] zyga: so you give them enough information to pare down the list [10:50] anything networkish, they're probably going to have to look at all the network* things the first time to figure out what they need [10:50] the second time it'll be enough as is [10:57] PR snapd#3381 closed: logger (& many more, to accommodate): drop explicit syslog [11:02] zyga: hmmm [11:02] yes? [11:03] $ snap remove http [11:03] error: cannot perform the following tasks: [11:03] - Remove security profile for snap "http" (x1) (cannot setup mount for snap "core": cannot update mount namespace of snap "core": cannot update preserved namespace of snap "core": cannot update snap namespace: not implemented) [11:03] Chipaca: you are using snapd from master but the rest from package [11:03] Chipaca: just buind and install snap-update-ns [11:03] ah ok [11:03] also [11:03] our error stacking sucks [11:03] i'll get there eventually [11:04] Chipaca: you want discard-ns too [11:04] FYI [11:04] and snap-confine [11:04] zyga: so, make hack [11:04] or you may hit weird issues [11:04] Chipaca: yes, and I need to teach it to build the go parts maybe :) [11:04] nah, those are easy :-) [11:06] just 4 left [11:08] ok, i've got to run [11:08] got physio [11:08] take care! [11:08] I probably won't be back in time for standup [11:09] morphis: is this https://forum.snapcraft.io/t/run-configure-hook-of-core-snap-if-present-runs-seemingly-forever/674/2 the snapctl/bind issue likely, or something else? [11:09] so: i'm working on "snap refresh core --channel bogus" (which currently, erroneously, works) [11:09] got a fix already, fixing tests and cleaning it up, should have pr before eod [11:09] also, got some work to do on go patch [11:11] pedronis: it looks different here as from the log output it hangs at "Mount snap" [11:11] so the topic is wrong? [11:11] pedronis: and the bind() issue shouldn't occur for the core snap as the core snap now plugs network-bind [11:11] it's not about configure [11:11] pedronis: the first post in that topic is [11:12] "[-] Run configure hook of "core" snap if present" [11:12] ok, confusion === tinwood is now known as tinwood_afk [11:43] zyga: https://github.com/snapcore/snapd/pull/3394 is ready for a merge unless you want someone else to have a look [11:43] PR snapd#3394: interfaces/seccomp: add bind() syscall for forced-devmode systems [11:43] morphis: can mvo ack it please? [11:43] mvo: ^^ === chihchun_afk is now known as chihchun === tinwood_afk is now known as tinwood [13:57] * zyga prepares the three banches [14:00] zyga: btw. when you try lxd on suse just don't forget to disable AppArmor for the containers [14:01] niemeyer: Spread-32 is now down to 1.3G, ok for you? [14:07] morphis: aha [14:07] morphis: how do I do that? [14:08] zyga: --with-less-security [14:09] zyga: --make-hackable [14:09] zyga: --wanton-insecurity [14:09] zyga: --make-chipaca-stop [14:09] Chipaca: sudo make me a sandwitch [14:09] PR snapd#3395 opened: many: remove interface meta-data from list of connections [14:09] * zyga didn't eat anything but breakfast today [14:09] fgimenez, how do you usually retriger a ci check when there is a failure which doesn't need any change in the code? [14:09] Chipaca: ^^ please land that [14:09] fgimenez, I got this https://travis-ci.org/snapcore/snapd/builds/235333896?utm_source=github_status&utm_medium=notification [14:10] zyga: you're going off? [14:12] zyga: https://github.com/lxc/lxd/issues/3096 [14:18] cachio: i can't retrigger on travis sorry, i think only people in the snapd-commiters team can retrigger [14:18] cachio: usually, if your PR needs to wait, merging master is a good idea in general, that push retriggers the build [14:22] zyga: https://github.com/lxc/lxd/issues/3345 [14:27] fgimenez, nice, tx [14:37] morphis: Any chance we can take it a bit further down? This will likely be something like ~2G for imaging purposes [14:38] * zyga -> car & kids [14:48] PR snapd#3396 opened: interfaces: add summary to each interface [14:50] PR snapd#3397 opened: interfaces: disable "mknod |N" in the default seccomp template again [14:51] niemeyer: ^^ [14:55] morphis: Also sent a review on the no-apparmor bind issue.. we should probably have forum conversation about that one.. given the comment is not entirely clear that we're in agreement about what's going on in that case [14:57] cachio: hey [14:57] cachio: some failing tests https://travis-ci.org/snapcore/snapd/builds/235657720?utm_source=github_status&utm_medium=notification [14:57] cachio: please collect the log if you want to keep inspecting it, as I want to restart the test [14:59] PR snapd#3393 closed: tests: add retry mechanism for snap install command [15:01] zyga: I found a workaround for my issue, so I'm no longer blocked, but I added what useful information I could to this bug: https://bugs.launchpad.net/snappy/+bug/1687079 [15:01] Bug #1687079: cannot change profile for the next exec call: No such file or directory [15:01] good luck! [15:02] ryebot: thank you! [15:02] np! [15:05] cachio: This one might be easy to address: http://paste.ubuntu.com/24644624/ [15:06] Chipaca: silly systemd question, we have this problem currently that when a new snapd boots it will rewrite the apparmor/seccomp profiles. however snap services start in parallel so there is a race condition here and they could start before snapd finished updating the profiles. I wonder if we could use systemd to sync this, i.e. have a way to say for all snap service that they may only start after snapd-regenerate-profiles was run. wdyt? [15:06] mvo: is snapd-regenerate-profiles a systemd service, or is it a task? [15:06] (the answer is yes either way) [15:07] Chipaca: it is something that happens in the daemon at startup, so no proper task just a part of the init of snapd [15:07] Chipaca: I like the "yes" here [15:07] mvo: ok, so it's going to be a little bit of work [15:07] mvo: we need to make systemd be a "notify" daemon [15:08] (btw, I don't know if this'll work in 14.04) [15:08] um [15:08] i mean we need to make *snapd* be a notify thing [15:08] ah [15:08] Chipaca: aha, there is a branch for that [15:08] there is? nice [15:09] Chipaca: but of course that branch can not do much currently except for telling systemd that it is still alive in a crude way [15:09] 3111 [15:09] I think the idea is that snapd wouldn't notify until it has rewritten the profiles [15:09] that it's started [15:09] yes! [15:10] I don't know how that vs the watchdog stuff interact though [15:10] pedronis: aha, maybe I misunderstand, I thought the notify thing was the same socket, let me double check [15:11] so [15:11] pedronis: i.e. that we can reuse code, but I will double check :) [15:11] they both do sd_notify [15:11] one with WATCHDOG=1 [15:11] one with WHATUPDOG=1 [15:11] i mean READY=1 [15:12] so, the watchdog code needs rewriting to support this [15:12] or at least refactoring [15:12] yep [15:13] but relatively easy i think [15:14] mvo: that thing can be a snap internal command [15:14] mvo: it would be easier to get this right [15:14] mvo: essentially a snapd ping of any kind is sufficient [15:14] mvo: then no need to make snapd notify anything [15:14] mvo: it's just a "snap ensure-soon" or whatever [15:14] mvo: and we're done [15:15] another call-for-feedback on https://github.com/snapcore/snapd/pull/3396 [15:15] PR snapd#3396: interfaces: add summary to each interface [15:16] I'd like to land it once green, I'm happy to iterate on the summary lines [15:16] once it lands I can propose the command that uses this and needed plubming [15:16] Is there a way to prevent automatic refreshes occurring? === cachio is now known as cachio_afk [15:17] ryebot: no [15:17] ryebot: you can set a refresh schedule [15:17] ryebot: but refreshes are mandatory [15:17] alright, where can I find docs on the refresh schedule? [15:18] nvm, I found a forum post [15:18] zyga: I'm not sure I understand. all snaps that have services will need to have an After=snapd-regenerated-service in their auto-generated systemd service and we need to emit that when we are finished with the generation. how is this related to an internal command? [15:18] Chipaca: thanks a bunch for your input, I will start a forum thread about it [15:19] zyga: what do you mean 'no need to make snapd notify anything'? [15:19] right [15:20] mvo: my point is that that After= can refer to a new unit that just says "snap ping" to wake up snapd [15:20] mvo: AIUI it'd be just After=snapd, with snapd not considered "up" until it's sd_notified of READY=1 [15:20] mvo: no need to involve more complex tooling [15:20] ahh [15:20] Chipaca: that the more complex solution is probably fine but this feels easier [15:20] Chipaca: it could just be after=snapd really [15:21] So, this is a little problematic for my use case. I'm helping the LF build multiple k8s clusters into an image for use in their k8s certification exam. During image build, the k8s bin versions might be something like 1.6.2. During testing (which could be any time, afaict), bam! 1.6.4 comes out and disrupts services etc. [15:21] Chipaca: I do agree that snapd needs to say "I'm up" to systemd for an "after=snapd" approach [15:21] agreed about the ping, now that i got you, but it's more hackish, and we want to have the sd_notify thing already [15:21] (for the watchdog) [15:21] so might as well do it [15:21] imho [15:21] Chipaca: yes, that's fine [15:21] fwiw [15:21] So the refresh schedule doesn't actually aid me in this case. [15:22] ryebot: hold on [15:22] kk [15:22] ryebot: can you re-phrase that, are you testing an image with snapd on it or is k8s just a snap of some kind [15:22] * zyga is unsure what it is [15:22] We [15:22] We are building an ami running a bunch of k8s clusters under lxd [15:23] the k8s clusters use CDK, which use snaps to install the k8s bins/services [15:23] * zyga sees techno-babble [15:23] ami's I grok [15:23] k8s is that kubernetes? [15:23] kubernetes* [15:23] yes sorry [15:23] ryebot: what's the problem with the refresh schedule? [15:23] * zyga reverses shield polarity and re-reads [15:24] Just that (afaik) people could be taking the test at any time of day, so we can't really pick a time to refresh that we can be sure won't interrupt them. [15:25] ryebot: can you tell me what is the test? [15:25] kubernetes certification exam [15:25] ryebot: when would you want them to test old lxd/kubernetes? [15:25] aha [15:25] I see [15:25] still in development [15:29] zyga: the version probably won't matter most of the time, so long as it's compatible with the test constraints [15:29] ryebot: so here's an idea [15:30] ryebot: just track the stable channel [15:30] ryebot: and a specific version therein [15:30] ryebot: sure, you may get updates [15:30] ryebot: but only within that channel [15:31] and the channel can track, say, version 2.x or 2.3.x (numbers are just examples) [15:31] Yes, we currently have them track 1.6.x, the problem is that some of these snaps are services, and so the update can interrupt them [15:32] kyrofa, o/ hey, fyi, I'm looking at the r1418 issue with nextcloud, trying to leave it in a consistent/coherent state (just in case you get some emails about state changes); automated review passed, so it should get approved (and then, it will be available through the branch it is released) [15:32] aha [15:32] matiasb, oh thanks, I saw that it got put into the manual review queue, so I rejected it again and was gonna ping someone asking about that :P [15:32] matiasb, makes sense if you're poking at it [15:33] kyrofa, oops, sorry! give me a min and it should get fixed (and run the review for the newer upload automatically) [15:34] matiasb, oh no problem [15:34] matiasb, is this one of the higher rev-count snaps? [15:36] kyrofa, there, r1418 back in track, everything should be ok there [15:36] kyrofa, it is one of the highest, for sure [15:37] * mcphail notes kyrofa's nextcloud snap is at revision 1337 :) [15:37] leet :) [15:38] mcphail, yes I was convinced that rev was bug-free. Unfortunately I was wrong :( [15:38] matiasb, thank you! [15:38] ryebot: how urgent is this concern of yours btw? [15:39] ryebot: because there is the idea of being able to tell snapd to not refresh things for a bit, i think (but i'd need to check) [15:39] but it's not even scheduled afaik [15:39] Chipaca: Honestly not sure. I mean, it may never happen, or we could really screw up some candidate's tests. The risk appears high enough that the LF devs brought it up. [15:40] Chipaca zyga: crazy idea - would disabling the snapd daemon work? :| [15:40] Chipaca: yes [15:40] er [15:40] ryebot: yes [15:40] ryebot: we also have refresh timer that fires very infrequently [15:41] ryebot: so you'd have to disable both [15:41] also a service? [15:41] ryebot: yep, you can even say 'systemctl disable --now snapd.*' and it'd TTRT [15:41] DDRT even [15:41] perfect [15:41] DTET [15:41] thanks guys!! [15:41] do the evil thing [15:41] hahaha [15:41] note that * is for systemctl itself, so you might need to protect it from your shell [15:41] gotcha +1 [15:42] alternatively systemctl's tab completer is good enough that you can do snapd. and it'll complete it for you [15:42] alright great I'm going to try this out :) [15:44] Chipaca: fancy looking at https://github.com/snapcore/snapd/pull/3396 [15:44] PR snapd#3396: interfaces: add summary to each interface [15:44] Chipaca: I'd like to do at most one bikeshed iteration, land it, propose more, than iterate ad-infinitum on the language [15:44] Chipaca: I essentially want to land the first two patches there [15:54] niemeyer: I've already drop most things I could, trying to remove anything further leads to removal of base system components [15:55] morphis: Can you please paste "rpm -qa" somwhere? [15:57] scaleway non-compliant "ubuntu" kernel: https://github.com/RocketChat/Rocket.Chat/issues/7000#issuecomment-303668983 [16:11] PR snapd#3395 closed: many: remove interface meta-data from list of connections [16:12] Chipaca: I'll land the branch and change the things as suggested after opening another PR, thus completing the set [16:12] mvo: what is the state of the CE blocker? [16:17] pedronis, hmm, did the timing of the device initialization on firstboot change recently ? my hummingboard image shows some weird behaviour [16:17] after console-conf: [16:17] ogra@localhost:~$ snap list [16:17] No snaps are installed yet. Try "snap install hello-world". [16:17] ogra@localhost:~$ sudo reboot [16:17] and after the reboot: [16:18] ogra@localhost:~$ snap list [16:18] Name Version Rev Developer Notes [16:18] core 16-2.26.3+git209.2f19233 2020 canonical - [16:18] hummingboard-gadget 16.04-1 x1 - [16:18] hummingboard-kernel 4.1rc x1 - [16:18] ogra_: nothing that I know of [16:18] weird [16:18] i have never seen something like that ... either snap list is broken and stays that way all the time ... or it works right after console-conf [16:19] having it only work after a reboot is very weird [16:20] ogra_: snap changes should tell when things happend [16:20] yeah, i just dont know when exactly the reboot happened :P [16:20] ogra@localhost:~$ snap changes [16:20] ID Status Spawn Ready Summary [16:20] 1 Done 2017-05-24T16:16:11Z 2017-05-24T16:16:26Z Initialize system state [16:20] 2 Error 2017-05-24T16:16:24Z 2017-05-24T16:16:49Z Initialize device [16:21] (the error is indeed the serial ... i'm working with --extra-snaps atm) [16:21] well, it never finished [16:22] if it's in error [16:22] so it probably just retrying again and again [16:22] so sometimes there's no snap [16:22] and sometimes there are some [16:22] yeah, i see change 3 now ... same error [16:22] seems to loop [16:22] interesting [16:22] well, it errors [16:23] also erroring there might also explode [16:23] (which is something I managed to reproduce but need to look into) [16:23] http://paste.ubuntu.com/24645391/ [16:23] well, the stuff doesnt look unusual [16:23] ah, no [16:23] sorry [16:23] yes, that's expected [16:24] nothing to do with list [16:24] we are interesting in change 1 [16:24] not change 2 [16:24] i'm not canonical and kernel/gadget are sideloaded [16:24] when the bits of change 1 happened [16:24] whether something was unusually slow there [16:24] OOOH ! [16:24] interesting [16:25] s/interesting/interested/ [16:25] http://paste.ubuntu.com/24645393/ [16:25] the config hook [16:25] * ogra_ seens "namespace" in the error and blames zyga [16:25] :P [16:26] niemeyer: hm, lost the instance somehow, do you still see it? [16:26] ogra_: anyway the times there look immediate [16:26] bit confused about snap list [16:26] morphis: If it's around for more than two hours and there's contention for machines, spread will kill it [16:27] ogra_: I suppose you to look at those times vs reboots times [16:27] yeah [16:27] s/to look/have to look/ [16:27] i guess i need to start over for that and actually capture timestamps [16:27] ogra_: which kernel are you on? [16:27] indeed i didnt at the first run [16:27] zyga, "some" kernel ... :P [16:27] ogra_: 3.x or 4.x? [16:28] zyga, could well be missing some bits [16:28] 4.1 [16:28] I see [16:28] let me know if you need help next week [16:28] will do [16:28] * ogra_ is happy to finally have a working gadget ... sanitizing the kernel is next [16:29] getting uboot to DTRT was rally painful this time [16:30] ogra@localhost:~$ htop [16:30] cannot perform readlinkat() on the mount namespace file descriptor of the init process: Permission denied [16:31] yeah, definitely kernel bits missing [16:31] ogra_: can you look for dmesg | grep DENIED [16:31] May 24 16:16:25 localhost kernel: [ 58.405309] audit: type=1400 audit(1495642585.323:10): apparmor="DENIED" operation="ptrace" profile="/usr/lib/snapd/snap-confine" pid=1563 comm="snap-confine" requested_mask="read" denied_mask="read" peer="unconfined" [16:33] zyga, dont bother yet ... i havent touched the kernel at all yet and only went with what was there for this board ... i'm actually surprised it works so well [16:34] ogra_: you can help yourself out by editing /etc/apparmor.d/usr.lib.snapd.snap-confine.real (just copy it) [16:34] ogra_: you can switch snap-confine to complain mode [16:35] ogra_: and load it into the kernel with apparmor_parser -r [16:35] ogra_: let me know if you need the exact syntax of the file [16:35] why does it have that .rela suffix ? [16:36] *real [16:36] war scars [16:36] hahaha [16:36] yes [16:36] because dpkg bugs [16:36] some backward incompatibility, dependency issue [16:36] looong story [16:37] ogra_: just add ",complain" after "attach_disconnected [16:37] ogra_: that should fix it [16:39] zyga, so it does [16:39] htop starts [16:41] morphis: We seem to have machines available, and Spread-32 is still running with opensuse [16:41] morphis: Created 8h44 ago === grumble2 is now known as grumble [17:03] niemeyer: it was 61, need to shift that to next monday, running out of time here [17:13] is it known that cleanbuild is somewhat incompatible with version-script? http://pastebin.ubuntu.com/24645640/ [17:20] morphis: Sorry, I'm getting a bit lost.. you said 32 above, and 32 indeed has opensuse on it.. === JanC_ is now known as JanC [17:21] 11:01:35  niemeyer: Spread-32 is now down to 1.3G, ok for you? [17:35] zyga: hi, sorry for the slow reply. state of the blocker is that there are two branches up for review, I summarized the short term fix in https://forum.snapcraft.io/t/snapd-2-25-blocked-because-of-possible-revert-race-condition/722 - ideally we would have the input from jdstrand on this too. alternatively we could just revert both mknod |N and quotactl and then we just need to merge a single branch not the seccomp-resolver one (hope this explaina [17:35] tion makes sense?) [17:36] mvo: as it happens, I'm here today after all [17:37] jdstrand: aha, were you supposed to be off? [17:37] jdstrand: please let me know if the above forum post makes sense, there is a bit of a problem with the snap-confine syntax/symbols, its summarzied there [17:38] mvo: ok, so I can ignore backscroll and only look at the forum? [17:38] mvo: no worries :) [17:38] mvo: yes, I'll follow the post shortly [17:38] jdstrand: thats probably enough [17:39] jdstrand: unless the post summarizes the problem not well enough :) [17:40] ok, let me read it [17:41] mvo: oh, I thought there was something in place to say that snapd always needed to start before services. when did that change? [17:41] jdstrand: we only had that in 15.04 for services AFAIR [17:42] jdstrand: it got lost along the way, but even if we had it it would still be racy, because snapd needs some (small amount of) time to write the new profiles [17:42] commands shouldn't start before snapd if the login session is supposed to start after snapd [17:42] jdstrand: so we really need something that is ready only after snapd has completed that profile generation [17:42] mvo: one more idea [17:42] mvo: is to have snap run ask snapd to do this [17:43] mvo: if services start after snapd [17:43] mvo: then there's no need to have snap run not wake snapd up [17:43] mvo: it's a bit ... well, but it has some good things as well [17:43] mvo: like we could refresh on demand this way [17:43] mvo: or warn about CVEs that snapd synced [17:43] mvo: lots of ideas are possible [17:43] zyga: yeah, I think we discussed this but we don't want snapd in the critical path for snap run all the time [17:43] mvo: but we are adding it back now [17:43] mvo: after=snapd does that already [17:43] mvo: but the ordering solution does that anyway [17:44] mvo: unless I'm mistaken and after=snapd won't start snapd if it doesn't run already [17:44] mvo: then we have the problem again [17:44] mvo: (which is interesting in fedora land for instance, where we don't start on boot) [17:45] zyga, pedronis: yes, for daemons we would add this dependency. true. it would be better to avoid it IMO but I don't see a way currently [17:45] (well, seccomp bpf would solve it for seccomp profiles only but that is not a general solution) [17:45] well, isn't that the plan of having a profiles/current [17:46] mvo: one more idea is to laizly mount snaps if snap run talks to snapd [17:46] then the dep is just changing that symlink early [17:47] that I though was what we discussed the other day [17:47] ok, that was annoying. the power went out [17:48] pedronis: we did, to me its not fully clear yet if it solves the various cases. but I probably just have not thought enough about it. TBH I like the suggestion of snapd backup-of-state-and-profiles and restore that backup on revert more, at least its clearer in my mind how that would work and what the constrains are [17:49] zyga: if we put snapd into the snap run path, then yes, it is an interessing idea [17:49] zyga, pedronis: lets continue in the forum, I need to step out for some minutes and can only reply async [17:51] ok [17:51] * zyga is preparing for the event [17:57] PR snapd#3398 opened: env: set XDG_DATA_DIRS for wayland et.al === cachio_afk is now known as cachio [18:24] mvo: ok, I responded in the forum. ping me when you want to discuss more [18:43] PR snapd#3396 closed: interfaces: add summary to each interface [19:01] jdstrand: thank you! [19:21] PR snapd#3399 opened: many: add the interface command [19:22] Chipaca: ^^ [19:31] ogra_, are you still around? [19:32] zyga: could you review snapd#3369? [19:32] PR snapd#3369: many: make shell scripts shellcheck-clean === mup_ is now known as mup [19:43] Chipaca: yes [19:54] Chipaca: what's going on with GOPATH vs GOHOME? [19:54] zyga, do you know how long does it take to retry when it is downloading a snap? [19:54] zyga: GOHOME I made up [19:54] zyga: GOPATH is a PATH, ie a list [19:55] ah [19:55] zyga: we were using it wrong [19:55] right [19:55] zyga: so I either had to change all the $GOPATH to ${GOPATH%%:*} [19:55] or, what i did [19:56] (%% and not ## because in go the first element in the list is where new stuff goes) [19:58] Chipaca: done [20:01] Chipaca: thank you for the review, I tried to make it work well in all cases [20:01] Chipaca: question about tab completion, where can I test the tab completer returning not only the item but also the description? [20:03] Chipaca, do you know how long does it take to retry when it is downloading a snap? [20:03] cachio: we time out after some reasonable time (<< minute) [20:04] cachio: exponential, starting at 200ms [20:04] delay [20:04] no sorry starting at 100ms [20:04] and then 2.5× every time [20:04] cachio: it's in store/retry.go [20:05] Chipaca, ah, ok, thanks [20:07] zyga: anything you'd like changed, wrt your comments in the pr? [20:08] I need to repush anyway because of conflicts, so i can tweak something :-) [20:09] Chipaca: no, it is fine [20:09] thanks :-) [20:09] ok [20:24] zyga: the 32k limit on commandlines is rather dated, but it was bytes not lines of input [20:24] by-the-way :-p [20:26] Chipaca: yes, I know [20:27] Chipaca: but I think it's not dated, is it changed? [20:28] zyga: it's about 2M [20:28] I think it's around 2M [20:28] nowadays [20:28] zyga: unless you have a _lot_ of stuff in your environment [20:30] Chipaca: aha, I didn't know [20:30] * zyga works on slides === sergiusens_ is now known as sergiusens [20:39] * Chipaca works on riffs [21:05] PR snapd#3390 closed: tests: remove additional setup for docker on core [21:14] niemeyer: problem on my end is that I lost the connection the spread-32, looks like I removed a bit too much [21:15] niemeyer: best is if I redo the cleanup on monday and give you a fresh instance to snapshot [21:16] niemeyer: lost the connection = it got dropped from .spread-reuse.yaml so I don't have the connection information anymore [21:25] PR snapd#3369 closed: many: make shell scripts shellcheck-clean [21:26] niemeyer: ping [21:26] https://travis-ci.org/snapcore/snapd/builds/235777521?utm_source=github_status&utm_medium=notification [21:26] ^^ something is or was awry with spread/linode [21:30] Chipaca: Yo [21:31] niemeyer: just travis weirdness i hope [21:31] * zyga -> food [21:42] niemeyer, is it any way to run many times the same test with spread? [21:43] niemeyer, I mean, using the same node [21:45] Chipaca: Thanks, pretty strange indeed [21:56] zyga: thank you for pointing out the "git log --oneline" thing [21:56] zyga: it's looking better already (far from perfect still) [22:04] niemeyer: ok to restart that run? [22:15] PR snapd#3400 opened: many: stop "snap refresh $x --channel invalid" from working [22:16] cachio: Other than -reuse, there's no way to retry in a loop right now [22:16] Chipaca: Yeah, no much to do there [22:17] Chipaca: Something awkward seems to have happened.. hard to believe all allocations would fail at the same time silently [22:17] niemeyer: network hiccup? [22:18] Perhaps.. or something else made the process stop [22:33] cachio: Btw, we can easily extend spread to repeat the same test over and over until it fails [22:34] niemeyer, yes, I'll need that because otherwise I spend to much time retrying manually [22:35] niemeyer, I'll make the change here [22:35] niemeyer, most of the failures are very difficult to reproduce [22:36] cachio: Thanks, let me know if you need a hand on that [22:38] niemeyer, sure [22:56] PR snapd#3400 closed: many: stop "snap refresh $x --channel invalid" from working === Elleo_ is now known as Elleo [23:29] niemeyer, the change worked verywell [23:30] niemeyer, I am rexecuting 100 or until the test fails [23:30] niemeyer, i could reproduce an error by doing that [23:31] niemeyer, I am not sure if you are interested on add that change to the master, are you? === cachio is now known as cachio_afk