[00:56] PR snapcraft#1907 closed: tests: setup the correct environment for adt [01:13] cprov, any chance you're around? [01:17] kyrofa: hey, how can I help ? [01:19] Something interesting has come up and I'd like your advice. You're familiar with the format of the snapcraft credentials file? [01:19] Turns out there are two ways to utilize export-login, one anticipated, and one not [01:20] kyrofa: not by heart, go on [01:20] One way is to encrypt the credentials file, save the decryption key in CI, and use it to decrypt the file, then use the file to log in [01:20] (it's an ini file basically) [01:20] That ^ is the anticipated usage [01:20] Right [01:21] However, some folks here are building tooling that actually saves the individual macaroons out of the credentials file, putting THOSE into the CI environment, and then reconstructing the file itself in order to login [01:21] This works of course, but is quite fragile. What if we change the login URL, or any part of that file format [01:22] So I'm trying to think of a way to make it more... token-ish [01:23] PR snapcraft#1908 opened: tests: update tests to work in adt [01:24] But it's weird because we have a root macaroon and a discharge [01:24] (or however that works) [01:24] I dunno, I guess I just wanted your thoughts [01:25] It would be cool if people could just paste the entire creds into an environment variable instead of needing to encrypt a file [01:26] kyrofa: we can serialize/discharge it and have the authorization token [01:27] So is the discharged one the only one we need? [01:27] But i am not sure it is that easier to use [01:29] cprov, here's another use-case for something like that: https://pastebin.ubuntu.com/26503341/ [01:29] snappy-m-o autopkgtest 1908 xenial:armhf xenial:arm64 [01:29] I'm writing a Travis deployer [01:29] Computer says nooo. See logs for details: [01:29] Command '['/tmp/tmpb03l5uir/retry_autopkgtest.sh', '1908', 'xenial:armhf', 'xenial:arm64']' returned non-zero exit status 1 [01:29] cprov, that's an example of AWS, note how they handle the access keys [01:34] kyrofa: i guess we snacraft can consume the a env var with the final secret [01:35] cprov, what would that final secret be, though? The unbound_discharge that we have in the config file now? [01:36] cprov, sorry, still pretty dense on macaroons :D [01:36] kyrofa: no, we have to bind it [01:37] snappy-m-o autopkgtest 1908 xenial:armhf [01:37] Computer says nooo. See logs for details: [01:37] Command '['/tmp/tmp1bapez85/retry_autopkgtest.sh', '1908', 'xenial:armhf']' returned non-zero exit status 1 [01:37] It is called serialize_for_request(), iirc [01:38] deserialize_macaroon, maybe? [01:38] Let me find it, one sec [01:38] Ah, no: root_macaroon.prepare_for_request [01:39] kyrofa: yup, https://github.com/cprov/surl/blob/master/surl.py#L130 [01:40] Okay so if I save that `bound` value then, that's all I need? [01:40] Does that have the same expiration time as the macaroon? [01:41] kyrofa: not only the bound discharge, you also need the root to build an 'Authorization' header the store can validate [01:42] kyrofa: 'Macaroon root={}, discharge={}', then we can probably hex encode the text and pass it in as a ready to use secret [01:45] cprov, ah, okay gotcha [01:46] cprov, I think that's a good idea. And `login --with -` will read from stdin [01:47] cprov, thank you for letting me borrow your brain :) [01:48] kyrofa: good idea about allowing stdin for pipe-ing evn vars [01:48] env, even [01:49] Yeah that already works [01:54] kyrofa: the b64 encoded auth is about 4k, a bit too long to be a secret. That's the only problem I can see. [03:06] PR snapcraft#1909 opened: Add dotnet command to path and enable developer scenario for snap [04:09] PR snapcraft#1910 opened: tests: expect in-snap unsquashfs when using docker [04:13] PR snapd#4591 opened: interfaces/desktop-legacy,unity7: support gtk2/gvfs gtk_show_uri() [04:17] PR snapd#4592 opened: interfaces/desktop-legacy,unity7: support gtk2/gvfs gtk_show_uri() for 2.31 [04:36] PR snapcraft#1911 opened: Add support enable configurable runtime version for .NET Core applications [05:58] o/ [05:58] * zyga preps breakfast for kids [06:12] morning [06:16] hey [06:17] zyga: o/ [06:18] zyga: on the same note as this https://forum.snapcraft.io/t/spotify-cannot-create-tmpfs-target/3805/7 do we create /var/lib/snapd/lib/gl automatically? [06:40] mborzecki: no, we need to do that [06:40] I think we create one of those [06:40] but not all perhaps [06:40] and this needs to be rc3 IMO [06:40] I'm going to take a shower, let's discuss soon [06:40] look at snap-confine's tree that handles that [06:40] ok [06:41] let's add the mkdir's there [06:43] zyga: we should cover lib/gl, lib/gl32 and lib/vulkan then [06:47] all of the variants enumerated from the regular expressions in the apparmor profile [06:47] well [06:47] I assume there are variables for that [06:48] and that on any given system there's a fixed amount [06:57] right [06:57] zyga: will you be looking into this? [07:00] mborzecki: mmm, maybe, depends on timing [07:00] I need to discuss with mvo first [07:29] mvo: good morning [07:29] mvo: I have great news and bad news [07:29] mvo: pick :) [07:29] hey zyga [07:29] zyga: bad first [07:29] mvo: we need to add some code that creates new directories in /var/lib/snapd/lib/{gl,vulkan,...} [07:30] mvo: people are running into this if they use rc2 and have nvidia and reexec [07:30] zyga: *nod* [07:30] mvo: it's small but a little annoying that we found out this late [07:30] mvo: ikey can help I'm sure, with his array of hardware [07:31] mvo: the good news is that if we wait for monday (jdstrand is off today) we can get fully functional layouts tooo [07:31] though that is more complex than we initially discussed because of confinement [07:32] in any case, it will be ready today and it's an option [07:32] (maybe "layouts beta" [07:32] anyway, that's the news, how are you feeling today? [07:32] zyga: hm, we want to be in candidate by monday [07:32] so thats a tough one [07:33] so let's fix the bad news and play safe [07:33] we can be in edge with layouts [07:33] yeah [07:33] I'd love 2.31 but I agree that this is an added risk [07:33] mvo: perhaps we should disable layouts for 2.31 [07:33] he will also not be able to review before .eu evening, right? [07:33] (entirely with some early return) [07:33] I think he will, he is just traveling today [07:33] he was super exited about what we did last night [07:34] maybe over weekend, we'll see [07:34] we just realized we need per-snap snap-update-ns profiles [07:34] because otherwise snap-confine becomes unconfined [07:34] we discussed the details already and implementation is straightforward [07:35] my plan today is to get layous ready in master, do the per-snap snap-update-ns profiles done, write spread tests and merge that [07:35] if you want I can help with missing directories [07:41] zyga: help> would be great, just catching up with mails and stuff [07:41] same here :) [07:58] hey spineau [07:59] morning zyga [08:07] good morning everyone [08:08] mornings! [08:09] hey pstolowski [08:10] sorry I didn't finish that review yesterday, I wasn't feeling good and ended up being unproductive for first half of the day [08:10] pstolowski: I read the code and it was OK so far, I wanted to see if there's anything missing by checking the non-diff parts and stopped there [08:11] zyga, 4551? [08:11] yes [08:13] zyga, ok, thanks for looking at it. Gustavo had objections against not reusing "conns" in the state for that, but perhaps I explained my reasoning purely in the standup (and he didn't look at the code yet afaict) [08:15] zyga, feeling better today? [08:16] yes [08:16] my back is okay, not sure why it hurt so much yesterday [08:17] it's not like broken bones that hurt when weather changes [08:17] thank you :) [08:17] glad to hear! [08:18] grats on skype [08:19] PR snapd#4593 opened: advisor: ensure commands.db has mode 0644 and add test [08:30] hey ikey [08:30] :) [08:30] snaps are starting to get popular :) [08:30] ikey: any issues on solus, I cannot wait to get skype off from classic confinement [08:38] zyga, not tried the one in the store yet [08:38] got back home late yesterday [08:38] PR snapd#4594 opened: systemd, wrappers: start all snap services in one systemctl call [08:39] ^^ more thinking than coding :/ [08:39] zyga: about those lib/gl[32], lib/vulkan directories, shall i look into it? [08:40] * ikey perks up ears at mention of the one domain he looks at [08:40] mvo: pstolowski: ikey: morning guys [08:40] morning [08:40] o/ [08:42] mborzecki, might i query whats changing in gl/vulkan land? [08:42] ikey: https://forum.snapcraft.io/t/spotify-cannot-create-tmpfs-target/3805 [08:42] hey mborzecki [08:42] is there some missing stuff on the hostside? [08:43] i.e missing host dir [08:43] ikey: yup [08:43] so packaging fixes, gotcha [08:43] thought the dirs were changing :D [08:43] mborzecki: thanks for the PR [08:43] technically the vulkan stuff would be my fault, sorry guys [08:44] mvo: i'll try to check it with that guy's snaps, but i'm missing some of the stuff he added in his snapd fork (btw. it'd be great if he opened an PR :P) [08:50] mborzecki: mmm, yeah go for it please, I'm working on new profile generation [08:51] thank you for taking care of that [08:59] PR snapd#4595 opened: snap: improve validation of snap layouts [08:59] mvo: this one is for 2.31, it will ensure 2.32 won't reject snaps accepted by 2.31 (layouts) [09:10] moin [09:12] hey Chipaca ! good morning [09:15] Chipaca: morning [09:16] Chipaca, o/ [09:16] hey Chipaca [09:16] hm no luck with syncloud's platform snap either https://paste.ubuntu.com/26504652/ [09:19] mborzecki: you could try with layouts soon :) [09:20] *today* [09:20] you can remap /opt/disk to $SNAP_DATA/opt/disk [09:20] (or to snap common) [09:20] mborzecki: try with my branch from last night [09:23] fyi https://www.reddit.com/r/linux/comments/7ukrbq/microsoft_releases_skype_as_a_snap_for_linux/ [09:24] yay for skype snap ^_^ [09:26] btw there was a post on slashdot too [09:26] don't know if anyone visits the site still, but here it is https://linux.slashdot.org/story/18/02/01/1644250/microsoft-releases-skype-as-a-snap-for-linux [09:27] the year of linux desktop is upon us :P [09:27] :) [09:31] * zyga reads slashdot [09:31] mborzecki: haha, yes [09:31] now that we've had at 's' with releases we'll probably never get to see photoshop as snap, and i can't really think of any interesting application that starts with 't' [09:33] mborzecki: total commander ;-) [09:33] mborzecki: we'll wrap around and 'a' will come with adobe busyware [09:33] total commander - the 'ever' shareware [09:34] mborzecki: thunderbird :-p [09:34] zyga: i'd swap adobe for autodesk, see their A* suite running on linux [09:35] mborzecki: yeah, I remember using *commander style programs since forever essentially [09:35] when I was 9 and we got a 386 at home [09:36] haha, remember 'dos navigator'? [09:36] * zyga googles [09:36] maybe not by name [09:36] I didn't speak english back then [09:36] so things were "magic" [09:36] so the name doesn't ring any bells but 1st screenshot I see is "yep, used that" [09:43] mborzecki, mvo thanks for you comments to 4584, i've implemented your suggestions (and the aspects we discussed yesterday during standup) [09:46] pstolowski: nice, thank you [09:50] hmm so a friend of mine tried installing skype on 17.10: https://paste.ubuntu.com/26504804/ [09:51] mborzecki: server side overload? we see a lot of failures in spread as well [09:52] he got: 2018/02/02 10:39:47.590935 backend.go:152: cannot create host snap-confine apparmor configuration: cannot replace snap-confine apparmor profile: AppArmor parser error for /etc/apparmor.d/snap.core.3887.usr.lib.snapd.snap-confine in /etc/apparmor.d/snap.core.3887.usr.lib.snapd.snap-confine at line 11: Could not open '/var/lib/snapd/apparmor/snap-confine [09:54] is /var/lib/snapd/apparmor/snap-confine a directory? [09:54] it should be [09:54] ogra_: quick question about the gadget defaults for core> you say this worked for 2.30? are you sure? then I will check the diff between the 2.30->2.31, so far I have not found a smoking gun (but adding a test case right now :) [09:55] mvo, not 100% ... i have definitely checked it a week or two ago and there it was behaving [09:55] i'm not 100% sure about the exact version though [09:56] ogra_: how much work is it for you to run this again with stable (2.30)? if trivial it would be nice to double check [09:57] i currently have the system completely messed up (trying out grub-uboot atm) ... it would take a bit [09:59] hmm, tests are sad today [10:15] Chipaca: snapshots it's big, usually indeed we would split overlord, daemon/client and cmd/snap bits in different PRs [10:16] pedronis: yep, thus the commits [10:16] pedronis: having 4-5 RFC PRs didn't seem to me to be something that'd progress [10:17] I'm not expecting a deep nitpicky review of this, just a "yeah that's what it should look like" / "WTF are you on can i have some" thing [10:18] Chipaca: can I have some [10:18] it must be good :) [10:19] zyga: diclofenac + pridinol (but not too often) [10:19] ogra_: re 1746714> there seems to be typo there, in your gadget it should read "rsyslog:\n disable: true" - your gadget has disable*d* [10:19] oh man [10:20] * Chipaca ~> physio [10:23] ogra_: no validation? [10:23] zyga, obviously not [10:23] zyga: config schemas is on the roadmap for some time [10:24] for core we could (nowdays) probably "emulate" them [10:24] * mvo thinks a bit [10:24] well [10:25] * zyga feels like having a hangover, needs more water [10:25] even an error when it is attempted to be set would already help here [10:25] (weather is terrible lately) [10:26] zyga: you realize you just said you want to have a hangover? ;-) [10:26] zyga: should't the vulkan code be outside of nvidia specific pieces? [10:30] kalikiana: oh yes [10:30] it came out wront [10:30] wrong [10:30] see! [10:31] I feel like I have a hangover, not like I would like to have one [10:31] :-D [10:31] mborzecki: not sure, good question [10:31] * kalikiana coffee break, back in a bit [10:31] mborzecki: I think not [10:32] mborzecki: it's not about nvidia but about external drivers [10:35] oh the tests are very unhappy today, indeed [10:46] ogra_: I wrote a testcase and I think once the typo is fixed this will work [10:48] mvo: I don't know if we can error, but we could at least log a warning if we see core configs that we don't know about [10:48] niemeyer: quick question - we just had this issue that an invalid core config option was used, for core itself we could validate if its a supported option, do you think that is worth persuing? probably not hard, we just need to expose something like config.Transaction.ChangesKeys() or something [10:49] pedronis: yeah, that was what I was thinking [10:50] pedronis: also it kind of buged me a bit that we do not test setting core options right now in spread, unfortunately because core has no snap-id its not easy to test (I hacked a bit and with https://paste.ubuntu.com/26505078/ it works) [10:50] because of nesting is not super obvious what it means though [10:50] mvo: I wonder if we should have a special key for that could be used there and would work also for non store core [10:50] mvo: like $core or something [10:51] pedronis: oh, nice [10:51] pedronis: I like that idea [10:51] it also would solve the issue that core doesn't have the same id in prod and staging (not super important but it's an issue for testing) [10:52] anyway just an idea [10:53] pedronis: I can prepare the pastebin as a proper PR with $core [10:54] the other option would be to invent a general syntax to use name instead of id, it's a bit a pity if we go there because then it would have been saner to have a special syntax for ids instead [10:54] we will have the same issues with the connect functionality [10:54] btw [10:56] mvo: anyway $core is also interesting because is a bit unclear what should happen when we split core vs base or have core-16 etc [11:00] pedronis: excellent point [11:01] pedronis: could we use heuristics? i.e. is the snap-id longer than the allowed snap names? [11:01] I don't remember what are the rules [11:01] ids are fixed lentgh [11:01] I think [11:02] we don't have a size limit in snapd for names [11:02] pedronis: yeah, that is what I remember too, might be worth looking at. anyway, I make this $core thing and will ask about validation in the standup, nested values will be a bit nasty but iirc we don't use those for the core config [11:02] but I think there is one in the store [11:02] pedronis: yeah, thats what I remember, I had to shorten some test snaps [11:03] because the store was unhappy :) [11:03] I can check that [11:03] (in a bit) [11:05] pedronis: no rush. it looks like snap-id is 32char, I think snap names can be larger so a heuristic may not be easily possible [11:05] * mvo is slightly sad about this [11:06] anyway using ids was quite intentional I think, just a bit annoying [11:06] but core is a bit special because reasons [11:06] anyway something to discuss with Gustavo [11:07] * mvo nods [11:10] mvo: snap name max length is 40 [11:10] (in the store) [11:12] pedronis: ta [11:43] mvo: does this fail on your machine? [11:43] ---------------------------------------------------------------------- [11:43] FAIL: cmd_userd_test.go:62: userdSuite.TestUserd [11:43] cmd_userd_test.go:84: [11:43] c.Assert(err, IsNil) [11:43] ... value *errors.errorString = &errors.errorString{s:"cannot obtain bus name 'io.snapcraft.Launcher'"} ("cannot obtain bus name 'io.snapcraft.Launcher'") [11:45] PR snapd#4596 opened: osutil: allow using many globs in EnsureDirState [11:57] Chipaca: ^ perhaps for you, it's pretty short and I need it badly [12:00] * Chipaca looks [12:01] so, question [12:01] are the /etc/apparmor.d files meant to include /var/lib/snapd/apparmor/profiles ? [12:01] because it seems those guys will only be loaded when i start snapd [12:02] but if i snap run something, those profiles don't exist yet [12:02] i.e. AVC apparmor="DENIED" operation="change_onexec" info="label not found" [12:06] ikey: I think that the thing that loads profiles on boot looks at /var/lib/snapd/apparmor/profiles [12:06] yeah thats kinda the thing here [12:06] ikey: we reload them in snapd if needed but it should happen during boot to let services start without starting snapd [12:06] "the thing" is slow as ass [12:07] and massively regresses boot time [12:07] ikey: that's orthogonal, I'm explaining how it functions, not why it is slow [12:07] and the only "solution" is private to the debuntu packaging world [12:07] oh? [12:07] what solution is that? [12:07] your package hooks for apparmor [12:07] maybe I'm missing the point somewhere [12:07] and the boot load service [12:07] the /var/lib/apparmor/profiles path isnt referenced in the /etc/apparmor.d/* files [12:07] so they won't be compiled in with a boot time service [12:08] until snapd is started and does the reload you mentioned [12:08] but with socket activation, snapd.service itself wont be started [12:08] it should be referenced by the arcane /etc/init.d/apparmor script [12:08] and a snap run of some snap will fail [12:08] i use systemd. :) [12:08] /etc/apparmor.d is more like a /usr/lib/apparmor.d [12:08] yes, we too [12:08] /etc/init.d is a banned path in solus [12:08] so i know we dont have it [12:08] it's a shell script and runs with sysv compat generator AFAIK [12:08] so are nasty shell scripts during boot [12:08] lol [12:09] and that's what makes it slow for you perhaps [12:09] so [12:09] im adding a workaround [12:09] I think you want to write a small tool (that would go upstream) [12:09] that 1) loads and 2) caches profiles [12:09] by including the private apparmor profiles directory into our aot portion [12:09] ya ive done that [12:09] loading cached profile is very fast [12:09] way ahead of ya [12:09] cool, [12:09] but please share that with jdstrand and upstream, [12:09] https://github.com/solus-project/aa-lsm-hook [12:09] that script has some reason for not being a systemd job sadly [12:09] and it's probably some messy complexity [12:09] maybe it's just legacy complexity that can die [12:10] but maybe there's some genuine thing to care about and you'd just run into it all over again [12:10] working out the kinks with this in solus atm [12:10] jdstrand is off today (traveling) but next week we can sync on that [12:10] so we have a -compile and a -load binary [12:10] -compile is called during package operations [12:10] and -load at boot [12:10] which if it fails to load, will recompile [12:10] to handle kernel abi changes and such [12:10] which takes my whopping 1.3s regression to to 32ms [12:11] in my VM over half the boot time /was/ spent in apparmor.service ^^ [12:12] but now im doing the AoT/cache approach and making sure it doesn't bust snapd :P [12:12] yeah, it also doesn't help that that is a shell script and it wasn't a one liner [12:12] look at how it gets invalidated [12:12] one of the things that the old code cared for was apparmor base things gettingchanged [12:12] so all the #include files [12:12] this sucks IMO [12:12] right [12:12] and it could be nice to even ship _precompiled_ profiles and build-depend on the include files [12:12] so this is why our AoT is called by usysconf during package changes [12:12] but that's not how it's done in ubuntu [12:12] only if /etc/apparmor.d changes [12:13] yes [12:13] and then the boot uses the cache [12:13] and really /etc/apparmod.d is a messy zoo that should live in /usr/lib [12:13] yeah [12:13] and not in /etc for the vast majority of content [12:13] it's just in dire need of a gardener [12:13] /usr being shipped, /etc being local vendor/etc [12:13] yeah, exactly [12:13] * zyga wrote https://disqus.com/home/discussion/omgubuntu/it_just_got_a_lot_easier_to_install_skype_on_ubuntu/#comment-3738650041 [12:14] people still raging about getting the stuff they want? .. [12:14] yeah, but I think that unless we put accurate comments and explanations people will run off with the wildest conspiracy theory they can come up with [12:15] yeah true [12:15] snaps are from SATAN or whatever [12:15] 8ms aa-lsm-hook.service [12:15] thats more like it [12:15] wooot [12:15] thank you [12:15] I actually think that's something we could adopt in base18 [12:15] mvo: ^^ [12:15] we want empty /ec [12:15] https://hastebin.com/mukeneheha.coffeescript [12:15] _empty_ /etc [12:16] all loaded properly [12:16] oh [12:16] minecraft!!! [12:16] so for now we load /from/ etc path, *but* [12:16] * zyga snap installs it [12:16] we support optional paths [12:16] man snaps are really a double-edged sword [12:16] i mean my eventual thinking is we'd merge the hook into apparmor userspace after discussion [12:16] zyga, productivity [12:16] how do you stay focused when you can install cool software and use it :) [12:16] xD [12:16] yes, that sounds good to me [12:17] soooooo... I have a new computer now [12:17] we have some path magic here: https://github.com/solus-project/aa-lsm-hook/blob/master/src/lib/hook.c#L24 [12:17] oh? [12:17] feeling very tempted to open it ^_^ [12:17]