[02:15] PR snapd#7580 opened: spread.yaml: exclude vendor dir [02:29] PR snapd#7581 opened: tests: add netplan test on ubuntu core [02:36] PR snapcraft#2748 opened: meta: warn about command mangling [05:17] morning [05:20] PR snapd#7564 closed: interfaces: add cgroup-version to system-key [05:20] school run, back in 30 or so [06:02] re [06:02] mvo: morning [06:05] PR snapd#7568 closed: snap: when running `snap repair` without arguments, show hint [06:06] PR snapd#7567 closed: snap-repair: error if run as non-root [06:07] hey mborzecki [06:07] mborzecki: good morning [06:08] mvo: can you take a look at https://github.com/snapcore/snapd/pull/7571 ? [06:08] PR #7571: sandbox/cgroup: refactor process cgroup helper to support v2 and named hierarchies [06:09] mborzecki: uh, I think I wanted to do this yesterday already :/ [06:09] mborzecki: let me do this *now* [06:16] sorry, had super stubborn kids [06:16] one is still at home [06:16] ugh :| [06:17] ok, let's try to focus... [06:19] mvo: thanks! [06:19] mvo: https://twitter.com/samerhaj/status/1182121464813756419 [06:20] and check out the next tweet there, vmware working on this? [06:20] esxi for raspi anyone? [06:20] zyga: wow, nice [06:21] I, for one, would not mind a raspberry pi with a proper boot stack [06:21] zyga: any clue how does uefi/acpi combo could handle hats? [06:21] no idea [06:26] mborzecki: added a comment, please let me know if it makes sense or if its too terse. happy to talk here as well if you want :) [06:26] mvo: looking [06:28] mvo: i see, it's a bit yagni right now, once we land the named hierarchy, the string will no longer the be controller name, but insteadf a named hierarchy name ('snapd' in this case) [06:29] mborzecki: right, landing it is fine [06:29] mborzecki: I was mostly wondering if consumers of the ProcGroup() need to know what matcher to create [06:30] mborzecki: or if we could simply keep using a string here [06:30] mborzecki: and the string in the future might be "snapd" [06:30] mborzecki: or is this not how it will work :) ? [06:30] * mvo obviously has less context [06:33] mvo: hm you'd have to pass name=snapd to be explicit, or "" when you mean a unified hierarchy, feels a bit too magical [06:35] mborzecki: hm, but we know internally in the cgroup module if we are unified or mixed so could we leverage this knowledge? [06:35] mvo: we use cgroup v1 even in unified mode [06:35] so perhaps it's all going to be simplified eventually [06:35] s/use/going to use/ [06:37] ok, I don't want to rathole this too much, sry! it just seems to me right now right now we only have one user of "cgroup.ProcGroup" and we always use freezer here so I wonder if we could encapsulate the knowledge that its freezer/something-else inside the cgroup package itself maybe? [06:37] we could but the cgroup package doesn't know that, it's the overlord that knows this; I think it's fine as is, it will change shortly, perhaps before the next release [06:38] mvo: zyga: hm so perhaps we could have a separate call cgroup.SnapProcGroup which internally knows whether to ask for freezer or a named group? [06:38] it is my understanding that by the time we are done it will use a different selector or a hard-coded string for the new location [06:38] I really don't think we need to do more now [06:38] just land and iterate [06:38] meanwhile, I'm back to https://github.com/snapcore/snapd/pull/7576 [06:38] PR #7576: spread.yaml,.gitignore: sync ignored files [06:39] why does something so simple have to be so hard :D [06:39] zyga: userd is currently using it and it does not know about freezer either, does it? its just snap-confine that knows about freezer (or am I missing something?) [06:40] userd is using cgroups? [06:40] zyga: I did a "git grep ProcGroup" and a match there [06:40] zyga: yeah, it is, that's the next thing, we'll probably need some api that encodes information about how snap cgrups are made [06:41] zyga: fwiw, I disagree with "why does something so simple have to be so hard" [06:41] zyga: it's for checking whcih snap the talking to userd over dbus comes from [06:41] oh, I didn't notice that [06:41] I just read that code [06:41] mborzecki: sry again for lack of context - but do we need (external) api for how cgroups are made? mostly wondering if we can keep most of this knowledge internal to the cgroup package [06:42] mvo: (I was referring to the gitignore PR) [06:42] zyga: oh, sorry [06:42] zyga: then I misunderstood :) [06:42] mvo: ignoring in spread and git consistently is haaard :) [06:42] zyga: I thought it was about my bikesheding api here :) [06:42] and it should not [06:42] no no, sorry [06:42] zyga: git hard> makes sense ;) [06:42] mborzecki: I wonder if the pid usage in userd could go away [06:42] and use /proc/pid/root instead [06:43] but that's a separate conversation [06:43] we discussed this with james, as long term API for discovering snap processes [06:43] brb, need tea, coffee [06:43] I'll catch up on the thread here [06:43] mvo: we don't, that's why the cleanups are being done, afaik we have 3 places rightr now that made up the grup names themseleves, userd, snapstate/refresh and s-u-n [06:43] but mostly want to unbreak that gitignore branch [06:43] * zyga loves the acronym s-u-n [06:44] mvo: 'groups are made' as in how the path under the relevant group mount point is constructured for a snap :) [06:44] mborzecki: oh, this is spread out right now? yeah, a good reason to consolidate it :) [06:47] mborzecki: anyway, my concern with the PR as it is is just that the current code in userd has cgroup.ProcGroup(pid, "freezer"). that seems to require knowledge in userd about freezer that should probably not be needed there. the new code now also adds the fact that its a v1controller there. so it seems like it makes this slightly worse. I think we can land but we should reverse the direction in the next prs to require less knowledge about how exa [06:47] ctly we do cgroups outside of the cgroup package. does that make sense? [06:49] back with coffee [06:50] mvo: I agree that eventually we should instead have an API that refers to app / hook tracking [06:50] and similar pid discovery [06:50] and encapsulate the cgroup logic there [06:50] I still would merge this as-is and iterate because velocity and not-worse-than-before [06:51] mborzecki: last night I needed to sober up after looking at 22MB binaries [06:51] so I started making those few-kb, built-with-C, hello-world programs that don't need libc [06:51] that was refreshing [06:52] offtopic: I found a great use for raspi zero -- it's the cheapest reliable serial logging device I've had [06:53] just plug a wire and log away [06:53] mvo: right, i mean it makes sense, we should be able to figure everything out in cgrup package given the security tag [06:53] (hopefully) [06:53] mborzecki: I would not even use cgroup as the wrapper [06:53] we may end up not using cgroups eventually [06:54] the package should be conceptual - pid tracking and discovery, it may use cgroups internally [06:54] zyga: snaptrack? :P [06:54] but what it should really have is functions like PidsOfSnap and SnapOfPid or similar [06:54] pidof ;-) [06:54] mvo: since you're doing reviews already, can you take a look at this one too? https://github.com/snapcore/snapd/pull/7578 again it's moving some pieces scattered around into the cgroup pacakge [06:54] PR #7578: sandbox/cgroup, overlord/snapstate: move helper for listing pids in group to the cgroup package [06:55] ok [06:55] back to gitignore [06:58] mborzecki: 7578> sure - the title is promising for sure! [06:58] mvo: hope the content doesn't disappoint :P === pstolowski|afk is now known as pstolowski [06:59] morning [06:59] pstolowski: hey [06:59] hey pawel :) [07:04] * zyga reads how git does .gitignore [07:06] /o\ [07:06] git has a screen full of global variables [07:15] mborzecki: love it [07:15] mborzecki: I would even go a bit further and hide pidsCgroupDir from the outside [07:16] mborzecki: but fine as is [07:21] PR snapd#7571 closed: sandbox/cgroup: refactor process cgroup helper to support v2 and named hierarchies [07:47] mborzecki: so I had a look at those exclude patterns [07:47] and I have the following compromise [07:47] mborzecki: git needs /foo to exclude a file ./foo won't cut it [07:47] mborzecki: spread's use of tar is exactly the opposite [07:47] hahah [07:48] mborzecki: it requires ./foo and /foo won't cut it [07:48] I think it's fine if I document it [07:48] it's better then sending garbage [07:48] and better than not working [07:48] I don't see any way out of this for now [08:22] * mvo has some eoan problems today and regrets upgrading to it [08:23] mvo: what happens? [08:23] mvo: I'm running on a similar-ish thinkpad and apart from suspend resume killing input (ha ha) it works okay (sob) [08:25] fitting that you regret upgrading to eoan in the morning [08:26] zyga: its on my desktop, sound issues, stuttering and it always select the wrong output on login so I need to go to gnome-settings etc [08:26] mborzecki: oh god [08:26] mborzecki: tar has --exclude-vcs-ignores [08:26] mborzecki: it reads .gitignore directly [08:26] can I just path spread instead? [08:27] mvo: man, that sucks :/ [08:27] mvo: it's the future though, better bite the bullet and report them [08:28] zyga: doing that but its a bit of a time sink this is why I regret it [08:29] indeed [08:29] Chipaca: hahaha [08:29] Chipaca: took a while until the penny dropped [08:32] mvo: no regrets [08:32] zyga: hm can we teach spread about --exclude-vcs-ignores then? [08:32] I just did locally [08:32] it'll take a while to land it though [08:33] it doesn't seem to work [08:33] hahahah [08:33] running a debug session to see if the files really got copied [08:33] it's driving me insane [08:33] I hate doing this, it breaks all C testing I work on [08:33] well, i'm trying to teach postrm and snap-mgmt about the snap.mount unit :/ so i feel you [08:34] * zyga hugs mborzecki :) [08:35] btw. noticed something intersting in the tests, we've supposedly stopped lxd service (and i guess any containers it ran), but then removing /var/snap/lxd/comon/ns/mntns fails with EBUSY [08:35] are you sure it stopped containers? [08:36] well, i guess i hope it did [08:40] --debug doesn't work if project prepare fails [08:40] WAAAAAAAAAAT [08:40] ah, no, spread is picky where --debug is [08:40] man [08:43] mborzecki: the plot thickens, the ignore patterns work [08:43] but it's removal that matters now :D [08:43] man, dpkg, you sure are picky [08:46] zyga: what're you fighting against? I think I missed that bit [08:47] Chipaca: being able to spread a tree with built files while also passing debian build package thing that hates them [08:47] Chipaca: almost there though [08:47] now that I see what the error message is [08:48] zyga: ah [08:48] ondra: if/when you have more time to poke at that slow boot, given your last pastebin, the next step would be to do a profiling of the slow step [08:49] Chipaca I did some progress with more logs [08:49] Chipaca in the meeting now, but will report back [08:49] k [08:57] zyga: there was a snap.mount unit before? [08:57] mborzecki: I got it :) [08:58] mborzecki: so... back to your question [08:58] there used to be but we nuked it and got the snap.mount.service [08:58] AFAIR [08:59] mborzecki: why? [08:59] mborzecki: but I don't recall exactly why [08:59] mborzecki: gobs of complexity [08:59] mborzecki: note there's also a snapd-generator [08:59] mborzecki: look at cmd/snapd-generator/ [08:59] it creates a mount unit AFAIR [08:59] looking at some postinst helpers, and there's a comment about snap.mount in 2.31 [08:59] I think we used to have snap.mount [08:59] deb postinst [09:00] but it was too "strong" [09:00] so we moved it to the generator conditionally [09:00] or something like that [09:00] and the snap.mount.service is a separate beast for $REASONS [09:00] I honestly don't recall, I'd have to dig [09:00] * Chipaca steps away for a bit (washing machine beeping) [09:01] zyga: no need, just curious [09:04] one more place and I think this is good [09:09] mborzecki: heh, now the delta code sends the crap again [09:09] or is it... [09:10] I don't see it in the tree anymore [09:10] but I do see it in the tarball we make early on [09:34] ehh debhelper [09:37] https://bugs.launchpad.net/snapd/+bugs?field.status%3Alist=NEW is a thing of beauty [09:38] Chipaca, try that on the ubuntu package :) [09:38] * Chipaca kickbans seb128 [09:38] sorry *g* [09:38] mborzecki: it's fun, disabled the repack helper for a second and I see tar is still ask to send the stuff over [09:38] :*) [09:38] mborzecki: maybe it's a bug in tar [09:38] checking [09:42] yeah [09:42] tar is not respecting those [09:42] looking deeper [09:42] man this used to be easier [09:44] Chipaca: no open bugs [09:45] Chipaca: \o/ [09:46] mvo: no open bugs? [09:46] did we just close them for 5 minutes to feel better ;) ? [09:46] zyga: thats what LP told me [09:46] " [09:46] There are currently no open bugs. [09:46] " [09:46] zyga: in https://bugs.launchpad.net/snapd/+bugs?field.status%3Alist=NEW [09:46] zyga: its a lie of course, it should be "no open bugs in this filtered view" [09:47] :D [09:47] I see it now [09:47] but I like the other string better [09:47] :) [09:47] 369 open bugs make a man feel honest about himself and that to err is human [09:47] being a software developer humbles me every single day^Wminute [09:48] zyga: #1761621 sounds like fun for you [09:49] mmm [09:49] zyga: it's private because of a stacktrace, i presume, let me know if you can't see it [09:49] I can [09:50] I'll look later todaty [09:50] ah i see you in the list, even [09:57] anybody with better stacktrace-reading chops than myself (basically all of y'all), #1762686 is a puzzle [10:01] hah! I'd forgotten the "Ubuntu was created for the red-eyed onanists engaged in anal balancing act" [10:08] I wonder if there's a non-escalating way of telling them that was unappropriate [10:08] inappropriate, even [10:13] That isn't just obvious ban-fodder? [10:13] mvo (and maybe cachio when he's around): anything we should do about #1734845 ? [10:13] Bug #1734845: hook (core) configure → exit status 1 cannot create lock directory /run/snapd/lock → Permission denied [10:14] cjwatson: I don't know the policy, nor the tools available to enforce it; maybe i should, if i'm going to start calling people out on this kind of stuff [10:14] and I am going to call people out on it [10:14] Was this on Launchpad? [10:14] yes [10:14] I failed to find it [10:15] cjwatson: #1818584 [10:15] Bug #1818584: snaps applications can't open files on an USB key

[10:16] OK, your response is probably appropriate for what seems like a first offence from a frustrated person. If they escalate let me know and we'll take administrative action [10:17] cjwatson: will do [10:26] mvo: your input (or your redirect to somebody else) on #1747735 would be good [10:26] Bug #1747735: command-completion for 'snap' on 14.04 does not recommend snapd

[10:27] Chipaca: in a meeting, will get back to you [10:31] a bug for snapd/ubuntu: gnome display manager it's slow with devuan [10:32] * Chipaca is nearly WATed out [10:32] mborzecki: debugged it [10:32] maaaan [10:32] zyga: ok, i think i have worked around dh_systemd_start, but heh it's ugly [10:33] Chipaca, stop complaining, just fix it ! [10:34] (its just a quick port to sysvinit to fix that devuan problem) [10:34] ogra: in the _description_ of the bug they say "it was only solved by installing the Nvidia drivers." [10:35] ogra: so the bug is... probably? that nouveau drivers are slow? [10:35] ah, then it is easy, just make the nvidia drivers a hard dep of snapd [10:35] chocolate por la noticia [10:36] (how would it work on devuan *at all* ? there is no systemd, logind or any other of the lennartisms in it) [10:40] PR snapd#7578 closed: sandbox/cgroup, overlord/snapstate: move helper for listing pids in group to the cgroup package [10:49] PR snapcraft#2743 closed: meta: warn about command mangling [10:52] PR core18#140 closed: run-snapd-from-snap: check for snapd.service existing too [10:53] Chipaca hi [10:54] Chipaca I added more logs and essentially this is all down to sha384 :( [10:54] ondra: I feel like you owe me a beer or something for not believing me [10:54] Chipaca as when we preparing seed, we calculate sha384 while we are parsing seed.yaml [10:55] ondra: would've saved us both a few hours of work :) [10:55] Chipaca I'm happy to get you beer :) [10:55] Chipaca but it's not key generation, this is snap assertion validation [10:55] ondra: strange that you don't see the io or cpu usage of it though, maybe things aren't lining up properly in the bootchart? [10:56] Chipaca also interesting one is, if I calculate it from shell, I get 100% cpu load and I get it also 2 as fast for all snapc [10:56] ondra: _before_ even looking at key gen etc i said it was probably still the sha3 [10:56] :-/ [10:56] Chipaca I loop through all snaps, and it's not taking me as much time [10:57] Chipaca ah, sorry for that, I did miss sha3 message from you [10:57] Chipaca I did not know we calculate sha3 before we start seeding, I though we do that at time of actual snap seeding [10:57] * pstolowski walk, bbiab [10:57] Chipaca but cpu load is indeed something puzzling [10:58] PR snapd#7580 closed: spread.yaml: exclude vendor dir [10:58] ondra: look for an email with subject 'booting the pi', 26 Jan 2017 [10:58] Chipaca: snap on trusty - oh well [11:00] mvo: ikr [11:00] Chipaca ahh nice, found it === asymptotically is now known as Guest37136 === asymptotically2 is now known as asymptotically [11:03] sil2100: thanks for merging the latest core18 PR from ijohnson ! our of curiosity, when do we plan the next stable core18 snap release? [11:04] sil2100: we have this fix and the netplan update in edge, would be great if those would make it to stable soon(ish) [11:04] ondra: wrt looping through all the snaps: did you unmount them and drop caches first [11:04] ondra: it's possible a large chunk of it is i/o [11:04] depends on how slow the sd card is maybe [11:04] Chipaca no I did not umout them [11:05] ondra: i'm assuming having them mounted keeps them in the cache somehow but could be wrong [11:05] ondra: also, what are you using to hash them from shell? [11:05] mvo: yw! In theory today's a Thursday, so if the timing is right it should get picked up by automation into beta and then to certification testing - we might be able to get it to stable around Monday [11:05] sil2100: amazing, thank you [11:05] Chipaca may be some, but I doubt entire snap and all, that would be half of all mem available [11:06] ondra: because sha3-384 is not sha384 [11:06] Chipaca I just called sha384sum [11:06] ondra: yeah, that's the wrong hash [11:06] Chipaca ah [11:06] that's an easy hash :) [11:06] Chipaca which one to use then? [11:06] ondra: snap install sha3384 ? [11:07] pedronis: 7556 has two +1, my two nitpicks can easily be done in a followup, I would love to see this landing so that the next pr is smaller, wdyt? [11:09] re: https://discourse.ubuntu.com/t/testing-default-snap-apps-against-equivalent-deb-apps-slow-start-times/12875/17 the desktop launcher script runs gdk-pixbuf-query-loaders and gtk-update-icon-cache and gio-querymodules.. could we utilise the $SNAP_DATA or $SNAP_COMMON directories for these caches and generate them in an install and/or refresh hook? [11:10] PR snapcraft#2746 closed: elf: handle missing dependencies not found on system [11:15] Chipaca yep. now time alines more [11:15] Chipaca damn it, no cheap wins :( [11:16] diddledan: is there a way to do it incrementally so a user can also have local stuff? [11:17] those caches only parse files that are distributed within the snap or runtime-snap so the user won't have their own local stuff, AFAIK [11:18] then why is it done at install at all? [11:18] why isn't it in the snap itself? [11:18] anyway, ⇝ lunch [11:18] it is in the snap currently as part of desktop-launch script which wraps the actual command [11:19] I was thinking we could extract it from running every launch to only running once when the snap is installed or refreshed [11:20] at the moment it conditionally runs on refresh at launch time. so we can move it to refresh-time so that the initial launch after a refresh is perceviably faster [11:22] i.e. in the launch script it has equivalent of `if refreshed, rebuild caches, then run the app` [11:23] mborzecki: did you had a chance to look at the error log in 7572? [11:24] PR snapd#7572 closed: gadget: rename "boot{select,img}" -> system-boot-{select,image} [11:30] yeah, but it dodn't make any sense though, if it keeps repeating we'll probably need to take a closer look at either glibc or go in arch [11:30] mvo: ^^ [11:32] mborzecki: ok, thats fine. lets assume its just a glitch [11:33] mvo: might be serious though, afaict it was related to thread local storage [11:33] mborzecki: hm, ok [11:37] zyga: tests/main/lxd is leaking this: https://paste.ubuntu.com/p/zkY96DMsT6/ [11:38] zyga: or, iow lxd is leakingthis [11:38] i suppose the debug output should include findmnt [11:38] that's curious [11:38] the common/ns/mntns thing is a bind mounted mount namespace [11:38] I really really wish lxd removal would clean up [11:39] mborzecki: can we paper over this with lxd-tool patch? [11:39] mborzecki: tests/lib/bin/lxd-tool [11:39] though it's more serious as it seems to break actual "snap remove", right? [11:39] zyga: we can, stil, removing lxd snap does not clean this up [11:39] yup [11:42] PR snapd#7582 opened: spread: include mounts list in task debug output [11:42] zyga: ^^ [11:49] mborzecki: +1 but would you mind changing that to co via prepare-restore.sh --debug-each and move the code there. This way more shellcheck love is applied. [11:50] zyga: this is covered by spread-shellcheck already [11:50] debug: | ? [11:50] zyga: i mean spread.yaml [11:50] hmm let me double check [11:52] mborzecki: https://github.com/snapcore/spread/pull/89 [11:52] PR spread#89: runner: pass --exclude-vcs{,-ignores} to tar [11:52] zyga: yup, debug && debug-each are covered by spread-shellcheck [11:52]