[00:23] PR snapd#4471 closed: cmd/snap-update-ns: refactor and improve Change.Perform to handle EROFS [00:32] PR snapcraft#1897 closed: docker: user proper tags in Readme.md [00:50] PR snapcraft#1898 opened: docker: don't rely on snapcraft-classic [00:53] kyrofa: the fixes are merged, please try edge and report any weird stuff [00:53] kyrofa: rc2 tomorrow [00:53] kyrofa: (early morning) [00:53] so you can be on beta to see it [00:57] PR snapd#4561 closed: tests: generic detection of gadget and kernel snaps [00:58] zyga, you got it, thanks man :) [00:59] Yeah looks like RC1 now, when I see RC2 I'll give it a shot [01:04] kyrofa: tomorrow morning [01:04] I'm baby sitting the last PR [01:05] PR snapd#4559 closed: snap-confine/nvidia: Support legacy biarch trees for GLVND systems [01:49] PR snapd#4342 closed: userd: add support for a simple UI that can be used from userd [01:52] i wonder what's going on here https://launchpadlibrarian.net/355232767/buildlog_snap_ubuntu_xenial_armhf_go-tip_BUILDING.txt.gz [01:52] i bet it's made an arm64 binary for some reason [02:00] mwhudson: hey [02:00] PR snapcraft#1898 closed: docker: don't rely on snapcraft-classic [02:00] zyga: hello [02:00] exec format error, never saw that one [02:00] yeah, feels like wrong arch [02:00] (but how?) [02:00] no idea [02:00] mwhudson: offtopic, can you please look at https://bugs.launchpad.net/snapd/+bug/1745584 - maybe it's just a missing policykit file in packaging? [02:00] Bug #1745584: pkexec dialog doesn't appear for 'snap install' on Debian [02:01] * zyga should go to sleep [02:02] zyga: mmm could be [02:02] mwhudson, i dunno, it doesn't work in centos even with the polkit file installed [02:03] yeah, looks like the polkit file is installed in debian too [02:03] data/polkit/io.snapcraft.snapd.policy /usr/share/polkit-1/actions/ [02:03] same goes for Fedora [02:03] I've been installing it for a while, no effect [02:04] mwhudson: wait, this bug is about "snap install"? [02:04] snap install doesn't query polkit at all [02:04] like, zilch [02:04] only GNOME Software does [02:04] if that's supposed to work with "snap install" too, then this is horrifically broken [02:07] Hey tyhicks, if you're around, what is the status of using errno instead of kill for seccomp denials? [02:07] mwhudson: https://src.fedoraproject.org/rpms/snapd/c/98b119302ae0d8d502f20877e0ecbf76a64ac1c9 [02:07] yeah, I've been installing it and it doesn't have effect :/ [02:09] kyrofa: hey - it sounds like it is stuck waiting for a libseccomp-golang release and for other distros to include that release :( [02:09] kyrofa: https://github.com/snapcore/snapd/pull/3998/#issuecomment-358028579 [02:10] PR #3998: snap-confine, snap-seccomp: utilize new seccomp logging features [02:10] kyrofa: I haven't had a chance to look into falling back to KILL if the available libseccomp-golang isn't new enough [02:10] all underlying work upstream and in Ubuntu is done [02:11] tyhicks, excellent, thank you very much :) [02:11] no problem [02:12] I wish it was already working but there were a lot of moving parts [02:12] Oh totally, glad to see it moving along though [02:13] We were just chatting about it at the sprint and none of us knew where the work stood. So now we do! [02:22] kyrofa: either you or zyga could poke the maintainers of libseccomp and golang-github-seccomp-libseccomp-golang about backporting these things to current Fedora releases if it's ready to go [02:22] (since both of you have FAS accounts, you're also able to do PRs against the packaging repos for both of these [02:22] ) [03:00] PR snapcraft#1899 opened: elf: readelf dependency [06:15] morning [06:52] mborzecki: morning [06:52] Chipaca: hey [07:57] Chipaca: sorry :) [07:59] o/ [07:59] drat [07:59] I wanted to sleep [08:00] mborzecki: :-) [08:00] Chipaca: fwiw will looking at ProcessState be enough? [08:00] last night tests were utterly broken on networking [08:01] let's see how if today they fare better [08:01] mborzecki: not really, i think using a flag variable is the only way [08:02] mborzecki: the command just knows it was killed, not by whom [08:02] Chipaca: right [08:02] Chipaca: otoh maybe it's not worth it, unless the caller does something silly with the error the way it's now should be fine [08:03] * zyga has zero PRs open, that's ... new [08:04] mborzecki: that was what I thought when I wrote it, but then I thought maybe we want to behave differently if the Run error is context.Canceled [08:04] Chipaca: imo adding a comment instead of a flag will be ok too :) [08:04] good morning [08:04] hey paweł! [08:04] zyga: pstolowski: morning [08:05] https://launchpad.net/ubuntu/+source/snapd/2.30+18.04 ftbfs on most archs [08:08] mborzecki: it'd look like https://pastebin.ubuntu.com/26493696/ [08:08] doko: thank you for the heads up, we'll look into it [08:11] mornin' [08:11] hey kalikiana [08:12] zyga: now given back. but seriously, hit the uploader for each day and one arch that this wasn't given back ... [08:17] Chipaca: right, so there's still a race but at least you'll get ctx.Err() if cancel was called or timeout was hit [08:18] mborzecki: I _could_ check that if the flag is set, the exec error looks like a 'killed' error [08:18] * Chipaca bbiab [09:00] PR snapd#4570 opened: release: snapd 2.31~rc2 [09:40] * Chipaca grins as he writes TestRunRace [09:41] mvo: Chipaca: hi, what's the status of #4459 [09:41] PR #4459: snap: add support for `snap advise-snap pkgName` [09:41] pedronis: what do you mean? [09:42] * Chipaca looks [09:42] it seems to be sitting there [09:42] since a while [09:42] but is not blocked [09:42] huh, i thought this was merged [09:42] see [09:43] pedronis: the status is it only has one +1 [09:43] Chipaca, pedronis I think it should get a ack/nack from gustavo [09:43] i guess :-) [09:43] mvo: it was not marked as such though [09:43] I suppose I cand do that [09:44] Chipaca, pedronis it might be considered as redundant functionality. thanks, if you mark it that would be great, otherwise I will do it [09:44] mvo: I requested a Gustavo reivew on it [09:44] mvo: redundant with what? [09:44] probably needs a poke also [09:46] Chipaca: redundant with snap advice-snap --comand [09:47] Chipaca: oh, hold on a sec [09:47] Chipaca: I'm looking at the wrong PR, I think this one is uncontroversial mostly [09:47] heh [09:47] too many PRs [09:47] sorry, I was looking at the `snap run` one that advises on available commands [09:47] pedronis: yes :( [09:48] mvo: I still recommend(very gently) trying to be a bit more agressive about closing PRs, otherwise it gets confusing [09:48] pedronis: agreed, let me start right now! [09:49] close as in get merged or close as in close, whatever makes sense [09:49] PR snapd#4424 closed: corecfg: pam_env can only handle 1024 byte [09:49] to be clear [09:50] :+1: [09:50] * Chipaca hugs mvo [09:50] PR snapd#4477 closed: snapenv: add SNAP_ARCH_TRIPLET [09:52] o. [09:53] (which is like o/ but with me being more sleepy) [09:53] mvo: so I woke up at 7 ... and ... yeah [09:53] man [09:53] 4521 needs a second review probably an easy win [09:53] zyga: I woke at 4 [09:53] zyga: neener neener [09:53] zyga: good morning (and you looked at this already) [09:53] zyga: good morning [09:53] haha :) [09:53] zyga: rc2 is branched and uploaded to the ppa, waiting for the build [09:53] that's what peeking into scary finances does to you [09:54] excellent [09:54] I wonder if openoffice is building right now, build slots are in short supply it seems :/ [09:54] Chipaca: I was merging things at 3am [09:54] Chipaca: :( that sounds scary [09:55] zyga: i saw you restarting things at my 1am, so yeah :-) [09:55] Chipaca: yeah, travis and spread didn't love each other last night [09:56] very ironc that 2.31~rc2 is only build on powerpc yet (which used to be one of the slowest arches) [09:56] ;D [09:56] mvo: no spectre for ppc [09:56] heh, yeah [09:56] i was wondering if it is build on a 32bit chroot on ppc64el or something, that would also explain it [09:56] anyway [09:57] mvo: someone just pressed the "turbo" button [09:58] haha [09:59] * zyga remebmers his 386 now [10:01] * Chipaca -> physio [10:01] mvo: yes, pretty sure powerpc builds in a chroot on a 64 bit box now [10:01] ha! [10:01] thanks mwhudson [10:03] Kernel version: Linux bos02-powerpc-008 4.4.0-103-powerpc64-smp #126-Ubuntu SMP Mon Dec 4 16:57:45 UTC 2017 ppc64 [10:03] PR #126: Ensure squashfs snaps keep all uid/gid and permissions on build [10:04] ogra@localhost:~$ htop [10:04] snap-confine has elevated permissions and is not confined but should be. Refusing to continue to avoid permission escalation attacks [10:04] hmm [10:04] * ogra_ reboots the board [10:04] hmmm? [10:05] edge ... not sure if i tinkered the istall to death here or if there is actually a prob in the core snap from tonight [10:05] ogra_: let me know if it happens, I can look at my boards [10:05] Chipaca: hm, silly question. when we run command-not-found (snap advise-snap --command echo) - should we return test-snapd-tools because we have echo in there as a command? even though its not a toplevel? [10:07] mvo: btw, doko said that snapd 2.30 builds are failing [10:07] 09:05 < doko> https://launchpad.net/ubuntu/+source/snapd/2.30+18.04 ftbfs on [10:07] most archs [10:07] zyga: checking [10:08] mvo: no, i don't think so, not unless it's got an alias [10:08] zyga: fwiw https://launchpad.net/ubuntu/+source/snapd/2.30+18.04 is all green afaict [10:09] indeed [10:09] no idea, not very helpful today [10:09] (yet) [10:10] mvo, zyga: yes, given back. you should not upload and run away, but read your build failure messages [10:11] but maybe snaps don't have those anymore ... [10:12] * mvo wonders why he did not get those [10:12] mborzecki: pushed a tweak to RunWithContext, including a test for the race [10:13] and now yes, to physio [10:15] Chipaca: thanks, will take a look [10:17] PR snapd#4571 opened: data, cmd, packaging: use autotools to generate artifacts under data [10:17] zyga: ^^ [10:17] zyga, i had temporary disabled the apparmor init script on that device (because apparmor stalls the boot for ~1min), apparently snapd doesnt create the snap-confine (i think on core it simply should do that) [10:17] *doesnt create the profile for ... [10:18] that actually seems the only thing this init script is needed for [10:19] ogra_: it does and that thing you disabled loads it [10:20] zyga, well, it seems to be the only profile this is needed for ... IMHO snapd could simply invoke the parser directly on core installs [10:21] core does not have any other profiles outside of snap apps ... where snapd already takes care correctly [10:21] ogra_: it has to be done on boot [10:21] ogra_: by that script [10:21] that script compiles and loads all the profiles (including snapd generated ones) [10:21] i understand that it has to be done o boot for classic installs ... [10:21] no [10:21] on all installs [10:22] I mean, snapd doesn't handle that at all [10:22] but on core there is only one profile that gets generated [10:22] which is the snap-confine one [10:22] no, all the per-snap profiles are handled by that guy [10:22] we could remove it but it's ... already doing it [10:22] hmm, i dont see it doig that [10:22] and there's no clear reason why that would help [10:22] (look at /var/lib/snapd/apparmor [10:22] that's also loaded by the script [10:23] well, the profiles are generated during snap install [10:23] just not for snap-confine [10:23] yes but on reboot they are not loaded [10:23] ogra_: I'm telling you the script does useful work on boot, if we drop that script we have to put some thing on early boot, before _any_ snap can start [10:24] ogra_: I'm not sure that should be snapd; it could be but I don't see a clear advantage [10:24] ogra_: not on boot speed for sure [10:24] well, thats different to what i see :) but i belive you :) [10:24] ogra_: there's a variable somewhere that makes it look at /var/lib/snapd/apparmor/profiles [10:24] ogra_: we also load profiles on startup if they _changed_ [10:24] (all snaps functioned fine for the last week while the script was off ... only a core refresh made snap-confine fail then) [10:25] ... but probably the profiles simply never changed in that time despite me randomly installing/removing7refreshing apps [10:26] anyway, re-enabling the script gets me 3min boots back but also makes snap-confine work again ... so its a user error [10:26] ogra_: how slow is it btw? [10:26] Chipaca: ppc does suffer from Spectre, FWIW (maybe not the actual 32-bit-only chips, I'm not sure, but certainly more modern ones) [10:26] maybe there is something wrong? [10:26] cjwatson: hey, I still _run_ one :) [10:26] zyga, lol ... yes ... thats what i'm trying to find :P [10:27] ogra_: maybe add a echo to the script (or something like that) [10:27] ogra_: so that we know when it starts and when it stops [10:27] ogra_: not sure if that's clearly readable from logs [10:27] (it is a 200-500MHz 256M ram board ... but 3mins are definitely to long even for that) [10:27] ogra_: so, there's a cache thing there but maybe it's broken and not used [10:27] systemd-analyze is usually pretty correct in its timings [10:27] ogra_: normally apparmor should read the profile, check with the cache and insert a cached compiled blob if one is recent [10:28] yeah, the cache is definitely empty on all my boards [10:28] ogra_: otherwise it would compile the profile, which is slow, save the cache and load it in the kernel [10:28] jamie pointed that out before [10:28] ogra_: not sure how much of that is done in intrd [10:28] ogra_: and how much in early systemd [10:28] (seems there is some general issue with the cache ) [10:28] ogra_: oh? [10:28] initrd doesnt do any apparmor stuff [10:28] only the mounting [10:28] there are two cache locations [10:28] for some historic reason AFAIR [10:28] did you check both? [10:28] well, jamie pointed me to /etc [10:29] look at /var/cache/apparmor [10:29] and that one is definitely empty on all boards i have [10:29] yeah, there are files [10:29] see if it makes any difference [10:29] even with a hand watch [10:29] if you remove them [10:29] if so then the cache works [10:29] if not ... well, slow boards are slow [10:29] well [10:30] snapd probing mmcblk1rpmb devces for assertions takes quite some time too ;) [10:30] there are some obvious issues (like this one) ... but they are not enough to get me to a sane boot speed yet [10:30] how much? [10:31] dunno, several tens of seconds [10:31] 15-20 i'D say ... thats sadly a service that doesnt log anything ... [10:31] is that blocking boot or does it happen in parallle? [10:31] *parallel [10:31] (i havent debugged that one yet, its an obvious one) [10:32] others are 30sec for mounting loop devices [10:32] thats more worrying [10:32] console-setup 20sec ... keyboard-setup 15sec ... [10:33] the device probing is blocking boot [10:34] ogra_: could you run systemd-analyize plot and copy the output somewhere? [10:35] well, the system is havily modified atm ... i'll do that once i flashed it freshly ... here is a top list of blame though https://paste.ubuntu.com/26494246/ [10:35] ogra_: the probing issue [10:35] ogra_: I honestly think it should not be a toggle [10:35] ogra_: but there should be a way to tweak this in the gadget [10:35] zyga, ? [10:36] ogra_: so that you can use udev tagging to blacklist media [10:36] ah [10:36] yeah [10:36] ogra_: so that certain internal devices are just skipped [10:36] ogra_: it's a simple system, extensible to devices and makes sense as a design IMO [10:36] well, you never ever want to probe mmcblkXrpmb or mmcblkXbootX [10:37] ogra_: I don't think that's true, that's absolute and those rarely turn out true [10:37] but thats also something systemd needs to learn, not only snapd [10:37] ogra_: thanks, the plot would be nice at some point but not urgent [10:37] PR snapd#4572 opened: mir: software clients need access to shared memory /dev/shm/#* [10:37] ogra_: as long as you can set an attribute that makes it skip probing and you can express that in the gadget it feels sufficient [10:38] zyga, right ... systemd doesnt have such a flag yet ... which is why i ignored that bit for now for snapd as well [10:38] i'll get to that later ... [10:38] thats one of the bits that are understood :) [10:38] ogra_: it's not a systemd flag [10:38] ogra_: it's an udev attribute [10:38] (i have many in this particular boot process that arent :) ) [10:38] ogra_: like MM_SKIP_PROBING (or whatever it was called) [10:41] zyga, hmm, i only know UDISKS_IGNORE ... but that wouldnt help with systemd fsck or snapd autoimport [10:41] ogra_: fsck I cannot speak of, those are separate issues [10:41] ogra_: but the job that tries to look for assertions is under our control [10:41] ogra_: so it could just skip devics with a given attribute [10:43] zyga, would be helpful if the autoimport udev rule could mention the variable in the comment somehow ... how would porters know about it ? [10:44] (without reading snapd's source) [10:44] ogra_: sure that's documentation; I'm describing a feature that doesn't exist [10:44] oh [10:44] ogra_: it's just something that we would doucment in device maker's book or similar [10:45] i thought you said there is "MM_SKIP_PROBING" [10:45] ogra_: I'm saying that such an attribute, IMO, makes more sense than a big bool flag for _users_ [10:45] well [10:45] ogra_: that was a reference to existing similar attribute for modem manager and serial ports [10:45] that flag will only make the udev rule skip [10:45] it wont prevent the unit fro running [10:45] *from [10:46] yes but we could make that unit run the prober with a device, so that control is reversed [10:46] systemd picks the devices, prober probes one [10:46] most customers i worked with til now want that stuff completely off so people cant tinker with USB sticks [10:46] those could then even run faster as they would happen in parllel [10:46] ogra_: that's okay, but it should be a gadget decision, not a toggle for users [10:46] it simply wastes cycles, even if it is a no-op [10:47] yeah [10:47] indeed it should be a gadget thing [10:47] (like rsyslog or disabling ssh access) [10:49] though here the issue is more of a subsequent thing ... i want the deices to be hidden completely during boot so fsck will ignore them too ... [10:50] (autoimport is just the next thing in the chain here ... ) [10:57] * zyga -> physical activities [10:57] ogra_: I'm so mentally tired I will do some house chores to vent my mind [10:57] ogra_: I agree on fsck [10:58] ogra_: though I would discuss it separately [11:07] cachio: good morning! 2.31~rc2 for armhf/arm64 is ready, the other ones are still building [11:11] mvo, good morning, already downloading images [11:11] cachio: great, I publish the other ones as soon as I can [11:11] mvo, great [11:17] * zyga hopes for one smooth release [11:17] 2.30 was not too bad [11:18] mvo: going out with rc2 would be awesome :) [11:19] cachio: i386/amd64 ready now as well [11:25] * zyga refreshes to beta [11:25] mvo: oh [11:25] mvo: we didn't do one thing [11:26] that piece of code that wrote the snap.mount unit (under proper name [11:26] we don't have that [11:26] for reexec [11:26] but I guess it's okay for now, as long as packages are coming out, not a regression in any way [11:27] zyga: yeah, it will only be fixed in lxd hosts with the right snapd version [11:27] zyga: eh deb/rpm [11:27] zyga, pstolowski there is a bugreport that we should document that hooks use dash - I wonder if we should just switch them to bash. wdyt? [11:28] oh, please dont [11:28] pstolowski: do you think you could check 1745015? [11:28] ogra_: why not? [11:28] (dealing with a 200MHz 256M single core device here ... such a switch would have some impact) [11:28] mvo: is there any mandate for the hooks to be anything? they are just executables, could be dash, python or elf [11:29] bash eats about 5Mb while dash lives in the kilobyte range [11:29] (and used to be massively slower as well, back when debian did the research ... that might have changed since 2007 though) [11:30] zyga: there is no mandate for this [11:30] mvo, assuming you dont mean live-build hooks here but app hooks [11:31] mvo, interesting. will look into 1745015 [11:31] ogra_: snap hooks, yes. well, fair enough. [11:31] mvo, most upcoming ubuntu core customers in the near future will be in a similar HW range, every byte of ram counts here [11:32] mvo, I agree with zyga; i think documenting is enough [11:32] pstolowski: ok [11:32] (and on single core-low-speed cpus also every herz :) ) [11:32] s/herz/hertz/ [11:39] mvo, oh, during my debugging i noticed that "systemctl list-dependencies" on core devices shows quite a bunch of red dots, i wonder if we need to handle some units more gracefully or need to disable them [11:40] i.e. "systemd-machine-id-commit.service" ... completely useless since we set the machine-id from the initrd ... but it gets run nontheless [11:41] or systemd-hwdb-update.service ... (the hwdb is readonly) [11:41] mvo: I think the thing that bug reporter is missing is that /bin/sh is the default in normal Ubuntu too [11:41] mvo: err /bin/sh -> dash [11:42] they've obviously just configured it differently [11:46] cjwatson: wrt spectre on ppc, I was thinking exclusively of the pre-2006 chips (have there been 32-bit ppc chips since then?) [11:47] also I thought powerpc meant 32 bits, and the newer ones were ppc64le or w/e [11:48] yeah but you said ppc which is a bit ambiguous [11:49] ah, my bad then (i meant powerpc i think) [11:49] in my defence my only powerpc machine was a nubus-ppc one [11:49] there've certainly been embedded 32-bit chips since then, and e.g. the e600 has out-of-order execution, branch prediction, and such; wouldn't surprise me if you could get something like Spectre to work but I don't know if anyone's bothered [11:51] * zyga starts to see the light at the end of the tunnel :) [11:52] my office gets messy quickly [11:52] somebody's got to be trying to get spectre to work on a router [11:52] mborzecki: you around? [11:53] Chipaca: yeah, trying to go through #4551 [11:53] PR #4551: ifacestate: do not auto-connect manually disconnected interfaces [11:53] mborzecki: just a quick Q: your version of the if in #4569 implies two class assertions, right? [11:53]