[05:57] PR snapd#7745 opened: snap-bootstrap: add go-flags cmdline parsing and tests [06:25] morning [06:29] Hey [06:29] zyga: hey [06:30] hey zyga [06:31] zyga: taking swap days? [06:32] Need to decide when first [06:32] mvo: how is your jet lag today? [06:32] zyga: horrendous [06:33] I feel I can only sleep for two or three hours at a time [06:33] zyga: couldn't sleep most of the night [06:33] Barely slept last night [06:33] zyga: same here, super nasty [06:33] I had dinner at 4am [06:34] I need to sort out my stuff and pick up the laptop from service [06:34] I will do H-BS-R training today if you don’t mind [06:34] zyga: lol@training [06:34] zyga: sure, go for it [06:35] zyga: I try to start with some simple uc20 things, like the initramfs mount helper stuff we talked about with xnox, I hope to have an initial PR up today [06:36] zyga: want to talk about the mount thing from last week later today? (i'm sitting at mcdonald's right now, car in the shop) [06:54] mborzecki: yeah [06:54] I have a 121 at 10:00 [06:54] When would be a good time? [06:55] zyga: 12? [06:55] Sounds good [06:55] zyga: i should be back home around 1030 hopefully [06:55] I need a moment before I start work today [06:56] Maybe an hour of sleep [06:58] hmm wonder whther cachio managed to update the arch images on friday [07:02] PR snapd#7739 closed: tests: moving arch linux to unstable [07:10] duh, our spread images of debian sid have fwupd-refresh.service enabled :/ [07:10] mborzecki: there are prs for this [07:10] mborzecki: 7742 fixes debian to be less unstable [07:11] mvo: right, though, we don't really need fwupd running :P [07:11] mborzecki: yeah, it seems to be running on a lot of the images [07:11] mborzecki: but yeah, I'm all for killing it :) [07:11] mborzecki: arch works currently btw, the degraded test is disabled there. but yeah, revreting this would be nice [07:12] mborzecki: fwiw, 7745 is green (fresh from this morning) and should be a super simple review *nudge* :) [07:12] mvo: already got it opened in a tab [07:12] mvo: going trough ian's PR first [07:15] mborzecki: sure, there is no hurry for mine, the followup will be more interessting hopefully [07:28] mvo: think we can land #7742 ? [07:28] PR #7742: tests: reset failing "fwupd-refresh.service" if needed [07:30] mborzecki: yeah, lets unblock the world and then fix it [07:30] mborzecki: I think sergio can just remove the package from the images hopefully [07:31] PR snapd#7742 closed: tests: reset failing "fwupd-refresh.service" if needed [07:32] mvo: yeah, not much use for fwupd in the cloud, though there might be a spread test that checks fwupd, so just disabling/masking fwupd-refresh should be enough [07:44] mborzecki: +1 [07:44] mborzecki: ok [07:51] mvo: https://paste.ubuntu.com/p/8f2MyCMSdr/ duh [07:57] mborzecki: fun [07:57] mborzecki: I think there was a recent merge for some motd-fixes, looks like the fix does not work. tests ftw! === pstolowski|afk is now known as pstolowski [08:02] morning [08:10] pstolowski: hey [08:14] hey mborzecki [08:14] mvo: welcome back! [08:14] hey pstolowski - good morning! [08:15] oh wow, i really love my flight option to frankfurt, and it is cheap [08:29] pstolowski: heh, wish they didn't kill all flights to/from LCJ [08:29] off to pick up the car and driving back home [09:48] PR snapd#7737 closed: overlord: mock device serial in gadget remodel unit tests [10:18] re [10:21] welcome back mborzecki [10:29] still wondering why people would have both docker.io and docker snap installed on a single host [10:29] mvo, zyga, i see some very weird system-files interface behaviour here ... i tried locally building an app using system-files giving access to a file in /etc/systemd ... connected the interface wrote the file ... purged the snap ... then i changed the snap to use a layout instead and dropped the interface completely ... the layout worked and didnt complain [10:30] mvo, zyga, trying teh same snap on another machine rightly complains that layouts in /etc/systemd affect the host ... yet there is no system-files interface to be seen anywhere [10:30] interesting question in the forum: https://forum.snapcraft.io/t/ubuntu-core-vs-yocto-and-resinos/5266/13 [10:30] i.e. the interface doesnt seem to have been disconnected when i purged the package on the original machine [10:30] and snap conenctions does not show it anymore [10:31] PR snapd#7746 opened: snap-bootstrap: implement "snap-bootstrap initramfs-mounts" [10:32] ... but i can still fully access the file on this machine [10:32] xnox: hey, are you around today? [10:41] ogra: uh, that sounds like a interessting question - maybe pstolowski can have a look, I'm way too jetlagged today. or I can dig later this weeek [10:46] mvo, ogra sure, i can take a look. ogra, can you grab state.json and apparmor profile of that snap from the original machine? can you share the snap so i can try it? [10:58] pstolowski: some comments under #7686 [10:58] PR #7686: systemd: handle preseed mode [10:59] mborzecki: ty === mborzeck1 is now known as mborzecki [11:05] PR snapd#7730 closed: spread: fix typo in spread suite [11:09] pstolowski, https://github.com/slimjim777/logsync/pull/1 i first added the top three commits ... the issue occured with the fourth commit of this PR (when i found i can use a layout (which wasnt really true)) [11:09] PR slimjim777/logsync#1: some fixes [11:10] ogra: ack [11:11] PR pc-amd64-gadget#24 opened: gadget.yaml: do not use CamelCase for names [11:12] pstolowski, https://people.canonical.com/~ogra/snappy/state.json [11:14] ogra: uh, that reveals your macaroon, better take it down [11:15] oops [11:15] or logout [11:15] yeah, and logout [11:31] cjwatson, i know there was a bug for this but i can not find it (though it might be a new issue) ... https://forum.snapcraft.io/t/getaddrinfo-enotfound-github-com-builds-failing-on-snapcraft-io-due-to-network-problems/14164 [11:31] (seems npm doesnt pick up the proxy settings on the lp builders) [11:36] ogra: That just means some bit of their build system is ignoring the configured proxy [11:36] ah, so its an electron-packager bug then [11:37] Not specifically npm - you can see that registry.npmjs.org requests are working just fine and getting 200 responses [11:37] k [11:37] It'll be some random node module's postinstall script probably [11:37] Maybe electron-packager, I'm not sure [11:38] I'll respond on the forum [11:38] thanks ! [11:44] mbriza: hiya. [11:46] popey: hi! i'm getting the 'system does not fully support snapd: cannot mount squashfs image using "squashfs": mount' error [11:46] What OS? [11:46] but i can't find the reason it's failing actually, there's just about 5 different possible reasons listed in the error message [11:46] fedora 31 [11:47] is there a verbose mode or something [11:48] there is SNAPD_DEBUG=1 and SNAPD_DEBUG_HTTP=7 for debugging store conversations. [11:49] systemd.setenv=SNAPD_DEBUG=1 might do it? [11:50] well ... does that kernel actually have squashfs support ? (grep squash /proc/filesystems) [11:51] ogra: yes, i googled it for a bit and i made sure squashfs and loop devices are available [11:52] also, does it support loop devices ... (ls /dev/loop-control ... or ls /dev/loop*) [11:52] ah, snap :) [11:53] popey: https://paste.centos.org/view/0919773d this is the output when running install with SNAPD_DEBUG=1 [11:53] i also added it to the snapd service and restarted it but there doesn't seem to be anything very interesting in the journal [11:53] ok, so you can ignore the reexec issue [11:54] i'm also running with selinux in permissive mode to be sure [11:54] are you able to manually mount a random squashfs file? (like your own home-created snap)? [11:54] did you try to manually mount a snap ? [11:54] haha [11:54] you type too slow :) [11:54] * ogra shuts up now [11:55] yes, i can [11:56] which instructions are you using to install snapd? [11:56] https://snapcraft.io/docs/installing-snap-on-fedora [11:56] oh, also `snap version`? [11:56] snap 2.42.1-1.fc31 [11:57] ok, installing fedora 31 on my big machine to test [11:57] i had a completely different set of issues on my f31 machine on friday though... i was at least able to install snaps :D [11:58] this is a different machine [11:59] yeah, very odd [11:59] well, virtualbox is broken for me on 19.10, so that stops that. Bah [12:00] i thought you said you'D install on your BIG computer ! https://pl.wikipedia.org/wiki/Cray_X-MP#/media/Plik:EPFL_CRAY-I_1.jpg [12:00] mbriza: what version of snapcraft built the snap you're having trouble with? [12:00] not that big, but it's heavy :) [12:00] popey: i can't install any snap [12:01] ok [12:01] not even "snap install hello-world" [12:01] yup [12:01] https://www.entroware.com/store/athena - big machine :D [12:01] ok, this is more serious. Do you have a funky kernel, or a stock Fedora supplied one? [12:02] popey, yeah, but it has a keyboard on the seat ! [12:02] nothing funky, not even closed source graphics drivers or anything [12:02] i just updated the machine this morning [12:02] hmmm [12:02] do you know what exactly got updated ? [12:03] (and did it work at any point in time on that machine) [12:03] basically everything, i was gone for almost 3 weeks :) i didn't have snap installed for quite some months but i remember it working at some point in the past [12:04] i think i removed it either because of a package conflict between upgrades (i upgrade rather soon in the fedora devel cycle) or disk space [12:06] Looks like I need virtualbox test build to get this working on linux 5.3. On that then installing fedora 31 and updates to see what happens here. Will take a while. Sorry. [12:07] np, i'll just lurk here or on the forums to work this out [12:07] actually, i can install clean on my x220. will do that from a usb key. [12:07] anyway, there's nothing too funky about neither of my systems, i think i'm fairly capable of keeping the system in a workable state [12:08] are there any conflicts with virtual machines or something like that? [12:08] i'm running something in qemu-kvm most of the time [12:09] no, I run in all manner of distros in vm's all the time [12:10] mbriza: I'm sorry if you already answered this: can you mount a snap by hand? [12:10] Chipaca: i can [12:11] mbriza: what does dmesg say when you do? [12:11] Chipaca: [ 2973.344224] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop1, type squashfs) errno=-22 [12:11] but i'm in permissive mode [12:12] the error message from snap itself is extremely vague, listing like 5 different possible reasons why it could be failing... pretty hard to debug from my side [12:12] mbriza: mount is very vague with its errors [12:13] (I'm inclined to agree with that. The error about re-exec not working should be informational, but it doesn't make that clear) [12:13] mbriza: which is why that check exists in the first place (getting that vague an error during install is even worse0 [12:14] mbriza: are you also able to mount a snap by hand if you put that snap in /tmp ? [12:14] popey: which error about re-exec not working? [12:14] https://paste.centos.org/view/0919773d [12:14] Chipaca: yes [12:14] that's mbriza's output [12:15] (I am still confused why you're getting a _multi_ snap) [12:15] ah [12:15] it should be amd64 for your machine. Must be erroneous binaries in there somehow [12:15] popey: you get multi when you have architectures: [amd64,i386], don't you [12:15] i can fetch the other one but both of them worked on the laptop [12:15] and both of the machines are amd64 [12:15] popey: and you'd have that architectures when it was i386 binaries for example [12:16] and then again, this is failing even when i try to install hello-snap from the store [12:17] mbriza: that error you get is not about the snap [12:17] mbriza: that error is from a sanity check performed before any operation is attempted [12:17] mbriza: did you go into permissive mode after starting snapd? [12:17] i restarted snapd multiple times in the meantime [12:19] mbriza: as I say, that error comes from a sanity check, and what it's trying to do is just a 'mount -t squashfs /tmp/' [12:19] mbriza: and that is failing, with the error from mount as reported to you [12:20] oh, oh [12:20] also maybe in selinux systems there's a context= appended? [12:21] mbriza: I don't know selinux enough to know what context that is [12:21] maybe mborzecki knows [12:21] it's trying to access a file called 'snap-failure' [12:22] How ironic :) [12:22] which is i guess an error reporter? [12:22] that's a thing you don't need to worry about :-) [12:22] it's an auto-reverter for snapd [12:23] for (core, or) systems with reexec [12:24] is setting permissive mode a runtime thing (not an selinux user here) ... i.e. did you reboot after setting it to make sure the kernel picked it up ? [12:24] there are 3 selinux alerts for the time of attempting to install a snap and all 3 point to this so i guess that's not relevant [12:25] ogra: permissive mode is a runtime thing, selinux won't block anything in this mode but it will report detected... uhh, breaches? don't know how to call it [12:25] mbriza: at a guess, we're adding a context= to mount, because selinux.ProbedLevel() isn't "unsupported", and that is getting into trouble because of it being permissive [12:25] ok [12:25] mbriza: but I know nothing about selinux :) so it's a question for mborzecki [12:25] Chipaca: i can run in enforcing if that's a problem [12:26] still the same error [12:26] mbriza: if you add -o context= to the mount of the .snap, does it still work? [12:26] that was really quick for enabling enforcing + restarting snapd + trying to install something [12:27] Chipaca: the daemon is not really running, it's systemd socket-activated [12:27] anyway, with gibberish in the mount command i'm getting the same error as above [12:30] mbriza: hm what about selinux? [12:30] ogra: hmm i cannot reproduce the issue you saw. on same host i got "cannot update snap namespace: cannot write to "/etc/systemd/journal-upload.conf" because it would affect the host in "/etc/systemd" when trying your snap with layouts, after installing and then removing the one with system-files interface [12:30] ogra: i'm on 19.10 [12:31] mborzecki: to be honest i'm not sure what the issue is - there's a lot of repetitive information but the gist is i'm not able to install any snap on fedora 31 because `mount` fails [12:31] not even sure if it's related to systemd, it fails regardless of running in permissive/enforcing mode [12:32] ogra: i suspect something got stale on your box but it's unclear how. your state.json looks sane, it doesn't have system-files connection [12:32] err i mean selinux [12:32] mbriza: reading backlog, looks like snappy_snap_t isn't defined? that'd be weird it's part of the policy shipped with the pacakge [12:33] ogra: is your first box still in this state (e.g. the snap with layout succesfully installed)? [12:33] mbriza: that's 31 right? [12:33] mborzecki: yes [12:33] mbriza: ok, let me check rawhide first [12:35] pstolowski, i'm on 16.04 here and i think the machine is still in that state ... [12:37] ogra: good. could you get me /var/lib/snapd/apparmor/profiles/snap.logsync-james.upload profile? [12:40] mborzecki: just reinstalled the snap packages and rebooted to be extra sure everything is up to date but install is still failing [12:40] mbriza: same thing in dmesg as you posted before? [12:40] yes [12:41] pstolowski, just to prove it still works ... https://paste.ubuntu.com/p/7vtsXM3Kvm/ ... one sec, getting the file [12:42] pstolowski, and the apparmor profile ... http://paste.ubuntu.com/p/gz5DfScfQ7/ [12:42] ogra: did you use --devmode or try at any point? [12:44] only in the very first run i think ... then i removed the snap, re-built, re-installed and connected system-files ... then dropped system-files for layout, removed/purged the snap and installed with the layout === ricab is now known as ricab|lunch [12:44] ogra: ok, so the profile has /etc/systemd/journal-upload.conf mrwklix, from the layout; but it doesn't have sytem-files interface anymore in the profile [12:44] PR snapd#7738 closed: overlord/state: panic in MarkEdge() if task is nil [12:45] it's weird it didn't complain [12:45] right ... [12:45] it complains if i copy the snap to another machine (also 16.04) [12:46] ogra: yep, got it. interesting. ok, i think i need to look at different place in the code now.. thanks [12:46] should oh also ... [12:47] ogra@anubis:~/datengrab/logsync:master$ sudo snap remove --purge logsync-james [12:47] logsync-james removed [12:47] ogra@anubis:~/datengrab/logsync:master$ cat /etc/systemd/journal-upload.conf [12:47] [Upload] [12:47] URL=http://www.example.com:19532 [12:47] -should [12:47] shouldnt the file be removed whan the snap gets purged ? [12:50] mbriza: installing core snap [12:51] ogra: --purge is only for avoiding automatic snapshots, nothing else [12:51] well, yeah ... but remove should remove the file, or not ? [12:52] mbriza: and it installed fine, ok, so we need to debug your system [12:52] mbriza: anything useful ausearch -m AVC ? [12:52] i.e. i dnot think a layouted file should stay behind when removing the snap that used the layout [12:53] that will result in all kinds of cruft after a while [12:53] (at least in the case where the layout actually created the file from scratch) [12:54] ogra: yes i think you're right, i'm not sure why layout isn't cleaning after itself [12:54] mbriza: while at it, does seinfo -t |grep snappy_snap_t list anything? [12:55] ogra: something for zyga (maybe there was a reason it's like that) [12:56] mborzecki: ausearch doesn't report anything new when running sudo snap install hello-world [12:56] mborzecki: snappy_snap_t is not present in the output of seinfo .-t [12:56] -t [12:57] ogra: one more question.. on the misbehaving box, is snapd at the current stable version? [12:57] checking [12:57] ogra: same version of snapd on both boxes? [12:57] mbriza: rpm -q snapd-selinux show anything? [12:57] ogra@anubis:~/datengrab/logsync:master$ snap version [12:57] snap 2.42.1 [12:57] snapd 2.42.1 [12:57] series 16 [12:57] ubuntu 16.04 [12:57] kernel 4.4.0-141-generic [12:57] kernel is a bit behind ... but the rest should be recent [12:58] the other box uses the hwe kernel but beyond this the same versions [12:58] mborzecki: snapd-selinux-2.42.1-1.fc31.noarch [12:58] mborzecki: there's this line in the journal though [12:58] mborzecki: Nov 18 13:37:27 localhost.localdomain kernel: SELinux: Context unconfined_u:object_r:snappy_home_t:s0 is not valid (left unmapped). [12:58] mbriza: seinfo -t snappy_home_t ? [12:59] mborzecki: Types: 0 [12:59] pstolowski, the main difference is that the system-files interface was never connected on the box that behaves correct [12:59] uhm [12:59] (well, i also have never used --devmode with this snap on the behaving machine) [13:00] mborzecki: https://paste.centos.org/view/e51f0cc4 this is in the journal too [13:00] fwiw, i just installed f31, and updated it to latest, and can install and run snaps fine [13:01] wow, my really sucks today [13:02] mborzeck1: https://paste.centos.org/view/e51f0cc4 this is in the journal too [13:02] mborzeck1: also as for the output for seinfo-t snappy_home_t, it was Types: 0 [13:02] mbriza: right, so looks like the policy wasn't loaded for some reason [13:03] mbriza: ok, can you try dnf history list snapd-selinux and then dnf history info ? === mborzeck1 is now known as mborzecki [13:05] mborzecki: there's this little line in there, yes: [13:05] mborzecki: 16 /usr/sbin/semodule: Failed on /usr/share/selinux/packages/snappy.pp.bz2! [13:05] mborzecki: and i get the same error when trying to reinstall the package [13:06] mbriza: yeah, i suppose if you do semodule -l |grep snappy there'll be nothing listed there [13:06] mborzecki: semodule -l doesn't even start [13:06] libsemanage.semanage_direct_get_module_info: Unable to read flatpak module lang ext file. (No such file or directory). [13:07] now this is not related to snappy at all [13:07] mbriza: hm, did you upgrade 30 -> 31? [13:07] yeah [13:07] dnf reinstall *selinux* [13:07] mbriza: did you do distro-sync afterwards? [13:08] mborzecki: i'm distro-syncing all the time [13:08] ogra: ok, i know what's going on and why it didn't work on second box [13:08] mbriza: tried with --allowerasing? [13:08] \o/ [13:09] mbriza: btw. you need to use sudo/su to run semodule [13:09] yeah, of course... seems there's something wrong about my selinux policies though [13:09] ogra: it only succeeds with layout-based snap if you already have /etc/systemd/journal-upload.conf in place (leftover from previous snap). trespassing-detection code is then happy. otherwise it complains. i'll pass it to zyga to decide what to do with iy [13:09] *it [13:10] yay, thanks [13:10] mbriza: can you post the output of sestatus somewhere? [13:10] so let me remoe that file ... to prove it ;) [13:10] mborzecki: https://paste.centos.org/view/2e2c930d [13:11] ogra@anubis:~/datengrab/logsync:master$ sudo snap install --dangerous logsync-james_0.2_amd64.snap [13:11] error: cannot perform the following tasks: [13:11] - Run configure hook of "logsync-james" snap if present (run hook "configure": [13:11] ----- [13:11] cannot update snap namespace: cannot write to "/etc/systemd/journal-upload.conf" because it would affect the host in "/etc/systemd" [13:11] snap-update-ns failed with code 1: File exists [13:11] -----) [13:11] yay ! [13:12] yup ... touching the file before snap install then makes it work again [13:14] mbriza: doesn't feel like it's related to snapd policy in particular [13:14] mborzecki: yeah, definitely somewhere else, thank you for helping me debug this [13:14] and to the rest of the channel, too :) [13:14] mbriza: last thing, can you try installing flatpak-selinux, and see if it works? [13:14] mbriza: they also ship the policy in a separate module [13:16] mborzecki: i have it but it (and mysql, too) complains [13:16] ogra: good, thanks for confirming [13:16] ... even after reinstalling [13:16] ogra: it is a bug [13:16] Trespassing should never surface as an error [13:17] mbriza: yeah, i'd try checking in #fedora-devel, and maybe trying .autorelabel too [13:17] It is a duplicate of another bug on launchpad already [13:17] zyga, yeah, after two days cuddling with it i guessed that much :) [13:17] Give us some time, we have a plan that fixes a large class of those issues [13:18] yeah, no worries, i'm surely not in a hurry ... just ran into it and found it weird [13:19] pstolowski, one other (unrelated) thing ... should "listen-stream:" still be supported ? i was trying it yesterday and snapcraft complains about things like "listen-stream: 8080" not "being an object" [13:20] (iirc you initially implemented it) [13:21] ogra: listen-stream: [13:22] Chipaca, uhm, so network sockets got removed ? [13:23] https://snapcraft.io/docs/snapcraft-yaml-reference thinks is a valid thing [13:26] mborzecki: ha, after deleting the lang files in /var/lib/selinux/targeted/active/modules/200 and reinstalling the packages, i'm able to install snaps again [13:26] thank you! [13:27] mbriza: does it still work when you reboot? [13:28] i just reinstalled everything with 'selinux' in its name and there were no errors so i assume it'll be fine [13:29] also my app actually starts on this machine when installed from snap [13:29] ogra: i quickly grepped the code and we have tests for listen-stream: ; should work. not sure why snapcraft rejects it [13:30] hi [13:30] mbriza: hah, that's a good sign ;) [13:31] is there a problem with latest stable base core18 ? [13:31] pstolowski, hmm, well, then it is most likely a snapcraft bug with the yaml parser ... [13:31] i'm getting error about this one since yesterday [13:31] error: unable to contact snap store [13:31] An error occurred when trying to execute 'sudo -i env SNAPCRAFT_HAS_TTY=True snap install --channel stable core18' with 'multipass': returned exit code 1. [13:31] $ SNAPCRAFT_BUILD_ENVIRONMENT=host snapcraft [13:31] Issues while validating snapcraft.yaml: The 'apps/remote/sockets/listen-stream' property does not match the required schema: 19532 is not of type 'object' [13:31] thats what i guess [13:31] zyga: re trespassing, shouldn't it behave consistently whether /etc/systemd/journal-upload.conf (that with bind via layout: bind-file: $SNAP_DATA/journal-upload.conf) exists or not? at the moment it is happy if it exists and you only get trespassing error if doesn't [13:31] *s/guess/get/ [13:32] i've tried to clean all mutipass vms, and launched a clean snapcraft, no luck [13:34] aha ... https://bugs.launchpad.net/snapd/+bug/1612440 [13:34] Bug #1612440: Full systemd socket activation support [13:34] looks like the snapd task is fix-released but the snapcaft one is still new [13:36] cachio is off today, isn't he? [13:39] hey hey [13:39] pstolowski: it's a bug, it's actually diagnosed already AFAIK [13:39] pstolowski: yes but $COMPLEXITY [13:39] pstolowski: it's not what it seems at first [13:40] ogra: did you report a bug for this? [13:40] ogra: it would help with tracking [13:40]