[05:35] <mup> PR snapd#3782 closed: client: fix go vet 1.7 errors <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3782>
[07:33] <mvo> jamesh: hey, I added some small comments for 3581, very excited that this can land
[07:33] <jamesh> mvo: okay.  I was just writing a reply to niemeyer's last comment.
[07:34] <mvo> jamesh: great!
[07:36] <mvo> pedronis: how are things looking from your side for 2.28 - did we need to make the tweaks you mentioned yesterday?
[08:02] <Chipaca> Buon giorno, principesse e principi!
[08:04] <zyga-suse> good morning
[08:04] <zyga-suse> hey Chipaca, good to see you
[08:04] <zyga-suse> how were your holidays?
[08:04] <Chipaca> tiring :-)
[08:04] <Chipaca> how were things?
[08:04] <zyga-suse> I think mostly okay
[08:05] <pedronis> mvo: hi, not we didn't  yet
[08:05] <zyga-suse> tests are greener than earlier
[08:05] <Chipaca> anything igneous that shouldn't be?
[08:05] <zyga-suse> Chipaca: haha, yes, mystery of my VM with high error rate
[08:05] <zyga-suse> Chipaca: I have a VM that when I run tests on, things massively fail, randomly
[08:05] <zyga-suse> Chipaca: (in unit tests)
[08:05] <pedronis> mvo: I need to prepare a PR with a change this morning
[08:06] <mvo> pedronis: ok, please tag with 2.28 then
[08:06] <Chipaca> zyga-suse: is it accidentally a Win10 vm?
[08:06] <zyga-suse> Chipaca: haha, no, it's my Arch VM
[08:06] <mvo> pedronis: I went over the open ones to see what we want for 2.28 and milestoned a bunch
[08:06] <zyga-suse> Chipaca: tests failures are pretty odd, like "cannot parse this constant yaml string"
[08:07] <zyga-suse> Chipaca: the host has good memory (tested)
[08:07] <Chipaca> zyga-suse: how big is the vm image?
[08:07] <zyga-suse> Chipaca: let me look
[08:07] <zyga-suse> 14GB
[08:07] <zyga-suse> Chipaca: note that it fails randomly but mostly so when I unpack the release tarball again
[08:07] <zyga-suse> so feels like something gets corrupted randomly
[08:08] <zyga-suse> Chipaca: the releast tarball md5sum checks out (weak but useful)
[08:08] <pedronis> Chipaca: buon giorno
[08:09] <Chipaca> zyga-suse: and it's a regular VM thing like kvm?
[08:09] <zyga-suse> yes
[08:09] <Chipaca> zyga-suse: you're not emulating ryzen or something
[08:10] <zyga-suse> Chipaca: no, on a 3rd gen core i5
[08:10] <zyga-suse> and this machine has loads of other VMs with nothing failing anywhere
[08:10] <zyga-suse> Chipaca: the kernel there is fresh, I forgot how fresh
[08:10] <zyga-suse> Chipaca: the golang bits are 1.8.3
[08:11] <zyga-suse> Chipaca: if you want I can show you
[08:11] <Chipaca> zyga-suse: can you build a fresh arch vm of the same version and compare checksums?
[08:11] <zyga-suse> Chipaca: checksums of all the files on disk?
[08:11] <Chipaca> yes
[08:11] <zyga-suse> Chipaca: arch is hand made (early parts) so I bet things will turn out different
[08:11] <Chipaca> well, not _all_, but say /lib
[08:11] <zyga-suse> Chipaca: I could reinstall all packages
[08:11] <zyga-suse> Chipaca: one thing I did
[08:11] <zyga-suse> Chipaca: that is interesting
[08:11] <mvo> pstolowski: 3569 is ready, right? test look a bit unhappy right now, but I can baby sit, I think we wan tthis for 2.28 :)
[08:11] <zyga-suse> Chipaca: is to save yaml to /tmp/FAILING.yaml whenever I cannot parse it
[08:12] <zyga-suse> Chipaca: and I got the string ` - : `or something like that
[08:12] <zyga-suse> o_O
[08:12] <zyga-suse> I should do incremental files to see what other strings may fail
[08:12] <zyga-suse> and note that it is not random at all
[08:12] <zyga-suse> er
[08:12] <zyga-suse> it **is** random
[08:12] <mvo> zyga-suse: do you always get the same yaml? hexdump -C on the file would be great
[08:12] <zyga-suse> mvo: so far yes but as I said, I only save one file
[08:12] <zyga-suse> mvo: so I could get the same one because it's the first/last
[08:12] <mvo> zyga-suse: but that file is always the same?
[08:12] <zyga-suse> yes
[08:13] <zyga-suse> each time I saw it (~10
[08:13] <mvo> zyga-suse: thats a good start
[08:13] <jamesh> mvo: I've pushed some changes based on your review comments.  Assuming CI is happy with it, can you merge it?
[08:13] <zyga-suse> and note that since I built the package so I managed to run everything at least once without any failures
[08:13] <Chipaca> zyga-suse: ` - :` probably is bad yaml tho
[08:13] <Chipaca> :-p
[08:13] <zyga-suse> speaking of which, I wanted to upload that package
[08:13] <zyga-suse> Chipaca: I figure :)
[08:13] <mvo> pstolowski: does 3526 needs a review from gustavo? or just a second review?
[08:13] <zyga-suse> let me get more facts
[08:13] <zyga-suse> I find it annoying and curious at the same time
[08:14] <mvo> jamesh: \o/ thank you
[08:14] <mvo> jamesh: I have a look and if CI is green I will merge it, really nice to see this sorted, this is a huge win for usability
[08:14] <pstolowski> mvo, hey, yes 3569 is ready but has been failing tests for random reasons unrelated to the PR itself, i restarted it a couple of times in the last few days
[08:14] <mvo> zyga-suse: you sure its ` - :` and not `( - :` ;) ?
[08:14] <zyga-suse> mvo: hahahaha
[08:15] <jamesh> mvo: I've got more I'd like to do w.r.t. polkit support: this was just a minimal change to get the foot in the door.
[08:15] <mvo> pstolowski: yeah, I pushed master, maybe today is a good day
[08:15] <zyga-suse> mvo: yaml-smiley-virus-2017
[08:15] <pstolowski> mvo, and yes, 3526 needs a review from Gustavo
[08:15] <zyga-suse> mvo: replaces random in-memory yaml with a non-parsing smile
[08:15] <zyga-suse> :D
[08:15] <mvo> jamesh: anything that is ready or almost ready that we want for 2.28 (which we plan to tag in the next few days)?
[08:15] <pstolowski> mvo, :) yes let's hope today is THE DAY ;)
[08:16] <mvo> pstolowski: I added gustavo for 3526, lets poke him in the standup :)
[08:16] <pstolowski> mvo, thanks@
[08:16] <jamesh> mvo: It's probably best not to wait.  I'd like to add polkit auth for a few more APIs, but I understand that is somewhat controversial
[08:17] <jamesh> mvo: less controversial is having the snap client support the textmode polkit agent and indicate whether it is interactive
[08:17] <mvo> jamesh: ok, sounds good! what is needed for the textmode polkit agent? I assumed this is transparent for the API user, no?
[08:18] <mvo> morphis_: want me to have a look at the open comments for 3260? it seems its almost there
[08:18] <jamesh> mvo: essentially just have it fork "pkttyagent --fallback", like systemctl does
[08:19] <zyga-arch> re
[08:19] <zyga-arch> Chipaca: have a look
[08:19] <jamesh> mvo: this would let you run privileged operations without sudo, having pkttyagent prompt if needed
[08:19] <zyga-arch> https://paste.gnome.org/pecbkpb4e
[08:19] <morphis_> mvo: if you have spare cycles for that, that would awesome
[08:19] <zyga-arch> this is from
[08:19] <zyga-arch> https://paste.gnome.org/prw9lmfbx
[08:19] <mvo> jamesh: aha, nice!
[08:19] <morphis_> mvo: but more in general I still waiting for niemeyer to approve this from an architectural point
[08:20] <zyga-arch> Chipaca: ideas appreciated, I can patch and rebuild easily
[08:20] <mvo> jamesh: if this PR is there before 2.28 is taged I will milestone it so that it gets at least priority reviews :)
[08:21] <jamesh> mvo: okay
[08:21] <mvo> morphis_: I can raise it in the standup, I was uner the impression he is happy with it now but I have not talked with him in detail
[08:21] <morphis_> mvo: that was my guess too, but he still has his denial on the PR
[08:22] <morphis_> mvo: otherwise let me try to get through zyga-arch's comments this afternoon
[08:22] <mvo> morphis_: I tagged it for 2.28, lets see if that helps speeding it along. I also check the open question and will push changes
[08:22] <morphis_> :-)
[08:22] <Chipaca> zyga-arch: and which test is it that fails with this?
[08:22] <mvo> morphis_: lets make a deal, I look at things now and what is left you can have in the afternoon :)
[08:22] <morphis_> sounds good :-)
[08:22] <zyga-arch> Chipaca: *any* tests that touches yaml, I saw it fail all over the source code
[08:23] <Chipaca> zyga-arch: an example?
[08:26] <zyga-arch> essentially any that calls snap.InfoFromSnapYaml for insatance FirstBootTestSuite.TestPopulateFromSeedHappyMultiAssertsFiles
[08:26] <zyga-arch> (note that this is a first that I saw this particular test failing)
[08:28] <zyga-arch> mvo: question, did you upgrade your laptop to >> xenial?
[08:28] <zyga-arch> (hence the spellchcek and govet changes)
[08:33] <zyga-arch> Chipaca: I just patched it to collect each failure, let's see what we get
[08:45] <zyga-arch> https://paste.gnome.org/pjiw7rerp
[08:45] <zyga-arch> more interesting failure log
[08:46] <Chipaca> zyga-arch: what filesystem is this on?
[08:47]  * Chipaca thinks "lardfs"
[08:48] <zyga-arch> Chipaca: ext4
[08:48] <zyga-arch> as boring as it gets, right?
[08:48] <zyga-arch> Linux archlinux 4.12.8-2-ARCH #1 SMP PREEMPT Fri Aug 18 14:08:02 UTC 2017 x86_64 GNU/Linux
[08:48] <zyga-arch> the kernel is fresh though
[08:50] <Chipaca> zyga-suse: can you try with a kernel that isn't newer than the host's? (not that it should matter)
[08:52] <zyga-arch> Chipaca: not sure, I don't know how to get old arch kernels
[08:53] <Chipaca> zyga-arch: snap install linux-2.0.43
[08:53] <Chipaca> no?
[08:53] <Chipaca> aw :-(
[08:54] <zyga-arch> heh, not yet
[08:54] <zyga-arch> I improved logging to save the failing yaml and the error message
[08:54] <zyga-arch> maybe some of those are genuine errors from the test suite
[08:54] <Chipaca> “snap set core locales en_GB.UTF-8”, WTDT?
[08:54] <Chipaca> WDYT*
[08:55] <ogra_> Chipaca, wont work
[08:55] <Chipaca> ogra_: wy?
[08:55] <ogra_> because there is no locale-gen
[08:56] <Chipaca> ogra_: not _yet_ :-)
[08:56] <ogra_> Chipaca, i wont add 20MB (or what was the locales stuff)
[08:56]  * ogra_ forgot the exact size but it wasnt small iirc
[08:57] <Chipaca> ogra_: oooh, we can discuss this like civilized people, or we can fight!
[08:57]  * Chipaca gets his gloves
[08:57] <zyga-arch> *ding* FIGHT
[08:57] <zyga-arch> and you konw
[08:57]  * Chipaca runs the fastest
[08:57] <zyga-arch> the finishing move will be called LOCALITY!
[08:58]  * Chipaca makes chicken noises and waves his elbows while running
[08:58]  * ogra_ throws a de_DE.UTF-8 ... in a perl variant !
[08:58]  * Chipaca is in a perl hackers list somewhere on the internet still
[08:59]  * Chipaca mutates the glob behind the variant and throws it back
[08:59] <zyga-arch> Chipaca: on the interenet nobody knows you are a perl interpreter
[08:59] <ogra_> Chipaca, i'll trade though ... find something that compensates enough for the added stuff and we can talk ;)
[08:59] <Chipaca> ogra_: but, serioulsy, if it's 20MB i'd agree, but i think what we want is the locale-gen, not the locales themselves
[09:00] <ogra_> there is definitely still other stuff we can drop
[09:00] <zyga-arch> ogra_: here's $0.01 for the flash cost ;)
[09:00] <zyga-arch> Chipaca: here's another interesting failure
[09:00] <zyga-arch> ----------------------------------------------------------------------
[09:00] <zyga-arch> PANIC: firstboot_test.go:445: FirstBootTestSuite.TestPopulateFromSeedMissingBootloader
[09:00] <zyga-arch> ... Panic: cmd: "mksquashfs . /tmp/check-717607699634245797/61/snapsrc/core_1.0_all.snap -noappend -comp xz -no-xattrs" failed: exit status 1 ("Parallel mksquashfs: Using 4 processors\nCreating 4.0 filesystem on /tmp/check-717607699634245797/61/snapsrc/core_1.0_all.snap, block size 131072.\n\r[[09:00] <zyga-arch> ptor, offset=0x60\nFATAL ERROR:Probably out of space on output filesystem\n\n\nExportable Squashfs 4.0 filesystem, xz compressed, data block size 131072\n\tcompressed data, compressed metadata, compressed fragments, no xattrs\n\tduplicates are removed\nFilesystem size 0.35 Kbytes (0.00 Mbytes)\n\t101.72% of uncompressed filesystem size (0.34 Kbytes)\nInode table size 98 bytes (0.10 Kbytes)\n\t100.00% of uncompressed inode table
[09:00] <zyga-arch> size (98 bytes)\nDirectory table size 55 bytes (0.05 Kbytes)\n\t100.00% of uncompressed directory table size (55 bytes)\nNumber of duplicate files found 0\nNumber of inodes 3\nNumber of files 1\nNumber of fragments 1\nNumber of symbolic links  0\nNumber of device nodes 0\nNumber of fifo nodes 0\nNumber of socket nodes 0\nNumber of directories 2\nNumber of ids (unique uids + gids) 1\nNumber of uids 1\n\tzyga (1000)\nNumber of gid
[09:00] <zyga-arch> s 1\n") (PC=0x458E88)
[09:01] <zyga-arch>  /dev/sda2        20G  9,5G  9,1G  52% /
[09:01] <Chipaca> hmmmm
[09:01] <Chipaca> zyga-arch: how
[09:01] <Chipaca> 's space doing?
[09:01] <zyga-arch> tmpfs           1,5G  2,1M  1,5G   1% /tmp
[09:01] <Chipaca> ah
[09:01] <zyga-arch> not bad
[09:01] <Chipaca> zyga-arch: what does dmesg say?
[09:01] <zyga-arch> really weird, right?
[09:02] <ogra_> did you add locales back ?
[09:02] <ogra_> :P
[09:02] <zyga-arch> nothing at all
[09:02] <zyga-arch> https://paste.gnome.org/psa6vgmnj
[09:02] <zyga-arch> the last timestamp is old there
[09:02] <mvo> zyga-arch: yeah, I run on zesty
[09:02] <zyga-arch> mvo: I'm tempted to update my xenial x250
[09:02] <mup> PR snapd#3569 closed: snapd,snapctl: decode json using Number <Created by stolowski> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3569>
[09:02] <zyga-arch> any issues?
[09:02] <mvo> zyga-arch: its doing fine on my x250
[09:03] <Chipaca> zyga-arch: i'll move this weirdness to the backburner for a bit
[09:03] <Chipaca> let see if something percolates
[09:04] <zyga-arch> Chipaca: yeah, me too; thank you for the time!
[09:04] <Chipaca> is it my machiine, or are the travis logs so slow they're almost unusable?
[09:07] <zyga-arch> my network is shitty enough not to handle 2 IRC connections (and music streaming :P)
[09:09] <pstolowski> yay! thanks mvo :)
[09:09] <mvo> pstolowski: thank you :)
[09:58] <zyga-arch> janisozaur: hey, around?
[09:59] <janisozaur> o/
[09:59] <zyga-arch> hey :)
[09:59] <zyga-arch> do you have a moment to help me out?
[09:59] <janisozaur> with?
[10:00] <morphis> zyga-arch: I am seeing https://paste.ubuntu.com/25375579/ right now on my production system
[10:00] <zyga-arch> janisozaur: I'd like you to build a package for me a few times
[10:00] <zyga-arch> I'm seeing random bugs that feel like kernel memory corruption or something similar
[10:00] <zyga-arch> and I need 2nd opinions
[10:00] <janisozaur> on Arch?
[10:00] <zyga-arch> yes
[10:01] <zyga-arch> morphis: oh that's a new thing, feels I dind't patch the apparmor profile
[10:01] <zyga-arch> morphis: where are you seeing this?
[10:01] <morphis> zyga-arch: Ubuntu 16.04
[10:01] <janisozaur> I don't have access to my system right now, maybe in the evening?
[10:01] <morphis> I think with proposed enabled
[10:01] <zyga-arch> janisozaur: that's ok, thank you!
[10:01] <zyga-arch> morphis: I think we need to fix this, mvo ^^^
[10:01] <zyga-arch> mvo: we may need to patch 2.27.x
[10:01] <zyga-arch> let me check though
[10:01] <morphis> 2.27.3 is the version of the snapd deb
[10:02] <janisozaur> zyga-arch: that's on fully updated system, right?
[10:02] <zyga-arch> yes
[10:03] <zyga-arch> morphis: can you look at your apparmor profile
[10:03] <zyga-arch> morphis: look for     mount options=(rw bind) @SNAP_MOUNT_DIR@/{,ubuntu-}core/*/etc/nsswitch.conf -> /tmp/snap.rootfs_*/etc/nsswitch.conf,
[10:03] <zyga-arch> morphis: this should be in /etc/apparmor.d/
[10:03] <morphis> yeah, it's there
[10:03] <zyga-arch> morphis: can you check that it is present for the core snap version as well
[10:04] <morphis> zyga-arch: hm, an apparmor_parser -r /etc/apparmor.d/usr.lib.snapd.snap-confine.real fixed it
[10:04] <zyga-arch> morphis: !!!
[10:04] <morphis> zyga-arch: the core snap is at 16-2.26.14
[10:04] <zyga-arch> morphis: can you please open a forum topic
[10:04] <zyga-arch> I think we've seen it before
[10:04] <zyga-arch> but now it's a smoking-gun-bug
[10:04] <zyga-arch> morphis: and please save your journal
[10:04] <morphis> sure, but can't share that in the public
[10:05] <zyga-arch> as there are log entries for each time the profiles are loaded
[10:07] <zyga-arch> jdstrand: ^
[10:07] <zyga-arch> mvo: ^ (possible bug in our packaging or in dh_apparmor in general)
[10:09] <morphis> zyga-arch: mvo: https://forum.snapcraft.io/t/apparmor-profile-for-snapd-confine-not-reloaded-on-deb-update/1820
[10:10] <morphis> zyga-arch: if I remember right I've rebooted in between
[10:10] <mvo> zyga-suse: ok, we need to figure out if its a regression, fortunately its not too late yet to do another 2.27
[10:11] <zyga-arch> mvo: I don't think so, we've seen this randomly a few times
[10:11] <mvo> morphis: what is the timeline, i.e. did you apt upgrade and after that the snapd services did stop working?
[10:11] <zyga-arch> mvo: but it is serious, it means we are running on a broken configuration
[10:11] <mvo> zyga-arch: or did a reboot happen in between?
[10:11] <zyga-arch> mvo: I don't know, it's morphis that is affected
[10:11] <mvo> zyga-arch: yeah, definitely serious
[10:12] <morphis> mvo: if I remember right, I did a manual apt upgrade
[10:12] <morphis> did a reboot and then after some time wondered why specific services coming from a snap were not running
[10:12]  * zyga-arch wonders where to upload the binary release for Arch
[10:12] <zyga-arch> :/
[10:13] <mvo> morphis: can you please add the info that there was a reboot in between? that might be useful to know
[10:13] <morphis> done
[10:13] <mvo> morphis, zyga-*: I think we want the input from jdstrand on https://forum.snapcraft.io/t/apparmor-profile-for-snapd-confine-not-reloaded-on-deb-update/1820
[10:13] <janisozaur> zyga-arch: github releases?
[10:14] <zyga-arch> janisozaur: oh, good idea
[10:14] <zyga-arch> mvo: can I add a file to our 2.27.3 github release page?
[10:15] <mvo> zyga-arch: I bet its apparmor cache related
[10:15] <zyga-arch> mvo: I checked that the debhelper scripts uses an option to skip the cache
[10:15] <mvo> morphis: silly question, but another reboot will not bring the problem back, right? I assume apparmor_parser -r will update all caches
[10:15] <mvo> zyga-arch: oh? woah
[10:15] <morphis> mvo: didn't tried yet
[10:15] <zyga-arch> mvo: not sure, caches are writen to explicitly
[10:15] <mvo> zyga-arch: so the file on disk is correct but the wrong profile is loaded even after reboot?
[10:16] <zyga-arch> I think the early startup script that loads profiles
[10:16] <zyga-arch> mvo: not sure how it uses or not uses caches
[10:16] <mvo> zyga-arch: re file for release> what file exactly?
[10:16] <zyga-arch> mvo: binary build for arch
[10:16] <zyga-arch> and a source package for arch
[10:16] <mvo> zyga-arch: sure
[10:16] <zyga-arch> thank you
[10:16] <mvo> zyga-arch, morphis: so just to double check - the apparmor profile on disk is correct, the reboot has happend and yet the kernel has an incorrect profile loaded?
[10:17] <morphis> mvo: yes, that is what happened
[10:17] <morphis> can it be a problem that the core snap was still at 2.26?
[10:17] <zyga-arch> morphis: next time, don't reload the profile
[10:17] <zyga-arch> morphis: please collect the loaded profile from sysfs
[10:17] <mvo> morphis: would you mind also putting this as a TL;DR in the forum post :) ?
[10:17] <zyga-arch> morphis: no, each profile is distinct
[10:18] <zyga-arch> it applies to a specific binary via path
[10:18] <mvo> morphis: this makes it also sound as alarming as it should sound :)
[10:18] <zyga-arch> morphis: profiles can be extracted from memory
[10:18] <zyga-arch> morphis: via /sys/kernel/apparmor/profiles/
[10:18] <zyga-arch> morphis: but you need to copy the raw policy
[10:18] <mvo> zyga-arch: 3260 is ready for a re-review
[10:18] <zyga-arch> morphis: you cannot tarball the directory
[10:18] <zyga-arch> (or you may need to give tar a smart option to ignore size)
[10:18] <zyga-arch> mvo: noted, I'll open it now
[10:19] <mup> PR snapd#3581 closed: daemon: add polkit support to /v2/login <Created by jhenstridge> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3581>
[10:19]  * mvo hugs jamesh
[10:20] <jamesh> mvo: thanks!
[10:20] <morphis> mvo, zyga-arch: btw. shouldn't I better file a bug report on launchpad for this?
[10:21] <zyga-arch> morphis: I prefer the forum really
[10:21] <zyga-arch> morphis: people look there x10 more often
[10:21] <morphis> zyga-arch: so is that were you now track bugs?
[10:21] <zyga-arch> morphis: track TODOs really, bugs and other
[10:22] <zyga-arch> morphis: a bug is fine as well but my point is that exposure there is low
[10:22]  * zyga-arch spends $ETERNITY to upload the release
[10:22] <morphis> just wondering what the actual policy is
[10:22] <zyga-arch> morphis: no idea
[10:22] <morphis> :-D
[10:22] <zyga-arch> morphis: forum >> bug for sure but bugs are used too
[10:26] <zyga-arch> I'll focus on the review from mvo/morphis now, ping me for attention
[10:31] <mup> PR snapd#3784 opened: cmd/snap: in "snap info', don't print a newline between tracks <Created by chipaca> <https://github.com/snapcore/snapd/pull/3784>
[10:37] <mvo> morphis: quick question - there is code in cmd_userd.go that looks like you created unittests, there is no cmd_userd_test.go on your hdd that is not git added ;)
[10:38] <Chipaca> mvo: the bigger question is, how can you see morphis' hdd
[11:14] <mup> PR snapd#3785 opened: asserts,overlord/devicestate: accept generic serials only if the model has generic-serials: true <Created by pedronis> <https://github.com/snapcore/snapd/pull/3785>
[11:15] <pedronis> mvo:  this is the PR ^ ,  marked for 2.28  and needs Gustavo to look at it
[11:21] <morphis> mvo: need to check but I guess I never ported them over from the former implementation when it was snapd --user
[11:22] <morphis> mvo, zyga-arch: there seems to be another problem, I am now seeing seccomp enabled for my devmode snaps with 2.27.3
[11:24] <zyga-arch> morphis: can you please collect as much stuff as you can
[11:24] <morphis> sure
[11:24] <zyga-arch> morphis: can you look at the bpf profile
[11:24] <morphis> zyga-arch: how do I see if a process is running with seccomp attached?
[11:24] <zyga-arch> morphis: and at the legacy profiles that we don't remove
[11:24] <zyga-arch> morphis: let me check, I don't know
[11:25] <zyga-arch> mvo: devmode snaps get an ALLOW bpf, right?
[11:25] <zyga-arch> mvo: or nothing at all?
[11:25] <morphis> hm, the profile has @complain
[11:25] <zyga-arch> morphis: which specific file has this?
[11:26] <zyga-arch> morphis: note that we have new profile location now so you may be looking at the old palce
[11:26] <morphis> zyga-arch: the profile for my snap app
[11:26] <zyga-arch> place
[11:26] <zyga-arch> morphis: right, _which_ file? :-)
[11:26] <morphis>  /var/lib/snapd/seccomp/profiles/snap.anbox.container-manager
[11:26] <zyga-arch> that's the old file
[11:26] <zyga-arch> it's not used anymore
[11:26] <zyga-arch> and not removed either
[11:26] <zyga-arch> morphis: look at seccomp/bpf
[11:26] <zyga-arch> there are two files per profile
[11:26] <zyga-arch> binary and source
[11:27] <zyga-arch> those are compiled with snap-seccomp
[11:27] <morphis> hm, /var/lib/snapd/seccomp/bpf/snap.anbox.container-manager.src has @complain too
[11:27] <zyga-arch> okay so _so far_ so good
[11:27] <zyga-arch> mvo: is the person to talk as he wrote that part
[11:27] <morphis> but I see a lot denials in dmesg: https://paste.ubuntu.com/25375999/
[11:27] <zyga-arch> I'll return to your code review
[11:28] <zyga-arch> mq_timedreceive ?
[11:28] <zyga-arch> or is this an i386 binary?
[11:28] <morphis> it's x86_64
[11:28] <zyga-arch> so this is mq_timedreceive, weird
[11:28] <zyga-arch> I'll leave that to mvo for now
[11:29] <zyga-arch> but feels like another forum topic to open
[11:30] <morphis> zyga-arch: can I reliable switch back to the older snapd or will that break things even more?
[11:30] <zyga-arch> you should be
[11:30] <zyga-arch> (able to)
[11:31] <morphis> ok, but lets wait for mvo before I do that
[11:36] <pedronis> seems there are still issues with the fakestore sometimes
[11:39] <pedronis> we probably want to print the logs there
[11:39] <zyga-suse> morphis: I commented on the 3260 PR about one aspect of the spread test
[11:39] <zyga-suse> morphis: can you look please
[11:39] <zyga-suse> I'm looking at the rest
[11:47] <morphis> zyga-suse: when I get to it today, I will for sure :-)
[11:47] <morphis> zyga-suse: thanks
[12:02] <mvo> zyga-suse, morphis: seccomp is allow all mode
[12:03] <zyga-suse> mvo: aha, then something wonky is going on
[12:04] <mvo> morphis: strange, @complain should be fine and should not kill anything, the generated bpf file ".bpf" would be nice, could you mail that to me?
[12:05] <mvo> zyga-suse, morphis: I can look at the test issue in 3260, I'm keen to land that, will also look at the cmd_userd.go bits and see if I can come up with a test
[12:06] <pedronis> mvo: Chipaca: as I told Gustavo yesterday I have errands today that overlap with the standup so I won't be there
[12:07] <mvo> pedronis: ok, thank you
[12:11] <morphis> mvo: sure
[12:12] <morphis> mvo: sent
[12:12] <morphis> mvo: so these messages in dmesg are no denials?
[12:14] <zyga-suse> morphis: those are hard denials
[12:14] <zyga-suse> (SIGKILL)
[12:14] <zyga-suse> there's no complain mode for seccomp being used yet
[12:15] <morphis> good, that is what I thought
[12:18] <mvo> morphis: bpf disassembled looks ok, what architecutre is this? amd64?
[12:19] <morphis> yes, amd64
[12:19] <morphis> mvo: installed with snap install --devmode --edge anbox
[12:20] <mvo> morphis: ok, let me try that
[12:22] <mvo> morphis: I just did a snap install --devmode --edge anbox and systemctl status snap.anbox.container-manager is happy
[12:22] <mvo> morphis: what should I see?
[12:34] <ogra_> ppisati, ".config:9871:warning: override: reassigning to symbol ARCH_MEDIATEK" what does that mean ?
[12:34] <ogra_> (building a patched arm64 mediatek kernel here, is the above fatal in any way ? i have never seen it before)
[12:41] <jdstrand> morphis: fyi, syscall 243 is mq_timedreceive and that isn't included in any interfaces. the denial indicates you are not actually in complain mode
[12:42] <jdstrand> mvo, zyga-arch: ^
[12:42] <jdstrand> also, I commented in the forum topic
[12:42] <jdstrand> morphis: ^ questions for you there
[12:42] <zyga-arch> jdstrand: looking
[12:47] <jdstrand> mvo: based on morphis' denial, it seems clear that seccomp.NewFilter(seccomp.ActAllow) isn't being run
[12:48] <jdstrand> mvo: that should be a very small bpf based on lines 616-624 (as you know)
[12:49] <jdstrand> morphis: can you paste the entirety of /var/lib/snapd/seccomp/bpf/snap.anbox.container-manager.src?
[12:53] <mvo> jdstrand: morphis mailed me the bpf and it looks like this http://paste.ubuntu.com/25376337/
[12:53] <zyga-suse> I get a change after running get-deps.sh
[12:53] <mvo> jdstrand: so it looks like the normal allow bpf
[12:53] <zyga-suse> golang.org/x/sys/unix changes checksum
[12:57] <mup> PR snapd#3786 opened: vendor: update vendor.json after (presumed) manual edits <Created by zyga> <https://github.com/snapcore/snapd/pull/3786>
[12:57] <zyga-suse> mvo: ^
[12:57] <jdstrand> mvo: that's interesting. seems that the old non-bpf profile is being preferred, which suggests a problem with the reexec code
[12:59] <mvo> jdstrand: oh, that is interessting
[12:59] <mvo> morphis: what do you get with `snap version` on the system that has the anbox problem?
[13:00] <zyga-suse> jdstrand: yes, I was thinking the same thing
[13:01] <zyga-suse> jdstrand: the old profile gets used because the old snap-confine runs
[13:11] <jdstrand> zyga-suse: based on the forum topic, it looks like morphis has the stable core installed (16-2.26.14) and upgraded to 2.27.3. I tried that exact upgrade path and couldn't reproduce the apparmor bits
[13:11] <jdstrand> (upgraded t 2.27.3 deb)
[13:11] <zyga-suse> jdstrand: I haven't been able to reproduce that
[13:11] <zyga-suse> jdstrand: but I've seen it happen once in front of my eyes
[13:12] <morphis> mvo, jdstrand: back now, let me read through the messages
[13:13] <morphis> mvo: snap version output is on the forum thread
[13:14] <jdstrand> zyga-suse: based on the forum topic alone, it looks like the system (not the reexec'd) snap-confine is used
[13:14] <zyga-suse> yes, I agree
[13:15] <jdstrand> zyga-suse: I wonder if it is a race between the snapd restart and the deb packaging
[13:15] <zyga-suse> mmmm
[13:16] <zyga-suse> re-execing snap (as in snap run) will always use the new snap-confine
[13:16] <jdstrand> zyga-suse: but in this case, new is the deb (2.27 deb vs 2.26 snap)
[13:17] <zyga-suse> riiiight
[13:17] <zyga-suse> so there's no reexec
[13:17] <zyga-suse> so there should be stuf from the distro
[13:17] <zyga-suse> aha
[13:17] <zyga-suse> I see
[13:17] <jdstrand> so re-exec should not be in effect (indeed, the denial clears shows /usr/lib/snapd/snap-confine has the denial, not /snap/core/.../snap-confine
[13:17] <zyga-suse> so half-way unpacked snap-confine binary?
[13:17] <jdstrand> clearly*
[13:17] <zyga-suse> yes, I noticed that
[13:18] <jdstrand> I'm trying to think through that. let me look at the postinst again
[13:21] <morphis> jdstrand: mvo: added requrested details to the forum post
[13:25] <jdstrand> zyga-suse: so, looking at postinst, snapd looks to be started *before* the dh_apparmor hooks
[13:25] <zyga-suse> oh,
[13:25] <zyga-suse> in the maintainer script
[13:26] <jdstrand> zyga-suse: line 125: deb-systemd-invoke start snapd.service >/dev/null || true
[13:26] <jdstrand> zyga-suse: line 160:  apparmor_parser -r -T -W "$APP_PROFILE" || true
[13:28] <mvo> morphis: the `snap version` output of the anbox issue
[13:28] <mvo> morphis: that is the bit I care most about
[13:28] <mvo> jdstrand: ohhhh, ok. will you do a PR or shall I?
[13:28] <jdstrand> zyga-suse: now, the new profile should be on disk so, technically, snapd would load the new profile and it shouldn't matter. *but* if there was a problem there with snapd using the wrong profile and updating the cache, by the time apparmor_parser dh postinst is run, it wouldn't update the cache, so on reboot, it would be out of date
[13:29] <jdstrand> zyga-suse: *if* the start won the race that is
[13:30] <jdstrand> since we are seeing some reexec issues with seccomp, I'm not sure we can say for sure that snapd is doing the right thing here, so perhaps the postinst ordering exposes something
[13:31] <jdstrand> morphis, zyga-suse: it does seem most correct to have the dh bits for apparmor before the dh bits for systemd, but I don't know if it will fix this issue
[13:31] <zyga-suse> I'll read your conversation with focus in a moment (on a call)
[13:32] <morphis> mvo: it's on the thread
[13:33] <morphis> mvo: within the first post: https://forum.snapcraft.io/t/apparmor-profile-for-snapd-confine-not-reloaded-on-deb-update/1820?u=morphis
[13:34] <mvo> morphis: I mean, you mentioned an "anbox" problem and seccomp not being in @complain mode but in enforce mode. that was a different issue, no?
[13:34] <morphis> mvo: ah, I have both issues I've reported today (apparmor + seccomp) on the same system after the deb upgrade to 2.27.3
[13:35] <morphis> so snap version output is the same for both
[13:35] <mup> PR snapd#3784 closed: cmd/snap: in `snap info`, don't print a newline between tracks <Created by chipaca> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3784>
[13:47] <mup> PR snapd#3787 opened: cmd/snap-repair: filter also on models and architecture <Created by pedronis> <https://github.com/snapcore/snapd/pull/3787>
[13:51] <zyga-suse> + curl -sS -o - https://codeload.github.com/snapcore/snapd/tar.gz/2.27
[13:51] <zyga-suse> gzip: stdin: not in gzip format
[13:51] <zyga-suse> unhappy github?
[13:51] <zyga-suse> only half of spread tests finished
[13:51] <zyga-suse> in https://travis-ci.org/snapcore/snapd/builds/267571170?utm_source=github_status&utm_medium=notification
[14:00] <Chipaca> zyga-suse: i got that one too for a lil' bit
[14:00] <Chipaca> seems to be better now
[14:00] <Chipaca> i'm assuming network hiccup
[14:09] <zyga-suse> https://plus.google.com/114015603831160344127/posts/MdNjzwKvknX
[14:09] <zyga-suse> jdstrand: ^ perhaps something to keep an eye out for
[14:10] <zyga-suse> jdstrand: specifically https://bugs.freedesktop.org/show_bug.cgi?id=101354
[14:12] <mup> PR snapd#3788 opened: cmd/snap-repair: ignore superseded revisions like the rest of the assertion infra <Created by pedronis> <https://github.com/snapcore/snapd/pull/3788>
[14:17] <jdstrand> zyga-suse: thanks
[14:23] <zyga-suse> jdstrand: were you familiar with those efforts?
[14:33] <mvo> zyga-suse: 3260 should now address all questions/suggestions
[14:33] <mvo> zyga-suse: why the question about artful?
[14:33] <zyga-suse> mvo: run xdg-open on artful
[14:33] <zyga-suse> last time I looked it was a broken shell script
[14:33] <zyga-suse> (that didn't work)
[14:34] <zyga-suse> recommending to switch to gio-open or something like it
[14:34] <zyga-suse> it feels like an upstream change that will go our way sooner or later
[14:35] <mvo> zyga-suse: looks ok to me (xdg-utils 1.1.1-1ubuntu2) - what system did you try this on?
[14:35] <zyga-suse> mvo: maybe it was fixed
[14:36] <zyga-suse> mvo: what happens when you run it? (note: artful, not zesty)
[14:36] <zyga-suse> mvo: does it print any advice?
[14:36] <mvo> zyga-suse: I only tested it in a chroot so far but it look like the normal xdg-open. version is the same as the zesty version
[14:37] <zyga-suse> mvo: is it a shell script?
[14:37] <mvo> zyga-suse: yes
[14:37] <mvo> zyga-suse: it always was one
[14:37] <ogra_> are oyu surte you are not looking at snapd-xdg-open ?
[14:37] <mvo> (afaik)
[14:37] <ogra_> *you sure
[14:37] <zyga-suse> ogra_: no
[14:38] <ogra_> (awful shell script talking about being replaced sounds exactly like it)
[14:38] <zyga-suse> ogra_: no, that was a 4 line script
[14:38] <zyga-suse> ogra_: that had a typo that made it broken
[14:38] <zyga-suse> ogra_: maybe this update was pulled from artful since, I saw it on my desktop at home
[14:38] <zyga-suse> (I cannot check that now)
[14:38] <mvo> zyga-suse: xdg-open prints helpful advice in my cloudvm as well
[14:38] <zyga-suse> it printed a message and ran something else
[14:39] <zyga-suse> mvo: if it works and will stay working then fine
[14:39] <mvo> zyga-suse: we can always tweak further, but if xdg-open is broken we have a bigger problem in artful
[14:39] <zyga-suse> mvo: I was worried since it felt like it is going away
[14:39] <hallyn> ...  say, the snappy vagrant box, is there a default username/pwd? vagrant wants those to get in to setup authkey
[14:39] <mvo> zyga-suse: lots of 3rd party things use it
[14:39] <zyga-suse> I see it even supports flatpak now
[14:39] <jdstrand> zyga-suse: I wasn't, no. Simon, the submitter, is active with apparmor so I'm hopeful it won't conflict with anything. I haven't looked at any of this yet (I created a card)
[14:40] <zyga-suse> thank you jdstrand
[14:41]  * zyga-suse experiences dial-up speeds :/
[14:44] <ogra_> use LTE !
[14:48] <Chipaca> ogra_: i agree! Liquid Tension Experiment makes everything better. Evidence: https://www.youtube.com/watch?v=edqH0ofRQrM
[14:48] <Chipaca> of course, zyga can't watch that because he's on GPRS
[14:49] <Chipaca> zyga-suse: it could be worse: you could be tethered over bluetooth to an GPRS nokia. rfcomm still haunts me.
[14:49] <mup> PR snapd#3786 closed: vendor: update vendor.json after (presumed) manual edits <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/3786>
[14:49] <zyga-suse> Chipaca: and the nokia might be wrapped in plastic and connected to usb power pack
[14:49] <zyga-suse> Chipaca: out there glued to a tree
[14:50] <zyga-suse> Chipaca: in the rain
[14:50] <zyga-suse> Chipaca: to get the signal
[14:50] <zyga-suse> Chipaca: that's close to how I'm connected
[14:50] <Chipaca> zyga-suse: outside? you'd get several kbps more from that!
[14:50] <Chipaca> like, 20% more! or 3kbps
[14:50] <zyga-suse> haha
[14:50] <zyga-suse> my network is so slow
[14:50] <zyga-suse> that I see a merged branch (3786) in the list of open PRs
[14:50] <zyga-suse> and that list has just loaded :D
[14:58] <ogra_> lol
[15:15] <jdstrand> morphis (cc mvo and zyga-suse): https://forum.snapcraft.io/t/apparmor-profile-for-snapd-confine-not-reloaded-on-deb-update/1820/8
[15:15]  * zyga-suse looks
[15:15] <morphis> jdstrand: just read it, interesting
[15:21] <jdstrand> morphis: if you wanted to propose a PR, I'd be happy to review it
[15:22] <jdstrand> morphis: maybe mvo or zyga-suse would want to propose that...
[15:22] <morphis> jdstrand: sadly not enough time to do that
[15:22] <mvo> jdstrand, morphis I can look at this in a bit
[15:22] <zyga-suse> I can look
[15:22] <jdstrand> ok, thanks guys
[15:22] <zyga-suse> wow, git push takes noticeable time
[15:23] <morphis> mvo: jdstrand: the other issue is seccomp being in enforce mode for devmode snaps
[15:23] <mvo> zyga-suse: I take a break now, so feel free
[15:23] <morphis> mvo: you found anything in the profiles I've sent you?
[15:23] <mvo> morphis: its using the wrong profile, that seems to be the issue, does reloading the unit fixes the issue?
[15:23] <mup> PR snapd#3789 opened: cmd/snap-confine: genearlize apparmor profile for various lib layout <Created by zyga> <https://github.com/snapcore/snapd/pull/3789>
[15:23] <mvo> morphis: the profile looks correct, its the normal allow profile
[15:24] <morphis> mvo: you mean restarting the systemd unit?
[15:24] <mvo> morphis: I suspect what happens is that the old profile is loaded or was loaded
[15:24] <zyga-suse> jdstrand: I'll propose a small PR with the order of calls in the maintainer script changed
[15:24] <mvo> morphis: yeah, the systemd unit for the failing service
[15:25] <morphis> mvo: nope, that doesn't help
[15:25] <morphis> let me try to remove and install the snap again
[15:27] <jdstrand> mvo: keep in mind seccomp doesn't work like apparmor-- snap-confine loads what is on disk into the kernel on each invocation (ie, there is no binary attachment or anything). either an old snap-confine that doesn't know bpf is parsing the profiles/* file, or the new snap-confine is preferring profiles/8 over bpf/*.bin for some reason
[15:27] <zyga-suse> hmm, this debian packaging change will require some more surgery it seems
[15:27] <jdstrand> mvo: the only other possibility would be *if* bpf/* on disk at the time snap-confine ran was strict mode, then updated after the fact for devmode
[15:28] <zyga-suse> jdstrand, mvo: it seems that all of the magic is in #DEBHELPER# expanding, I wonder how we can fix that
[15:28] <pedronis> zyga-suse: what's the status of snapd#3617, wasn't it split in a lot of smaller PRs ? or is still wip ?
[15:28] <mup> PR snapd#3617: Using udev tagging for snap interfaces <Created by adglkh> <https://github.com/snapcore/snapd/pull/3617>
[15:29] <zyga-suse> pedronis: it's good for review I think, all the small PRs were merged and integrated there now
[15:29] <zyga-suse> pedronis: I haven't checked where things go
[15:29] <zyga-suse> (as in what needs doing there)
[15:29] <jdstrand> zyga-suse: so, if you put dh_apparmor before dh-installinit, it still ends up after?
[15:30] <zyga-suse> jdstrand: no, I mean that there is no explicit dh_apparmor there, it's all done at build time
[15:30] <jdstrand> zyga-suse: ie, all in override_dh_installint
[15:30] <zyga-suse> jdstrand: quickly looking at that file shows that it is post-processed during the build to contain the actual "meta"
[15:30] <zyga-suse> "meat"
[15:30] <jdstrand> zyga-suse: ?? I'm looking at debian/rules from xenial-- line 188 has dh_apparmor
[15:31] <zyga-suse> I was looking at .postinst in packaging/ubuntu*/
[15:31] <jdstrand> zyga-suse: if you are talking about packaging/*...
[15:32] <jdstrand> zyga-suse: you have to look at packaging/ubuntu*/rules
[15:33] <zyga-suse> ok, I have that open
[15:33] <jdstrand> zyga-suse: you can change things in there to affect #DEBHELPER# in postinst
[15:33] <zyga-suse> I see, I'm not familiar with that, let me experiment
[15:33] <jdstrand> zyga-suse: alternatively, you can just drop dh_apparmor, then modify snapd.postinst directly to put what dh_apparmor would put in #DEBHELPER# before #DEBHELPER#
[15:35] <jdstrand> zyga-suse: looking at that file, you just need to make it so dh_apparmor happens before dh_systemd_start -psnapd data/systemd/snapd.service
[15:35] <jdstrand> cause it is deb packaging, you have lots of choices :)
[15:36] <zyga-suse> yes :-)
[15:36]  * jdstrand notes that the xenial Uuntu archive packaging does not seem to be derived from packaging/ubuntu*
[15:36] <jdstrand> Ubuntu*
[15:37] <zyga-suse> if that's the case we should fix that for 2.28
[15:37] <jdstrand> I don't know what is used where, so to fix this in SRU, just make sure things are updated everywhere it needs to be
[15:38] <jdstrand> zyga-suse: that might violate the SRU process. I'm not familiar with the agreements there. please discuss with mvo before transitioning 2.28 in the Ubuntu archive to packaging/ubuntu*
[15:38] <zyga-suse> ah, indeed... man this stuff is so hairy
[15:38] <zyga-suse> mvo:  I thought that each upload to xenial proposed was using this packaging
[15:38] <zyga-suse> are you maintaining a separate package somewhere for xenial?
[15:40] <jdstrand> oh, maybe it is
[15:40] <Pharaoh_Atem> mvo is using packaging/ubuntu*
[15:40] <jdstrand> I looked at the one from -updates rather than -proposed
[15:40] <Pharaoh_Atem> that's why he's got the weird symlink for gbp
[15:41] <jdstrand> the one from -proposed looks like packaging/ubuntu*
[15:41] <jdstrand> zyga-suse: ^
[15:41] <zyga-suse> aha
[15:41] <jdstrand> zyga-suse: ie, I think you are good with updating packcaging/ubuntu*
[15:41] <zyga-suse> well, that's nice then
[15:41] <jdstrand> yeah, sorry for the brief noise
[15:42] <zyga-suse> jdstrand: can you look at https://github.com/snapcore/snapd/pull/3789 while I'm fiddling with debian please?
[15:42] <mup> PR snapd#3789: cmd/snap-confine: genearlize apparmor profile for various lib layout <Created by zyga> <https://github.com/snapcore/snapd/pull/3789>
[15:43] <jdstrand> zyga-suse: so, you can move 'dh_apparmor --profile-name=usr.lib.snapd.snap-confine.real -psnapd' from override_dh_installdeb to the top of override_dh_systemd_start, *or* modify postinst to put what dh_appamor would do up above #DEBHELPER# in snapd.postinst
[15:44] <jdstrand> the former is probably best, since you will benefit from bug fixes to dh_apparmor
[15:46] <jdstrand> zyga-suse: yes, I can do that
[15:58] <jdstrand> zyga-suse: you dropped a rule (libdl is specified in one but not the other)
[15:59] <zyga-suse> oh, ineede
[16:00] <zyga-suse> corrected
[16:00] <zyga-suse> I just pasted your earlier rules from solus PRs
[16:01] <zyga-suse> good catch!
[16:02] <jdstrand> zyga-suse: yes, but you pasted only one of the comments :) there were two comments, one for each section :)
[16:03] <jdstrand> zyga-suse: approved
[16:05] <zyga-suse> thank you
[16:05] <zyga-suse> I see, I will recheck that comment then
[16:06] <zyga-suse> jdstrand: can you please look at the topmost three patches in https://github.com/zyga/snapd/commits/tweak/opensuse-snap-confine-aa-profile
[16:06] <zyga-suse> I think the deeper two are okay
[16:06] <zyga-suse> but the topmost one is surely not landable
[16:06] <zyga-suse> ideas welcome
[16:06] <zyga-suse> I will post them later today (I need to break now)
[16:12] <mup> PR snapd#3790 opened: cmd/snap-confine: allow reading /proc/filesystems <Created by zyga> <https://github.com/snapcore/snapd/pull/3790>
[16:12] <pedronis> niemeyer: both me and noise][ added some comments on the 'generic'/generic-serials topic,  I made the PR but I marked it blocked because it's a bit unclear whether is really what we want atm
[16:13] <niemeyer> pedronis: Thanks, will follow up there
[16:13]  * zyga-suse -> off for supper/break
[16:15] <pedronis> mvo: I reviewed a couple fo small 2.28 PRs,  as I said mine is blocked atm,  I also created some small repair PRs, OTOH snapd#3616 is the next important unreviewed one
[16:15] <mup> PR snapd#3616: cmd/snap-repair: implement Runner.Verify and use it to check signatures of repairs from Next <Critical> <Created by pedronis> <https://github.com/snapcore/snapd/pull/3616>
[16:17] <Chipaca> zyga-suse: how do i fix the “cannot update snap namespace: cannot switch mount namespace: invalid argument” again?
[16:20] <mvo> pedronis: does 3616 needs a normal review or a gustavo reivew?
[16:21] <mvo> morphis, jdstrand: I'm very confused about the seccomp issue, what does "SNAPD_DEBUG=1 snap run anbox" show? this should print some re-exec related debug
[16:23] <mup> PR snapd#3756 closed: corecfg: deal with system.power-key-action="" correctly <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3756>
[16:24] <mup> PR snapd#3754 closed: corecfg: fix proxy.* writing and add integration test <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3754>
[16:36] <Chipaca> niemeyer: why does “N** become “write: (N-1)” and not “write: N”?
[16:36] <Chipaca> the forum takeover of ^F stopping from doing actual useful in-page search is annoying
[16:37] <niemeyer> Chipaca: Re. forum, sometimes it indeed feels awkward, but the thinking is probably that things both load and unload dynamically on the page, so CTRL-F is not super useful as it is either
[16:37] <niemeyer> Chipaca: About the **, I suggest seeing the video on the first message
[16:37] <Chipaca> i know -- still annoying :-)
[16:37] <niemeyer> It is
[16:38] <morphis> mvo: 2017/08/23 18:37:15.673897 cmd.go:103: DEBUG: core snap (at "/snap/core/current") is older ("2.26.14") than distribution package ("2.27.3")
[16:38] <niemeyer> Chipaca: It explains in detail what it's supposed to do and why it's sort of a unique case
[16:38] <niemeyer> Chipaca: I'd guess it's about half-way through the video
[16:38] <Chipaca> fair 'nuf (i thought i knew, but ¯\_(ツ)_/¯)
[16:38] <niemeyer> Chipaca: CTRL-F won't help on the video either, unfortunately :P
[16:39] <Chipaca> niemeyer: but “play at 2× speed” does :-)
[16:40] <niemeyer> Chipaca: Yeah, love it as well
[16:40] <Chipaca> niemeyer: also love “play at 0.75×” when learning music
[16:40] <niemeyer> Chipaca: Recently I also used /2 for the first time
[16:41] <Chipaca> niemeyer: also: mplayer -af scaletempo=scale=1.0 -speed 0.75
[16:42] <niemeyer> Chipaca: It was a super dense topic, and the speaker felt like speaking at 3x naturally
[16:42] <niemeyer> Chipaca: I actually doubted he was able to speak with so much precision about so many things so fast, and thought the video was sped up.. but looking at other hints the video, no.. the guy was just inhuman..
[16:44] <niemeyer> Chipaca: Nice, I didn't know about scaletempo
[16:44] <pedronis> mvo: both
[16:45] <Chipaca> niemeyer: the order of -af and -speed is important in that (if you set the speed before doing the scale, it's does the opposite and you need to specify speed=pitch
[16:46] <niemeyer> Aha, makes sense
[16:46] <niemeyer> Chipaca: This is the one if you're curious: https://www.youtube.com/watch?v=j-A0mwsJRmk
[16:46] <pedronis> mvo: it has zero reviews atm
[16:51] <Chipaca> niemeyer: yeah, as i understand it 1** meant “reads 0 and 1, writes 0 and/or 1”
[16:52] <niemeyer> Chipaca: Yeah, I'm trying to avoid [0, 1], but so far no good ideas have shown up
[16:52] <niemeyer> Perhaps we don't need to avoid it.. this is really an extremely atypical case and I expect a handful of snaps to use it
[16:53] <Chipaca> niemeyer: “and/or” is perhaps too handwavy to codify though
[16:53] <Chipaca> niemeyer: is there a resason read is an ordered array instead of two endpoints?
[16:53] <niemeyer> Chipaca: Yeah, the thinking will need to change from write N to write compatible with N
[16:53] <Chipaca> do we really want to support a snap reading 1, 2 and 4 but not 3/
[16:54] <Chipaca> to me an epoch/read_{min,max} and epoch/write, epoch/compat, would cover how it's in my head
[16:56] <niemeyer> Chipaca: We went from a simplistic view of the world to one that respects its craziness while keeping a way to represent it simplistically.. we should probably not give up on acking that the world is crazy :)
[16:56] <niemeyer> Chipaca: e.g.
[16:57] <niemeyer> Chipaca: epoch 1 is a stable, epoch 2 was an alpha revision that broke the format, epoch 3 is the new stable
[16:57] <niemeyer> Chipaca: It's quite plausible to have a 3 that can handle read 1 but not 2
[16:58] <Chipaca> niemeyer: true dat
[16:58] <Chipaca> niemeyer: wrt crazy, thinking about people that actually want a version of “master@87da2fcf1d0925f489985323303531507b5ed537”
[17:00] <mvo> morphis: do you get output or is anabox immediately killed when you run the above command?
[17:01] <niemeyer> Chipaca: I don't think we should allow those
[17:02] <Chipaca> niemeyer: i agree -- we want them human readable
[17:02] <niemeyer> Yeah, this is clearly oriented for a machine to read
[17:02] <mvo> morphis: fwiw, this debug output looks sane SNAP_CONFINE_DEBUG=1 snap run anabox
[17:02] <niemeyer> and will corrupt the output of everybody else if shown entirely
[17:03] <morphis> mvo: https://paste.ubuntu.com/25377571/
[17:03] <morphis> mvo: no, the process doesn't get killed
[17:03] <morphis> the process will spawn an LXC container
[17:03] <morphis> and processes inside that will get killed later on
[17:06] <Chipaca> mvo: is test-snapd-dbus-service yours?
[17:19] <mvo> Chipaca: I think so
[17:20] <Chipaca> mvo: i was going to ask you to build it for i386, but i changed my test instead
[17:20] <Chipaca> mvo: because i'm lazy _and_ impatient
[17:20] <mvo> morphis: oh, that is interessting, so it does not get killed but the daemon part of it is?
[17:20] <mvo> Chipaca: heh, ok
[17:20] <mvo> Chipaca: I think we just need to upload it to LP to make it build there
[17:24] <mup> PR snapcraft#1501 closed: tests: use assertThat instead of assertEqual <Created by elopio> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/1501>
[17:26] <mvo> morphis: I think we need to continue to debug this tomorrow morning, I will ping you if thats ok
[17:56] <niemeyer> morphis, mvo: Provided some feedback on the userd PR..
[17:56] <niemeyer> Some simple recommended changes and one issue to think through
[18:18] <zyga-suse> re
[18:18] <zyga-suse> Chipaca: hmmmm, not sure, is this on 14.04?
[18:19]  * zyga-suse gives up on backlog
[18:20] <zyga-suse> Chipaca: did you resolve that issue?
[18:20] <kyrofa> roadmr, any idea what's happening here? https://pasteboard.co/GH176X7.png
[18:21] <mup> PR snapd#3789 closed: cmd/snap-confine: genearlize apparmor profile for various lib layout <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/3789>
[18:22] <mup> PR snapd#3791 opened: cmd/snap-confine: allow using additional libraries required by openSUSE <Created by zyga> <https://github.com/snapcore/snapd/pull/3791>
[18:22] <zyga-suse> jdstrand: ^ a small PR with extra libraries for snap-confine
[18:23] <kyrofa> roadmr, I can't upload that snap again, but I should not have to request manual review
[18:27] <zyga-suse> jdstrand: thank you for the review
[18:27] <zyga-suse> jdstrand: last of the trivial PRs that' s likely to go in is https://github.com/snapcore/snapd/pull/3790
[18:27] <mup> PR snapd#3790: cmd/snap-confine: allow reading /proc/filesystems <Created by zyga> <https://github.com/snapcore/snapd/pull/3790>
[18:28] <zyga-suse> jdstrand: this is the RFC branch that I don't expect to land https://github.com/snapcore/snapd/pull/3792
[18:28] <mup> PR snapd#3792: cmd/snap-confine: allow running snap-exec without confinement <Created by zyga> <https://github.com/snapcore/snapd/pull/3792>
[18:29]  * zyga-suse gets back to layout
[18:29] <mup> PR snapd#3792 opened: cmd/snap-confine: allow running snap-exec without confinement <Created by zyga> <https://github.com/snapcore/snapd/pull/3792>
[18:29] <jdstrand> roadmr: can you sync r919? (fix for 1712476, unrelated to my previous request to you)
[18:33] <zyga-suse> I just found a huge downside of using snaps :-(
[18:33] <zyga-suse> when you are working on snap-confine and other lower parts
[18:33] <zyga-suse> and you tweak things
[18:33] <zyga-suse> and your music player or video player breaks
[18:33] <zyga-suse> man that is annoying
[18:33] <zyga-suse> (those apps are snaps)
[18:34] <roadmr> jdstrand: sure, I'll put it in the queue
[18:34] <roadmr> kyrofa: let me see
[18:34] <jdstrand> zyga-suse: done
[18:34] <jdstrand> zyga-suse: you should use a vm :)
[18:34] <zyga-suse> jdstrand: thank you again
[18:34] <zyga-suse> jdstrand: I have plenty but I also like to work natively
[18:34] <roadmr> kyrofa: which snap is this?
[18:35] <kyrofa> roadmr, revisions 2692, 2678, and 2684 of nextcloud
[18:38] <jdstrand> zyga-suse: play music from your vm :P
[18:38] <jdstrand> roadmr: thanks!
[18:38] <zyga-suse> man, did you try that? it's terrible
[18:38] <zyga-suse> choppy and no video acceleration that works
[18:38] <roadmr> jdstrand: btw the latest update (913, right?) was deployed earlier today \o/
[18:39] <mup> PR snapd#3790 closed: cmd/snap-confine: allow reading /proc/filesystems <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/3790>
[18:40] <jdstrand> roadmr: awesome, thanks! :)
[18:40] <jdstrand> zyga-suse: I was joking :)
[18:41] <jdstrand> I've gotten to where I don't do any dev work on my system except for the occasional live policy updates
[18:41] <Chipaca> zyga-suse: no, i just used a different snap to test
[18:41] <jdstrand> everything is vm
[18:42] <jdstrand> s
[18:42] <zyga-suse> Chipaca: I'd appreciate if you could run that again
[18:42] <zyga-suse> Chipaca: with SNAP_CONFINE_DEBUG=1
[18:42] <Chipaca> sure, 1 sec
[18:44] <Chipaca> zyga-suse: SNAP_CONFINE_DEBUG=1 doesn't seem to have changed anything
[18:44] <Chipaca> zyga-suse: what a i looking for?
[18:44] <zyga-suse> Chipaca: what's the last few things
[18:44] <Chipaca> zyga-suse: where
[18:44] <zyga-suse> and did you get any denials/messages logged?
[18:44] <zyga-suse> like nothing at all?
[18:44] <zyga-suse> SNAP_CONFINE_DEBUG=yes hello
[18:45] <zyga-suse> or whatever that snap was
[18:45] <Chipaca> zyga-suse: "hello"?
[18:45] <Chipaca> i mean
[18:45] <Chipaca> the snap is not installed
[18:45] <Chipaca> zyga-suse: http://pastebin.ubuntu.com/25378106/
[18:45] <zyga-suse> Chipaca: any snap that you ran this with before that failed with EINVAL
[18:46] <zyga-suse> I see
[18:46] <zyga-suse> hmm hmm hmm
[18:46] <zyga-suse> I see
[18:46] <Chipaca> (as per usual, this might be me needing to install the right thing)
[18:46]  * zyga-suse assumed something else
[18:46] <Chipaca> (but i thought we'd fixed that)
[18:46] <zyga-suse> is this a make-hack experience?
[18:46] <Chipaca> it is
[18:46] <zyga-suse> I think it was a different case before
[18:46] <zyga-suse> wait
[18:46] <Chipaca> zyga-suse: you want me to see if doing this with stock snapd works?
[18:46] <zyga-suse> unless it's one of the versions
[18:47] <zyga-suse> just install snap-update-ns in distro location
[18:47] <zyga-suse> and see if that fixes it
[18:47] <Chipaca> 1 sec
[18:47] <roadmr> jdstrand: ironic :) "Store upload scan failed for review-tools [...] Unexpected output from click-review.
[18:47] <roadmr> jdstrand: (which btw looks very similar to what kyrofa reported)
[18:48] <kyrofa> roadmr, jdstrand yeah I just got several more emails about it as well. Something is busted
[18:48] <Chipaca> lel
[18:48] <roadmr> kyrofa: indeed
[18:48] <Chipaca> zyga-suse: yes that fixed it
[18:48]  * zyga-suse wants that release to go out one day
[18:48] <Chipaca> zyga-suse: i thought we'd changed 'make hack' to copy that one as well?
[18:48]  * Chipaca makes a note to push a PR for this
[18:48] <zyga-suse> no, I don't think we did
[18:48] <zyga-suse> we should
[18:48] <zyga-suse> thanks!
[18:48] <Chipaca> ok, i'll do it
[18:48] <Chipaca> but after dinner and dogwalk
[18:54] <jdstrand> roadmr: that is probably the cleanup code. it will output stuff if there are lingering things to clean
[18:54] <roadmr> jdstrand: aha. Is that in r913?
[18:55] <roadmr> jdstrand: it happened in your snap and kyrofa's but not on mine, which is a trivial minimal snap... which kinda makes sense
[18:56] <jdstrand> roadmr: yes, 913 (also 919)
[18:56] <jdstrand> roadmr, kyrofa: r2698 passed review
[18:57] <roadmr> yep, that's slightly suspicious - why some and not others?
[18:57] <jdstrand> roadmr: you have more than one review machine?
[18:57] <Chipaca> zyga-suse: how do i rebuild the makefile?
[18:57] <roadmr> jdstrand: yes, reviews run on app servers and there are 4 of them
[18:57] <Chipaca> just ./autogen.sh?
[18:57] <zyga-suse> Chipaca: automake
[18:58] <zyga-suse> Chipaca: I _may_ have a nicer  build system on the horizon
[18:58] <jdstrand> roadmr: right, so the one that tried the libreoffice snap will have a directory that didn't get cleaned
[18:58] <zyga-suse> Chipaca: but for now you just need automake
[18:58] <jdstrand> roadmr: (the bug r919 fixes)
[18:58] <roadmr> click-review returned unexpected output for package /tmp/tmplOHGz7/nextcloud.nextcloud_master-2017-08-23_i386.click: ERROR: Could not find '/tmp/review-tools-p1e8wr9d' Removing old review '/tmp/review-tools-skxfgd28'
[18:58] <Chipaca> in my experience "nicer build system" just means "build system that is more naive about the twisted ways of the world"
[18:58] <roadmr> jdstrand: that's for kyrofa's package, this is for yours: click-review returned unexpected output for package /tmp/tmpwU84wp/review-tools.jdstrand_0.47+git_i386.click: ERROR: Could not find '/tmp/review-tools-skxfgd28'
[18:59] <jdstrand> roadmr: so it will try to clean it. until r919 is rolled out, it will fail eith PermissionError. when r919 hits, a msg() to stdout will state it is being cleaned
[18:59] <zyga-suse> no, just it doesn't have a 5000 line shell crap in case you are on that broken release of AIX on this 36 bit processor using that non-ascii encoding
[19:00] <jdstrand> roadmr: I don't understand the review-tools one... that is suggesting the directory went away
[19:00] <roadmr> jdstrand: interestingly, note that both log messages mention /tmp/review-tools-skxfgd28
[19:01] <roadmr> hm, but that doesn't seem to be it. This is another nextcloud one:   click-review returned unexpected output for package /tmp/tmpF5KzHz/nextcloud.nextcloud_11-2017-08-23_i386.click: ERROR: Could not find '/tmp/review-tools-7p_jt1us'
[19:01] <roadmr> click-review returned unexpected output for package /tmp/tmpbFMjHa/zeronet-js.mkg20001_0.0.1-alpha17_armhf.click: ERROR: Could not find '/tmp/review-tools-vuwvtkw3' ??
[19:01] <jdstrand> roadmr: do multiple reviews happen concurrently on these servers?
[19:02] <roadmr> jdstrand: hm, potentially, yes
[19:02] <roadmr> are they racing each other? :(
[19:02] <jdstrand> roadmr: ok, r919 fixes that
[19:02] <roadmr> jdstrand: yay :) well it's already in the pipeline, if so, the fastest way is to just stay the course until I can roll out r919
[19:02]  * roadmr checks for clean runway and so on
[19:03] <jdstrand> roadmr: I'll get rid of that msg() too and turn it into debug() for r920
[19:03] <roadmr> thanks... ugh I hope it doesn't continue to output spurious stuff which confuses sca :(
[19:05] <jdstrand> roadmr: that is what r920 will address. r919 fixed a short-circuit value for 'maxage' in the reaping code. it should've been 'only reap if the stale review dir is older than 3 hours'
[19:05] <jdstrand> roadmr: I can up that to anything. I figured a review that was 3 hours old was clearly hung
[19:05] <roadmr> yes, I agree
[19:06] <roadmr> jdstrand: so is it that two concurrent reviews are trying to reap the same temporary dir?
[19:07] <jdstrand> roadmr: in r913, yes
[19:07] <jdstrand> roadmr: in r919, no. but, please hold off r913 for a moment
[19:08] <jdstrand> I'll have an r920 in just a moment
[19:08] <roadmr> oh you meant "hold off r919" :)
[19:09] <jdstrand> roadmr: yes. want to test the use of debug()
[19:09] <roadmr> ok
[19:15] <Chipaca> zyga-suse: comme çi? https://github.com/snapcore/snapd/compare/master...chipaca:make-hack-update-ns
[19:15] <zyga-suse> mmm
[19:16] <zyga-suse> nice use of $(or)
[19:16] <zyga-suse> you want to use $(wildcard *.go) I think
[19:16] <zyga-suse> same for others
[19:17] <zyga-suse> I don't think *.go does what you want
[19:17] <zyga-suse> and I think you want to use 755
[19:17] <zyga-suse> not 644 :D
[19:17] <roadmr> jdstrand: merges are kinda fire-and-forget, I did my best to try and stop r919 from landing so we don't waste time testing/building that. Not sure I succeeded :)
[19:17] <zyga-suse> other than that I think it looks ok
[19:18] <jdstrand> roadmr: no big deal. that will fix the concurrency issue (sorry about that)
[19:19] <Chipaca> zyga-suse: lel
[19:19] <Chipaca> zyga-suse: about the 755
[19:19] <Chipaca> zyga-suse: about *.foo AIUI it works in prereq's
[19:19] <jdstrand> roadmr: r920 is now committed. it will fix the output to not confuse SCA and should properly cleanup the bad libreoffice snap review directory
[19:19] <zyga-suse> I somewhat dobut that (in automake many things are weridly broken)
[19:20] <zyga-suse> though... let mecheck
[19:20] <roadmr> jdstrand: woohoo, I'll propose the r920 merge
[19:20] <zyga-suse> Chipaca: ah, you are corret
[19:20] <zyga-suse> *correct
[19:21] <zyga-suse> Chipaca: globs work when used directly
[19:21] <Chipaca> yup
[19:21] <zyga-suse> Chipaca: don't work when using variables, hence the function $(wildcard)
[19:21] <Chipaca> zyga-suse: if i'm editing a makefile, i have 'info make' open :-)
[19:21] <zyga-suse> haha, guess what I did
[19:21] <Chipaca> because otherwise i've got to _remember_ this stuff
[19:22] <zyga-suse> I love make though
[19:22] <zyga-suse> even if it sucks
[19:22] <roadmr> haha
[19:22]  * Chipaca is not arguing
[19:22] <kyrofa> roadmr, can you let me know when it's deployed? I don't know which snap goes into which channel, so I'll need to re-spin all my builds
[19:23] <jdstrand> roadmr: it is weird that the bum libreoffice snap happened at the same time as the r913 rollout, but it is good that it did-- it made it clear what was happening there
[19:23] <roadmr> kyrofa: sure thing. Sorry about that :( it'll be about 2 hours but I'll ping here
[19:23] <kyrofa> Thank you
[19:24] <roadmr> jdstrand: I saw that error in a bunch of units (I think all of them actually) - so unless libreoffice built e.g. 4 snaps and they all hit different units, I think it was more about the race condition (the "unexpected output" thing, I mean)
[19:24] <roadmr> at least I do get visibility on that unexpected output via kibana
[19:25] <jdstrand> roadmr: yes, there were two issues. one was concurrent, one was the lo snap
[19:25] <roadmr> jdstrand: when it rains it pours eh? :) (and 2 drops constitute a downpour haha)
[19:25] <mup> PR snapd#3793 opened: cmd: "make hack" now also installs snap-update-ns <Created by chipaca> <https://github.com/snapcore/snapd/pull/3793>
[19:25] <jdstrand> but both would've given unexpected output
[19:25] <Chipaca> zyga-suse: ^
[19:26] <jdstrand> anyway, r920 should fix that
[19:26] <roadmr> great :)
[19:26] <zyga-suse> thank you
[19:27] <Chipaca> now let's go walk that hound
[19:27]  * roadmr barks in approval
[19:28] <Chipaca> roadmr: I only have the short lead for you i'm afraid
[19:28] <Chipaca> also, i'm not picking up your poo
[19:28] <Chipaca> just FYI
[19:29]  * roadmr wags tail
[19:29] <roadmr> haha
[20:02] <mup> PR snapd#3794 opened: asserts,overlord/devicestate: revert allowing 'generic'-signed serials <Created by pedronis> <https://github.com/snapcore/snapd/pull/3794>
[20:24] <mup> PR snapd#3526 closed: hooks: support for refresh hook <Created by stolowski> <Merged by niemeyer> <https://github.com/snapcore/snapd/pull/3526>
[20:46] <janisozaur> zyga-arch: sorry, can't help today
[21:02] <zyga-suse> ikey: hey, offtopic, how to debug solus getting no keyboard input when installed in gnome-boxes?
[21:02] <zyga-suse> ikey: could it be related to my locale (pl)?
[21:21] <mup> PR snapcraft#1504 opened: project loader: refactor into package <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/1504>
[21:32] <roadmr> kyrofa: that rollout you're waiting for is in progress, should be about 10-15 mins. I'll confirm when it's done
[21:33] <kyrofa> Great, thanks roadmr
[21:42] <roadmr> kyrofa: ok, done - let it rip.
[21:42] <roadmr> jdstrand: ^^ same
[21:49] <jdstrand> roadmr: thanks!
[21:51] <jdstrand> roadmr: fyi, there is r922 which is not urgent. It makes it so any error during the reap results in a syslog entry (instead of a traceback)
[21:51] <roadmr> nice!
[21:51] <roadmr> I'll make a note to put in a merge for that tomorrow; I'm almost EOD and don't want to leave it dangling today :(
[21:52] <jdstrand> roadmr: r921 has a new yaml check (ie, standard stuff)
[21:52] <jdstrand> roadmr: sure, np at all. thanks for doing it! :)
[21:52] <jdstrand> stgraber: fyi ^
[22:00] <stgraber> jdstrand: sounds good