[06:05] <mborzecki> morning
[06:13] <zyga> good morning :)
[06:14] <zyga> mborzecki: I pushed missing patches from master into https://github.com/snapcore/snapd/pull/6235
[06:14] <mup> PR #6235: overlord,apparmor: new syskey behaviour + non-ignored snap-confine profile errors (2.36) <Created by zyga> <https://github.com/snapcore/snapd/pull/6235>
[06:14] <zyga> the reference is https://github.com/snapcore/snapd/pull/6233
[06:14] <mup> PR #6233: overlord: don't write system key if security setup fails <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6233>
[06:14] <mborzecki> oh, and it's green now
[06:14] <zyga> there are a few extra lines of diff
[06:15] <zyga> because that branch combines another small patch from master
[06:15] <zyga> namely https://github.com/snapcore/snapd/pull/6235/commits/cbc6e3e029e36e2d120a3dbb2edad39f689da479
[06:15] <mup> PR #6235: overlord,apparmor: new syskey behaviour + non-ignored snap-confine profile errors (2.36) <Created by zyga> <https://github.com/snapcore/snapd/pull/6235>
[06:15] <zyga> I think it's good to go :)
[06:19] <zyga> mborzecki: I think we can merge https://github.com/snapcore/snapd/pull/6248
[06:19] <mup> PR #6248: tests/lib/reset: restore context of removed snapd directories <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6248>
[06:20] <mup> PR snapd#6248 closed: tests/lib/reset: restore context of removed snapd directories <Simple 😃> <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/6248>
[06:21] <zyga> a pair of simple looking conflicts on https://github.com/snapcore/snapd/pull/6238
[06:21] <mup> PR #6238: [RFC] many: add minimal SELinux support, refactor the policy <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6238>
[06:22] <zyga> mborzecki: did you pick that keyboard? :)
[06:22] <mborzecki> zyga: no, not yet
[06:23] <zyga> mborzecki: pawel has apparently moved his work onto the new machine
[06:23] <zyga> expectedly, because it's the quietest and fastest machine he has now
[06:23] <zyga> I'm curious if you will experiment with vmware as well :)
[06:24] <mup> PR snapd#6256 closed: snap: add new `snap run --trace-exec` call (2.36) <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/6256>
[06:34] <zyga> mborzecki: I resolved conflicts on https://github.com/snapcore/snapd/pull/6193/files
[06:34] <mup> PR #6193: spread: drop Fedora 27, add Fedora 29 <⛔ Blocked> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6193>
[06:34] <zyga> though I made F29 manual so this can land
[06:34] <zyga> it should pass with just F28
[06:52] <zyga> Quick school run
[07:28] <zyga> Re
[07:50] <mup> PR snapd#6258 opened: WIP: interfaces: support D-Bus activation <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/6258>
[07:58] <zyga> and back now
[07:58] <zyga> jamesh: hey :)
[07:58] <jamesh> hi zyga
[07:59] <zyga> I will try to review all branches this week
[07:59] <zyga> but we have a backlog and limited capacity
[08:00] <jamesh> zyga: no problem.  I'm still working on the dbus activation one: the PR is more for CI feedback
[08:00] <jamesh> (that's why I marked it WIP)
[08:00]  * zyga loves to see :: targets used in makefiles
[08:06] <pstolowski> mornings
[08:08] <mborzecki> pstolowski: hey
[08:11] <zyga> hey pawel :)
[08:22] <zyga> https://github.com/snapcore/snapd/pull/6257 is green
[08:22] <mup> PR #6257: testutils: split checkers, tweak tests <Created by zyga> <https://github.com/snapcore/snapd/pull/6257>
[08:22] <zyga> and very long :-)
[08:24] <zyga> nvidia physix is BSD now
[08:24] <zyga> that's impressive
[08:24] <zyga> wonder if nvidia will have a change of strategy for their other software
[08:25] <zyga> mborzecki: hey
[08:25] <zyga> mborzecki: I think we should do something about that snapd.mk PR
[08:26] <mborzecki> did that land?
[08:26] <zyga> I would like to resurrect it, make minimal modifications as discussed (move variable generation back to .spec file and have snapd.mk include vars.mk)
[08:26] <zyga> and land it
[08:26] <zyga> looking at the PR from james, there will be new files, new things to look at
[08:28] <zyga> uh oh
[08:28] <zyga> The job exceeded the maximum log length, and has been terminated.
[08:28] <zyga> travis now kills jobs that log too much
[08:28] <zyga> perhaps we should tweak spread to log less stuff
[08:28] <zyga> and definitely make rpm builds quieter
[08:29] <mborzecki> zyga: yeah, 2.36 branch is failing atm because of this i think
[08:30] <zyga> one other thing I would silence is the package name selection
[08:30] <zyga> that is, mapping from ubuntu names to other names
[08:30] <mborzecki> zyga: or sort of, something else fails, but travis terminates the job :)
[08:30] <zyga> that's extremely verbose
[08:30] <mborzecki> set +too-much-gibberish-x
[08:50] <zyga> trying some simple changes to reduce the log size now
[08:54] <zyga> mborzecki: do you know if it is possible to query the state of "set" settings?
[08:55] <mborzecki> zyga: snap get ?
[08:55] <mborzecki> snap get -d
[08:55] <zyga> no no
[08:55] <zyga> I mean in shell
[08:55] <zyga> set -x vs set +x
[08:55] <mborzecki> aah
[08:55] <zyga> there's nothing I can find quickly
[08:59] <mborzecki> zyga: set -o should do the trick
[08:59] <zyga> ah, neat, thanks!
[09:00] <mborzecki> zyga: xtrace is apparently the option
[09:00] <zyga> yes
[09:00] <mborzecki> and even supported by zsh :)
[09:02] <zyga> quick patch for reduced verbosity https://www.irccloud.com/pastebin/o15U1EqK/
[09:06] <mup> PR snapd#6259 opened: tests: reduce verbosity around package installation <Simple 😃> <⚠ Critical> <Created by zyga> <https://github.com/snapcore/snapd/pull/6259>
[09:06] <zyga> mborzecki, pstolowski: ^
[09:07] <pstolowski> ok
[09:08] <zyga> mborzecki: do you want to discuss the connections API?
[09:08] <zyga> just looking at some random part of trace log
[09:08] <zyga> I would love to have
[09:08] <zyga> snap debug is-connected
[09:08] <zyga> snap debug connected-slots SNAP:PLUG
[09:08] <zyga> snap debug connected-plugs SNAP:SLOT
[09:09] <zyga> snap debug connecte-snaps SNAP:<PLUG-OR-SLOT>
[09:09] <zyga> (typos aside, you get the idea)
[09:09] <mborzecki> mhm
[09:09] <zyga> ideally all fuelled by a new API call
[09:10] <zyga> perhaps those could also be visible via snapctl
[09:10] <zyga> pstolowski: ^ do we have something like that for snapctl today?
[09:11] <pstolowski> zyga: no, unfortunately not
[09:11] <zyga> we might expose it in both places eventually
[09:11] <zyga> feels like something useful to snap developers
[09:12] <zyga> mborzecki: the general idea feels like
[09:12] <zyga> mborzecki: connections <snap:plug> <snap:slot> <interface> <string> <flags>
[09:12] <zyga> interface optionally limits to interface type
[09:12] <zyga> string optionally filters by name of plug/slot
[09:13] <zyga> snap:plug and snap:slot can be empty or partially empty
[09:13] <zyga> (perhaps string is useless actually)
[09:13] <zyga> flags can alter the set of returned data (e.g. attributes) or visible things (e.g. show disconnected things as well as connected, for hotplug)
[09:13] <zyga> it seems it could power all the commands mentioned above
[09:14] <zyga> all of the options would probably be packed into a query structure
[09:14] <zyga> we might also have a flag for controling core/gadget connections
[09:14] <zyga> (e.g. skip them or include them)
[09:15] <zyga> so that the client code can stop having to special case "system/snapd/core/ubuntu-core"
[09:17] <mborzecki> zyga: let's have a chat ~11, 1115?
[09:17] <zyga> sure
[09:22] <mborzecki> #6193 is green
[09:22] <mup> PR #6193: spread: drop Fedora 27, add Fedora 29 <⛔ Blocked> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6193>
[09:32] <zyga> merge it
[09:32] <zyga> we will solve the 29 blocked aspect separately
[09:34] <pstolowski> zyga: looking at 6257
[09:34] <mborzecki> zyga: about fontconfig again, desktop interface mount of /var/cache/fontconfig is probably a problem
[09:34] <zyga> mmm
[09:34] <zyga> but that only applies to strictly confined apps
[09:35] <zyga> isn't the problem you experienced only affecting classic confined apps?
[09:38] <mborzecki> zyga: i'm still bit worried that we're generating that cache in a location that does not match distro choice, with the way things are set up now it should work, we're going to make /var/cache/fontconfig ourselves after all
[09:38] <zyga> mborzecki: yes
[09:39] <zyga> maybe we should really build a cache in /var/cache/snapd/fontconfig
[09:39] <mborzecki> zyga: righ, but the the classic snaps that use libfontconfig from core will end up poking /var/cache/fontconfig ;)
[09:40] <zyga> yes
[09:40] <mborzecki> tradeoffs everywhere
[09:40] <zyga> but for those we have no way out
[09:40] <zyga> until we can attack  mount namespaces for classic confined software
[09:41] <mborzecki> mhm
[09:42] <zyga> mborzecki: updated https://github.com/snapcore/snapd/pull/6251
[09:42] <mup> PR #6251: cmd/snap-confine: refactor calling snapd tools into helper module <Created by zyga> <https://github.com/snapcore/snapd/pull/6251>
[09:42] <pstolowski> #6180 is green and needs (re)reviews
[09:42] <mup> PR #6180: snap/info: bind global plugs/slots to implicit hooks <Created by stolowski> <https://github.com/snapcore/snapd/pull/6180>
[09:47] <mborzecki> zyga: pstolowski: can you take a look https://github.com/snapcore/snapd/pull/6193 and +1 ?
[09:47] <mup> PR #6193: spread: drop Fedora 27, add Fedora 29 <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6193>
[09:48] <pstolowski> k
[09:50] <zyga> mborzecki: aha
[09:51] <mborzecki> pstolowski: and this one needs a 2nd review too: https://github.com/snapcore/snapd/pull/6246 super simple :)
[09:51] <mup> PR #6246: spread: show AVC audits when debugging, start auditd on Fedora <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6246>
[09:52] <mup> PR snapd#6193 closed: spread: drop Fedora 27, add Fedora 29 <Created by bboozzoo> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6193>
[09:52] <zyga> mborzecki: replied on https://github.com/snapcore/snapd/pull/6251
[09:52] <mup> PR #6251: cmd/snap-confine: refactor calling snapd tools into helper module <Created by zyga> <https://github.com/snapcore/snapd/pull/6251>
[09:53] <mup> PR snapd#6246 closed: spread: show AVC audits when debugging, start auditd on Fedora <Simple 😃> <Created by bboozzoo> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6246>
[09:53] <mborzecki> anyone knowledgeable of assertions left around? https://forum.snapcraft.io/t/cant-install-or-refresh-snaps-on-arch-linux/8690 the device is seeded, however there is no model assertion even though data.auth.model is already set to generic-classic
[09:53] <zyga> hmmm
[09:53] <zyga> I guess we just have to man up and read that code
[09:54] <zyga> pstolowski: on https://github.com/snapcore/snapd/pull/6257, shall I update all headers to 2018?
[09:54] <mup> PR #6257: testutils: split checkers, tweak tests <Created by zyga> <https://github.com/snapcore/snapd/pull/6257>
[09:55] <pstolowski> zyga: i'd use this opportunity
[09:55] <zyga> sure
[10:00] <zyga> pstolowski: done :)
[10:00] <pstolowski> ty !
[10:04] <mborzecki> suprising how travis jobs are succeeding now
[10:04] <zyga> pstolowski: this is not super urgent but it needs a 2nd review https://github.com/snapcore/snapd/pull/6244
[10:04] <mup> PR #6244: release: detect too old apparmor_parser <Created by zyga> <https://github.com/snapcore/snapd/pull/6244>
[10:05] <zyga> a much simpler variant of this went into 2.36
[10:05] <zyga> this one is just tidier and less noisy
[10:08] <mup> PR snapd#6090 closed: Granted 'account_control' access to /home/** <Created by stefannilsson> <Closed by zyga> <https://github.com/snapcore/snapd/pull/6090>
[10:10] <zyga> pstolowski, mborzecki: what do you guys think about https://github.com/snapcore/snapd/pull/6250
[10:10] <mup> PR #6250: data: set KillMode=process for snapd <Created by mvo5> <https://github.com/snapcore/snapd/pull/6250>
[10:11] <mup> PR snapd#6200 closed: tests: fix for tests test-*-cgroup <Created by sergiocazzolato> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6200>
[10:11] <zyga> perhaps cherry pick the detector I suggested and LGTM?
[10:11] <zyga> I'm sure mvo can iterate on top
[10:11] <zyga> but feels like the right thing to do
[10:19] <pstolowski> zyga: do apparmor_parser and snap-seccomp lock profiles? any risk of having profiles messed up if new instances start writing profiles while old ones are still running?
[10:20] <pstolowski> (but in general yes, i think that's the right step what this PR proposes)
[10:20] <zyga> they don't lock profiles because they don't have to AFAIK:
[10:20] <zyga> snap-seccomp writes a new file with atomic move
[10:21] <zyga> snapd writes source profiles with atomic move
[10:21] <pstolowski> ah, good then
[10:21] <zyga> apparmor_parser loads profiles into the kernel, kernel is the point of mutual exclusion
[10:21] <zyga> snap-confine loads profiles from disk only at startup
[10:21] <zyga> that holds the lock but that's not really a factor
[10:21] <zyga> seccomp profiles persist until process death
[10:22] <zyga> apparmor profiles are changed dynamically
[10:22] <mborzecki> uhh reached asserts db trying to find out what's going on
[10:22] <zyga> and?
[10:25] <mborzecki> got lost :)
[10:25] <mborzecki> back to devicemgr
[10:31] <mborzecki> i bet that device serial is set on that guy's machine, but it's unclear why that'd be the case and the model assertion would not be present
[10:33] <zyga> feels like we could use some commands to show stuff
[10:33] <zyga> more than "snap known"
[10:33] <zyga> which feels like a debug command, not an end-user thing
[10:33] <mborzecki> super unpleasant user command if anything :)
[10:34] <zyga> https://www.irccloud.com/pastebin/xNFfHNva/
[10:34] <zyga> hmmm
[10:34] <zyga> something really broken there :)
[10:34] <zyga> looking
[10:34] <mborzecki> nice
[10:35] <zyga> note that it uses dpkg on centos
[10:35] <mborzecki> heh right
[10:35] <mborzecki> snapd-xdg-open? i don't recall this in rpm
[10:36] <zyga> tests/upgrade/snapd-xdg-open/task.yaml
[10:36] <zyga> it runs on centos
[10:36] <zyga> but is clearly specific to ubuntu
[10:36] <mborzecki> is that a new task?
[10:36] <zyga> nope
[10:36] <zyga> look at it
[10:36] <zyga> wonder why the error never surfaced?
[10:36] <mborzecki> haha so how did it not fail before?
[10:36] <zyga> exactly
[10:36] <zyga> set -e is easy to lose
[10:37] <zyga> well
[10:37] <zyga> it failed now
[10:37] <zyga> I really wonder what I changed
[10:38] <zyga> I don't quite understand some of the logic in the pkgdb.sh
[10:46] <zyga> mborzecki: removed cloexec handling in https://github.com/snapcore/snapd/pull/6251
[10:46] <mup> PR #6251: cmd/snap-confine: refactor calling snapd tools into helper module <Created by zyga> <https://github.com/snapcore/snapd/pull/6251>
[10:47] <mborzecki> zyga: what was it?
[10:47] <zyga> mborzecki: will to fix that centos issue or shall I?
[10:47] <zyga> mborzecki: with cloexec?
[10:47] <zyga> mborzecki: just my debug wrappers
[10:47] <mborzecki> mhm
[10:47] <mborzecki> aah
[10:47] <mborzecki> shell scripts?
[10:47] <zyga> I didn't realise this is needed for scripts but not for programs
[10:47] <zyga> yep
[10:47] <mborzecki> yup, they mention that in the manpage
[10:47] <zyga> I test this way locally
[10:48] <zyga> yeah, in the BUGS section below :)
[10:48] <zyga> I missed that
[10:54] <mup> PR snapd#6235 closed: overlord,apparmor: new syskey behaviour + non-ignored snap-confine profile errors (2.36) <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6235>
[10:57] <zyga> mborzecki, pstolowski: my main feature branch that can land https://github.com/snapcore/snapd/pull/6149
[10:57] <mup> PR #6149: cmd/snap-confine: capture initialized per-user mount ns <Per-user mount ns  🐎> <Created by zyga> <https://github.com/snapcore/snapd/pull/6149>
[10:57] <zyga> I will have follow ups and tests once the feature snapd side is ready
[10:58] <pstolowski> zyga: k, will take a look
[10:58] <zyga> thank you!
[11:00] <zyga> mborzecki: I have a debug shell on centos 7 with that dpkg-query script
[11:00] <zyga> no idea why it would ever pass before
[11:01] <mborzecki> heh
[11:01] <zyga> disabled it, pushed
[11:01] <zyga> magic
[11:01] <mborzecki> btw. i'm playing with spread on fedora 29
[11:01] <mborzecki> DEBUG: running snap-device-helper add snap_test-snapd-udev-input-subsystem_slot /sys/class/mem/null 1:3
[11:01] <mborzecki> execl failed: No such file or directory
[11:01] <zyga> do you know my main issues with spread on fedora 29
[11:01] <zyga> right
[11:01] <mborzecki> snap-device-helper is a script, wtf?
[11:01] <zyga> I know about htis
[11:01] <zyga> *this
[11:01] <zyga> yes
[11:01] <zyga> but that's not the main problem
[11:01] <zyga> the problem is that we must never repackage core or snapd snaps
[11:01] <zyga> with stuff we've built on a random distro
[11:01] <zyga> that's a bad test
[11:01] <zyga> bad logic
[11:02] <zyga> and not representative of what ever happens in production
[11:02] <zyga> locally built snapd package gets injected into core snap
[11:02] <mborzecki> yes, wanted to reach that point and inspect what happens exactly
[11:02] <zyga> so libc from 2019 will be in a 2016 system
[11:02] <zyga> I mean, progs linked to 2019 will be in a 2016 system
[11:02] <zyga> what is wrong is that rpm helpers change the #!/ line
[11:03] <zyga> look at the shebang line in the source and in that system that is broken
[11:03] <zyga> the short term resolution:
[11:03] <zyga> stop repackaging core / snapd anywhere outside of 16.04
[11:03] <zyga> we can perhaps repackage snap-exec
[11:03] <zyga> but nothing more
[11:07] <mborzecki> woot, rpm learned some new tricks
[11:27] <zyga> mborzecki: mkosi got a huge static typing boost
[11:28] <zyga> https://github.com/systemd/mkosi/pull/300
[11:28] <mup> PR systemd/mkosi#300: Modify so that `flake8` and `mypy` are happy <enhancement> <Created by LukeShu> <https://github.com/systemd/mkosi/pull/300>
[11:31] <mborzecki> hm still got mixed feelings about python and type hints
[11:38] <zyga> really? that's like 99% of the error checking there :)
[11:41] <mup> PR snapd#6260 opened: spread, tests: use audit checkpoints when dumping audit log <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6260>
[11:41] <mborzecki> simple one ^^
[11:41] <mborzecki> zyga: it's just the type enforce part is missing
[11:42] <zyga> mborzecki: no need
[11:42] <zyga> mypy checks that it is true :)
[11:42] <zyga> if mypy says it's ok, the types match
[11:42] <zyga> reality is somewhat different as sometimes there are bugs in mypy and sometimes there are cases that are valid but the type system cannot express it
[11:42] <zyga> but really, apart from changing python in py4
[11:43] <zyga> this is the best way to write python now
[11:52] <zyga> eh
[11:52] <zyga> untangling the mess in config manager
[11:52] <zyga> sucks
[11:53] <zyga> cyclic import mess, that is
[11:53] <mborzecki> hah :)
[11:54] <zyga> I will never finish user mounts this way
[12:03] <zyga> I think I got it
[12:03] <zyga> I will propose a small branch with just the tweak to config
[12:06] <mup> PR snapd#6261 opened: tests/lib/prepare: make sure that SELinux context of repacked core snap is controlled <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6261>
[12:06] <mborzecki> another simple PR ^^
[12:06] <mborzecki> this time not dropping xattrs
[12:07] <zyga> eh
[12:07] <zyga> more cyclic shit
[12:07] <zyga> I hate this part of golang
[12:08] <zyga> LGTM with capitalisation of sentences
[12:10] <zyga> mborzecki: hmm
[12:10] <zyga> does set -o
[12:10] <zyga> does that set something in practice
[12:10] <zyga> because i have a feeling that that patch actually uncovered real bugs
[12:10] <cachio> zyga, hey
[12:10] <cachio> error: Failed build dependencies:
[12:10] <cachio> 	compiler(go-compiler) is needed by snapd-1337.2.36.2-0.el7.x86_64
[12:10] <cachio> I get this error on centos 7.6
[12:10] <mborzecki>        −o    Write the current settings of the options to standard  output  in
[12:10] <mborzecki>              an unspecified format.
[12:11] <zyga> cachio: ehey
[12:11] <mborzecki> cachio: hmm interesting
[12:11] <cachio> zyga, I made sure go is instlaled
[12:11] <zyga> mborzecki:  https://www.irccloud.com/pastebin/3I0NnCO5/
[12:11] <cachio> mborzecki, yes
[12:11] <zyga> cachio: I don't follow EPEL work so no idea
[12:11] <cachio> mborzecki, do you?
[12:11] <cachio> right
[12:11] <cachio> ?
[12:12] <mborzecki> cachio: any chance i could use the image?
[12:12] <cachio> I have a debug session already openned
[12:12] <cachio> mborzecki, do you want to use it?
[12:12] <zyga> mborzecki: my guess is that sourcing some of our packages nukes set -e
[12:12] <zyga> and this is now changed _somehow_
[12:13] <cachio> mborzecki, otherwise the image name is test-centos-1
[12:13] <zyga> sourcing vs executing
[12:13] <zyga> sourcing is evil
[12:13] <mborzecki> cachio: yes, i'll try to build snapd there, i recall chasing down something similar
[12:14] <mborzecki> zyga: hence, 'evil sourcerer' :)
[12:14] <zyga> haha
[12:14] <zyga> we should really stop doing that
[12:15] <zyga> or add sanity checks on 1st/last line of each file
[12:15] <zyga> this really sucks
[12:15] <zyga> I wonder what else is hidden by this
[12:15] <mborzecki> shell sucks
[12:16] <cachio> mborzecki, great, thanks
[12:19] <cachio> zyga, are you working with this error on snap-run right?
[12:20] <zyga> which error?
[12:20] <cachio> zyga, I have seen many times the error :
[12:21] <cachio> snap run --trace-exec test-snapd-tools.echo hello
[12:21] <zyga> I haven't touched that code
[12:21] <zyga> nor seen the error
[12:21] <zyga> that's mvo's new thing, if it fails perhaps we should revert for now
[12:22] <zyga> mborzecki: https://github.com/snapcore/snapd/pull/6190 should now pass and has the configuration untangled
[12:22] <cachio> zyga, ok, I'll collect the logs since yesterday to check it is failing on the same place
[12:22] <mup> PR #6190: overlord/configstate,features: expose features to snapd tools <⛔ Blocked> <Created by zyga> <https://github.com/snapcore/snapd/pull/6190>
[12:22] <zyga> how is it failing?
[12:24] <cachio> zyga, well, it is failing on arm devices
[12:24] <zyga> _how_
[12:24] <cachio> zyga, https://paste.ubuntu.com/p/4RGQB3VmCv/
[12:25] <zyga> error: unknown flag `trace-exec'
[12:25] <mborzecki> that's the new thing
[12:25] <zyga> looks like combination of old binaries and new tests
[12:25] <zyga> so?
[12:26] <ackk> hi, I opened https://bugs.launchpad.net/snapcraft/+bug/1802345 a while ago. now, with snapcraft building by default in VMs, is there a way to export that env var?
[12:26] <mup> Bug #1802345: can't build python part with C modules <Snapcraft:New> <https://launchpad.net/bugs/1802345>
[12:26] <zyga> ackk: that's a question for sergiusens
[12:26] <cachio> zyga, ahh, ok, in that case the core on edge did't get the last updates
[12:29] <cachio> zyga, sorry, I'll check what happended there
[12:33] <zyga> mborzecki: I added some sanity detectors to sourcing pkgdb
[12:33] <xnox> hi! can anybody please verify https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1778936 ?
[12:33] <mup> Bug #1778936: please re-add Support-system-image-read-only-etc.patch <patch> <verification-needed> <verification-needed-bionic> <systemd (Ubuntu):Fix Released> <systemd (Ubuntu Bionic):Fix Committed> <systemd (Ubuntu Cosmic):Fix Released> <https://launchpad.net/bugs/1778936>
[12:33] <xnox> otherwise, i guess i will need to drop the writable-etc patch to get systemd to be released =/
[12:34] <zyga> xnox: what's the timeline?
[12:34] <xnox> zyga, well, mvo has been chasing me about this for a long time, and as far as i understand it affects/blocks usablity of core18 by default.....
[12:35] <zyga> yes, I mean, how soon do you need this verified
[12:35] <zyga> mvo is away this week
[12:35] <xnox> zyga, i have a new SRU ready to upload with severe regression fixes.
[12:35] <xnox> zyga, thus i do need to release this update soon.
[12:35] <ackk> sergiusens, hi, do you have any insight on #1802345
[12:35] <mup> Bug #1802345: can't build python part with C modules <Snapcraft:New> <https://launchpad.net/bugs/1802345>
[12:35] <xnox> zyga, i'm away from thursday.... today/tomorrow would be really nice
[12:36] <zyga> xnox: I see, I will get back to you shortly
[12:36] <xnox> zyga, mostly i have no idea how to build and boot core18 with systemd from bionic-proposed.
[12:36] <xnox> otherwise i can validate this issue myself....
[12:36] <xnox> zyga, are there core18 images built with bionic-propsoed somewhere?
[12:36] <zyga> sadly, I don't know this
[12:36] <zyga> I'm trying to find out
[12:37] <zyga> the pipeline around this is a bit of a black magic
[12:37] <sergiusens> ackk: might just be a python plugin bug; the xenial issue feels tricky as snapcraft wouldn't be able to build as we use yaml, macaroons and other things so I will have to look into it
[12:39] <ackk> sergiusens, wdym "snapcraft wouldn't be able to build" ? you mean build itself?
[12:39] <sergiusens> yes, build itself
[12:40] <xnox> zyga, also the systemd update, triggered snapd autopkgtests with regressions on all arches.
[12:40] <diddledan> sergiusens, ackk, try adding libc6-dev if you want the headers for libc
[12:40] <ackk> sergiusens, why is that relevant? the bug is about building a python snap
[12:41] <ackk> diddledan, the problem is not that headers are not there, it's that they are not found without that export
[12:41] <xnox> zyga, is 2018-11-20 16:29:44 Failed tasks: 1
[12:41] <xnox>     - autopkgtest:ubuntu-18.04-i386:tests/main/dirs-not-shared-with-host:alternatives
[12:41] <xnox> known?
[12:41] <sergiusens> ackk: snapcraft is a python snap, what do you mean it is not relevant?
[12:41] <xnox> in that case i can ask release team to add a hint to ignore that.
[12:41] <ackk> sergiusens, does it specify that env var in override-build?
[12:42] <diddledan> oh, ackk, you're building with python2 but supplying the python3-dev package
[12:42] <diddledan> add `python-version: python3` to your part yaml
[12:43] <ackk> diddledan, oh lemme try that
[12:43] <zyga> xnox: hmmm, probably tests out of sync with code
[12:43] <zyga> xnox: we no longer have /etc/alternatives management in snapd
[12:44] <zyga> and core was changed to not use such symlinks
[12:44] <zyga> xnox: it's a bit unfortunate because I don't participate in that part of snapd (release) and this is really something mvo should answer
[12:44] <zyga> but he's unlikely to this week
[12:44] <sergiusens> ackk: envvars do not make it to plugin logic, if you are doing something complex, script the whole thing or write a custom plugin
[12:46] <ackk> diddledan, same error
[12:46] <diddledan> sergiusens: we should make available a bare-bones template/example for people to base their custom plugins on
[12:46] <diddledan> i.e. a template that does nothing without adding code but shows all the standard functions
[12:46] <diddledan> stubs all the standard functions**
[12:46] <sergiusens> diddledan: we had that in the old docs, I cannot for the life in me find anything in the new ones...
[12:49] <ackk> diddledan, fwiw it'd be nice if you could specify the actual python version, like python3.7  as both 3.6 and 3.7 are available on bionic
[12:51] <mborzecki> off to pick up the kids
[12:52] <ackk> sergiusens, why am I doing something complex? it's just building a pyhton app, with C modules
[12:52] <ackk> sergiusens, seems something that the python plugin should support, which it could by just exporting C_INCLUDE_PATH to /usr/include/<python-version>
[12:53] <sergiusens> well, I said I would look at the bug later, all you will get from me now are shallow replies
[12:55] <diddledan> https://media2.giphy.com/media/JMoyE48gJqivS/giphy.gif?cid=3640f6095c06792f566577304103b645
[13:00] <cachio> mborzecki, hey, could you please take a look to #6217
[13:00] <cachio> it is close
[13:00] <cachio> thanks
[13:00] <mup> PR #6217: tests: reset snapd state on tests restore <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/6217>
[13:01]  * pstolowski lunch
[13:09]  * zyga -> cofee
[13:10] <zyga> *coffee
[13:12] <diddledan> ackk: move `python3-dev` from `build-packages` to `stage-packages`.
[13:29] <ackk> diddledan, ah, that worked! out of curiosity, why is that needed?
[13:42] <zyga> cachio: hey, can you please work with xnox to run regression tests on https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1778936
[13:42] <mup> Bug #1778936: please re-add Support-system-image-read-only-etc.patch <patch> <verification-needed> <verification-needed-bionic> <systemd (Ubuntu):Fix Released> <systemd (Ubuntu Bionic):Fix Committed> <systemd (Ubuntu Cosmic):Fix Released> <https://launchpad.net/bugs/1778936>
[13:42] <zyga> cachio: it needs a core18 snap with systemd from proposed
[13:44] <zyga> (back from coffee and extra dog walk that he demanded)
[13:44] <zyga> mborzecki: I added set -o traces in various points
[13:44] <zyga> running a test now, let's see
[13:45] <cachio> zyga, sure
[13:45] <zyga> thank you
[13:45] <zyga> xnox: ^ cachio will help with testing that
[13:45] <ackk> diddledan, thanks for the tip btw
[13:52] <zyga> mborzecki: set -e is set when the test executes, as expected
[13:53] <zyga> mborzecki: comparing that to what happens in master
[13:53] <zyga> this is really puzzling
[13:56] <mborzecki> 2018-12-04 14:54:52 Failed tasks: 3 <-- that's with fedora 29
[13:56] <zyga> mborzecki: which tasks failed?
[13:57] <mborzecki> zyga: tests/main/document-portal-activation tests/main/prepare-image-uboot and tests/main/snap-run
[13:58] <zyga> interesting
[13:58] <zyga> what did you change?
[13:58] <mborzecki> added a workaround for snap-device-helper
[13:59] <zyga> mborzecki: heh
[13:59] <zyga> man
[13:59] <zyga> it passes on master
[13:59] <zyga> so yes
[13:59] <zyga> something is broken in our spread test suite
[13:59] <zyga> running with --shell-after now
[13:59] <mborzecki> hahah
[13:59] <zyga> I will shortly know what's the cause
[14:00] <mborzecki> we should write go scripts
[14:00] <zyga> with the tracing added, all I need is to diff the right spot
[14:00] <zyga> :/
[14:00] <mborzecki> #!/usr/bin/go run
[14:00] <zyga> we should not source stuff
[14:00] <zyga> write tools
[14:00] <zyga> not shell functions
[14:00] <zyga> and migrate away from that
[14:00] <zyga> then tools can become non-shell
[14:01] <mborzecki> damn, standup
[14:01] <zyga> joining
[14:02] <zyga> pstolowski: standup
[14:04] <zyga> mborzecki: can you ack/merge https://github.com/snapcore/snapd/pull/6257
[14:04] <mup> PR #6257: testutils: split checkers, tweak tests <Created by zyga> <https://github.com/snapcore/snapd/pull/6257>
[14:05] <mup> PR snapd#6257 closed: testutils: split checkers, tweak tests <Created by zyga> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/6257>
[14:20] <pedronis> zyga: mborzecki: hi, I don't the situation of the arch user that lost the model assertion, afaict the code sets the state only after having added the assertion
[14:20] <pedronis> * I don'tunderstand
[14:20] <ppisati> ogra: where can i find the ubuntu core arm64 image for the raspberrypi3?
[14:20] <pedronis> mborzecki: if you can spend some thinking what could delete it,  I don't have brain cycle for it atm
[14:20] <zyga> pedronis: note that i have a core device in similar state, I have the state of that if someone wants to attack that at some point
[14:21] <ogra> ppisati, somewhere in mvo's people account
[14:21] <ogra> IIRC
[14:22] <ogra> ppisati, https://forum.snapcraft.io/t/pi3-64-unofficial-image/8317
[14:22] <pedronis> zyga: that sound bad, I don't know any code that would remove a model assertion that was set
[14:22] <pedronis> sounds a serious bug
[14:22] <ppisati> ogra: ack, thank you
[14:22] <ogra> ppisati, did you get anywhere with A+ and vc4 ?
[14:23] <ppisati> ogra: oh yeah, it already spawned 2/3 lp bugs
[14:23] <ppisati> ogra: cma=256M is the problem
[14:23] <ogra> well, thats builtin
[14:23] <ppisati> ogra: this is the oops - https://bugs.launchpad.net/ubuntu/+source/linux-raspi2/+bug/1805117
[14:23] <mup> Bug #1805117: Xenial/raspi2 and RaspberryPi3a+: OOPS during boot <patch> <linux-raspi2 (Ubuntu):New> <https://launchpad.net/bugs/1805117>
[14:24] <ogra> the driver has it hardcoded somewhere ... IIRC
[14:24] <ppisati> ogra: and this is the cma / deadlock: https://bugs.launchpad.net/ubuntu/+source/linux-raspi2/+bug/1805174
[14:24] <mup> Bug #1805174: Xenial/raspi2 and RaspberryPi3A+: video not working <linux-raspi2 (Ubuntu):New> <https://launchpad.net/bugs/1805174>
[14:24] <ppisati> ogra: yeah. but you can specify cma=a on the dtparam
[14:24] <ppisati> ogra: wait, let me find
[14:24] <ogra> btw, the closed driver works really fine
[14:24] <ogra> yeah, i know
[14:28] <ppisati> ogra: https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/xenial/tree/arch/arm/boot/dts/overlays/README?h=raspi2
[14:28] <ppisati> ogra: search for "vc4-kms-v3d"
[14:28] <ppisati> ogra: Params: cma-256                 CMA is 256MB, 256MB-aligned (needs 1GB)
[14:28] <ppisati> ogra: while the rpi3a+ has 512MB
[14:28] <ppisati> ogra: pass cma-128 and it will work
[14:29] <ogra> yeah, i got that ... my prob is that i have no conditional i could use in config.txt to actually set it dynamically
[14:29] <ogra> and i dont want an A+ sepcific image
[14:31] <pedronis> mborzecki: zyga: notice that those systems could be in that state since long, is just that 2.36 is surfacing now the bad state
[14:31] <zyga> pedronis: in my case I updated a device after fresh install
[14:31] <ogra> the alternative would be to simply default to cma-128, but that degrades the actual pi3 i guess
[14:31] <zyga> this was circa ~ 3  weeks ago
[14:31] <zyga> the device is off since then
[14:32] <mborzecki> pedronis: right, that's my hunch as well, added a note in the forum
[14:32] <pedronis> zyga: as I said the update would be make the bad state appear
[14:32] <pedronis> I don't think it removed the model
[14:32] <ppisati> ogra: the problem is that the fw set cma=256 even on that board, and the drivers locks up with that much memory
[14:32] <pedronis> zyga: I might be missing something
[14:32] <pedronis> tough
[14:32] <ppisati> ogra: i can fix the oops, but then ther's another problem after that
[14:32] <ogra> yeah
[14:32] <ogra> right
[14:32] <zyga> pedronis: I will look at that device tomorrow
[14:33] <zyga> working with maciek on mount error now
[14:33] <ogra> i understand ... that why i would like to simply do it via a conditional in config.txt and have it pick the right value ... https://www.raspberrypi.org/documentation/configuration/config-txt/conditional.md
[14:33] <ogra> *that's
[14:33] <ogra> but i can neither filter on pi3 vs A+ nor on available RAM
[14:34] <ogra> (would be all easier if we could apply the overlays from u-boot ... then i could just check what board i'm on easily)
[15:08] <mup> PR snapd#6262 opened: apparmor: allow snap-update-ns access to common devices <Simple 😃> <Created by jdstrand> <https://github.com/snapcore/snapd/pull/6262>
[15:14] <sergiusens> having a .snapcraft as an alternate to snap is going to be a bigger code change than I wanted
[15:15] <zyga> sudo umount /tmp/dupa-* &&  sh -c 'for i in $(seq 1000); do mkdir -p /tmp/dupa-$i && umount /tmp/dupa-$i 2>/dev/null; cp foo.snap foo-$i.snap && LIBMOUNT_DEBUG=all mount foo-$i.snap /tmp/dupa-$i >mount.$i.log 2>&1 & done'
[15:16] <zyga> er
[15:16] <zyga> sudo sh -c 'umount /tmp/dupa-* 2>/dev/null; for i in $(seq 1000); do mkdir -p /tmp/dupa-$i && cp foo.snap foo-$i.snap && LIBMOUNT_DEBUG=all mount foo-$i.snap /tmp/dupa-$i >mount.$i.log 2>&1 & done'
[15:17] <zyga> with waiting: sudo sh -c 'umount /tmp/dupa-* 2>/dev/null; for i in $(seq 1000); do mkdir -p /tmp/dupa-$i && cp foo.snap foo-$i.snap && LIBMOUNT_DEBUG=all mount foo-$i.snap /tmp/dupa-$i >mount.$i.log 2>&1 & done; wait'
[15:18] <zyga> mborzecki: ^ those take a lot of CPU from systemd and various mount-observing tools
[15:23] <zyga> ha
[15:23] <zyga> when running many mount in parallel
[15:23] <zyga> I got:
[15:23] <zyga> systemd[PID]: Failed to get udev event
[15:25] <zyga> sudo sh -c 'umount /tmp/dupa-* 2>/dev/null; for i in $(seq 1000); do mkdir -p /tmp/dupa-$i && cp foo.snap foo-$i.snap; done; for i in $(seq 1000); do LIBMOUNT_DEBUG=all mount foo-$i.snap /tmp/dupa-$i >mount.$i.log 2>&1 & done; wait'; grep 'final srcpath'  mount.*.log | cut -f 6 -d ':' | sort | uniq -c | cut -c 7-  | cut -d ' ' -f 1 | sort | uniq -c
[15:27] <zyga> mborzecki: can you copy any snap you have on hand to foo.snap
[15:27] <zyga> mborzecki: and run that
[15:27]  * zyga -> dinner
[15:38] <cjwatson> sergiusens: please tell me this isn't another place where snapcraft.yaml can live?  we already have three :(
[15:47]  * cachio lunch
[16:03] <zyga> re
[16:03] <zyga> jdstrand_: thank you for the review
[16:03] <zyga> I will jump to that shortly
[16:08] <sergiusens> cjwatson: the snapd team does not want to rename their in code "snap" directory
[16:09] <sergiusens> so it came as a requirement to change it
[16:09] <cjwatson> Well, just remember that adding more locations for snapcraft.yaml requires changing at least LP and BSI too
[16:10] <zyga> BSI?
[16:10] <cjwatson> And may cause operational problems since it will increase the number of queries BSI is making to GitHub (although hopefully not by very much)
[16:10] <cjwatson> build.snapcraft.io
[16:10] <zyga> ah
[16:10] <sergiusens> cjwatson: yeah, it sucks
[16:10] <sergiusens> just to avoid a big sed in the code, we get to add logic all over the place
[16:10] <cjwatson> sergiusens: can you file bugs on LP and BSI once the snapcraft change lands, if it's unavoidable?
[16:19] <sergiusens> sure thing
[16:19] <cjwatson> thanks
[16:28] <mup> PR snapcraft#2399 closed: cmake plugin: use build snaps to search paths <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2399>
[16:36] <zyga> mborzecki: still there?
[16:36] <zyga> no luck on 16.04 either
[16:39] <zyga> mborzecki: but interestingly I can easily trigger this:
[16:39] <zyga> https://www.irccloud.com/pastebin/UFMlCwov/
[16:39] <zyga> so that part is working as intended
[16:47] <zyga> gru 04 08:44:10 ubuntu snapd[897]: 2018/12/04 08:44:10 Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 408 Request Time-out\r\nCache-Control: no-cache\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n<html><body><h1>408 Request Time-out</h1>\nYour browser didn't send a complete request in time.\n</body></html>\n"; err=<nil>
[16:48] <zyga> hmm?
[16:55] <katamo> aloha, I need to change ownership of a snap, specifically https://snapcraft.io/scout-plane/listing, from myself to a colleague. Can I get help with that here?
[16:55] <mborzecki> zyga: nice
[16:55] <zyga> katamo: yep, but probably requires a forum topic too,
[16:55] <popey> katamo: best to ask in the store category on the forum
[16:55] <zyga> mborzecki: yep
[16:55] <zyga> mborzecki: so, ...
[16:55] <katamo> popey ack, ty
[16:55] <zyga> mborzecki: I'm running this in parallel with systemd restarts now
[16:55] <zyga> no luck
[16:55] <mborzecki> restarts?
[16:56] <zyga> daemon-reload
[16:56] <mborzecki> ah ok
[16:56] <mborzecki> zyga: have you tried the reproducer script yet?
[16:56] <mborzecki> zyga: https://gist.github.com/bboozzoo/d4b142229b1915ef7cc0cf8593599ad9
[16:56] <zyga> yeah but I want to find a deeper angle on this :), the erorr was, clearly, from libmount, not from systemd itself
[16:56] <zyga> system caught and reported it
[16:56] <zyga> that's fine
[16:59] <mborzecki> zyga: maybe a problem is with loading mountinfo in systemd?
[16:59] <zyga> hmmm
[16:59] <zyga> interesting idea
[16:59] <mborzecki> zyga: iirc that's done separately in io loop
[16:59] <zyga> worth looking
[16:59] <mborzecki> zyga: and there's that super ineresting comment:
[16:59] <mborzecki>         /* Note that due to the io event priority logic, we can be sure the new mountinfo is loaded
[16:59] <mborzecki>          * before we process the SIGCHLD for the mount command. */
[16:59] <zyga>  yeah, I read that
[16:59] <zyga> it looks correct though
[16:59] <zyga> one more thing to do
[17:00] <zyga> is to breakpoint there
[17:00] <zyga> when it cannot find it
[17:00] <zyga> and rewind enough state to look around
[19:11] <mup> PR snapcraft#2419 opened: project: state file path change <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2419>
[19:17] <mup> PR snapd#6263 opened: Retrieving all pages for 'find' command, not only one <Created by WiseRaccoon> <https://github.com/snapcore/snapd/pull/6263>
[20:46] <zyga> jdstrand: hey, updated https://github.com/snapcore/snapd/pull/6244
[20:46] <mup> PR #6244: release: detect too old apparmor_parser <Created by zyga> <https://github.com/snapcore/snapd/pull/6244>
[20:46] <zyga> I'd like to land it as-is, I'm still doing one aspect
[20:47] <zyga> when we have a object (kernel or parser) and some features
[20:47] <zyga> there are features we want to have to be fully operational
[20:47] <zyga> features we want to have to be partially operational
[20:47] <zyga> and features we must have to be in any way operational
[20:47] <zyga> you are right this is not captured
[20:47] <zyga> I'd like to do that properly with a follow-up
[20:47] <zyga> I added a few missing tests, got coverage to 100%
[21:16]  * jdstrand responded in the pr
[21:32] <zyga> jdstrand: thanks, lookin g
[21:35] <zyga> jdstrand: replied there
[21:35] <zyga> jdstrand: totally offtopic
[21:35] <zyga> do you know what's the status of the upstreaming effort
[21:36] <zyga> I'm running tumblweed instead of ubuntu recently and I keep taking notice that we're not fully supported yet
[21:51] <jdstrand> zyga: af_unix is still missing. it is the last thing. I know jjohansen was working to get things into the latest merge window and af_unix was on the table, but not sure if he got it in
[21:51] <zyga> cool, fingers crossed :)
[21:51] <zyga> using TW is like being on debian sid
[21:51] <zyga> but without any debian specific things I may assume are true universally
[21:51] <jdstrand> zyga: note that once af_unix is in, will need apparmor 3 userspace for (at least) the networking bits
[21:51] <zyga> it's been a fun journey
[21:52] <zyga> is apparmor userspace 3 released?
[21:52] <jdstrand> not yet
[21:53] <jdstrand> it is also planned, but behind the af_unix mediation and a few other things. it, those things and af_unix are all for this cycle
[21:53] <jdstrand> and by cycle, I mean our dev cycle, not the upstream kernel or anything else
[21:53] <zyga> do you know if the eventual release will affect ubuntu that has carried the earlier patches in any way?
[21:53] <zyga> aha
[21:54] <jdstrand> zyga: I don't hve the details on that, but we'll definitely make sure we have things working everywhere in Ubuntu
[22:21] <zyga> jdstrand: I'm about to EOD but I'd love a one last look at https://github.com/snapcore/snapd/pull/6244
[22:21] <mup> PR #6244: release: detect too old apparmor_parser <Created by zyga> <https://github.com/snapcore/snapd/pull/6244>
[22:21] <zyga> ah, I see you replied
[22:22] <zyga> again, stale page :/
[22:22] <zyga> jdstrand: I'll tweak this tomorrow, thank you :)
[22:23] <jdstrand> zyga: np. happy to review again tomorrow. take it easy :)
[23:16] <roadmr> jdstrand: the tools r1167 are now merged in the store's trunk and should be deployed some time this week, ideally tomorrowish
[23:21] <jdstrand> roadmr: thanks! :)