[02:34] <cachio> Saviq, hey, please try the image fedora-rawhide-64, tomorrow I'll ping you
[06:12] <mborzecki> morning
[06:19] <zyga> Hi
[06:20] <zyga> Some late night pull requests
[06:20] <zyga> With green 2.36
[06:20] <zyga> :-)
[06:37] <mborzecki> heh :) went to a meetup yday, then i stayed until midnight making that damn selinux work, audit actually showed some interesting behavior by journald poking some attributes in /proc all the time
[06:39] <zyga> Cool, looking forward to that
[06:39] <zyga> My son is 1/3rd my age today
[06:40] <zyga> Feeling old or young? :-)
[07:23] <zyga> re
[07:23] <zyga> mborzecki: hey
[07:23] <zyga> so today we will try to stabilize 2.36 and merge the bits back into master
[07:23] <zyga> mborzecki: this is part of the issue https://github.com/snapcore/snapd/pull/6233
[07:23] <mup> PR #6233: overlord: don't write system key if security setup fails <Created by zyga> <https://github.com/snapcore/snapd/pull/6233>
[07:24] <zyga> mborzecki: this is a more complete view but as you will see there it cannot land just yet https://github.com/snapcore/snapd/pull/6235
[07:24] <mup> PR #6235: overlord,apparmor: new syskey behaviour + non-ignored snap-confine profile errors <Created by zyga> <https://github.com/snapcore/snapd/pull/6235>
[07:24] <zyga> each commit has some description that explains what was going on
[07:24] <zyga> Chipaca: good morning sir!
[07:25] <zyga> I need to run an errand in the morning (wife doctor checkup) but I will be back as soon as I can
[07:34] <Chipaca> zyga: goodmorning
[07:35]  * Chipaca not legally awake yet
[07:40] <mup> PR snapd#6236 opened: Staticcheck fixes <Created by chipaca> <https://github.com/snapcore/snapd/pull/6236>
[07:58] <zyga> Hey mvo
[07:58] <zyga> I’m afk on the way to a doctor with my wife (all is good, no worries)
[07:59] <zyga> I sent some PRs last night
[07:59] <zyga> Please have a look
[07:59] <mvo> zyga: cool
[07:59] <mvo> zyga: thanks, I check the PRs
[08:00] <zyga> If you have time I would love more eyes on the trio of pastebins from yesterday
[08:00] <zyga> To be confident we understand how things happen
[08:01] <zyga> After I am back I will post leap fixes
[08:01] <zyga> Together this will fix 2.36 as we understand it so far
[08:06] <mvo> zyga: great
[08:11] <pedronis> zyga: I asked a question in one of them
[08:14] <pstolowski> morning
[08:15] <mvo> hey pstolowski
[08:18] <zyga> pedronis: replied now
[08:18] <zyga> Actually
[08:18] <zyga> Hmmm
[08:18] <zyga> More problematic
[08:19] <zyga> Oh boy :-)
[08:19] <zyga> Something to think about
[08:19] <pedronis> zyga: I thought we already added the waiting to snap-confine, is that false?
[08:21] <zyga> No system key, no snap run
[08:21] <mvo> pedronis: thanks for adding the todo to 6195
[08:21] <zyga> I’m afk partially sorry for laggy replies
[08:21] <pedronis> zyga: ?
[08:21] <pedronis> I thought we did something saner
[08:22] <zyga> I’m not at home, doctor visit with wife
[08:22] <pedronis> zyga: ok, let' chat later
[08:22] <pedronis> mvo: do you know what snap-confine does if there's no system-key? I thought it would wait a while
[08:23] <mvo> pedronis: it will wait a while, do you see something different?
[08:23] <mvo> pedronis: it should try to talk to snapd in this case
[08:23] <zyga> This is really about a restart case
[08:23] <pedronis> zyga: is saying something else
[08:23] <zyga> So we error because we are restarting
[08:23] <zyga> It will be fixed by next startup
[08:23] <pedronis> ?
[08:23] <pedronis> error where
[08:23] <pedronis> something sounds wrong
[08:24] <pedronis> is snap-confine assuming that snapd runs all the time
[08:24] <pedronis> that's a bit optimistic
[08:24] <mvo> pedronis: well, we had a long discussion abut this when we introduced the system key
[08:24] <pedronis> mvo: I mean  if snap-confine needs to wait and can't talk to snapd it should wait a bit more and try again, no?
[08:25] <mvo> pedronis: I can look in a bit for the details, I think we wrote it down. at least gustavo and me, don't remember if you were part of it
[08:25] <zyga> I can hop on a voice call if you want to discuss this in detail
[08:25] <pedronis> until some timeout
[08:25] <mvo> pedronis: yes, thats correct
[08:25] <pedronis> is it simply dying if snapd is not there
[08:25] <pedronis> ?
[08:25] <mvo> pedronis: is this not what you see?
[08:25] <zyga> That is what currently happens
[08:25] <mvo> pedronis: no, it should not
[08:25] <mvo> zyga: oh?
[08:25] <zyga> It waits
[08:25] <zyga> And then might die if something required is missing
[08:25] <zyga> AFAIK
[08:26] <mvo> I am just trying this
[08:26] <mvo> it definitely waits
[08:26] <pedronis> then is as designed, no?
[08:26] <pedronis> I mean, I don't expect this to work 100%
[08:26] <zyga> Yes
[08:26] <zyga> I think this is as designed
[08:26] <pedronis> it should not fail on boot 100%
[08:26] <mvo> and when I start snapd again it will continue
[08:26] <pedronis> either
[08:26] <pedronis> we might have a problem in that
[08:27] <pedronis> the timeout might be shorter than it takes to regen all security if lots of snaps ?
[08:28] <zyga> Yes
[08:28] <mvo> yes, that could be a problem :(
[08:28] <pedronis> is not a problem for today tough
[08:28] <pedronis> but zyga scared a bit, like snap-confine would not to do what I remember was designed to do
[08:28] <mvo> it waits 60s right now
[08:29] <mvo> pedronis, zyga let me try this
[08:29] <Chipaca> what were we calling the feature where we'd delay a snap refresh until the app was closed?
[08:29] <mvo> 6195 still needs a second review
[08:30] <zyga> I think it is not perfect but as designed
[08:30] <pedronis> as I said I don't expect perfect (kind of impossible given the constraints), we might have to improve over time in some corners
[08:31] <pedronis> but for a second it sounded like it was broken
[08:31] <zyga> I was scared :-)
[08:31] <mvo> hm, it seems to error when it hits the timeout - iirc we wanted it to continue (best effort) - yes?
[08:32] <pedronis> Chipaca: we call it "Prevent refreshes while running"
[08:32] <pedronis> at least that's the name of the day
[08:32] <Chipaca> pedronis: is there a forum topic about it?
[08:33] <Chipaca> asking because https://forum.snapcraft.io/t/concerns-about-consistency-and-data-corruption-during-snap-refresh/8741
[08:33] <pedronis> yea, I saw that one
[08:33] <Chipaca> if there isn't, i'll just reply vaguely
[08:33] <Chipaca> :-)
[08:36] <pedronis> Chipaca: I don't see one, is mentioned in the last sprint topics but not as a discussed one
[08:36] <pedronis> anyway is planned for the cycle
[08:37] <Chipaca> pedronis: https://forum.snapcraft.io/t/concerns-about-consistency-and-data-corruption-during-snap-refresh/8741/2?u=chipaca
[08:37] <Chipaca> ¯\_(ツ)_/¯
[08:38] <Chipaca> mvo: #6195 gtg fwiw
[08:38] <mup> PR #6195: snapstate: update fontconfig caches on install <Created by mvo5> <https://github.com/snapcore/snapd/pull/6195>
[08:39] <pedronis> Chipaca: thx
[08:39]  * Chipaca wanders off in search of coffee
[08:41] <zyga> I could use some too
[08:41] <zyga> Waiting for blood tests now
[08:42]  * mvo gets some lovely tea
[08:43] <zyga> I’m happy nobody mentioned nutrient breakfast yet
[08:43] <Chipaca> mvo: I saw somebody using "material tea timer" and thought you might like it (if you didn't have it already)
[08:43] <Chipaca> mvo: https://play.google.com/store/apps/details?id=org.ligi.materialteatimer
[08:44] <mvo> Chipaca: nice!
[08:44] <mvo> Chipaca: I don't have it, I use the boring clock app
[08:44] <mvo> Chipaca: but this even has nice pics (teap0rn)
[08:45] <Chipaca> mvo: and it's FOSS
[08:45] <mvo> \o/
[09:00] <mup> PR snapd#6195 closed: snapstate: update fontconfig caches on install <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/6195>
[09:00] <mup> PR snapd#6218 closed:  snapstate: update fontconfig caches on install (2.36) <⚠ Critical> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/6218>
[09:00] <dot-tobias> good morning
[09:10] <zyga> Hey dot-tobias
[09:12] <mborzecki> hm finnally, clean installation, no denials, ending up with this: system_u:system_r:unconfined_service_t:s0 root 11016 1  0 09:11 ?      00:00:00 /bin/sh /snap/test-snapd-service/x1/bin/start
[09:13] <mup> PR snapd#6237 opened: client, store: don't use store from client (use client from store) <Created by chipaca> <https://github.com/snapcore/snapd/pull/6237>
[09:34] <zyga> Breakfast time
[09:38]  * Chipaca joins zyga
[09:38]  * Chipaca thinks hobbits had the right of it
[09:39] <zyga> Haga
[09:40] <zyga> Haha, yes :-)
[09:52] <mup> PR snapd#6189 closed: daemon, vendor: bump github.com/coreos/go-systemd/activation, handle API changes (2.36) <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/6189>
[10:03] <mup> PR snapd#6238 opened: [RFC] many: add minimal SELinux support, refactor the policy <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6238>
[10:05] <mborzecki> putting selinux aside for while now
[10:08] <mborzecki> any PRs needing 2nd reviews?
[10:09] <Son_Goku> mborzecki, you are literally my best friend right now!
[10:10] <Son_Goku> you have _no_ idea how happy I am to see this pull request!
[10:11] <mborzecki> Son_Goku: haha :) yw, would appreciate if you could find someone more familiar with this stuff to take a look at the policy
[10:11]  * Son_Goku dances in joy
[10:11]  * zyga goes to sob in the corner
[10:11] <zyga> :-)
[10:11] <Son_Goku> zyga, this is something I've been asking for two years for!
[10:11] <zyga> Heading home ttyl
[10:12] <Son_Goku> you can't blame me for being overjoyed :)
[10:12] <zyga> I know but I don’t make the schedule
[10:12] <mborzecki> Son_Goku: hope i got most of the things right, but figuring out the bits was like uhh
[10:12] <zyga> I’m happy to see this too :-)
[10:12] <Son_Goku> mborzecki, yeah I knew it was going to be like that
[10:12] <zyga> Beginning of a long journey
[10:12] <mborzecki> and the docs are few and useless really :P
[10:12] <Son_Goku> mborzecki, most security subsystems aren't well documented sadly :(
[10:13] <Son_Goku> I'll see if I can get Lukas from the Fedora SELinux team to do a review
[10:13] <zyga> Insecurity by security obscurity
[10:13] <mborzecki> Son_Goku: yeah, unfortunately true
[10:13] <mborzecki> some to think of it, SMACK is probably more obscure than SELinux and AppArmor
[10:13] <Son_Goku> yep
[10:13] <mborzecki> s/some/come/
[10:13] <Son_Goku> and TOMOYO even more so
[10:14] <Son_Goku> (TOMOYO is what Mageia has enabled, but no one knows what to do with it)
[10:14] <mborzecki> hahah
[10:14] <mborzecki> iirc AGL was supposed to use SMACK
[10:14] <Son_Goku> yep
[10:15] <Chipaca> Son_Goku: o/ !
[10:15] <Son_Goku> Chipaca: \o
[10:15] <Chipaca> Son_Goku: when you have a moment I'd like to pick you brain a tiny little bit
[10:15] <zyga> You know
[10:15] <Son_Goku> Chipaca, I have a moment now
[10:15] <zyga> Breakfast downtown is fun and fancy
[10:15] <zyga> Need to do this more often
[10:15] <mborzecki> zyga: nice, enjoy!
[10:15] <Son_Goku> mborzecki, and the biggest criticism of SMACK is that it's pretty much functionally identical to a minimal SELinux policy
[10:16] <Chipaca> Son_Goku: you remember, ages ago, when we talked about tracking what users had used snaps, to enable snap-user-enumeration without having to do system-user-enumeration?
[10:16] <Son_Goku> Chipaca, yeah? that was what, a year ago at the rally?
[10:16] <Chipaca> Son_Goku: you objected to that because you said a user's data should be the user's to control,  or something like that?
[10:16] <Son_Goku> yes
[10:16] <Chipaca> yeah probably more
[10:16] <Son_Goku> actually I don't think that was my only objection
[10:16] <Chipaca> Son_Goku: now we're again looking at things where we need to enumerate users
[10:17] <Chipaca> Son_Goku: and the same solution comes up, and so I wanted to revisit your objection to it
[10:17] <Chipaca> to see if I understood it correctly
[10:17] <Son_Goku> are we talking about ~/snap/* enumeration?
[10:17] <Son_Goku> or the user population/enumeration for services?
[10:17] <Chipaca> Son_Goku: "all users that have used snaps"
[10:18] <Chipaca> Son_Goku: like
[10:18] <Chipaca> Son_Goku: ls /home/*/snap/* but that's a bad way of doing it
[10:18] <Son_Goku> right
[10:18] <zyga> You need a dbus call that does a perl shell call to is
[10:18] <Son_Goku> this? https://forum.snapcraft.io/t/bug-with-cleaning-snap-data-in-home-dirs-proposed-solution/1201
[10:19] <zyga> Ls, darn shell typos
[10:19] <Son_Goku> Chipaca ^ ?
[10:19] <mvo> zyga: have you looked at the systemd-run change for the profile generation? if not I can do that while waiting for builds
[10:19] <Chipaca> Son_Goku: yeh
[10:19] <Chipaca> Son_Goku: i think that's it
[10:20] <Son_Goku> my problem was that the proposal as I understood it would break shared setups and make user data private from the user itself
[10:20] <Chipaca> Son_Goku: that's one bit I don't understand
[10:20] <Chipaca> Son_Goku: what's the user data that would be private from the user?
[10:20] <Chipaca> Son_Goku: the proposal is not to move all user data to /var/whatevs
[10:20] <Son_Goku> well, how I understood it is that data would move from ~/snap to /var/lib/snapd/<userid>/data
[10:21] <Chipaca> Son_Goku: but to create a stamp file in /var/whatevs "this user used a snap"
[10:21] <Son_Goku> or something like that
[10:22] <Son_Goku> Chipaca, my only objection there is that it makes network shared users ugly
[10:22] <Chipaca> Son_Goku: user data would remain unchanged; all this'd do is that 'snap run' would 'touch /var/lib/snapd/user/$USER' before running the thing
[10:22] <Chipaca> Son_Goku: how does it make network shared users ugly?
[10:22] <Chipaca> please expand :-)
[10:23] <Son_Goku> well, actually, if it's only that and doesn't effect how data "ownership" and migration occurs, then it's not an issue
[10:23] <Son_Goku> the way it was explained to me at the rally is that we wanted to do "smart things" by doing so
[10:23] <Son_Goku> for example, automatic cleanup of related user-data if the stamp didn't exist
[10:23] <Son_Goku> which would be extremely dumb for networked user case
[10:24] <Chipaca> er, no, that's the opposite of what I'd want to do
[10:24] <Chipaca> "related user-data" would not be even looked at if the stamp didn't exist
[10:24] <mup> PR snapd#6239 opened: packaging/fedora/snapd.spec: fix bogus date in changelog <Created by mvo5> <https://github.com/snapcore/snapd/pull/6239>
[10:25] <mvo> mborzecki: it looks like the selinux pr is failing because libselinux not found on centos afaict
[10:25] <Chipaca> as you say it breaks for networks, it also means that something is enumerating users from the system, which is a no-no
[10:25] <mborzecki> mvo: thanks, will look into it
[10:25] <mvo> mborzecki: really nice progress btw
[10:26] <Chipaca> Son_Goku: re-reading that topic now to find the cleanup-of-missing thing to address it there
[10:27] <mborzecki> mvo: thanks, and sorry for not being too responsive for reviews the last couple of days, didn't want to loose context
[10:27] <Son_Goku> mborzecki, it's because you've disabled libselinux right now
[10:28] <Son_Goku> mborzecki, you currently have selinux disabled for centos builds
[10:29] <mvo> mborzecki: no worries
[10:29] <Son_Goku> oh no, wait, it looks like you've switched it back
[10:29] <mborzecki> Son_Goku: --enable-selinux is behind %{?with_selinux:..}
[10:29] <mborzecki> Son_Goku: and the rest is if/endif'ed
[10:30] <mborzecki> Son_Goku: restored fedora 28 to spread specifically to build it
[10:36] <mborzecki> omg, with_selinux is always defined :/
[10:36] <mup> PR snapd#6240 opened: release: 2.36.2 <Created by mvo5> <https://github.com/snapcore/snapd/pull/6240>
[10:39] <zyga> mvo: I have not looked yet
[10:39] <mvo> zyga: ok, I give it a poke
[10:43] <zyga> Thank you
[10:47] <mborzecki> hmm selinux doesn't like the fontconfig bits
[10:53] <Chipaca> Son_Goku: https://forum.snapcraft.io/t/bug-with-cleaning-snap-data-in-home-dirs-proposed-solution/1201/14?u=chipaca
[10:56] <Son_Goku> mborzecki, will this work if both apparmor and selinux are available as options (i.e. SUSE distributions?)
[10:58] <mborzecki> Son_Goku: we'll probably need to disable one, i see that with_selinux is 1 by default
[10:58] <Son_Goku> well, I'm saying can the integration be compiled in?
[10:59] <Son_Goku> because there are folks that use SELinux instead of AppArmor on SLED/SLES, for example
[11:00] <mborzecki> Son_Goku: you can build both and there's a change it'll work out of the box, the libselinux bits do autodetection and so does libapparmor
[11:00] <Son_Goku> yeah, that's what I was getting at
[11:00] <mborzecki> Son_Goku: there's always a quesion of refpolicy they use, whether it's the same as fedora
[11:01] <Son_Goku> RH/Fedora and (open)SUSE use the same upstream: fedora-selinux
[11:01] <Son_Goku> actually, of all the distros that offer SELinux support, I think only Debian/Ubuntu don't offer the fedora-selinux variant of the policy
[11:02] <Son_Goku> (and "offer" is a _strong_ word here for Ubuntu... more like didn't purge from Debian impots)
[11:02] <Son_Goku> *imports
[11:04] <jamesh> Chipaca: if you've got time, could you have a look at https://github.com/snapcore/snapd/pull/5822 ?
[11:04] <mup> PR #5822: wrappers: allow user mode systemd daemons <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/5822>
[11:04] <mborzecki> Son_Goku: mhm, right so there's a change it'll work :) happy to learn how far it gets though
[11:04] <Chipaca> jamesh: whoops, yes
[11:04] <Son_Goku> mvo, niemeyer, I hope mborzecki's PR for selinux makes it in for the next release, this would drastically improve my confidence in shipping snapd for RHEL/CentOS through EPEL
[11:05] <Son_Goku> I'm actually pretty tempted to not ship until we get this in a release, because this is a *big* improvement across the board
[11:05] <jamesh> Chipaca: it should be fairly up to date now: I rebased it recently and spread seems happy with it again
[11:05] <Chipaca> jamesh: ok
[11:05] <cachio> Saviq, hey, could you try the rawhide image?
[11:05] <Son_Goku> mborzecki, how is transitioning from older policy rules to the new one?
[11:05] <Son_Goku> it works okay?
[11:06] <mborzecki> Son_Goku: haha, don't be, i see some issues on centos alredy, we need to have smarter runtime detectio of whether the policy is present and the latest bits with fontconfig are causing some issues
[11:06] <Son_Goku> ah
[11:06] <Son_Goku> mborzecki, well, it would at least mean the confinement _does_ something now :)
[11:06] <Son_Goku> not great, but it does something
[11:07] <mborzecki> Son_Goku: it's only confining snapd and helpers
[11:07] <Son_Goku> but now the framework is in place to confine snaps properly too
[11:07] <mborzecki> Son_Goku: but there's a path forward i guess at this point
[11:07] <mborzecki> Son_Goku: at least there are options to explore :)
[11:07] <Son_Goku> yep
[11:08] <Son_Goku> now that we have libselinux integration, we can look at things like mapping CIL and snappy policy knobs to the snappy security HLL
[11:09] <zyga> re
[11:11] <zyga> mvo: sorry for being absent so long, I'm finally home now
[11:11] <zyga> mvo: I will start by adding tests to https://github.com/snapcore/snapd/pull/6233
[11:11] <mvo> zyga: no worries
[11:11] <mup> PR #6233: overlord: don't write system key if security setup fails <Created by zyga> <https://github.com/snapcore/snapd/pull/6233>
[11:11] <zyga> then resume with leap fixes
[11:23] <cachio> mvo, hey, I saw 2.36.2 branch has failed
[11:23] <cachio> 2 tests with same error on configure hooks
[11:27] <mvo> cachio: failed in spread?
[11:28] <mvo> cachio: or failed somewhere else?
[11:28] <cachio> mvo, in spread
[11:29] <mvo> cachio: I have not looked yet, does the failure look familiar in any way?
[11:29] <cachio> https://paste.ubuntu.com/p/z7jh7KwJt9/
[11:29] <cachio> mvo, it is happening on 2.36 family
[11:30] <cachio> mvo, I am already trying to reproduce it
[11:40] <zyga> cachio: no  need
[11:40] <zyga> cachio: this is the problem we've been trying to understand recently
[11:41] <zyga> and now I believe we mostly do
[11:41] <cachio> zyga,ah, ok
[11:41] <cachio> zyga, It just failed here :)
[11:43] <Saviq> cachio: I did, it seems to have failed to install our deps https://travis-ci.org/MirServer/mir/jobs/461204937
[11:43] <Saviq> I need to add a spread timeout < 50 minutes so it bails out I suppose
[11:43] <mvo> zyga, cachio is it the permission denied bug?
[11:43] <zyga> yes
[11:44] <mvo> cachio, zyga thanks! in this case I hope to push a PR soon with a fix
[11:44] <cachio> mvo, yes
[11:44] <cachio> cannot create temporary directory for /var/lib/snapd mount point: Permission denied
[11:44] <zyga> mvo: does systemd-run play ball on 14.04?
[11:44] <mvo> cachio: yeah, I hope to have something ready before or shortly after lunch
[11:44] <mvo> zyga: I don't know, need to check but if not that would suck(tm)
[11:45] <mvo> zyga: 14.04 is just ~5 more month but still
[11:45] <zyga> mvo: the set of patches I proposed fixes this too
[11:45] <zyga> I will land all the bits needed today
[11:46] <zyga> I don't mind systemd-run being used as well,\ we may have other issues causing system key-based problem
[11:47] <mvo> zyga: hm, you mean 6234 - or more?
[11:47] <zyga> https://github.com/snapcore/snapd/pull/6235
[11:47] <mup> PR #6235: overlord,apparmor: new syskey behaviour + non-ignored snap-confine profile errors <⛔ Blocked> <Created by zyga> <https://github.com/snapcore/snapd/pull/6235>
[11:48] <zyga> that's the full set but it is blocked by leap
[11:48] <zyga> https://github.com/snapcore/snapd/pull/6235/commits/15cae44908738b6c8e1814563c0cae727594a3d7
[11:48] <zyga> have a look at each patch there
[11:49] <mvo> zyga: interessting - and its green, nice. is this reliable, i.e. did you run a couple of times?
[11:50] <zyga> not in that PR but yeah, a few times locally
[11:50] <zyga> feel free to restart it as many times as you like
[11:50] <zyga> it failed only on noise like mount error or store timeout
[11:50] <mvo> great
[12:08] <mvo> zyga: looks like systemd-run is out, a shame
[12:08] <zyga> oh, why?
[12:08] <mvo> zyga: not availalbe on trusty, it has 204 but we need 205
[12:08] <zyga> 14.04?
[12:08] <zyga> :////
[12:08] <zyga> bummer
[12:08] <zyga> well
[12:08] <zyga> 4 months
[12:08] <mvo> zyga: yeah, indeed
[12:08] <zyga> and I'm busy writing tests
[12:08] <mvo> oh well, I will shelve the PR
[12:08] <mvo> until then
[12:08] <zyga> I'm happy to finally end 2.36 drama ;)
[12:10] <mvo> zyga: indeed
[12:10] <mvo> zyga: well done!
[12:11] <cachio> zyga, :)
[12:21] <mup> PR snapd#6239 closed: packaging/fedora/snapd.spec: fix bogus date in changelog <Created by mvo5> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/6239>
[12:22] <Chipaca> mvo: systemd-run wha?
[12:31] <pedronis> Chipaca: it doesn't exist in 14.04 ?
[12:46] <zyga> mvo: I pushed unit test to https://github.com/snapcore/snapd/pull/6233
[12:46] <mup> PR #6233: overlord: don't write system key if security setup fails <Created by zyga> <https://github.com/snapcore/snapd/pull/6233>
[12:46] <zyga> mvo: I will now add a spread test
[12:46] <zyga> have a look please
[12:57] <mborzecki> off to pick up the kids
[13:07] <pstolowski> mborzecki: what's the story with https://bugs.launchpad.net/snapd/+bug/1759349 ?
[13:07] <mup> Bug #1759349: Confinement doesn't work on arch (manjaro) linux <snapd:Triaged> <https://launchpad.net/bugs/1759349>
[13:10] <mvo> zyga: sure, looking
[13:11] <zyga> thx
[13:12] <mvo> Chipaca: https://github.com/snapcore/snapd/compare/master...mvo5:systemd-run-security-gen?expand=1 <- the idea was to avoid that apparmor_parser and snap-seccomp processes get killed when we restart snapd
[13:12] <mvo> Chipaca: but a non-starter for now
[13:19] <jdstrand> mvo: curious how snapd fits into trusty/esm...
[13:20] <mvo> jdstrand: last time we talked about this someone mentioned that snapd would probably not included but that wasn't very official
[13:20] <zyga> jdstrand: gives mvo more gray hair?
[13:20] <jdstrand> mvo: probably want to talk to joe and amurray. they arer in the process of defining what's in trusty/esm and that is coming from stakeholders
[13:21] <mvo> jdstrand: ok
[13:22] <jdstrand> mvo: I hadn't really thought that snapd would be in esm, but it seems likely it will have a (much) expanded package set than precise/esm
[13:23] <jdstrand> mvo: I guess part of this would be looking at data for snapd on trusty and seeing how that maps to esm customers
[13:24] <jdstrand> mvo: which joe and amurray may need some help with (on the data collection bits)
[13:24] <mvo> jdstrand: makes sense. however the version of systemd (204) there is giving us a headache
[13:24] <jdstrand> mvo: not trying to give you gray hair, sorry :\
[13:24]  * jdstrand nods
[13:24] <zyga> mvo: wrote the spread test, running it now
[13:24] <zyga> I'll make something warm
[13:25] <mvo> zyga: thanks!
[13:25] <mvo> zyga: and sorry for commenting on the wrong PR :/ I commtned on the right now now too
[13:25] <zyga> mvo: with both spread and unit test I'm happy with this being merged into master
[13:25] <zyga> mvo: hah, no worries :)
[13:25] <zyga> least of our problems
[13:25] <jdstrand> mvo: I wouldn't necessarily consider also upgrading systemd in trusty/esm off the table. there is more flexibility there, so *if* snapd should be included in trusty/esm, perhaps there are things that can be done to make it easier on you
[13:25] <mvo> jdstrand: well, it will be a discussion but trusty is a real burden for us, that should be taken into consideration
[13:26] <jdstrand> mvo: sure, which is why I wanted to mention that you should let joe and amurray aware of that. you're obviously a stakeholder :)
[13:26] <mup> PR snapcraft#2380 closed: tests: use autopkgtest to leverage snap testing <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2380>
[13:40] <zyga> jdstrand: I would love 14.04 == 16.04 as far as systemd is concerned
[13:41] <zyga> would be much sweeter support target
[13:41] <zyga> we have plenty of // TODO because 14.04 ... in the code
[13:41] <zyga> mvo: I added 2.36.3 milestone
[13:41] <mvo> zyga: thanks
[13:42] <mvo> zyga: to LP?
[13:42] <zyga> yes
[13:42] <mvo> zyga: cool
[13:42] <zyga> mborzecki: probably want to update https://bugs.launchpad.net/snapd/+bug/1772016
[13:42] <mup> Bug #1772016: Mount snap "snapcraft" (1591) ([start snap-snapcraft-1591.mount] failed with exit status 1: <snapd:Triaged> <https://launchpad.net/bugs/1772016>
[13:53] <mup> PR snapd#6231 closed: data: set KillMode=process <⛔ Blocked> <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/6231>
[13:57] <zyga> mvo: I must say I really enjoy filing bugs and writing regression tests
[13:57] <zyga> that paper trail and non-main test means that we still get rapid testing of features (main) but get nice and good feeling that over time things don't explode on bugs that we fought before
[13:58] <zyga> and if it happens we have all the context to look at
[14:01] <pedronis> Chipaca: standup?
[14:01] <Chipaca> pedronis: omw
[14:02] <pedronis> cachio: ^
[14:02] <mvo> zyga: indeed
[14:09] <mvo> cachio: 2.36.2 is ready for beta validation now :)
[14:09] <cachio> mvo great
[14:09] <cachio> I'll start asap
[14:12]  * cwayne gets excited for some core results
[14:13] <mvo> kenvandine: the fontconfig fix is in the core beta now, if you could test and ask your team to test that would be awesome
[14:14] <kenvandine> mvo: awesome
[14:14] <kenvandine> thanks
[14:14] <kenvandine> mvo: how about core18?
[14:14] <mvo> kenvandine: it will work there too
[14:14] <kenvandine> sweet
[14:14] <mvo> kenvandine: I mean, it will work with snaps that use core18 as well
[14:16] <sergiusens> niemeyer: hello, just to clarify, is the conclusion from https://forum.snapcraft.io/t/snap-multi-line-descriptions-need-newline-trim-on-store-import/3934/12 to use | instead of > considering that the markdown filter will apply the appropriate formatting wrt (in this case) newlines?
[14:19] <mborzecki> zyga: can you try 'snap install hello-world_foo hello-world_bar'? does it work for you?
[14:19] <zyga> mborzecki: in a sec
[14:22] <zyga> mvo: if you can do a final ack on https://github.com/snapcore/snapd/pull/6233
[14:22] <mup> PR #6233: overlord: don't write system key if security setup fails <Created by zyga> <https://github.com/snapcore/snapd/pull/6233>
[14:23] <mup> PR snapcraft#2417 closed: Revert "lifecycle: make snapcraft init template use > not | (#2393)" <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2417>
[14:24] <zyga> mborzecki: yes
[14:24] <zyga> mborzecki: works ok
[14:24] <mborzecki> zyga: both snaps in a single line?
[14:24] <zyga> yes
[14:25] <mborzecki> zyga: are you running the latest master?
[14:25] <zyga> no
[14:25] <zyga> 2.36.1
[14:25] <zyga> sorry, too many machines :)
[14:25] <zyga> my master machine is on the right
[14:25] <zyga> this one was on the left
[14:25] <mborzecki> zyga: i get 'error: store.SnapNotFound with 2 snaps'
[14:26] <zyga> hmmmm
[14:30] <niemeyer> sergiusens: It looks like ">" (aka line folding) changes the output completely from what is presented to the user. It even corrupts the rendering once the text is interpreted as Markdown.
[14:30] <niemeyer> sergiusens: Isn't that true?  If it is, then it indeed looks like a terrible idea to use it.
[14:31] <sergiusens> niemeyer: yes, exactly why I am inclined to revert it https://www.irccloud.com/pastebin/WaNVs8vG/para%20que%20veas%20el%20efecto
[14:38] <zyga> mup: hello
[14:38] <mup> zyga: I apologize, but I'm pretty strict about only responding to known commands.
[14:39] <niemeyer> Heh..
[14:39] <zyga> 2fa?
[14:39] <niemeyer> No.. either chrome or hangouts itself is hanging on me
[14:41] <mup> PR snapd#6241 opened: tests/main/parallel-install-store: verify installation of more than one instance at a time <Parallel installs ⛓> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6241>
[14:45] <mup> PR snapd#6240 closed: release: 2.36.2 <Created by mvo5> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6240>
[14:56] <niemeyer> sergiusens: Why are you just inclined?  Is the behavior reasonable?
[14:58] <pedronis> pstolowski: niemeyer:  I wrote something here: https://bugs.launchpad.net/snapd/+bug/1777121
[14:58] <mup> Bug #1777121: Remove is called after snap services are stopped  <snapd:In Progress by stolowski> <https://launchpad.net/bugs/1777121>
[14:58] <niemeyer> sergiusens: To me it looks clearly like a bug.. ideally that sort of change should be carefully investigated before it takes place
[14:59] <niemeyer> pedronis: Thanks.. depending on their use case, we might have a hook like this, but I'd try to cook it in a way that detaches from the promise of running services
[15:03] <pedronis> niemeyer: let's see if they have further feedback, now that the picture is clearer
[15:03]  * zyga -> soup
[15:04] <pstolowski> pedronis: ty. i marked my PR blocked for now, we can decide if we want to close depending on feedback
[15:06] <Chipaca> brb, need to reboot - plugging in my headset hangs my trackpad
[15:06] <Chipaca> (WAT)
[15:09] <cachio> mvo, hey
[15:09] <cachio> https://paste.ubuntu.com/p/gNMHkR2Wpt/
[15:10] <cachio> any idea about this request
[15:10] <cachio> it is taking about 3/4 seconds
[15:10] <cachio> delaying the test suite
[15:12] <niemeyer> pedronis: We might call it "cleanup", for example.. we should just make its semantics more clear, including when exactly it's called, what are the promises or possibilities, and we might also consider what other use cases we might have for the same hook
[15:14] <mvo> cachio: not sure where it comes from, what happens before/after in the logs?
[15:16] <cachio> mvo, https://paste.ubuntu.com/p/ct763X3tj5/
[15:17] <cachio> mvo, it jumps from 14:50:20 to 14:50:24
[15:17] <cachio> with that request
[15:17] <jdstrand> mvo: hey, did you see my response regarding your compression exploration? we don't need to discuss it here, but if you start to narrow in on something, please perform resquash tests (I can show you how) on a variety of snaps (eg, large, like chromium as well as snaps built on one arch (eg, i386, armhf, arm64) and resquashed on amd64)
[15:17] <cachio> it happens when we stop the snapd.service
[15:21] <mborzecki> could we land https://github.com/snapcore/snapd/pull/6211 ? super simple
[15:21] <mup> PR #6211: spread: run tests on Fedora 28 again <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6211>
[15:22] <rbasak> "error: cannot install zero snaps" -- not something I ever thought I'd see a dedicated error message for :)
[15:23] <rbasak> (my fingers raced a middle click paste against the return key)
[15:24] <zyga> rbasak: attention to detail :-)
[15:27] <mvo> jdstrand: yeah, saw it but didn't look at the details yet
[15:28] <mvo> cachio: aha, looks like the catalog update
[15:28] <mvo> cachio: I guess we could disable that via some magic
[15:29] <mvo> cachio: but we need to write code for that, iirc we currently unconditionally run it
[15:29] <cachio> mvo, the problem is that we start / stop it on reset
[15:30] <cachio> and it takes like 7/10 seconds to stop it
[15:30] <jdstrand> mvo: ok, don't want to distract you, just want you to be aware that we should not regress resquash tests in the store and that, having been through this with our existing options, things are rather delicate
[15:30] <cachio> mvo, is it refreshing the catalog any time we start snapd?
[15:41] <Chipaca> cachio: catalog refresh happens every startup yes
[15:42] <Chipaca> cachio: also sometimes assertions
[15:42] <zyga> fun fact: on freebsd there's /.snap and then there's a /sys symlink to source code
[15:42] <Chipaca> zyga: what's /.snap ?
[15:42] <zyga> empty directory
[15:43] <zyga> maybe snapshot spot?
[15:43] <zyga> not sure yet
[15:43] <zyga> Chipaca: you will like this one
[15:43] <Chipaca> zyga: https://lists.freebsd.org/pipermail/freebsd-questions/2012-January/237296.html
[15:43] <zyga> Chipaca: /home is a symlink to /usr/home :D
[15:43] <cachio> Chipaca, mmm, ok, perhaps I can make small change to avoid stop snapd just after we start it
[15:46] <zyga> Chipaca: also /proc exists, but it is empty
[15:46] <zyga> freebsd is odd :)
[15:46] <zyga> Chipaca: there's /rescue which is much like /bin but everything is statically linked
[15:46] <zyga> I see what you did there freebsd
[15:47]  * cachio lunch
[15:48] <Chipaca> cachio: or we could make snapd check the timestamp on the catalog and not refresh too often
[15:54] <zyga> mvo, Chipaca found a fun bug
[15:54] <zyga> https://pastebin.ubuntu.com/p/xyNHG2Hqdk/
[15:54] <zyga> we mount core18 _after_ starting snapd
[15:54] <zyga> and things go south
[15:55] <zyga> is this expected?
[15:58] <Chipaca> zyga: all bugs are expected
[15:58]  * Chipaca puts down his tea and looks at it quizzically
[15:59] <pedronis> zyga: where? what's the context of that?
[15:59] <zyga> pedronis: regression test noticed that things misbehave on a core18 system
[16:00] <zyga> just vanilla run-off-the-mill core18 test system
[16:00] <zyga> looking at journal logs to see what was wrong
[16:05] <zyga> I guess this possible means that we mount any snap after snapd is started
[16:05] <zyga> which would be possibly quite grim
[16:05] <zyga> shall I report it and finish my current task?
[16:07] <cachio> Chipaca, yes, that too
[16:09] <pedronis> zyga: yes
[16:13] <mvo> zyga: yeah, thanks for finding this
[16:13] <zyga> reported as https://bugs.launchpad.net/snapd/+bug/1805866
[16:13] <mup> Bug #1805866: On core18 system core18 snap was mounted after snapd had started <snapd:New> <https://launchpad.net/bugs/1805866>
[16:14] <mvo> zyga: its slightly strange iirc we set Before=snapd.service in our mount units plus mounts happen before multi-user
[16:14] <zyga> and if I can have one xmas present, I would love to see markdown support on launchpad comments
[16:14] <mvo> zyga: but might be because of something in core18 that is special
[16:14] <zyga> yep
[16:15] <zyga> it's "fun" that this fix for 2.36 uncovers issues like that
[16:15] <mvo> zyga: yeah
[16:15] <mvo> zyga: is this first run on core18? I mean, very first start? there nothing is mounted yet, snapd needs to bootstrap itself
[16:15] <zyga> mvo: I just ran
[16:16] <zyga> spread -debug -v google:ubuntu-core-18-64:tests/regression/lp-1805838
[16:16] <zyga> maybe try with -shell-after
[16:16] <mup> PR snapd#6211 closed: spread: run tests on Fedora 28 again <Created by bboozzoo> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/6211>
[16:16] <mup> PR snapd#6242 opened: overlord/snapstate: use file timestamp to initialize timer <Created by chipaca> <https://github.com/snapcore/snapd/pull/6242>
[16:16] <mvo> zyga: ok, what PR was this again?
[16:16] <Chipaca> cachio: ^
[16:16] <zyga> https://github.com/snapcore/snapd/pull/6233
[16:16] <mup> PR #6233: overlord: don't write system key if security setup fails <Created by zyga> <https://github.com/snapcore/snapd/pull/6233>
[16:17] <zyga> the test passes on all systems now
[16:20] <mvo> zyga: thanks! running it now
[16:25] <mup> PR snapd#6243 opened: systemd: allow only a single daemon-reload at the same time <⛔ Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/6243>
[16:31] <pstolowski> zyga: can you take a look at #6180 (doesn't have to be today)
[16:31] <pstolowski> ?
[16:31] <zyga> k
[16:31] <mup> PR #6180: snap/info: bind global plugs/slots to implicit hooks <Complex> <Created by stolowski> <https://github.com/snapcore/snapd/pull/6180>
[16:31] <zyga> ah
[16:31] <zyga> sure
[16:32]  * pstolowski should probably remove complex label from it and have something that attracts people instead ;)
[16:32] <zyga> we need a <cookie> label
[16:32] <pstolowski> yay
[16:32] <roadmr> 🍪
[16:32] <pstolowski> it's not really complex btw.. just needs some insight maybe
[16:32] <zyga> roadmr: just need to cross check with kyrofa about how it renders
[16:33] <pstolowski> roadmr: lovely
[16:33] <mup> PR snapcraft#2418 opened: Release changelog for 3.0.1 <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2418>
[16:34] <pedronis> zyga: do we need #6230 ?
[16:34] <mup> PR #6230: spread: detect "signal: terminated" in journal logs <⛔ Blocked> <Created by zyga> <https://github.com/snapcore/snapd/pull/6230>
[16:34] <zyga> no, it was just experiments
[16:34] <zyga> closed now
[16:34] <mup> PR snapd#6230 closed: spread: detect "signal: terminated" in journal logs <⛔ Blocked> <Created by zyga> <Closed by zyga> <https://github.com/snapcore/snapd/pull/6230>
[16:35] <zyga> pedronis, mvo: though the forkstat stuff was amazing
[16:37] <mvo> zyga: yeah, I think we keep this in mind, something like this may come back later
[16:39] <pedronis> Chipaca: #6192 needs a 2nd review and I suggested a smal tweak
[16:39] <mup> PR #6192: overlord/snapstate: on refresh, check new rev can read current <Created by chipaca> <https://github.com/snapcore/snapd/pull/6192>
[16:40] <mvo> zyga: I suspect the issue with the core-18 is really that on first-run nothing is available yet. otoh the state should also be empty so slightly strange
[16:41] <zyga> maybe particular test setup is the culprit but I don't expect this to happen
[16:41] <mvo> zyga: I see "Reboot on qemu:ubuntu-core-18-64 ... is taking a while" and it seems like its hanging
[16:41] <zyga> we're either seeging
[16:41] <zyga> *seeding
[16:41] <zyga> and snaps are installed
[16:41] <zyga> or we know nothing about them
[16:41] <zyga> feels like a bug for real at some level
[16:41] <mvo> zyga: oh, they are installed, hm, hm, if so we should have a Before=snapd.service
[16:41] <zyga> mmm
[16:41] <zyga> yeah
[16:41] <zyga> I haz real bug
[16:42] <pedronis> zyga: mvo: we do have some tests that wipe the state
[16:42] <zyga> pedronis: I ran that single test in isolation
[16:42] <pedronis> and simulate the seeding again
[16:42] <zyga> it was the first code to run
[16:42] <pedronis> zyga: what test is it?
[16:43] <mvo> zyga: did you also get this wait?
[16:43] <zyga> spread -debug -v google:ubuntu-core-18-64:tests/regression/lp-1805838 from https://github.com/snapcore/snapd/pull/6233
[16:43] <mup> PR #6233: overlord: don't write system key if security setup fails <Created by zyga> <https://github.com/snapcore/snapd/pull/6233>
[16:43] <zyga> wait?
[16:44] <pedronis> am I reading it wrong, that info is not in the bug?
[16:45] <zyga> no, it's not in the bug but that test is arguably not doing anything, it just restarts snapd to see what the logs show -
[16:46] <pedronis> it's a new test?
[16:46] <pedronis> I don't have it here
[16:46] <zyga> yes
[16:47] <zyga> it's a regression test for that branch
[16:49] <zyga> mvo: I missed this part:
[16:49] <zyga> zyga: I see "Reboot on qemu:ubuntu-core-18-64 ... is taking a while" and it seems like its hanging
[16:49] <zyga> mvo: I didn't get that
[16:49] <zyga> is it still hanging
[16:49] <zyga> ?
[16:49] <mvo> zyga: hm, ok
[16:50] <mvo> zyga: yeah, I try again
[16:50] <zyga> eh, failed on mount error
[16:50] <zyga> - Mount snap "test-snapd-tools" (7) ([start var-lib-snapd-snap-test\x2dsnapd\x2dtools-7.mount] failed with exit status 1: Job for var-lib-snapd-snap-test\x2dsnapd\x2dtools-7.mount failed.
[16:50] <zyga> See "systemctl status "var-lib-snapd-snap-test\\x2dsnapd\\x2dtools-7.mount"" and "journalctl -xe" for details.
[16:51] <mvo> zyga: I was running it in qemu
[16:51] <zyga> aha
[16:51] <mvo> zyga: running it in google now
[16:54] <zyga> afk
[16:54] <zyga> going to my son's birthday :)
[16:54] <zyga> ttyl
[17:05] <kyrofa> Haha, zyga|afk looks like a rock
[17:07] <roadmr> https://www.youtube.com/watch?v=2NEbe_brJAQ
[17:18] <mvo> zyga|afk: enjoy!
[17:34] <mvo> zyga|afk: hm, hm, spread -v -debug  google:ubuntu-core-18-64:tests/regression/lp-1805838 just worked, I try again after dinner
[17:37] <zyga|afk> mvo: yes, it passes
[17:37] <zyga|afk> mvo: try -shell-after
[17:37] <zyga|afk> see what logs say
[18:47] <zyga> whee
[18:48] <mup> PR snapd#6233 closed: overlord: don't write system key if security setup fails <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6233>
[21:42]  * cachio afk
[23:14] <mup> PR snapd#6244 opened: release: detect too old apparmor_parser <Created by zyga> <https://github.com/snapcore/snapd/pull/6244>
[23:15]  * zyga really goes to sleep now