[06:08] <zyga> Good morning
[06:19] <mborzecki> morning
[06:22] <zyga> Hey
[06:22] <zyga> How are you?
[06:23] <mborzecki> zyga: fine, thanks
[06:24] <mborzecki> zyga: it's cold outside :/
[06:24] <zyga> Yeah
[06:24] <zyga> I was walking with the dog for two hours last night
[06:25] <zyga> All the way until midnight
[06:25] <mborzecki> zyga: nice
[06:31] <zyga> mborzecki: quick review to start the day? https://github.com/snapcore/snapd/pull/6587
[06:31] <mup> PR #6587: interfaces/apparmor: factor out test boilerplate <Simple 😃> <Created by zyga> <https://github.com/snapcore/snapd/pull/6587>
[06:31] <mborzecki> zyga: sure, why not
[06:34] <zyga> mborzecki: I changed some of the test function names but that is all sensibile in the follow-up patch
[06:48] <zyga> mborzecki: pushed
[06:52] <mborzecki> zyga: thanks!
[06:52] <zyga> thanks
[06:52] <zyga> I will push something that  enables apparmor on all of suse next
[06:53] <zyga> then I'll look at debian
[06:53] <zyga> I think we can handle debian (and derivatives), suse (and derivatives if any) to cover most of apparmor land
[07:05] <zyga> mborzecki: https://lwn.net/Articles/782786/
[07:05] <zyga> maybe mvo should apply
[07:05] <zyga> ;D
[07:21] <pedronis> zyga: hi, when are you going to address 6360 feedback ?
[07:21] <zyga> pedronis: hi
[07:22] <zyga> pedronis: most likely today
[07:22] <mborzecki> guys, think we can land this now if everyone is happing with the current phrasing? https://github.com/snapcore/snapd/pull/6556
[07:22] <mup> PR #6556: cmd/snap: hide 'interfaces' command, show deprecation notice <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6556>
[07:22] <zyga> pedronis: I already started yesterday but haven't finished yet
[07:25] <pedronis> mborzecki: I didn't see all the bikeshedding there
[07:26] <mvo> good morning
[07:27] <mvo> mborzecki: hey, yay (or :cry:) at 6585
[07:27] <pedronis> mborzecki: I fear the new sentence is too long
[07:27] <mvo> hey pedronis
[07:28] <pedronis> mborzecki: I would like John input on htat
[07:29] <pedronis> *that
[07:29] <zyga> hey mvo
[07:30] <pedronis> mborzecki: I commented there
[07:30] <mborzecki> pedronis: thans
[07:30] <mvo> hey zyga
[07:30] <pedronis> mvo: hi
[07:31] <zyga> mvo: quick review to start the day? https://github.com/snapcore/snapd/pull/6587
[07:31] <mup> PR #6587: interfaces/apparmor: factor out test boilerplate <Simple 😃> <Created by zyga> <https://github.com/snapcore/snapd/pull/6587>
[07:31] <mvo> zyga: sure
[07:31] <mvo> (when typing this the mouse focus was still on mutt - typing random chars into mutt is really scary)
[07:32] <zyga> mvo: lol
[07:32] <zyga> yes
[07:32] <zyga> kind of like vi but then git prevents critical damage
[07:32] <zyga> no such thing for mutt
[07:33] <zyga> mvo: do you want to travel more, apply now for the position of debian leader: https://lwn.net/Articles/782786/
[07:33] <mvo> zyga: *cough*
[07:34] <mvo> zyga: and no
[07:34] <zyga> :D
[07:34] <zyga> join debian they said
[07:34] <zyga> see the world the said
[07:34] <mvo> zyga: but it would be interessting, pushing some ideas to moderinize things would be nice
[07:34] <mvo> zyga: hahaha
[07:34] <zyga> yeah
[07:34] <zyga> mvo: I see the reference was recognized :)
[08:06] <pstolowski> morning
[08:13] <zyga> hey pawel
[08:37] <mup> PR snapd#6587 closed: interfaces/apparmor: factor out test boilerplate <Simple 😃> <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6587>
[08:38] <mup> PR snapd#6580 closed: cmd/snap-confine: drop unused dependency on libseccomp <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/6580>
[08:41] <mup> PR snapd#6588 opened: interfaces/apparmor: partial confinement on openSUSE <Created by zyga> <https://github.com/snapcore/snapd/pull/6588>
[09:14] <pedronis> zyga: did you change 6575 since I reviewed it?
[09:15] <zyga> pedronis: I applied the changes you asked for, fixed a typo that broke he build then merged master and pushed
[09:15] <zyga> I then noticed that I left duplicate include line and force pushed to clear it
[09:15] <zyga> that's all
[09:16] <pedronis> zyga: no, sc_init_invocation was not there when I reviewed it
[09:16] <zyga> oh? but it is added before your review?
[09:16] <pedronis> no
[09:17] <zyga> please double check, perhaps you reviewed a view that was stale
[09:17] <zyga> it is so according to the github timeline
[09:17] <pedronis> zyga: you force pushed, not point debating this
[09:17] <zyga> yes, I said that
[09:18] <zyga> but that was in the morning  after some things landed and the branch conflicted
[09:18] <zyga> you can compare the revisions if you want
[09:19] <pedronis> anyway I removed my +1
[09:20] <mvo> zyga: yeah, 6575 changed quite a bit since I looked at it - or so it felt to me this morning but I have not looked at the details
[09:20] <mvo> slightly sad because the previous one was ready for merging afaict
[09:21] <zyga> mvo: I implemented the changes you suggested
[09:21] <zyga> mvo: and made it clear in the log as well
[09:22] <pedronis> zyga: my emails says that stuff was pushed 20 hours ago, and my review was 21 hours ago
[09:22] <pedronis> but as I said hard to tell with force push
[09:22] <zyga> pedronis: but that force push was done this morning
[09:22] <zyga> and after I pushed yesteray I was happy to see your review
[09:23] <mvo> zyga: sorry, I'm confused, in 6575 I just wrote "looks good" - am I mixing something up?
[09:23] <zyga> I'm 100% sure github showed this the same way before i did that in the morning
[09:23] <zyga> mvo: we discussed this durinig the standup and  you suggested that I put additional patches to the same branch
[09:23] <pstolowski> pedronis: hello, sample output of new timings to discuss - https://paste.ubuntu.com/p/gXrk8sBwwB/ - contains timings of some tasks, ifacemgr startup, snapmgr ensure loop (nb, do we really run ensure so often?). combines doing/undoing times from tasks
[09:24] <zyga> pedronis: this is especially clear since the three patches starting with cmd/snap-confine: tweak ordering of fields in sc_invocation are added in response to your review
[09:25] <mvo> zyga: oh, sorry. my bad then. maybe we can go back to the point in time (via a PR) that had the +1 from pedronis  and the "looks good" from me and then land those bits easily?
[09:25] <ondra> zyga hey
[09:25] <ondra> zyga wanna talk about those PR for gadget snaps?
[09:26] <zyga> mvo: but the unclear aspect is what pedronis actually had reviewed
[09:26] <ondra> zyga unless you already had chat with pedronis
[09:26] <zyga> ondra: hey, I asked about it yestereday
[09:26] <pedronis> zyga: I didn't review the init helper
[09:26] <zyga> ondra: I don't know what I can do to help
[09:26] <pedronis> maybe it was stale
[09:26] <pedronis> whatever
[09:26] <ondra> zyga yeah I was already off when I saw you message
[09:26] <pedronis> it needs a review from scratch from both of us
[09:26] <ondra> zyga nothing apart of +1 :D
[09:27] <zyga> ondra: I don't fully understand the consequence of that, it is  something perdronis should look at first
[09:27] <ondra> zyga I think rest is on foundation
[09:27] <ondra> zyga yeah we already talked with him
[09:30] <pedronis> zyga: if I have too be honest I find snap-confine is starting to be too much split out in helpers, it's a maze to follow
[09:31] <pedronis> I'm sure they all make sense locally but if in a 2nd time you need to find out what's going on is a chase
[09:35] <Chipaca> mborzecki: I pushed a tweak to the interfaces deprecation pr
[09:35] <Chipaca> pedronis: ^
[09:35] <pstolowski> pedronis: let me know when/if you have a moment to discuss this
[09:35] <mborzecki> Chipaca: thanks!
[09:36] <pedronis> pstolowski: we call ensure loop each 5 mins but also each time we need to run more task with EnsureBefore(0)
[09:36] <mborzecki> Chipaca: is fill() aware of a sequence '..' and will not break it?
[09:36] <Chipaca> mborzecki: it is not
[09:37] <Chipaca> mborzecki: ¯\_(ツ)_/¯
[09:37] <Chipaca> I mean, breaking it isn't ideal but isn't the end of the world either
[09:37] <mborzecki> Chipaca: fits unboken in my term so :)
[09:38] <Chipaca> mborzecki: also breaks just right on 80
[09:38] <mborzecki> mhm
[09:38] <pedronis> Chipaca: I don't understand how using fill helps
[09:39] <pedronis> it's going to take up vertical space anyway
[09:39] <Chipaca> pedronis: your issue was with vertical space, not with improper word-splitting?
[09:39] <pedronis> yes
[09:39] <mborzecki> pedronis: https://paste.ubuntu.com/p/7qzSqPM4Zc/
[09:39] <pstolowski> ah, right, EnsureBefore(0)
[09:40] <Chipaca> pedronis: but the deprecation is only shown on 'snap interfaces'
[09:40] <pedronis> pstolowski: we really need to be careful with it
[09:40] <Chipaca> pedronis: which is not concerned with vertical space
[09:41] <pedronis> Chipaca: no, but is a table
[09:41] <Chipaca> pedronis: so?
[09:41] <mborzecki> the message goes to stderr
[09:42] <pedronis> I know
[09:42] <pedronis> we are still eating 3 lines
[09:42] <pedronis> seems too much
[09:43] <Chipaca> grah
[09:43] <pstolowski> Chipaca: have you by an chance implemented our TabWriter?
[09:44] <Chipaca> pstolowski: we don't have a TabWriter
[09:44] <Chipaca> pstolowski: why?
[09:44] <Chipaca> pstolowski: we do have a tabWriter, I think, that wraps TabWriter
[09:44] <Chipaca> as a helper function i mean
[09:44] <Chipaca> not as a type
[09:45] <pedronis> Chipaca: 'snap interfaces' is deprecated and replaced by the 'snap connections' command.  is 79
[09:45] <Chipaca> pedronis: in English
[09:45] <pedronis> Chipaca: they we just hide it
[09:45] <pstolowski> Chipaca: ah, nvm, it's a standard go lib thing
[09:45] <pedronis> thne
[09:45] <pedronis> Chipaca: then we just hide it
[09:46] <Chipaca> pedronis: I don't understand why
[09:46] <Chipaca> I don't understand why 3 lines are too many, on a command that produces 121 lines of output by default
[09:47] <pedronis> Chipaca: it doesn always produces 121 lines
[09:48] <pedronis> Chipaca: consider snap interfaces gnome-logs
[09:48] <Chipaca> pedronis: ok
[09:48] <Chipaca> pedronis: so what's the issue
[09:52] <pedronis> Chipaca: we started here  "The 'snap interfaces' command is deprecated, try the new 'snap connections'."
[09:54] <Chipaca> pedronis: yes
[09:54] <mborzecki> i don't mind going back to the first version
[09:55] <Chipaca> We could even go with
[09:55] <Chipaca> 'snap interfaces' is deprecated; use 'snap connections'.
[09:55] <Chipaca> this incorporates all the feedback so far
[09:55] <mborzecki> works for me
[09:55] <mborzecki> pedronis: ^^ wdyt?
[09:56] <mvo> Chipaca: +1
[09:56] <Chipaca> mborzecki: I'll push the change, plus adding the notice to the long help
[09:56] <mvo> (fwiw)
[09:57] <mborzecki> Chipaca: thank you!
[09:57] <mvo> I updated 6401 fwiw
[09:58] <pedronis> Chipaca: the last version is very long, but it's all facts, it doesn't recommend anything
[09:58] <pedronis> (except implicitly)
[09:59] <Chipaca> pedronis: the _last_ version? “'snap interfaces' is deprecated; use 'snap connections'.”?
[09:59] <pedronis> Chipaca: the last version in the PR
[09:59] <Chipaca> pedronis: you ok with the last one here? i think it covers all the issues we've come across so far
[10:00] <pedronis> Chipaca: it's very terse
[10:01] <pedronis> sorry, that's not the problem
[10:01] <Chipaca> I'd also add
[10:01] <Chipaca> NOTE this command is deprecated and has been replaced with the 'connections'
[10:01] <Chipaca>      command.
[10:01] <Chipaca> to the longHelp
[10:01] <pedronis> thanks
[10:12] <pedronis> Chipaca: have we goon too far the other way around though? is the new one too prompt?
[10:14] <Chipaca> pedronis: nah, it's fine
[10:14] <Chipaca> pedronis: adding 'please' and 'try' are both problematic for reasons degville explained in the pr
[10:15] <pedronis> Chipaca: wondering if we want "the new" , issue with that is that at some point we would need to remove "new" because is untrue
[10:16] <Chipaca> pedronis: I really think it's fine as is
[10:16] <pedronis> ok
[10:28] <mup> PR snapd#6589 opened: daemon: support returning assertion information as JSON with the "json" query parameter <Created by pedronis> <https://github.com/snapcore/snapd/pull/6589>
[10:29] <pedronis> Chipaca: ^  something for you when you have a bit of time
[10:29]  * Chipaca looks at next year's calendar
[10:29] <Chipaca> ok
[10:30] <pedronis> heh
[10:30] <Chipaca> :-)
[10:54] <mup> PR snapcraft#2499 closed: many: support for "base: core" in snapcraft.yaml <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2499>
[10:56] <ogra> mvo, here is another core18 regression https://bugs.launchpad.net/snappy/+bug/1819629
[10:56] <mup> Bug #1819629: no default locale in core18 causes pam errors <Snappy:New> <https://launchpad.net/bugs/1819629>
[10:56] <mup> Bug #1819629 opened: no default locale in core18 causes pam errors <Snappy:New> <https://launchpad.net/bugs/1819629>
[10:57] <mvo> ogra: thanks - that sounds hopefully straightforward to fix, maybe sil2100  can give 1819629  a stab
[10:59] <ogra> yeah, just some "echo C.UTF-8 >/etc/default/locale" somewhere would be enough
[10:59] <ogra> (in the build)
[11:00] <sil2100> Oh, indeed
[11:01] <mvo> ogra, sil2100 yeah, the new build system has a "static" dir, we could simply drop static/etc/default/locale into the core18 tree and its fixed :)
[11:01] <ogra> awesome
[11:02] <ogra> sil2100, also https://forum.snapcraft.io/t/cant-find-the-spi-interface/7523 ... you might need to pull the latest upstream into the CanonicalLtd branch for the pi3 gadget SPI interface fix
[11:03] <ogra> i wonder if we could put the snapcore/pi3-gadget tree to rest somehow witout breaking all the forks of it (not sure if github supports re-owning branches and keeps the references)
[11:26] <ondra> zyga what happens when I change base in snap between revisions, will snapd correctly run new revision agains different base?
[11:27] <zyga> ondra: when all processes of that snap terminate, yes
[11:28] <ondra> zyga is there way to check what base it's working
[11:28] <zyga> ondra: yes, sure, just nsenter and look
[11:29] <ondra> zyga hmm for me refresh failed, because I think in post-refresh hook it collided on libc versio
[11:30] <zyga> ondra: can you provide a reproducer?
[11:31] <ondra> zyga testing now, let me make sure I have it all right
[11:31] <zyga> ondra: you can set SNAPD_DEBUG=1 and see what happens
[11:31] <zyga> but I don't know how to set that to see hook output
[11:32] <zyga> ondra: there's a spread test for this feature if you want to see how it behaves
[11:32] <ondra> zyga I get hook error and that indicates there was libc collision, but let me validate clean install first
[11:33] <zyga> https://github.com/snapcore/snapd/blob/master/tests/main/stale-base-snap/task.yaml
[11:33] <cachio> jamesh, hey
[11:34] <cachio> jamesh, yesterday I was working with a test error
[11:34] <cachio> jamon,
[11:34] <cachio> jamesh, the test desktop-portal-filechooser on snapsd
[11:34] <cachio> jamesh, https://paste.ubuntu.com/p/wMhG5vdyzK/
[11:35] <cachio> the test just fails like this on ubuntu 18.04
[11:35] <ondra> zyga OK so install opengrok from stable, then try to refresh to edge
[11:36] <cachio> the error itself is that the snap can't write in the mount dir because of lack of permissions
[11:36] <cachio> jamesh, any idea if this has been already seen/reported?
[11:37] <ondra> zyga stable is with core16, edge it base core18, there is libpthread library which depends on libc version, this is used in post-refresh hook, which fails
[11:37] <cachio> jamesh, in ubuntu 18.10 works fine
[11:37] <ondra> zyga clean install from edge works
[11:37] <ondra> zyga so only difference I can think og
[11:39] <zyga> ondra: perhaps something was still running?
[11:39] <zyga> it's interesting
[11:39] <zyga> that post  refresh hook can run with old libc
[11:39] <zyga> I mean
[11:39] <zyga> it's possible to craft this
[11:39] <zyga> so that it will happen as you described
[11:39] <zyga> perhaps snapd should disallow base changes that don't discard the mount namespace
[11:40] <zyga> pedronis: ^
[11:40] <ondra> zyga so post-refresh runs with previous base?
[11:40] <zyga> ondra: please report this
[11:40] <zyga> ondra: yes, it is possible that this happens
[11:40] <zyga> ondra: when an app is still active
[11:40] <zyga> ondra: or perhaps hooks don't depend on their teremination and  we start post refresh hook while  another hook is running
[11:40] <ondra> zyga doesn't it stop all services before refresh?
[11:40] <zyga> ondra: it does, but there are hooks and apps
[11:41] <zyga> those can run longer
[11:41] <zyga> and there are enduring services
[11:41] <zyga> it warrants a bug
[11:41] <zyga> this is funny actually
[11:41] <ondra> zyga there should not be any app running, so just hooks
[11:41] <zyga> because it means we cannot correctly refresh in some cases
[11:41] <zyga> ondra: perhaps more than one bug  then (base awareness across refresh and hook ordering)
[11:41] <ondra> zyga have you managed to reproduce it?
[11:41] <zyga> no
[11:42] <zyga> I have not tried
[11:42] <zyga> I just explained how it is 100% possible to do that
[11:42] <ondra> zyga right :)
[11:42] <zyga> ondra: curious, what was the base change?
[11:42] <zyga> core -> core18
[11:42] <zyga> or back?
[11:44] <ondra> zyga core->core18
[11:46] <ondra> zyga a bit related, what is correct base in snap.yaml for core16? core16 or core?
[11:51] <zyga> ondra: and is libc incompatible across that hop?
[11:52] <ondra> zyga I'm using custom libpthread which I ship in snap, so it's missing things from libc
[11:54] <zyga> ahh
[11:55] <ondra> zyga this is error I'm getting mkdir: relocation error: /snap/opengrok/34/lib/x86_64-linux-gnu/libpthread.so.0: symbol __tunable_get_val, version GLIBC_PRIVATE not defined in file ld-linux-x86-64.so.2 with link time reference
[11:59] <zyga> ondra: please report a bug, after an errand later today I will write a test that reproduces thiis
[11:59] <zyga> ondra: it's unfortunate because it's a design omission
[12:00] <ondra> zyga sure I will file bug for it
[12:00] <zyga> thank you for another interesting case :-)
[12:01] <ondra> zyga you are welcome :)
[12:11] <pedronis> zyga: we always run at most one hook per single snap
[12:11] <pedronis> at a time
[12:12] <zyga> pedronis: and we consider the hook "done" when the initial process exits
[12:12] <pedronis> zyga: ?
[12:12] <zyga> pedronis: we probably can craft a hook that leaves a background process behind
[12:12] <pedronis> probably
[12:12] <pedronis> is that the case here though?
[12:13] <mup> PR snapd#6562 closed: timings: base API for recording timings in state <Created by stolowski> <Merged by stolowski> <https://github.com/snapcore/snapd/pull/6562>
[12:13] <zyga> no, I don't think so, unless that was accidentally what had occurred
[12:13] <zyga> but regardless of what happened here the issue of running a revision that clearly demands base snap  X with the stale base snap Y needs addressing
[12:13] <pedronis> yes
[12:13] <pedronis> shouldn't stale detection find that out?
[12:14] <pedronis> different base, different device?
[12:14] <zyga> it is different
[12:14] <zyga> I bet it's not changed because the namespace is still used
[12:14] <zyga> howerver here I'd argue that different base snap name should be detected
[12:14] <zyga> but we have no logic to recover base snap name from the mount namespace
[12:15] <zyga> (or if it can be done, in general, reliably, apart from parsing path names)
[12:20] <pedronis> pstolowski: the doc comments in the branch you merged still say Timing a lot
[12:20] <pedronis> instead of Span
[12:20] <pedronis> unless I'm confused
[12:22] <pstolowski> pedronis: uhm you're right, sorry.. i'll address in a followup, this doc will be updated anyway with new helpers
[12:22] <pedronis> pstolowski: yes, fine for a follow up
[12:26] <pedronis> Chipaca: the failure on the deprecate interface PR is real, something going with xgettext-go
[12:26] <pedronis> and the warning
[12:26] <pedronis> panic: unknown type: interfacesDeprecationNotice
[12:27]  * Chipaca kills all tests
[12:27] <Chipaca> pedronis: I'll fix
[12:27] <Chipaca> after lunch tho
[12:27] <pedronis> :)
[12:29] <Chipaca> niemeyer: https://pbs.twimg.com/media/D1dQcvqXcAAaQKs?format=jpg&name=large
[12:30] <Chipaca> since you were asking about this yesterday
[12:30] <cachio> mvo, is it ok to move 2.37.4 to stable right?
[12:30] <niemeyer> Chipaca: There we go.. :)
[12:30] <mborzecki> anyone up for a review on #6485 ?
[12:30] <mup> PR #6485: interfaces/seccomp: regenerate changed profiles only <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6485>
[12:41] <pedronis> mborzecki: it has grown quite a lot
[12:41] <zyga> mborzecki: I can enqueue that for later today
[12:42] <zyga> I need to leave in ~15 minutes
[12:42] <mborzecki> thanks!
[12:42] <zyga> mborzecki: enqueued :)
[12:44] <pedronis> mborzecki: it would be easier to review if the new package added was its own PR
[12:45] <mborzecki> pedronis: i can split it
[12:45] <pedronis> mborzecki: if it's not too much painful, I would prefer, it would go faster on the review side that way
[12:46]  * zyga goes to the doctor 
[12:50] <pedronis> cachio: let's chat about 2.37.4 in the standup
[12:51] <cachio> pedronis, ok
[13:02] <mborzecki> Chipaca: https://github.com/snapcore/snapd/pull/6556#issuecomment-471990121 could that be because of ; in the message?
[13:02] <mup> PR #6556: cmd/snap: hide 'interfaces' command, show deprecation notice <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6556>
[13:02] <Chipaca> mborzecki: no, we use ; elsewhere
[13:02] <Chipaca> mborzecki: it's probably the i18n.G() of const
[13:03] <Chipaca> checking now
[13:03] <mborzecki> off to pick up the kids
[13:04] <Chipaca> how
[13:04] <Chipaca> how does update-pot work any more
[13:04] <dot-tobias> ogra / mvo: I'm trying to rebase my fork of ogra's kiosk gadget snap (https://github.com/ogra1/pi-kiosk-gadget) onto the core18-related changes of the “universal Pi gadget snap” (https://github.com/snapcore/pi-gadget). Building the core18 gadget works fine, but my Pi 3B does not boot at all … green LED flickers shortly and then dies, no HDMI output whatsoever. I tried to selectively apply changes from both repos from the point where
[13:04] <dot-tobias>  they diverted (commit 013c2931230a4b4de419080f8499fca27c51727d), but no success. Do you know if the current revision of the universal Pi gadget works?
[13:04] <Chipaca> 'go install' now installs binaries in ~/bin/
[13:04] <Chipaca> :-|
[13:04] <ogra> dot-tobias, got a link to your current code ?
[13:05] <Chipaca> oh wait, no, i have GOBIN set for some reason
[13:05] <Chipaca> hrmm
[13:05] <ogra> arent you in england ? shouldnt that be GOWASTEBASKET instead ? :P
[13:06] <Chipaca> ogra: https://en.oxforddictionaries.com/definition/bin
[13:07] <Chipaca> ogra: 'rubbish bin' is british (vs 'trash')
[13:07] <ogra> oops
[13:07] <Chipaca> ogra: bloody europeans, coming over here, stealing our language
[13:08] <ogra> lol
[13:11] <dot-tobias> ogra: https://gitlab.com/glancr/gadget-snap-pi-kiosk/ -> master builds fine with LXD container without base (though the attempt to feed mir-kiosk the whole config file does not validate in ubuntu-image); branch core18-WIP has the core18 changes in snapcraft.yaml: https://gitlab.com/glancr/gadget-snap-pi-kiosk/commit/d91cc2386af0da24a8942e52053469f4b920ec7b (other files can be ignored)
[13:13] <ogra> dot-tobias, the linux-modules package contains dtb files ?!?
[13:14] <ogra> (i highly doubt that)
[13:14] <ogra> is that a change made by you or did you copy it from the universal branch ?
[13:15] <dot-tobias> ogra: Got that from the snapcore/pi-gadget upstream, see https://github.com/snapcore/pi-gadget/blame/988ee427691177de5cee84370b096be3c6a07951/snapcraft.yaml#L89
[13:16] <dot-tobias> but that would indeed explain why it's not booting … (I have close to zero knowledge about this stuff, so apologies and thank you for beginner explanations  in advance 😊 )
[13:18] <ogra> dot-tobias, ok, i checked, devicetrees are in that package, so this is fine
[13:20] <ogra> o dont see anything else significant that could prevent booting apart from the fact that you use the distro compiler, but that should be v7 or newer anyway in bionic
[13:20] <ogra> s/o/i/
[13:21] <ogra> how does your prime folder look like after build ?
[13:21] <ogra> you should fine the RPi bootloader blobs in there as well as uboot2.img and uboot3.img
[13:21] <dot-tobias> re: compiler: yeah, that's what I gathered from https://github.com/snapcore/pi-gadget/pull/9
[13:21] <mup> PR pi-gadget#9: RFC: use ubuntu cross gcc to build uboot <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/pi-gadget/pull/9>
[13:21] <dot-tobias> let me check
[13:22] <ogra> actually in prime/boot-assets/
[13:23] <ogra> $ ls ../../pi3/pi3-gadget/prime/boot-assets/
[13:23] <ogra> bcm2709-rpi-2-b.dtb  bcm2710-rpi-cm3.dtb  cmdline.txt  COPYING.linux  fixup.dat     fixup_x.dat       overlays     start_cd.elf  start.elf    uboot.bin
[13:23] <ogra> bcm2710-rpi-3-b.dtb  bootcode.bin         config.txt   fixup_cd.dat   fixup_db.dat  LICENCE.broadcom  psplash.img  start_db.elf  start_x.elf
[13:23] <ogra> thast what you want to see
[13:23] <ogra> *that's
[13:23] <dot-tobias> Ok, thanks. Rebuilding the snap just to be sure, BRB
[13:26] <dot-tobias> ogra: Side-note – Do you happen to know how I can feed a multiline file to a “defaults” entry in gadget.yaml? I'm trying to accomplish setting mir-kiosk's display config file through the device hook (see https://forum.snapcraft.io/t/prepare-device-hook-questions/9626/5?u=tobias , current attempt at https://gitlab.com/glancr/gadget-snap-pi-kiosk/blob/master/gadget.yaml#L12)
[13:27] <Chipaca> dot-tobias: > will mangle things for you
[13:27] <Chipaca> dot-tobias: also the quotes are spurious
[13:27] <Chipaca> dot-tobias: paste your yaml into http://yaml-online-parser.appspot.com to see what works :-)
[13:29] <Chipaca> dot-tobias: note these are comments about the yaml itself, I don't know if what you're trying to do should work :-)
[13:29] <ogra> i'm also not sure if the mir team has even tested this
[13:29] <ogra> probably ask in #mir-server
[13:30] <dot-tobias> I tried with a pipe before and the parser complained while building the snap, but maybe I did something else wrong. The online parser shows valid JSON with \n linebreaks at least, will test with ubuntu-image 😊 Yeah, but having a valid yaml result is at least one step closer to the goal 😄 Thank you!
[13:32] <mvo> cachio: 2.37.4 stable> yes - I see no reason why not
[13:33] <dot-tobias> ogra: Snap build fore core18 version is complete, here's the boot-assets folder:
[13:34] <dot-tobias> ➜ ls boot-assets/
[13:34] <dot-tobias> bcm2708-rpi-0-w.dtb     bcm2709-rpi-2-b.dtb       bcm2835-rpi-a.dtb       bcm2835-rpi-b-rev2.dtb  bcm2837-rpi-3-b.dtb  COPYING.linux  fixup_x.dat       start_cd.elf  uboot2.bin
[13:34] <dot-tobias> bcm2708-rpi-b.dtb       bcm2710-rpi-3-b.dtb       bcm2835-rpi-a-plus.dtb  bcm2835-rpi-zero.dtb    bootcode.bin         fixup_cd.dat   LICENCE.broadcom  start_db.elf  uboot3.bin
[13:34] <dot-tobias> bcm2708-rpi-b-plus.dtb  bcm2710-rpi-3-b-plus.dtb  bcm2835-rpi-b.dtb       bcm2835-rpi-zero-w.dtb  cmdline.txt          fixup.dat      overlays/         start.elf
[13:34] <dot-tobias> bcm2708-rpi-cm.dtb      bcm2710-rpi-cm3.dtb       bcm2835-rpi-b-plus.dtb  bcm2836-rpi-2-b.dtb     config.txt           fixup_db.dat   psplash.img       start_x.elf
[13:34] <ogra> dot-tobias, you are missing the files from the config dir (config.txt and cmdline.txt)
[13:34] <ogra> oh, no, i'm just blind
[13:35] <zyga> mvo: standup update: I’m working on splitting the big refactoring PR; going well and I have several patches that can be proposed but I’m waiting till the whole complete “split” branch is functioning. Apart from that I briefly interacted and tested MX Linux where there is some desire to run snapd. I will try one more time with init system set to systemd. I plan to re-review Maciej’s seccomp optimization PR.
[13:36] <mvo> zyga:
[13:36] <zyga> And as I shared I’m at a doctor undergoing small surgery (all is good, no limbs lost)
[13:36] <dot-tobias> ogra: I don't see uboot{3}.img however, or did you mean uboot.bin?
[13:36] <mvo> zyga: uh get well!
[13:36] <zyga> No no, all is good, nothing to worry about
[13:36] <ogra> dot-tobias, yeah
[13:37] <ogra> if this same stuff ends up properly in the system-boot partition of your image i dont really know what to say ... this should definitely boot
[13:37] <ogra> do you see these files in the toplevel of the boot partition if you plug the SD into your desktop ?
[13:40] <mvo> Chipaca: "there are files left in the git tree after the tests:" ?? cmd/snap-seccomp/... you talked aobut this before, didn't you?
[13:41] <Chipaca> mvo: where are you getting this?
[13:41] <mvo> Chipaca: at the end of a unit test spread run
[13:41] <mvo> Chipaca: I thought I saw you talking about something similar earlier
[13:41] <mvo> Chipaca: did I misremember?
[13:41] <Chipaca> mvo: not today, not for ages
[13:42] <mvo> Chipaca: ok, then please ignore me
[13:42] <Chipaca> mvo: 'git status' should list things
[13:42] <mvo> Chipaca: yeah, it happend on spread
[13:42] <mvo> Chipaca: maybe it was a hickup, no worries
[13:43] <zyga> pedronis: about that branch from this morning. I can split it exactly where it was approved before
[13:43] <ondra> zyga so I can only reproduce it on one machine, on other machines when I do clean install and then refresh it just works
[13:43] <zyga> ondra: very interesting
[13:43] <zyga> Is one machine slower?
[13:43] <ondra> zyga very likely much slower
[13:43] <zyga> pedronis: I didn’t mean to introduce bumps into the review process
[13:44] <ondra> zyga I can try to repro on other slow machine
[13:44] <zyga> ondra: can you please tell me more about the hooks in play
[13:44] <zyga> What kind of hooks are used
[13:44] <zyga> And how are the hooks implemented
[13:45] <ondra> zyga https://github.com/kubiko/opengrok-snap/blob/master/snap/hooks/install
[13:45] <ondra> zyga install and post-refresh are same
[13:45] <zyga> ondra: it’s hard for me to look now (I only have my phone and cannot move)
[13:45] <zyga> So please walk me through this
[13:46] <ondra> zyga nothing special there, couple of cp and mkdir and then snapctl get and snapctl set
[13:46] <ondra> nothing else
[13:46] <zyga> Ah
[13:46] <zyga> Hmmm
[13:46] <zyga> Anything using &?
[13:46] <zyga> (Shell job control and stuff)
[13:47] <ondra> zyga nope
[13:47] <ondra> zyga all running in straight line
[13:47] <zyga> So there are install and post-refresh only, right?
[13:47] <ondra> zyga also error is from first line of that hook
[13:47] <zyga> Are there any services?
[13:47] <zyga> Perhaps we don’t wait enough for service shutdown
[13:48] <zyga> We now kill just the main process, right?
[13:48] <ondra> zyga there is configure which does nothing
[13:48] <zyga> Inside services
[13:48] <ondra> zyga yes there is service
[13:48] <zyga> (Or am I mistaken?)
[13:48] <ondra> zyga I have stopped service with systemctl and it still happened
[13:48] <zyga> Aha
[13:49] <zyga> That is very useful data
[13:49] <zyga> Can you reproduce it on the slow machine if you remove the service from snap yaml?
[13:49] <ondra> zyga there is one time activated service and tomcat, I stopped both and same thing
[13:50] <zyga> I wonder if you can reduce this to just the hooks
[13:50] <zyga> Or if services play a role
[13:55] <ondra> zyga but if I remove service it will be new revision, so would that make difference as service from new snap should not be running before post-refresh hook, or am I missing sometning?
[13:55] <ondra> zyga so I cannot repro on another slow machine with clean istall
[13:58] <ondra> zyga BTW unrelated bug: ERROR cannot discard snap namespace "opengrok", will retry in 3 mins: cannot discard preserved namespace of snap "opengrok": cannot unlink opengrok.mnt: Device or resource busy
[13:58] <ondra> zyga not able to remove snap which was installed with try
[13:58] <ondra> zyga Remove security profile for snap "opengrok" (x4) (cannot find installed snap "opengrok" at revision x4: missing file /snap/opengrok/x4/meta/snap.yaml)
[13:59] <ondra> zyga but /snap/opengrok is empty dir, so something out of sync
[14:01] <jamesh> cachio: was out an event earlier.  I haven't seen that error before.  It looks like it is coming from the xdg-document-portal FUSE file system though
[14:05] <degville> mvo: I'm having trouble connecting to the stand up! I'll keep trying (I'm in London)
[14:05] <ondra> zyga removing service made no difference
[14:06] <jamesh> cachio: I suspect the ENOSYS is this one: https://github.com/flatpak/xdg-desktop-portal/blob/master/document-portal/document-portal-fuse.c#L730 -- but I'm not sure why the test would trigger that code path: the test should be running with a clean document portal DB
[14:06] <dot-tobias> ogra: So I rebuilt the core18 gadget from scratch, added “base: 18” to my model assertion and dd'd the image to an SD card. All files show up in the root folder, but there's an additional folder “pi2-kernel_83.snap” which I didn't notice before. Contains additional dtbs. And unfortunately, the Pi still won't boot 😞 HDMI output turns the screen on, but nothing visible and green LED doesn't blink once. IIRC that means “no no” …
[14:07] <zyga> ondra: excellent
[14:07] <mup> PR snapd#6556 closed: cmd/snap: hide 'interfaces' command, show deprecation notice <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/6556>
[14:07] <zyga> ondra: please report the try issue as well
[14:07] <ondra> zyga but as I said, this was service in snap to be installed, so it'd not affect already installed snap
[14:08] <zyga> I’m happy that the hook is sufficient to reproduce
[14:08] <zyga> Ah
[14:08] <zyga> Did you try removing the service from both revisions?
[14:08] <ogra> dot-tobias, anything on serial (i forgot if u-boot actually prints to the display or serial only)
[14:09] <dot-tobias> ogra: Bummed to say that I don't really know (or even have the necessary hardware?) how to do that. Will read up a bit on that and get back to you. Thank you!
[14:10] <cachio> jamesh, I suppose it is something not being cleaned up corrently in that case
[14:10] <ogra> dot-tobias, https://www.amazon.de/SainSmart-USB-TTL-Console-Raspberry/dp/B00KVUSI30 you want something like this
[14:11] <cachio> jamesh, it is weird because the package is installed/uninstall as part of the test
[14:11] <jamesh> cachio: I used to have some paranoid clean up in the prepare for these tests, but was asked to remove it and trust other tests to leave things in a pristine state
[14:11] <mvo> degville: thanks for letting me know
[14:11]  * dot-tobias tips hat to ogra
[14:12] <jamesh> cachio: I suspect it is some user-level configuration rather than something that would be affected by package install/uninstall
[14:14] <cachio> jamesh, it is weird that I can't reproduce that error on ubuntu 18.10
[14:14] <cachio> with the same code
[14:15] <jamesh> cachio: yeah.  We've got 1.0.3 on both 18.04 and 18.10
[14:16] <jamesh> cachio: scratch that.  On bionic 1.0.3 is still waiting in proposed
[14:17] <cachio> jamesh, mmm, I'll try to test with 1.0.3 on bionic to see it that fix the issue
[14:17] <cachio> jamesh, I'll let you know the results}
[14:18] <jamesh> cachio: we definitely had problems when Cosmic had xdg-desktop-portal 1.0.2, which were fixed by 1.0.3.  Bionic seemed fine with the older 0.11, but we've been trying to get 1.0.3 rolled out to all the old releases.
[14:19] <jamesh> It wouldn't surprise me if there is some other corner case in the older version that I didn't hit when testing
[14:23] <ondra> zyga and when I return 0 as result of the hook, then snapd is tricked and refresh works
[14:23] <zyga> ondra: well, that's expected
[14:23] <ondra> zyga though of course there are all the errors there
[14:23] <zyga> the beauty(*) of  shell
[14:23] <ondra> zyga yep
[14:24] <ondra> zyga so I have only one machine I'm able to reproduce this one
[14:26] <ondra> zyga ha, was did you say was trick to check what core I'm running against if I can snap run --shell
[14:26] <zyga> ondra: it's easier to use nsenter
[14:26] <zyga> ondra: nsenter -m/run/snapd/ns/foo.mnt
[14:26] <zyga> then see what / is
[14:27] <ondra> zyga nsenter: neither filename nor target pid supplied for ns/mnt
[14:27] <zyga> -marg, not -m ar
[14:28] <ondra> zyga because even now installed, it still seems to run with wrong base
[14:28] <zyga> nesenter cli parsing is very strict
[14:29] <ondra> zyga nsenter: reassociate to namespace 'ns/mnt' failed: Invalid argument
[14:29] <zyga> what is the command you ran exactly?
[14:29] <ondra> zyga when running nsenter -m/run/snapd/ns/opengrok.mnt
[14:29] <zyga> sudo?
[14:29] <zyga> actually
[14:29] <zyga> no
[14:29] <ondra> zyga same with sudo
[14:29] <zyga> EINVAL means it's not mounted anymore
[14:29] <zyga> that's ... odd
[14:29] <zyga> we used to unmount but keep the file
[14:29] <zyga> but recently we always unlink it for clarity
[14:31] <zyga> ondra: note that snap-confine may not save a new mount namespace if construction fails
[14:31] <zyga> ondra: and may discard a mount namespace when it is stale
[14:32] <zyga> ondra: but that should not happen here (construction should not fail)
[14:33] <ondra> zyga so I still get into that namespace despite the error
[14:33] <ondra> zyga and it's indeed using core16 base
[14:34] <zyga> ondra: uh?
[14:34] <zyga> the erorr is ... od
[14:34] <zyga> either setns worked or it did not work
[14:35] <zyga> ondra: I'm at risk of getting lost here, can you please collect this information in the bug report
[14:35] <ondra> zyga https://paste.ubuntu.com/p/99Pz3wQ79P/
[14:35] <zyga> ondra: I think the bug may be separate from the core -> core18 bug
[14:35] <zyga> ondra: that is, there is something wonky with hook dependency (CC pstolowski)
[14:35] <ondra> zyga on machine where it works I get: https://paste.ubuntu.com/p/jhCTxk7mwH/
[14:36] <zyga> ondra: that nsenter looks ok
[14:36] <zyga> no EINVAL anymore
[14:36] <ondra> zyga see the difference in library version
[14:36] <zyga> 2.23?
[14:36] <ondra> zyga yep, that is wrong glibc
[14:36] <ondra> zyga should be 2.27
[14:36] <zyga> ondra: please refresh my memory,  core16 has 2.23 and core18 has 2.27?
[14:37] <zyga> ondra: can you run findmnt and tell me what / is please?
[14:37] <ondra> zyga correct
[14:37] <zyga> ondra: so, one observation, it would be neat to always have logging in snap-confine and log that to syslog or something
[14:37] <zyga> ondra: this way we would know what happened for sure
[14:37] <ondra> zyga /                                              /dev/loop0
[14:37] <zyga> losetup  -l?
[14:38] <zyga> oh
[14:38] <zyga> one sec
[14:38] <zyga> so
[14:38] <ondra> zyga /dev/loop0         0      0         1  1 /var/lib/snapd/snaps/core_6130.snap (deleted)
[14:38] <zyga> SNAP_CONFINE_DEBUG=yes snap run --shell ...
[14:38] <zyga> oh?
[14:38] <zyga> deleted is certainly interesting
[14:38] <zyga> (the plot thickens)
[14:38] <zyga> ondra: that will now tell you what snap-confine detected
[14:39] <zyga> it should tell you that the mount namespace is stale but occupied
[14:39] <ondra> zyga on other machine it looks correct
[14:39] <zyga> is that snap really deleted on the fs?
[14:39] <ondra> zyga already seeing loop0 indicates something is hanged :)
[14:39] <zyga> deleted is very nasty :/
[14:40] <zyga> wait, this is core!
[14:40] <zyga> right?
[14:40] <zyga> ooooh
[14:40] <zyga> man I know
[14:40] <zyga> so
[14:40] <zyga> we have a bug
[14:40]  * zyga slows down to think 
[14:40] <ondra> zyga /var/lib/snapd/snaps/core_6130.snap does not exist at all
[14:40] <zyga> ondra: specifically this is ubuntu-core 16 right?
[14:40] <zyga> a core system
[14:40] <ondra> zyga classic
[14:41] <zyga> aha
[14:41] <zyga> in that case it's back to regularly interestngi
[14:41] <ondra> zyga 18.04
[14:41] <zyga> I suspected there  are bugs in core->core18 sans reboot refreshes
[14:41] <zyga> but let's think
[14:41] <zyga> do you have snap changes that indicate something happening to revision 6130?
[14:42] <zyga> I suspect it was just garbage collected
[14:42] <ondra> zyga yeah nothing in changes
[14:42] <zyga> ok
[14:43] <zyga> SNAP_CONFINE_DEBUG=yes snap run --shell ...
[14:43] <zyga> what does that say?
[14:43] <zyga> (use the  real snap name there)
[14:44] <zyga> ondra: (suspense music plays)
[14:45] <ondra> zyga https://paste.ubuntu.com/p/7w6nRX4Y4J/
[14:45] <zyga> DEBUG: sending information about the state of the mount namespace (discard)
[14:45] <cachio> jamesh, same error with 1.0.3
[14:45] <zyga> DEBUG: the mount namespace is stale and should be discarded
[14:45] <cachio> jamesh, I have a debug session open, do you want to take a look?
[14:45] <zyga> when you do this you must  keep something in there running
[14:46] <zyga> now you should see core18 /
[14:46] <zyga> ondra: as I said, when the error happens there are still some processes lingering
[14:46] <zyga> ondra: so snap-confine knows it's supposed to discard but chooses not to
[14:47] <zyga> ondra: can you confirm that / is now core18?
[14:47] <ondra> zyga no / is core16 still in that name space
[14:48] <zyga> ondra: DEBUG: found process 6540 belonging to user 0
[14:48] <zyga> I missed that
[14:48] <zyga> what is that process
[14:49] <ondra> zyga funny ps -aux does not show anything running from old revision
[14:49] <zyga> ondra: coin on the edge
[14:49] <zyga> ondra: it's a thread
[14:49] <zyga> can you find that in /proc please?
[14:49] <zyga> ondra: a bit of go thread is running and it inhabits that namespace somehow (wild guess)
[14:51] <ondra> zyga I assume reboot will fix this as whatever is hanging will die, that stale core revision seems pretty old
[14:51] <zyga> ondra: don't reboot
[14:51] <zyga> ondra: find out what process 6540 is
[14:51] <zyga> what's /proc/6540/exe ?
[14:52] <zyga> ondra: inspect it from the outside please)
[14:53] <ondra> zyga I refreshed back to base core18 snap, debug log was from revision running against core16 and that was functioning
[14:53] <zyga> ondra: can you reproduce this enough to see the dangling process and find out what it was please?
[14:54] <zyga> ondra: you can look at /sys/fs/cgroup/freezer/snap.$SNAP_NAME/cgroup.procs for the list of inhabitants
[14:54] <zyga> ondra: (to avoid using snap-confine)
[14:54] <ondra> zyga now same on broken revision, process id I see from snap confine is ssh-agent
[14:55] <zyga> oh
[14:55] <zyga> how did that get there?
[14:55] <pedronis> Chipaca: thanks for the review
[14:55] <ondra> zyga snap is running ssh-agent
[14:55] <zyga> ondra: so there's a ssh-agent running in that mount namespace
[14:55] <zyga> ondra: and that is a "service" (but not)
[14:55] <zyga> ondra: who starts it?
[14:55] <ondra> zyga it's forked
[14:55] <zyga> is that a service / app in the snap>?
[14:56] <ondra> zyga service will check if it's running and starts it
[14:56] <zyga> the service that is a part of the snap?
[14:56] <ondra> zyga I suppose I can add it as service
[14:56] <zyga> ondra: if you forgete refreshes for a second
[14:56] <zyga> if you stop the service who started i
[14:56] <zyga> does it stop ssh-agent?
[14:56] <zyga> or is that a process that is not stopped across restart (effectively)
[14:56] <ondra> zyga no it will not stop it when you stop service
[14:56] <zyga> ondra: so that's that
[14:56] <zyga> ondra: it's a feature
[14:57] <zyga> ondra: we don't discard the mount namespace when there are apps or hooks running that would then observe old/new fs together with other processes in that snap
[14:57] <ondra> zyga shall I kill the process and test refresh?
[14:57] <zyga> ondra: refresh-app-awareness would refuse to refresh your snap
[14:57] <zyga> ondra: please
[14:57] <zyga> ondra: we do have a bug
[14:57] <zyga> ondra: about core / core18
[14:57] <ondra> zyga not yet
[14:57] <zyga> or in general where base snap name changes
[14:58] <zyga> that should behave in a different (yet undesigned way)
[14:58] <mup> PR # closed: snapd#5644, snapd#5822, snapd#5915, snapd#6098, snapd#6108, snapd#6177, snapd#6238, snapd#6258, snapd#6270, snapd#6325, snapd#6327, snapd#6329, snapd#6341, snapd#6347, snapd#6360, snapd#6367, snapd#6401, snapd#6404, snapd#6410, snapd#6418, snapd#6436, snapd#6485, snapd#6491,
[14:58] <mup> snapd#6502, snapd#6541, snapd#6553, snapd#6559, snapd#6564, snapd#6575, snapd#6577, snapd#6583, snapd#6585, snapd#6588, snapd#6589
[14:59] <ondra> zyga lol different error cannot unlink opengrok.mnt: Device or resource busy
[14:59] <zyga> ondra: uname -r?
[14:59] <zyga> ondra: that's a known kernel behavior on old (3.10) kernels
[14:59] <mup> PR # opened: snapd#5644, snapd#5822, snapd#5915, snapd#6098, snapd#6108, snapd#6177, snapd#6238, snapd#6258, snapd#6270, snapd#6325, snapd#6327, snapd#6329, snapd#6341, snapd#6347, snapd#6360, snapd#6367, snapd#6401, snapd#6404, snapd#6410, snapd#6418, snapd#6436, snapd#6485, snapd#6491,
[14:59] <mup> snapd#6502, snapd#6541, snapd#6553, snapd#6559, snapd#6564, snapd#6575, snapd#6577, snapd#6583, snapd#6585, snapd#6588, snapd#6589
[15:00] <ondra> zyga I'm on 4.15
[15:00] <zyga> ondra: that's new then :)
[15:00] <zyga> ondra: new and not fun
[15:00] <zyga> ondra: can you run SNAPD_DEBUG=1  snap-discard-ns
[15:00] <zyga> with the snap name
[15:00] <zyga> well
[15:00] <zyga> specifically
[15:00] <zyga> unlink is expected to fail
[15:01] <zyga> but we should have unmounted it before
[15:01] <zyga> run stat on that file
[15:01] <zyga> and stat -f please
[15:01] <zyga> perhaps before you discard it
[15:01] <ondra> zyga but since I killed that process I'm now running core18
[15:01] <zyga> to see what's currently there
[15:03] <ondra> zyga nothing with stat -f
[15:03] <zyga> nothing as in?
[15:06] <ondra> zyga https://paste.ubuntu.com/p/7DmG45TYnz/
[15:06] <zyga>     ID: 0        Namelen: 255     Type: nsfs
[15:07] <zyga> this is relevant
[15:07] <zyga> it's a mounted namespace
[15:07] <zyga> so it cannot be unlinked
[15:07] <zyga> the question is: why did  snap-discard-ns not unmount it>
[15:07] <ondra> zyga so I will redo snap to run ssh-agent as service, I guess this should then fix it
[15:07] <zyga> which version of snapd are you on?
[15:08] <ondra> zyga 2.37.2
[15:09] <zyga> I forgot when snap-confine started  using snap-discard-ns but I suspect it's been before than that
[15:09] <zyga> what happens if you want to discard the mount namespace and use SNAPD_DEBUG=1 SNAP_CONFINE_DEBUG=yes snap-discard-ns?
[15:10] <zyga> (note: make sure to use the right binary, reexec wise)
[15:11] <ondra> zyga where do I get snap-discard-ns
[15:11] <zyga> ondra: it's right next to snap-confine
[15:11] <zyga> either in /usr/lib/snapd or in the core snap in the same path
[15:12] <ondra> zyga https://paste.ubuntu.com/p/fGPY9WNRrp/
[15:12] <zyga> ondra: what's the version of snapd on the hostt?
[15:12] <zyga> woah
[15:12] <zyga> that's neat
[15:12] <mup> PR snapd#6590 closed: daemon/api: filter connections with hotplug-gone=true <Created by stolowski> <https://github.com/snapcore/snapd/pull/6590>
[15:12] <zyga> so it unmounted it, supposedly without erorrs
[15:12] <ondra> zyga 2.37.2
[15:13] <zyga> and then it ... failed to unlink it because it is still mounted
[15:13] <zyga> can you re-run
[15:13] <zyga> with strace  please?
[15:13] <zyga> (ditch the debug options)
[15:13] <zyga> and report this as a new bug
[15:13] <zyga> it's a weird kernel behavior, unless we're missing something
[15:15]  * zyga cannot wait for the strace
[15:15] <ondra> zyga https://paste.ubuntu.com/p/SkBMzTPpHX/
[15:15] <ondra> zyga so you want bug with which exactly? :)
[15:15] <ondra> zyga I will paste those longs in, jut to know how to call it
[15:15] <zyga> woah
[15:15] <zyga> I see
[15:15] <zyga> fstatfs(5, {f_type=TMPFS_MAGIC, f_bsize=4096, f_blocks=255810, f_bfree=255557, f_bavail=255557, f_files=1279048, f_ffree=1278375, f_fsid={val=[0, 0]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_NOSUID|ST_NOEXEC|ST_RELATIME}) = 0
[15:16] <zyga> so...
[15:16] <zyga> findmnt?
[15:16] <zyga> it looks like a kernel bug
[15:17] <zyga> we look at the .mnt file, see it's a TMPFS (not nsfs), try to unlink it and hit a EBUSY error
[15:17] <zyga> can you repeat the stat -f call on the file agani
[15:17] <zyga> you reported it was nsfs before
[15:17] <zyga> is it still?
[15:18] <mup> PR core18#119 opened: Add a default locale <Created by sil2100> <https://github.com/snapcore/core18/pull/119>
[15:20] <zyga> ondra: ?
[15:20]  * zyga is dying from suspense
[15:20] <mup> PR snapd#6590 opened: daemon/api: filter connections with hotplug-gone=true <Created by stolowski> <https://github.com/snapcore/snapd/pull/6590>
[15:22] <ondra> zyga yeah now it's tmpfs
[15:22] <zyga> ondra: findmnt
[15:22] <zyga> is it a bind mount?
[15:23] <zyga> or /proc/self/mountinfo if you fancy that instead
[15:23] <ondra> zyga findmnt in the namespace?
[15:23] <zyga> outside, all of discard should be done outside
[15:23] <zyga> inside that _IS_ expected to happen
[15:23] <zyga> (AFAIK)
[15:23] <ondra> zyga https://paste.ubuntu.com/p/4S3VJyJJCv/
[15:24] <zyga> │   └─/run/snapd/ns/opengrok.mnt      nsfs[mnt:[4026532246]] nsfs       rw
[15:24] <zyga> outside it is mounted
[15:24] <zyga> were you running discard from within the mount namespace by any chance?
[15:24] <ondra> no from outside
[15:24] <zyga> and that strace where it showed to be a tmpfs, that was outside as well?
[15:25] <ondra> yes
[15:25] <zyga> ondra: please report this
[15:25] <ondra> all from outside
[15:25] <zyga> ondra: one bug for base core -> core18 design change on running processes
[15:26] <zyga> ondra: one bug on discard that we just talked about
[15:26] <zyga> ondra: I think there are no other things to report at this time, do you agree?
[15:26] <ondra> zyga agree
[15:26] <ondra> zyga I will do bugs tonight, now meetings
[15:26] <zyga> ondra: *thank you*
[15:27] <zyga> this is super valuable
[15:27]  * zyga hugs ondra 
[15:27] <pstolowski> zyga: sorry, i saw you ping but haven't followed the long discussion you had with ondra, is there anything i should look at re hooks?
[15:27] <zyga> pstolowski: no, it's all clear now
[15:27] <zyga> there was a hidden app running
[15:27] <zyga> all is good
[15:28]  * zyga heads home
[15:28] <zyga> afk for some time
[15:28] <pstolowski> ok
[15:29] <pedronis> zyga: I'm not sure that PR needs to be resplit but I made some comments there
[15:29] <ondra> zyga you are welcome and thank you!  this was intense :D
[15:36] <cachio> mvo, we have core 2.37.4 on stable now
[15:36] <cachio> pedronis, ~
[15:36] <pedronis> thx
[15:37] <mvo> cachio: thank you!
[15:39]  * cachio lunch
[15:48] <mup> PR core18#120 opened: Backport wpa_supplicant.service.d/snap.conf from core <Created by sil2100> <https://github.com/snapcore/core18/pull/120>
[15:50] <mup> PR core18#119 closed: Add a default locale <Created by sil2100> <Merged by sil2100> <https://github.com/snapcore/core18/pull/119>
[15:50] <mvo> sil2100: thanks!
[15:51] <sil2100> mvo: yw! The locale one was very trivial so I just ekhm, self-merged it
[15:51] <sil2100> We'll make sure it goes out with the nearest snap
[15:52] <mvo> xnox: do you mind if I do a systemd SRU? with the timedatectl fix and the new 1819728 fix?
[15:52] <mvo> sil2100: thank you!
[15:52] <xnox> mvo, go go go =)
[15:53] <xnox> mvo, but also if mark files a proposed regression bug report, it will be on you to revert ;-)
[15:53] <mvo> xnox: cool, thank you
[15:53] <mvo> xnox: sure, its perfectly safe ;)
[15:53] <mvo> xnox: already in the upstream systemd etc
[15:54] <xnox> yeah yeah, ddstreet thought so too =)
[15:54] <mvo> (since 1y almost)
[15:54] <xnox> yet got caught up by missing pieces of things =)))))
[15:54] <xnox> mvo, most of them do go through fine.
[15:55] <mvo> xnox: heh - thats fine, I need to prepare the details and then I will show you the final debdiff
[15:55] <mvo> xnox: so tomorrow
[15:56] <mvo> xnox: still thanks for the green light
[16:06]  * zyga is home
[16:17] <mup> PR snapd#6591 opened: [RFC] managers: basic measurements <Created by stolowski> <https://github.com/snapcore/snapd/pull/6591>
[16:22] <mup> PR snapcraft#2498 closed: python plugin: graceful ret when no packages set <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2498>
[18:36] <jdstrand> pedronis: hey, just for my own prioritizing, when are you shooting for a 2.38 upload to disco?
[18:39] <jdstrand> mvo: hey, maybe that is a question for you: just for my own prioritizing, when are you
[18:39] <jdstrand> shooting for a 2.38 upload to disco?
[18:41] <mup> PR snapcraft#2500 opened: DRAFT: Add remote build <Created by cmatsuoka> <https://github.com/snapcore/snapcraft/pull/2500>
[18:42] <mvo> jdstrand: probably end of this week, why? anything pending?
[18:43] <jdstrand> mvo: nothing for you. I can't upload apparmor 2.13 without 2.38 is all
[18:43] <jdstrand> (2.13 will break snap removes with system where apparmor is enabled)
[18:47] <mvo> jdstrand: ok
[19:04]  * cachio afk
[19:17] <mup> PR snapcraft#2501 opened: nodejs pluging: support for type str bin entries <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2501>
[19:26] <mup> PR snapcraft#2496 closed: many: support the use of build-aux/snap <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2496>
[19:26] <Pharaoh_Atem> I was hoping that 2.38 would have the selinux stuff :(
[20:50] <popey> jdstrand:  is it possible personal-files could grow exec to go with read and write?
[20:50] <popey> jdstrand: i have an app which uses a go library called byteexec which writes go byte arrays as executables in ~/.byteexec, but fails for me because while it can write, it can't exec
[20:55] <jdstrand> popey: it isn't really meant for that and that could easily lead to exec transition conflicts. why can't it write to SNAP_USER_DATA (which is $HOME to the snap)?
[21:00] <popey> jdstrand: no, this doesn't seem to work https://github.com/getlantern/byteexec/blob/master/byteexec.go#L111
[21:01] <jdstrand> popey: patch that or file a bug to have them look at HOME?
[21:03] <jdstrand> popey: alternatively, the snap could use personal-files for ~/.bytecode and then create a symlink from there to SNAP_USER_DATA. it is a hack, but should work
[21:04] <popey> huh, that's interesting
[21:04]  * popey breaks out the wrapper scripts
[21:05] <sergiusens> jdstrand: layouts!
[21:06] <sergiusens> user.Current opens up /etc/passwd to find out the user, so a code patch could work too
[21:07] <sergiusens> jdstrand: if we are going to commit to HOME being SNAP_USER_DATA perhaps it would be a good idea to have the bind mounted /etc/passwd path to that as home
[21:08] <jdstrand> sergiusens: it won't work for the user's home atm
[21:08] <jdstrand> better than bind mounting is likely an nss module that is snap aware
[21:13] <sergiusens> jdstrand: yeah, but the go std lib opens /etc/passwd and reads straight from there (I haven't looked in detail or seen how they solve nis and such)
[21:14] <jdstrand> that a dumb way to do it. the home may not even be in /etc/passwd
[21:14] <jdstrand> maybe they have a fallback that does nss
[21:20] <popey> should $HOME point to the users outside home dir?
[21:20] <popey> because now I'm wondering how my launcher can know what the home dir outside the snap is named
[21:22] <jdstrand> popey: $HOME should be set to SNAP_USER_DATA when the snap starts. you can use 'getent passwd $(id -u) | cut -d : -f 6' to see what the system has to say about it
[21:23] <popey> hah, that's memorable
[22:13] <popey> jdstrand:  ok, moving things and symlinking helped my next issue is that the binary which runs is trying to set the system proxy. I don't see any interfaces which can do that. Is it something I should request?
[22:21] <jdstrand> popey: that is a surprising request for your app but I see no reason why we couldn't have an interface for it. of course, it depends on what it is trying to actually do, but a forum post is good
[22:21] <popey> What should I capture? strace? :)
[22:22] <popey> it's a blob which tries to set the proxy
[22:26] <popey> hmm, looks like it rummages in gsettings. I'll start a thready, thanks
[23:23] <Guest92> hi all. i'm trying to build a snap that depends on mono. the version of mono from apt is too old, so trying to work out how to pull in the latest versions directly
[23:23] <Guest92> something roughly equivalent to => echo 'deb https://download.mono-project.com/repo/ubuntu stable-bionic main' | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
[23:24] <Guest92> from what i've read so far, this doesn't seem to be possible?