[02:33] <mup> PR snapcraft#2066 closed: errors: feature flag error reports <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2066>
[02:36] <mup> PR snapcraft#2069 opened: Reports <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2069>
[04:43] <zyga> jjohansen: artful is also affected. I will give you more results today
[04:44] <jjohansen> zyga: ah good, I was going through the diff and going htf is artful not affected and bionic is :)
[04:48] <zyga> Mainline is not affected or very little
[04:48] <zyga> Loading one profile over and over leaks very very quickly
[04:48] <zyga> Maybe some new table is not released?
[04:49] <zyga> I wrote some tests but went to sleep. I will keep looking today
[05:03] <jjohansen> mainline certainly has a leak
[05:03] <jjohansen> which kernel version for mainline are you not seeing it on?
[05:03] <jjohansen> zyga: ^
[05:05] <zyga> jjohansen: I built mainline from yesterday, I was at e241e3f2bf97
[05:05] <jjohansen> okay, thanks
[05:06] <zyga> jjohansen: when I say there was no leak I mean that loading a profile over and over (unchanged) ran for over 30 minutes with minimal memory bump (probably noise from other programs)
[05:06] <zyga> jjohansen: at most 300MB
[05:06] <zyga> jjohansen: on a affected kernel a few minutes of this would consume all my ram
[05:07] <zyga> jjohansen: I will give you more data soon, sorry, yesterday I just collapsed
[05:08] <zyga> jjohansen: this is the base64-encoded binary profile I was loading in a loop http://paste.ubuntu.com/p/Jfs3RRKcPw/
[05:11] <zyga> jjohansen: on 4.13.16 inserting that 10K times leaks 440MB
[05:11] <zyga> (on amd64)
[05:12] <zyga> jjohansen: perhaps other profiles we tested inside spread+snapd leaked more memory but I wanted to keep using one profile for experiments
[05:12] <mborzecki> morning
[05:13] <zyga> hey mborzecki, good morning
[05:13] <mborzecki> any fires today?
[05:14] <zyga> mborzecki: no, I think all is the same for now
[05:31] <zyga> jjohansen: on xenial kernel the jump is from 946 all the way up to ~2GB
[05:31] <zyga> (this time using distribution kernel, not my build of the corresponding tag)
[05:31] <zyga> jjohansen: xenial kernel is misleading, this was 4.13.0-37
[05:35] <zyga> jjohansen: 4.4.0-119 on xenial is also affected but very slightly so, same profile, same count, 626MB->660MB
[05:35] <zyga> jjohansen: I'll test intermediate kernels now
[05:40] <zyga> jjohansen: 4.8.0-58 goes from 699M -> 773M
[05:42] <zyga> jjohansen: 4.10.0-42 goes from 698M -> 723M
[05:42] <zyga> jjohansen: so that feels like noise so far
[05:42] <zyga> the real jump is in 4.13
[05:42] <zyga> where we drop significant amount of memory
[05:48] <zyga> jjohansen: 4.15.0-13 goes from 640M to 1.61G
[05:50] <zyga> jjohansen: so for all practical purpose the diff between 4.10 and 4.13 has introduced the major part of the leak
[05:51] <zyga> jjohansen: but note that even on 4.4 there's some memory going somewhere, maybe that's just slab growing
[05:51] <zyga> jjohansen: I'll introduce a variant that does 10M insertions to see if slab stabilizes
[06:12] <mborzecki> zyga: do you know if issue #3 'Memory use on minimal/constrained systems' had any further developments?
[06:12] <zyga> jjohansen: on the xenial 4.4 kernel 10M insertions doesn't seem to actually leak memory, after some initial growth (of non-free memory) slab stabilises at 929M and just stays there
[06:12] <zyga> mborzecki: no, I didn't focus on it
[06:13] <mborzecki> i know we said it's won'tfix for 18.04, but didn't see any messages that would indicate it was further discussed later yesterday
[06:16] <zyga> mborzecki: sorry, I don't know more than that
[06:16] <zyga> mvo: did you end up having the meeting with cloud guys?
[06:54] <zyga> I'll terminate testing 4.4., it's pretty much rock solid
[06:55] <pedronis> mborzecki: hi, some of the things you worked on recently (autostart, timers? ) need to be added here https://forum.snapcraft.io/t/the-snap-format/698 ?
[06:56] <mborzecki> pedronis: thanks, will do
[06:56] <pedronis> mborzecki: as usual we need put then version (2.xx+) where it starts working
[06:58] <mvo> zyga, mborzecki yeah, we had a meeting yesterday. we will not do anything right now, its too risky, but we want to prepare so that we can provide a fix post-release asap
[06:58] <zyga> jjohansen: 4.4 is rock solid, doesn't leak memory over extreme number of insertions, I'm looking at 4.10 now and it also looks good, memory use stops at ~1.03GB after 10s of thousands of insertions
[06:59] <zyga> jjohansen: I'll keep it running for some more time and then try 4.13 where I suspect we really leak memory constantly
[06:59] <zyga> mvo: sounds very godo
[06:59] <zyga> good
[07:03] <mborzecki> pedronis: mvo: one thing about exiting when idle, we don't have snap.refresh.timer anymore to wake us up, but we could schedule a command to run as a on-demand timer using systemd-run
[07:04] <kalikiana> moin moin
[07:07] <mvo> zyga: nice findings on the kernel mem leak front
[07:07] <mvo> mborzecki: indeed
[07:07] <zyga> mvo: i hope we can find the leak soon enough :-)
[07:13] <pstolowski> good morning
[07:14] <pedronis> mborzecki: it depends what's the goal
[07:14] <pedronis> mvo: is the plan to make exit on idle, generalized behavior?
[07:15] <pedronis> mvo: did you discuss just timings or also a bit the goals?
[07:17] <pedronis> mborzecki: is setting configuration with "set system"  landed?
[07:18] <mborzecki> pedronis: yes
[07:18] <pedronis> it's on edge but not 2.32 , right?
[07:18] <mborzecki> but iirc it's in master only
[07:18] <pedronis> ok
[07:19] <pedronis> bit unfortunate, but oh well
[07:20] <mborzecki> hm timer services are in egde too
[07:20] <mborzecki> but not in 2.32.*
[07:20] <pedronis> ah
[07:20]  * pedronis admits to have lost track of things a bit (2.32 being so long lived)
[07:21] <pedronis> mborzecki: anyway that's bit less of an issue
[07:21] <pedronis> the issue with set core vs set system is that it must work before one installs core
[07:21] <mvo> pedronis: gustavo preferes the wake up, do stuff, exit approach over not doing anything at all. but its still a bit undecided so worthwhile to have another meeting to discuss options. I personally still favour the "do as little as possible via units" approach
[07:21] <mvo> pedronis: yeah, 2.32 is the new 2.33 :/ its a bit annoying
[07:22] <pedronis> so basically we need to document set core  and support it
[07:22] <pedronis> for the life of bionic
[07:22] <pedronis> (more or less)
[07:23] <pedronis> mvo: discussion for monday I suppose?
[07:23] <mvo> pedronis: yeah, *maybe* today but I think gustavo is pretty busy today
[07:23] <mborzecki> pedronis: it's 2 small patches, should be easy to cherry-pick in case we want fixes in 2.32
[07:23] <pedronis> it's too late I think
[07:25] <mvo> mborzecki: you can prepare a PR and target it so that *if* we need to rebuild we have it. but I'm with pedronis probably too late
[07:25] <mborzecki> mvo: sure
[07:26] <pedronis> mvo: so I imagine we concluded that it's called 2.32.4 , not 2.33 ?
[07:27] <mvo> pedronis: yes, I had a call with Adam about it, the amount of work to make it 2.33 is just too high at this point
[07:28] <pedronis> I don't think we have promised/enforced minor releases to be small or have no features
[07:28] <pedronis> we try to
[07:29] <pedronis> in theory we have assumes , but seems they stay unused
[07:29] <pedronis> (anyway they are not relevant for the API, it's transparent)
[07:33] <mup> PR snapd#5044 opened: 'system' nickname for 'core' in snap get/set (2.32) <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5044>
[07:37] <pedronis> mvo: I'm going to merge my PRs about doMountSnap,  should I prepare cherry-picks?  or will you later?
[07:40] <mup> PR snapd#5036 closed: overlord/snapstate: allow to get an error from readInfo instead of a broken stub, use it in doMountSnap <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/5036>
[07:45]  * pedronis is clearly not rested enough
[07:56] <Caelum> zyga: I posted on opensuse-packaging, but it's a very low traffic list, will let you know if I get any replies
[07:59] <mborzecki> 5038 failing on travis, works if i run it from host, cleaned the git tree but still
[08:00] <mborzecki> and it's awkward, afaict all services are getting enabled by dh snippets in postinst, but the snapd.wakeup.service is disabled when the test starts
[08:01] <mborzecki> https://media1.giphy.com/media/12NUbkX6p4xOO4/giphy.gif probably
[08:04] <zyga> Caelum: perfect, thanks!
[08:17] <mborzecki> i really have packaging at times
[08:23] <cwayne> if only someone invented a simple way to package stuff!
[08:23] <zyga> slackware?
[08:24] <mborzecki> zyga: could you take another look at https://github.com/snapcore/snapd/pull/4989 later on?
[08:24] <cwayne> lol
[08:24] <mup> PR #4989: tests: add arch to CI <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/4989>
[08:24] <zyga> mborzecki: sure
[08:32] <Chipaca> moin moin
[08:33] <mvo> hey Chipaca
[08:33] <Chipaca> mvo: do you remember why switch didn't have --devmode and etc?
[08:33] <mvo> Chipaca: I think there is no reason, its a nice idea
[08:33] <Chipaca> also what we agreed about --no-devmode
[08:33] <Chipaca> ditto --classic vs --no-classic, etc
[08:33] <mvo> Chipaca: iirc nobody asked that switch would have this capability but I think its a nice idea
[08:33] <Chipaca> mvo: people have asked, we've just not been paying attention =)
[08:33] <mvo> Chipaca: I had this problem too (wanting to swtich from strict to devmode and vice-versa)
[08:34] <mvo> Chipaca: *cough*
[08:34] <mvo> Chipaca: details ;)
[08:34] <mvo> Chipaca: don't destroy my narative
[08:34] <Chipaca> mvo: https://forum.snapcraft.io/t/refresh-into-devmode/4130 and https://forum.snapcraft.io/t/refreshing-snaps-in-devmode/4942
[08:35] <Chipaca> mvo: but also I remember niemeyer had a reason for not having --no-devmode etc, but I don't remember it
[08:35] <Chipaca> and I'm not sure it wasn't due to him confusing go-flags and flags, wrt --<boolean flag>=<boolean>
[08:41] <zyga> mborzecki: https://github.com/snapcore/snapd/pull/4989/files#r181321369
[08:41] <mup> PR #4989: tests: add arch to CI <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/4989>
[08:43] <mvo> Chipaca: sorry, I don't remember why we would not want --no-devmode etc
[08:43] <zyga> mborzecki: after that +1
[08:47] <Chipaca> mvo: how do you get out of devmode?
[08:48] <mvo> Chipaca: I think you need to refresh to a new revision for this right now, no?
[08:52] <Chipaca> mvo: I think so yes
[08:56] <zyga> the memory leak was introduced in one of 144 patches
[08:57] <zyga> I will review them quickly and see if I can automate a test loop
[09:02] <zyga> interestingly this patch is in that list a7c3e901a46ff54c016d040847eda598a9e3e653
[09:02] <mvo> zyga: nice
[09:02] <mvo> zyga: bisect ftw
[09:03] <zyga> it's not the bug yet though
[09:08] <mup> PR snapd#5039 closed: overlord/snapstate: use the readInfo in doMountSnap as a check only, undo if it errors <Blocked> <Created by pedronis> <Closed by pedronis> <https://github.com/snapcore/snapd/pull/5039>
[09:08] <mup> PR snapd#5040 closed: overlord/snapstate: poll for up to 10s if a snap is unexpectedly not mounted in doMountSnap <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/5040>
[09:09] <zyga> jdstrand: I didn't know we already supported loading arbitrary extended data into profiles!
[09:11] <mvo> mborzecki: thanks for your review for 5043
[09:11] <mvo> mborzecki: I will dig a bit more in a bit but right now I think there is no way to disentangle --kill-who=main and KllMode=process
[09:11] <Chipaca> mvo: so! what can i help with today?
[09:12] <mvo> Chipaca: smart ideas about 5043 are in short supply right now :)
[09:13] <mup> PR snapd#5045 opened: overlord/snapstate:  poll for up to 10s if a snap is unexpectedly not mounted in doMountSnap (2.32) <Created by pedronis> <https://github.com/snapcore/snapd/pull/5045>
[09:14] <pedronis> mvo:  ^ prepared the backport
[09:14] <mvo> pedronis: \o/ thank you
[09:14] <Chipaca> mvo: I hear good things about runit
[09:14] <mvo> Chipaca: :-D
[09:14] <Chipaca> mvo: :-)
[09:15] <pedronis> so things that appear described as unrelated are interacting in obscure ways and are not orthogonal :/
[09:16] <Chipaca> mvo: so, if I understand the issue correctly, it's that if a daemon has refresh-mode=potato but does not hangle sigpotato and instead dies, then the whole service is killed?
[09:18] <mvo> Chipaca: yeah, all processes in the cgroup will be killed, that is my understanding
[09:18] <Chipaca> mvo: right
[09:19] <Chipaca> mvo: but isn't a daemon not handling the signal it asks to be delivered on refresh a bug?
[09:19] <mvo> Chipaca: from reading the source (my understaning is still a bit incomplete) I think what happens is that the main pid dies and that triggeres sigchld in systemd which notices that the main pid of the given unit died
[09:19] <mvo> Chipaca: this makes the unit enter "stop state" and systemd will do what it needs to do when this state appears. which includes the cleanup of the cgroup (AIUI)
[09:20] <mvo> Chipaca: in the sigterm case what we want is that the daemon restarts, I guess one could argue it should re-exec and use the same pid
[09:20] <mvo> Chipaca: this would solve the problem but I think this is not how most apps deal with it :/
[09:22] <Chipaca> mvo: what are we trying to accomplish? with refresh-mode=<foo>, when the snap refreshes, we do what? and the daemon does what? and systemd does what?
[09:22] <Chipaca> we=snapd there
[09:24] <pstolowski> zyga: have you seen jdstrand's comment to 5041?
[09:25] <mvo> Chipaca: on snap refresh with --refresh-mode=sigterm what we want is that the main process of the unit in question gets a sigterm. but that the other processes in the cgroup are left alone and survive
[09:25] <mvo> Chipaca: the use case is e.g. libvirt when it has a bunch of vms running that should not stop
[09:26] <mvo> Chipaca: we tell systemd to do "systemctl kill --kill-who=main -s TERM snap.name.app"
[09:26] <mvo> Chipaca: instead of the usual "systemctl stop snap.name.app"
[09:26] <mvo> Chipaca: making sense so far?
[09:26] <Chipaca> yes
[09:26] <mvo> now the problem seems to be that the option --kill-who=main is not orthogonal to KillMode= in a service file
[09:27] <mvo> Chipaca: or it is but in a different way, there is some entanglement
[09:27] <mvo> (the enganglement I described above, main pid dies, systemd wants to cleanup)
[09:27] <Chipaca> yes
[09:28] <Chipaca> mvo: what do we _want_ systemd to do?
[09:29] <mvo> Chipaca: on "systemctl kill --kill-who=main" we want it to kill the main pid and leave the rest alone
[09:29] <mvo> Chipaca: on unit stop we want it to stop everything
[09:30] <Chipaca> mvo: do we want it to restart the thing?
[09:30] <Chipaca> or _just_ kill it?
[09:30] <mvo> Chipaca: do whatever is defined in Restart=
[09:30] <mvo> Chipaca: it seems that this is a decision for the snap
[09:30] <mvo> Chipaca: but normally it would be Restart=on-failure (which is our default)
[09:31] <mvo> Chipaca: for a lot of things (SIGHUP) its a non-problem because the process will handle it and not die but sigterm is the problematic one
[09:31] <mvo> Chipaca: still making sense :) ?
[09:31] <Chipaca> restart=on-failure does not restart the thing when killed with TERM
[09:32] <Chipaca> might this be the reason it's entering stop mode at all?
[09:32] <mvo> Chipaca: I think I tested with "Restart=always"
[09:32] <mvo> Chipaca: and it had no effect but I can do so again
[09:33] <zyga> pstolowski: looking
[09:34] <Chipaca> mvo: sigterm is like sighup, in that the process can catch it etc (sigkill is the uncatchable one)
[09:34] <Chipaca> but, ok
[09:34] <mvo> Chipaca: I know
[09:34] <mvo> Chipaca: but it seems the services we care about do not catch it
[09:34] <Chipaca> mvo: a'ight
[09:35] <mvo> Chipaca: we could argue they should and the problem would go away
[09:35] <Chipaca> mvo: and AIUI the problem with using KillMode is that 'stop' will no longer work as expected?
[09:35] <zyga> jdstrand: I replied to your comments on 5041
[09:36] <mvo> Chipaca: correct
[09:36] <mvo> Chipaca: it will mean there are processes hanging around (potentially)
[09:37] <Chipaca> mvo: and can ExecStop itself call systemctl?
[09:40] <mvo> Chipaca: a good question, I think so, what do you have in mind?
[09:40] <Chipaca> mvo: wondering whether we can manually use ExecStop to get the 'stop' behaviour we want
[09:42] <Chipaca> mvo: as all the rest seems to be ok with the particular choice of restart/killmode
[09:44] <mvo> Chipaca: hm, won't systemd just call ExitStop= in both cases? when kill was used and when stop was used?
[09:50] <Chipaca> mvo: will it?
[09:53] <mvo> Chipaca: it seems to be, I added "ExecStop=/bin/sh -c "echo foo >/tmp/foo"" and ran a kill (with Restart=always) and /tmp/foo with the content got created
[09:55] <Chipaca> mvo: and is ExecStopPost _also_ run with kill?
[09:59] <mvo> Chipaca: let me check
[10:00] <mvo> Chipaca: yes, I also get a debug file with it
[10:01] <Chipaca> mvo: it sounds to me like the lesser weevil is to document that if you use refresh-mode, systemctl stop will be weird
[10:02] <mvo> Chipaca: yeah, and on remove do a extra kill --kill-who=all
[10:02] <mvo> Chipaca: I can't figure another way but I can poke at it a bit more after lunch
[10:02] <Chipaca> mvo: and do that on 'stop' itself also
[10:02] <mvo> Chipaca: on snap stop service?
[10:02] <Chipaca> ie 'snap stop' should work as expected even when systemctl stop doesn't
[10:03] <mvo> Chipaca: thats a nice idea
[10:03] <Chipaca> yeh
[10:44] <mborzecki> re
[10:49] <zyga> jjohansen, jdstrand: I took the profile with a leak and started removing features from it; I want to see if any of the newly-added features may be responsible
[10:50] <zyga> jjohansen, jdstrand: I also stress-tested all of the sysfs files in apparmorfs for extensive reading and can say that they are not a factor
[10:51] <zyga> (though I found one curious behaviour of the "revision" file, is that documented anywhere?
[10:55] <jjohansen> the revision file's behavior is known and going to change
[10:58] <zyga> jjohansen: that it "sleeps"
[10:58] <zyga> jjohansen: loading an empty profile leaks as well
[11:03] <zyga> jjohansen: https://github.com/zyga/apparmor-bug-leak
[11:04] <zyga> loading this is sufficient https://github.com/zyga/apparmor-bug-leak/blob/master/neutered-sample.aa
[11:04] <zyga> so it's not like a new optional feature is there and causes the leak
[11:04] <zyga> maybe something is not unref'd?
[11:18] <mborzecki> mvo: there's a failure in https://travis-ci.org/snapcore/snapd/builds/365982227?utm_source=github_status&utm_medium=notification in linode:debian-9-64:tests/main/snap-service-refresh-mode https://paste.ubuntu.com/p/cxTNbkBbtb/
[11:21] <mvo> mborzecki: thanks, looking
[11:22] <mborzecki> mvo: just the paste, i've restarted the build
[11:22] <mvo> mborzecki: thanks, I see it in the paste I will chase that
[11:23] <mvo> mborzecki: looking into this area anyway currently
[11:23] <mborzecki> mvo: yup, that's what i thought :)
[11:27] <zyga> jjohansen: I varied the test to insert a profile with ever-changing contents, I will check how that behaves across kernel versions
[11:29] <zyga> jjohansen: I noticed that one of the things that differs between the broken and working kernels is the presence/absence of symlinks in apparmorfs/policy/profiles/$PROFILE/
[11:40] <zyga> jjohansen, jdstrand: loading _different_ profiles forever on an affecter kernels doesn't leak memory
[11:41] <zyga> mvo: as a workaround I can generate random garbage rule like /tmp/.snapd.bug.1234.$RANDOM r,
[11:41] <zyga> mvo: and inject that into all profiles
[11:41] <zyga> mvo: we will lose all caching but we will not leak
[11:45] <mborzecki> pstolowski: would you like to review https://github.com/snapcore/snapd/pull/5034 ? :)
[11:45] <mup> PR #5034: userd: set up journal logging streams for autostarted apps <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5034>
[11:46] <mborzecki> pstolowski: it's based on 5024, so it's just the last 2 patches that are different 5034 specific
[11:47] <zyga> mvo: more ideas on v
[11:47] <zyga> https://forum.snapcraft.io/t/oom-for-interfaces-many-on-bionic-i386/4101/18?u=zyga
[11:49] <zyga> jjohansen, jdstrand: I'm now looking at error recovery in aa_replace_profiles
[11:49] <mvo> zyga: yay
[11:49] <mvo> zyga: let me know if you have anything to test
[11:50] <zyga> mvo: look at the options I gave
[11:50] <zyga> mvo: 1 is simple but wasteful
[11:50] <zyga> mvo: 2 should have no downsides but is complex
[11:50] <zyga> mvo: I can prepare a test with .1 quickly
[11:50] <mvo> zyga: if (1) is simple, we we just do it as an expeiment?
[11:50] <zyga> surte
[11:50] <zyga> *sure
[11:50] <mvo> zyga: yeah, lets do it if it does not take much time on your side
[11:57] <pstolowski> mborzecki: sure
[11:58] <zyga> mvo: after the break, now need to do stuff at home
[12:01] <pstolowski> Son_Goku: hey! am i missing something, or copr.fedorainfracloud.org doesn't actually offer an option for uploading spec+srpm as advertised on the wiki?
[12:01] <zyga> pstolowski: yes you are
[12:01] <Son_Goku> you can upload a srpm via the copr CLI tool
[12:01] <Son_Goku> or you can upload it somewhere and tell the web ui to fetch it
[12:02] <zyga> pstolowski: you should be able to from http://copr.fedorainfracloud.org/coprs/pstolowski/go-udev/packages/
[12:02] <Son_Goku> ah, and you can upload the srpm from the web ui too
[12:03] <Son_Goku> pstolowski: https://copr.fedorainfracloud.org/coprs/pstolowski/go-udev/add_build_upload/
[12:04] <pstolowski> Son_Goku: ah, there! I was staring at the Packages -> Add New Package where you need to have git/svn
[12:04] <pstolowski> thanks
[12:24] <mborzecki> reminds me to drop my copr packages, they're quite old anyway
[12:26] <mup> PR snapd#5046 opened: snap, wrappers, tests: rename refresh-mode -> stop-mode, endure -> skip-refresh <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5046>
[12:26] <mborzecki> mvo: ^^
[12:36]  * kalikiana lunch
[12:49] <zyga> re
[12:49] <zyga> mvo: looking at that workaround now, it will be a moment, I'm almost done
[12:52] <mvo> mborzecki: \o/
[12:52] <mvo> zyga: yay
[12:55] <mup> Issue snapcraft#2028 closed: When asking to release to a branch that's too long, a traceback is printed that gives no hints as to the source of the error <bug> <Created by sergiusens> <Closed by sergiusens> <https://github.com/snapcore/snapcraft/issue/2028>
[12:55] <mup> PR snapcraft#2059 closed: storeapi: handle 500 error response when releasing snap <Created by kyrofa> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2059>
[12:55] <Chipaca> mborzecki: haven't we had a release with refresh-mode already?
[12:59]  * Chipaca hunts for his headphones
[12:59] <mborzecki> Chipaca: yes, that's why i have some doubts about backwards compat
[13:02] <mborzecki> Chipaca: standup
[13:06] <mup> PR snapd#5047 opened: tests: removing linode-sru backend <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5047>
[13:20] <mup> PR snapd#5048 opened: tests: updating bionic version for spread tests on google <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5048>
[13:32] <popey> Is it "known" that opengl apps on 18.04 on binary nvidia are broken?
[13:33] <zyga> popey: no
[13:33] <popey> shotcut "could not initialize opengl" but works on intel
[13:33] <zyga> popey: on stable?
[13:33] <popey> i am on beta
[13:33] <popey> but i can go back to stable to test
[13:33] <zyga> popey: can you give us some more data, which hardware, which snaps, etc
[13:33] <zyga> mborzecki: ^
[13:33] <popey> sure
[13:34] <mborzecki> popey: after standup i'll reboot and check on bionic
[13:34] <popey> filing a bug to capture it
[13:34] <mborzecki> popey: how does it fail? is there any log?
[13:35] <popey> https://bugs.launchpad.net/snapd/+bug/1763717
[13:35] <mup> Bug #1763717: some opengl applications don't work on nvidia binary driver <snapd:New> <https://launchpad.net/bugs/1763717>
[13:35] <mborzecki> popey: try to /usr/lib/snapd/snap-discard-ns <snap-name>, and then capture the log with SNAPD_DEBUG=1 SNAP_CONFINE_DEBUG=1
[13:35] <popey> ok
[13:37] <popey> added to the bug
[13:37] <mborzecki> popey: it's a classic snap
[13:37] <kalikiana> re
[13:38] <popey> Is that bad? :)
[13:38] <mborzecki> popey: if it's a classic snap then it does not get any of the nvidia namespace setup treatment
[13:38] <popey> (I mean, we'd like it to not be classic)
[13:38] <zyga> popey: classic snaps don't have any support for that
[13:39] <popey> so it should "just work" right?
[13:39] <zyga> it depends on how it is made
[13:39] <zyga> but it's something for kalikana perhaps
[13:39] <zyga> we don't influence i
[13:39] <zyga> we don't influence it
[13:39] <zyga> it probably doesn't work
[13:39] <mborzecki> popey: shotcut snap right?
[13:39] <popey> yes
[13:39] <zyga> because 18.04 and 16.04 differ
[13:39] <popey> its in the store
[13:40] <zyga> host's handling of nvidia changed
[13:40] <zyga> there's nothing we can do IMO
[13:41] <mborzecki> i'll probably fail on arch too, let me try
[13:42] <mborzecki> popey: aborts on arch too https://paste.ubuntu.com/p/qPtZStGWth/
[13:42] <popey> :(
[13:45] <ogra_> you can surely do something about it in a wrapper script in the snap itself
[13:45] <ogra_> (hacks)
[13:52] <mborzecki> popey: i've installed all dependecies and can run shotcut directly outside of snap
[13:54] <popey> I'm at a loss what we suggest the developer does for a smooth experience installing snaps on multiple distros at different releases and have it work.
[13:54] <ogra_> do not use classic :)
[13:54] <zyga> mvo: I'll take the dog out now but then I will be back to propose the workaround
[13:54] <mvo> zyga: thank you
[13:54] <zyga> popey: you _can_ use nvidia but the snap needs some code for that
[13:54] <zyga> popey: talk to kalikana and sergiusens
[13:54] <zyga> popey: it's doable just nobody done it
[13:54] <popey> ok
[13:55] <zyga> popey: snapd is not preventing that
[13:55] <zyga> popey: it's just not enabling that (because it cannot)
[13:55]  * zyga -> afk
[13:55] <ogra_> yeah ... as i said, wrapper hackery
[13:55] <Chipaca> mvo: mborzecki: so...
[13:56] <Chipaca> mvo: mborzecki: removing catalogrefresh from snapd drops it (with an ~empty state) from 25MB rss to 15MB rss on start
[13:56] <Chipaca> pedronis: also ^
[13:56] <pedronis> catalogrefresh is all kind of evil :)
[13:56] <mborzecki> Chipaca: so basically .text + .data + .bss size
[13:56]  * Chipaca covers catalogrefresh's ears
[13:56] <mvo> Chipaca: woah
[13:57] <Chipaca> all kinds of .bs
[13:57] <pedronis> this is also because of bolt db code
[13:57] <mborzecki> Chipaca: heh ;)
[13:57] <pedronis> I suppose
[13:57] <Chipaca> pedronis: that's my assumption, yes
[13:57] <mborzecki> Chipaca: with GOGC=1 RSS was 19MB
[13:57] <mvo> Chipaca: nice, lets move it out into a separate helper
[13:57] <mvo> Chipaca: thanks, thats a huge win
[13:57] <Chipaca> mvo: now, or post-1804
[13:58] <pedronis> there are some issues around auth to move it out
[13:58] <Chipaca> pedronis: why? i thought it didn't auth at all
[13:58] <mvo> Chipaca: depends on your schedule for today, if you have free cycles I would say asap but does not have to be *now*
[13:58] <pedronis> Chipaca: we need to talk with nessita, apparently /names can be per store
[13:58] <pedronis> I don't know if it is yedt
[13:58] <mvo> oh
[13:58] <mvo> ok
[13:59] <Chipaca> mvo: I'll check with nessita and work on it today
[14:00] <mvo> mborzecki: I will merge your reresh-mode-endur-rename PR into my snap-rereshmode-fixes PR and work from that, ok?
[14:00] <mvo> Chipaca: sounds good, thank you
[14:00] <pedronis> mvo:  if we do a .5 we should consiser #5044
[14:00] <mup> PR #5044: 'system' nickname for 'core' in snap get/set (2.32) <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5044>
[14:01] <pedronis> I don't remember how risky it is
[14:01] <mvo> pedronis: +1 (cc mborzecki if you have cycles you could prepare a backport)
[14:01] <pedronis> but it would be good to have that in bionic from the start
[14:01] <mvo> pedronis: iirc not risky
[14:01] <pedronis> mvo: that is already the backport
[14:01] <pedronis> afaiu
[14:03] <mborzecki> mvo: sounds good
[14:03] <mborzecki> mvo: 5044 is a backport already
[14:03] <mvo> mborzecki, pedronis: aha, excellet
[14:03] <mborzecki> mvo: or did you mean a backport of the rename pr?
[14:04] <pedronis> mborzecki: he said he will merge the rename in his PR
[14:05] <mvo> mborzecki: I meant the system one
[14:05] <mborzecki> mvo: ok, so 5044 is good then ;)
[14:06] <mvo> mborzecki: yes!
[14:06] <mvo> mborzecki: if you want you can look into a smarter way to do https://github.com/snapcore/snapd/pull/5043/files#diff-08088787360fb3ca74a0aed4c6103b22R312
[14:06] <mup> PR #5043: many: fix "refresh-mode: sig{term,hup,usr[12]}" via KillMode=process <Created by mvo5> <https://github.com/snapcore/snapd/pull/5043>
[14:07] <jdstrand> roadmr: hey, can you pull r1025? this turns on by default (but that doesn't matter with your django changes), improves the resquash error message for snapcraft 2.38 and fixes a typo in overrides.py
[14:07] <mvo> mborzecki: i.e. what we want is to ensure that on remove everything in the unit gets killed eventually
[14:07] <roadmr> jdstrand: sure thing!
[14:07] <jdstrand> roadmr: thanks!
[14:08] <mvo> mborzecki: so something like "check unit, anything in there left? if so send sigterm. poll for up to 5 (or 10) seconds and check every ~1s if there is anything left. then do a hard sigkill on the group
[14:13] <mup> PR snapcraft#2070 opened: extractors: support for setup.py <enhancement> <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2070>
[14:14] <mborzeck1> gnome shell froze again :/
[14:14] <zyga> Pharaoh_Atem: Hey
[14:14] <zyga> I will build a test snap on top of the
[14:14] <zyga> Fedora base snap this weekend
[14:15] <zyga> And I will try to publish your base
[14:16] <mvo> mborzecki: did you got my last messages or shall I repaste?
[14:16] <mborzecki> mvo: repaste please
[14:16] <pedronis> mvo: ah, I wondering if we should operate a bit more like systemd when we really try to kill the whole service
[14:16] <mvo> mborzecki: so something like "check unit, anything in there left? if so send sigterm. poll for up to 5 (or 10) seconds and check every ~1s if there is anything left. then do a hard sigkill on the group
[14:17] <mvo> pedronis: yeah, I think on remove we must be
 mborzecki: if you want you can look into a smarter way to do https://github.com/snapcore/snapd/pull/5043/files#diff-08088787360fb3ca74a0aed4c6103b22R312
[14:17] <mup> PR #5043: many: fix "refresh-mode: sig{term,hup,usr[12]}" via KillMode=process <Created by mvo5> <https://github.com/snapcore/snapd/pull/5043>
 mborzecki: i.e. what we want is to ensure that on remove everything in the unit gets killed eventually
[14:17] <mvo> mborzecki: sorry, slightly out of order
[14:17] <mborzecki> that's ok
[14:17] <pedronis> I left a comment there
[14:18] <pedronis> about a code org question
[14:18] <pedronis> and John input
[14:18] <pedronis> Chipaca: you should also look at #5043
[14:18] <mup> PR #5043: many: fix "refresh-mode: sig{term,hup,usr[12]}" via KillMode=process <Created by mvo5> <https://github.com/snapcore/snapd/pull/5043>
[14:18] <Chipaca> pedronis: I did, had a long chat with mvo about it this morning
[14:19] <pedronis> heh, ok
[14:22] <popey> zyga: mborzecki I am not convinced this is a classic only issue. snap install mame --beta, that's not classic and doesn't work on 18.04/nvidia, but does on 16.04/intel.
[14:23] <popey> I am certain this worked previously on 18.04/nvidia (because both I and Wimpress have tested it on that platform)
[14:23] <zyga> Yes, It worked on 16.04 by accident
[14:23] <zyga> But Ubuntu changed so it no longer does
[14:24] <zyga> It probably still works on 16.04
[14:24] <zyga> Chipaca can confirm as he has the right software and hardware
[14:25] <Chipaca> i what now?
[14:25] <popey> This doesn't feel like an optimal response to "My snap worked and now it doesn't"
[14:25] <mborzecki> popey: updating my bionic install now, i'll check in a minute
[14:26] <Pharaoh_Atem> zyga: ?
[14:29] <mvo> pedronis: thank you, I have a look, I just added stop-mode and tweaked a bit but I need to add more tests and tweaks but I think the direction is nice
[14:29] <mup> PR snapd#5046 closed: snap, wrappers, tests: rename refresh-mode -> stop-mode, endure -> skip-refresh <Created by bboozzoo> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/5046>
[14:30] <Pharaoh_Atem> zyga: before we publish anything, we need permission from Fedora Council and Server WG for usage of the name 'fedora'
[14:34] <zyga> Pharaoh_Atem: that's a good point
[14:34] <zyga> Pharaoh_Atem: how can we get that?
[14:34] <Pharaoh_Atem> probably talk to sgallagh to see how to do that
[14:34] <zyga> Pharaoh_Atem: can we can publish it as phedora for now?
[14:34] <zyga> for development
[14:34] <zyga> before anyone can think it's ready for use
[14:35] <Pharaoh_Atem> fedora-release needs to be swapped for generic-release, and then we can call it whatever
[14:35] <zyga> Pharaoh_Atem: that's nice
[14:36] <zyga> Pharaoh_Atem: I'll make a test snap, if we agree on the name "phedora"
[14:36] <zyga> or something like thaT :)
[14:36] <Pharaoh_Atem> with generic-release, the system identifies as Generic Linux :)
[14:36] <Pharaoh_Atem> https://src.fedoraproject.org/rpms/generic-release/blob/master/f/generic-release.spec#_136-154
[14:36] <mborzecki> popey: ohmygiraffe works fine, trying out mame now, any roms you can recommend? :)
[14:36] <zyga> mborzecki: did you play golden sun?
[14:37] <zyga> mborzecki: I have it on an old Nintendo console, it's an amazing game
[14:38] <zyga> Pharaoh_Atem: generic is ... too genric
[14:38] <zyga> but we can come up with something I'm sure
[14:38] <mborzecki> i'm a metal slug fan ;)
[14:38] <popey> mborzecki: yes, omg works, but that was built and never rebuilt a year ago.
[14:38] <zyga> mborzecki: ah, I love that game too :) man I'm so old now that I think of it
[14:39] <Pharaoh_Atem> zyga: we can also fork generic-release and produce a <foo>-release package to brand as another distro that says "ID_LIKE=fedora"
[14:39] <Pharaoh_Atem> again, fairly trivial stuff
[14:39] <zyga> yeah, that's good
[14:39] <popey> mborzecki: Personally I like ghosts & goblins, r-type, r-type 2, rtype leo, scramble, nemesis, side arms and bomb jack - and I own the arcade boards so I'm (allegedly) legal ;)
[14:39] <zyga> I'll make some packages and stick it into copr
[14:39] <Pharaoh_Atem> ;)
[14:39] <zyga> popey: _wow_
[14:40] <Pharaoh_Atem> zyga: you could even call it Ubuntu RPM Edition if you wanted :P
[14:40] <mvo> pedronis: re https://github.com/snapcore/snapd/pull/5043#discussion_r181401324 - do you have a suggestion what place to use?
[14:40] <mup> PR #5043: many: add "stop-mode: sig{term,hup,usr[12]}{,-all}" instead of conflating that with refresh-mode <Critical> <Squash-merge> <Created by mvo5> <https://github.com/snapcore/snapd/pull/5043>
[14:40] <zyga> Pharaoh_Atem: but then ubuntu trademark ;-)
[14:40] <Pharaoh_Atem> though I think sabdfl would be a little bit peeved :P
[14:40] <zyga> Pharaoh_Atem: I'll call it zygoonix
[14:40] <Pharaoh_Atem> hehe
[14:40] <mvo> Pharaoh_Atem: hahaha
[14:40] <mborzecki> popey: yeah, mame does not work, but the nvidia libs get mounted at the proper location, i'll check the wrapper script in case it's doing something silly
[14:40] <popey> Orange Hat
[14:40] <Pharaoh_Atem> :D
[14:40] <zyga> OMG
[14:40] <mvo> lol
[14:40] <Pharaoh_Atem> why not, eh?
[14:40] <zyga> orange is the new hat ;-)
[14:41]  * zyga gets back to kernel bug fixing and workarounds
[14:41] <popey> mborzecki: I was using a manual launcher I made previously, I switched to desktop-glib-only
[14:41] <Pharaoh_Atem> if you want to have more fun, there's also a generic-logos package you can fork and make up some fun branding for
[14:41] <pedronis> mvo: I think the helper should just return if the mode if for all or main?  and then wrappers picks process based on that
[14:42] <mvo> pedronis: yes, that sounds good
[14:42] <mvo> pedronis: I will change it this way
[14:42] <Pharaoh_Atem> popey: Purple Cap :P
[14:42] <pedronis> mvo: thank you, is mostly for future readers, is a bit strange to be in snap
[14:42] <mvo> pedronis: I will call it "KillEmAll()"
[14:42] <mvo> pedronis: totally agreed, the wrong place
[14:42] <zyga> mvo: how about SecureKillEmAll
[14:43] <zyga> we want to be safe, after all
[14:43] <popey> Pharaoh_Atem: I like!
[14:43] <zyga> I think I have a name
[14:44] <zyga> but I'll announce it later :P
[14:44] <mborzecki> popey: strace shows it's not loading libGL from the right location
[14:44] <Pharaoh_Atem> zyga: now don't forget, we need logos and stuff too :P
[14:45] <Chipaca> mvo: it looks like I won't be splitting catalog refresh today
[14:45] <zyga> Pharaoh_Atem: and a boot up chime
[14:45] <Pharaoh_Atem> YES!
[14:45] <zyga> Pharaoh_Atem: it can be "SCO LINUX" played backwards
[14:45] <Pharaoh_Atem> haha
[14:46] <Pharaoh_Atem> it takes ~40 minutes to make your own branded derivative of Fedora, satisfying _all_ of the necessary trademark guidelines
[14:46] <Pharaoh_Atem> and with your derivative, you can do whatever you want
[14:48] <mborzecki> popey: this is ld.so log from when it tries to load libGL https://paste.ubuntu.com/p/S5MyfKdyXT/ SNAP LIBRARY_PATH is not in LD_LIBRARY_PATH anymore
[14:49] <mborzecki> that's why it does not pick up the right libGL
[14:50] <zyga> mborzecki: can we inject that for classic snaps?
[14:50] <popey> (this isnt classic)
[14:50] <zyga> oh
[14:50] <zyga> then we can definitely influence that
[14:50] <popey> so could I work around this by adding an environment stanza which prepends SNAP_LIBRARY_PATH to LD_LIBRARY_PATH?
[14:52] <zyga> popey: that's what snapcraft does
[14:52] <mborzecki> popey: i'm looking into the wrappers now
[14:52] <popey> ok
[14:52] <popey> thanks
[14:53] <Chipaca> huh, I did apt purge snapd and now have two loopback devices hanging around
[14:53] <zyga> Chipaca: can you run
[14:54] <zyga> losetup
[14:54] <zyga> and paste that
[14:54] <Chipaca> I can run, just not on open road just yet
[14:54] <Chipaca> too hard on the back
[14:54] <Chipaca> zyga: it lists two devices, both with "deleted"
[14:54] <zyga> Chipaca: super interesting
[14:54] <zyga> so two snaps were removed but their loop devices were not cleaned up
[14:54] <Chipaca> yes
[14:54] <zyga> what are those
[14:54] <Chipaca> and I can mount them :-)
[14:55] <zyga> some experiments or regular things?
[14:55] <Chipaca> core and lxd
[14:55] <zyga> intersting
[14:55] <zyga> maybe related to what pedronis is chasing
[14:55] <Chipaca> and i can't detach them even though they're not mounted
[14:55] <pedronis> do you have lxd containers running?
[14:57] <Chipaca> pedronis: I don't think so, but how do i check?
[14:57] <Chipaca> huh, i do have lxd stuff running
[14:57]  * zyga found one typo, closer to having a patch
[14:57] <pedronis> that might explain why didn't go away
[14:57] <pedronis> maybe
[14:58] <Chipaca> dbus, lxd itself, and dnsmasq
[14:58] <Chipaca> thinking about it I manually removed the lxd snap yesterday
[14:58] <zyga> as in rm -f
[14:58] <Chipaca> no, as in 'snap remove lxd'
[14:58] <Pharaoh_Atem> zyga, mvo, niemeyer: Flock 2018 is in Dresden, Germany: https://fedoramagazine.org/flock-2018-venue-announcement/
[14:59] <zyga> during holidays
[14:59] <zyga> nice
[14:59] <zyga> maybe I can come
[14:59] <zyga> even totally unofficialy
[14:59]  * pedronis is going to eow
[15:03] <mborzecki> zyga: does RPHAT have higher priority than LD_LIBRARY_PATH?
[15:03] <mborzecki> RPATH
[15:05] <zyga> mborzecki: thinking
[15:05] <zyga> mborzecki: I don't know :/
[15:05] <zyga> I need to check spec for that
[15:05] <sergiusens> rpath that has lower precedence than the LD_LIBRARY_PATH environment variable
[15:06] <sergiusens> googled and memory recollection seems to align with that
[15:06] <sergiusens> elf files can be disabled from looking at LD_LIBRARY_PATH though
[15:07] <mborzecki> sergiusens: zyga: this is what I see: https://paste.ubuntu.com/p/8mbNy545Z4/
[15:07] <mborzecki> if i LD_PRELOAD then mame works
[15:07] <mborzecki> and I can see the paths from LD_LIBRARY_PATH being used when searching for other libs
[15:09] <mup> PR snapd#5048 closed: tests: updating bionic version for spread tests on google <Created by sergiocazzolato> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5048>
[15:10] <sergiusens> mborzecki: don't you need to snap --shell run `mame` and set LD_LIBARARY_PATH in there?
[15:11] <mborzecki> sergiusens: i'm inside the snap ns
[15:11] <mup> PR snapd#5047 closed: tests: removing linode-sru backend <Created by sergiocazzolato> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5047>
[15:12] <sergiusens> oh, I read incorrectly
[15:12] <sergiusens> can I see all of readelf's output?
[15:12] <sergiusens> but will look shortly, need to do a pickup
[15:15] <mborzecki> sergiusens: http://paste.ubuntu.com/p/bGPHtsMHMs/
[15:21] <mborzecki> sergiusens: LD_DEBUG=all http://paste.ubuntu.com/p/Zm57vmQCwR/, at first it's only looking at RPATH, then around line 5217 it picks up LD_LIBRARY_PATH when it moves to loading deps of other libraries without RPATH
[15:22] <mborzecki> pff no clue what's happening, i could probably try to patch mame binary and strip/fixup rpath
[15:36] <zyga> mvo: testing the fix now
[15:36] <mvo> zyga: testing as in running spread with 18.04-32 ?
[15:36] <zyga> yes
[15:37] <mvo> cool
[15:37] <zyga> mvo: I've added a constraint that runs this only on classic
[15:37] <zyga> we can further refine it to look for apparmor "feature" non-leaking kernel or something
[15:39] <zyga> btw, where did you get bionic 32 image
[15:39] <zyga> I had to use my hacky scripts to make one
[15:39] <zyga> for qemu
[15:40] <mvo> zyga: autopkgtest-buildvm-ubuntu-cloud -r bionic -a i386
[15:41] <zyga> yeah
[15:41] <zyga> that's what I do
[15:41] <mvo> zyga: oh, and it did not work?
[15:42] <zyga> no, I mean, I hoped we had a google image
[15:43] <mvo> zyga: aha, not yet afaict
[15:43] <mvo> afaik
[15:44] <zyga> mvo: quick pre-review?
[15:44] <zyga> https://github.com/snapcore/snapd/compare/master...zyga:tweak/inject-random-profiles?expand=1
[15:44] <mvo> Chipaca, zyga 5043 is ready I think for a review, tests still running so maybe some tweaks needed but hopefully its ok
[15:44] <zyga> ok, looking
[15:45] <zyga> holly molly
[15:45] <zyga> that's not a small change sadly
[15:45] <zyga> but let me read on
[15:55] <zyga> jdstrand: hey
[15:55] <zyga> jdstrand: if around, can you look at the link above please
[15:55] <pedronis> zyga: did you see jdstrand commented in the forum?
[15:56] <zyga> checking
[15:56] <zyga> yeah, just saw
[16:00] <mvo> zyga: fwiw, it looks reasonable - does it help? i.e. is the test working now?
[16:00] <zyga> still going for now
[16:01] <zyga> it definitely does help in my simplified testing where I just insert random profile in a tight loop
[16:01] <zyga> that's stable over 100K insertions
[16:02] <mvo> nice
[16:58]  * kalikiana wrapping up for the week o/
[16:59] <ogra_> jdstrand, [Apr13 16:23] audit: type=1400 audit(1523636600.835:36): apparmor="DENIED" operation="open" profile="snap-update-ns.classic" name="/dev/urandom" pid=1811 comm="3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[17:00] <ogra_> jdstrand, looks like the classic-support interface would like /dev/urandom support ;)
[17:02] <zyga> snap-update-ns.classic
[17:02] <zyga> what would make it open /dev/urandom?
[17:02] <zyga> ogra_: that profile is applied to a program that modifies snap namespaces, not to the program itself
[17:02] <ogra_> zyga, thats the classic (developer chroot) snap on a core system
[17:03] <zyga> yes, but the denial is odd, this is not a profile for the application
[17:03] <ogra_> something inside tries to access urandom i guess
[17:03] <zyga> it is a profile for the tool that constructs the execution environment
[17:03] <zyga> it never should open /dev/urandom
[17:03] <zyga> can you tell me how to reproduce this?
[17:04] <ogra_> not really but i can try to proxy ... afaik ian only noticed it in his logs after working a day with his dragonboard
[17:04] <zyga> in any case it should not cause any actual harm
[17:04] <ogra_> no, thats clear
[17:04] <ogra_> but i thought it was just missing device access in the classic-support interface
[17:08] <mup> Issue snapcraft#1920 closed: Design error reporting <Created by sergiusens> <Closed by sergiusens> <https://github.com/snapcore/snapcraft/issue/1920>
[17:08] <mup> PR snapcraft#1951 closed: repo: do not pull in libc6-dev by default for stage-packages <bug> <Created by sergiusens> <Closed by sergiusens> <https://github.com/snapcore/snapcraft/pull/1951>
[17:09] <pedronis> zyga: snap-update-ns might load bits of snapd  that try to use random generation
[17:09] <zyga> ah, indeed
[17:09] <zyga> perhaps osutil does need it
[17:10] <pedronis> we would need to investigate, it might not be something that snap-update-ns uses but some init of something linked might use it
[17:11] <mup> Issue snapcraft#2071 opened: patchelf to handle RUNPATH <Created by sergiusens> <https://github.com/snapcore/snapcraft/issue/2071>
[17:42]  * cachio afk
[17:53] <mvo> hrm, hrm, no travis slots, the typical friday evening problem after a busy week
[17:53]  * mvo is slightly sad about this
[17:59] <kyrofa> roadmr, are there issues with the staging store right now?
[17:59] <kyrofa> roadmr, having trouble logging in, do you see any issues here? https://pastebin.ubuntu.com/p/KVXTxkYZBf/
[18:02] <roadmr> kyrofa: what's a 504?? hehe let me see
[18:03] <kyrofa> A timeout, indeed, that seemed odd to me as well
[18:03] <roadmr> weird
[18:03] <roadmr> a sec
[18:06] <mup> Issue snapcraft#1918 closed: Add y/n support for sending errors back <Created by sergiusens> <Closed by kyrofa> <https://github.com/snapcore/snapcraft/issue/1918>
[18:06] <mup> PR snapcraft#2069 closed: Reports <Created by sergiusens> <Merged by kyrofa> <https://github.com/snapcore/snapcraft/pull/2069>
[18:07] <roadmr> kyrofa: I'm able to "snapcraft login" to staging just fine
[18:08] <kyrofa> roadmr, I actually had a success with another account as well-- any chance there could be an issue with a specific account?
[18:08] <kyrofa> Either that or it doesn't always happen
[18:08] <kyrofa> Although I got it twice in a row with the same account
[18:09] <roadmr> kyrofa: could be! can you authenticate with that account at login.staging.ubuntu.com?
[18:09] <kyrofa> Let me try
[18:10] <roadmr> kyrofa: hm, the URL that's timing out on you is actually in the staging store / dashboard. Incremented Retry for (url='/dev/api/account'): Retry(total=4, connect=None, read=None, redirect=None)
[18:10] <roadmr> kyrofa: when was this?
[18:11] <kyrofa> Just now
[18:11] <roadmr> kyrofa: ok, let me check whether any of the app servers are wonky
[18:12] <roadmr> (they were earlier today, that's why I asked "when")
[18:14] <roadmr> kyrofa: everything looks healthy :(
[18:14] <roadmr> kyrofa: I'm about to go lunch but you can ask in #snapstore, anyone there should be able to look at this
[18:21] <kyrofa> roadmr, note that I can indeed login to login.staging.ubuntu.com
[18:23] <zyga> wooooot
[18:23] <zyga> first thunderstorm of the season
[18:23] <zyga> !!!
[18:23] <zyga> waterfall from the sky
[18:25] <popey> zyga: do you know if mb figured out what was breaking mame?
[18:25] <popey> (is there something i can do in the launcher?)
[18:25] <zyga> popey: sorry, I don't know
[18:25] <popey> ok
[18:25] <popey> I'll ask next week
[18:25] <zyga> I'm looking at a kernel leak all day
[18:27] <suebt> hey zyga, any news about the daemon thingy? :)
[18:27] <zyga> suebt: hey, no, I will try next week, sorry :/
[18:28] <suebt> ok np
[18:38] <mup> PR snapd#5049 opened: interfaces/apparmor: workaround kernel memory leak <Created by zyga> <https://github.com/snapcore/snapd/pull/5049>
[18:39] <roadmr> kyrofa: yes, per my earlier comment, it's not a login.staging.u.c issue, but a dashboard.snapcraft.io one
[18:40] <kyrofa> Ah, okay
[18:40] <popey> jdstrand: I can't upload with snapcraft 2.40 - despite your thread suggesting 2.40 implements this -no-fragments thing
[18:40] <popey> https://forum.snapcraft.io/t/automated-reviews-and-snapcraft-2-38/4982
[18:41] <popey> Hang on, this is an electron application - built with electron-builder. Does that bake in something older than 2.40?
[18:43] <roadmr> kyrofa: just so we're clear, things are working well for you now, and you'll let me (or anyone in store team) know if you see this again, right?
[18:43] <roadmr> kyrofa: (just to avoid an expectation mismatch "I thought you were looking into it" "I thought you'd tell me if it broke again" :)
[18:46] <kyrofa> roadmr, no, something is wrong. I can definitely login with other accounts, but not this one. Ever
[18:46] <kyrofa> Same thing happens every time
[19:01] <roadmr> kyrofa: oh! ok, let's look at it from this angle then
[19:05] <cprov> kyrofa: oi oi
[19:05] <kyrofa> Hey there cprov
[19:06] <cprov> kyrofa: the `account` endpoint (used on login to cache account-id, IIRC) is returning a lot more data in `snaps` and it's  undeniably slower, specially for crazy testing users we have in staging
[19:07] <kyrofa> cprov, if you look at the snap names, we testing things like registration
[19:07] <kyrofa> cprov, and then we never touch some again. Is there a way we can tell the store to toast them?
[19:07] <cprov> kyrofa: cleaning up snaps is not operational yet, for now the best suggestion I have is to create a new user
[19:07] <kyrofa> Ouch
[19:08] <kyrofa> This is all CI
[19:08] <kyrofa> Can we manuall clear out the snaps?
[19:08] <kyrofa> manually*
[19:08] <cprov> kyrofa: it's not that bad, a new user will be performing well up to 1000s snaps, which is weeks of testing
[19:10] <cprov> kyrofa: cleanup is too manual atm, via the UI one by one, we need come up with a script to do that properly
[19:12] <kyrofa> cprov, okay. Is there a different endpoint we could be using that won't give us every single snap?
[19:13] <cprov> kyrofa: not yet, but what do you have in mind ?
[19:13] <kyrofa> cprov, just seems silly to return a bunch of data we don't need
[19:13] <kyrofa> I don't want to create a new user every few weeks
[19:14] <cprov> kyrofa: why does it need the account/ on login ?
[19:15] <cprov> kyrofa: ftr, there are 4200 u1test* snaps in staging
[19:17] <mvo> zyga: I see #5049 - did it survive ubuntu-18.04-32?
[19:17] <mup> PR #5049: interfaces/apparmor: workaround kernel memory leak <Created by zyga> <https://github.com/snapcore/snapd/pull/5049>
[19:20] <kyrofa> cprov, no idea, do you think it shouldn't?
[19:21] <kyrofa> I can find out if you think it unnecessary
[19:21] <cprov> kyrofa: let me check the code, I don't see an obvious reason
[19:24] <zyga> mvo: my workaround worked
[19:24] <zyga> mvo: it failed on network on one test
[19:24] <zyga> mvo: I restarted but it works
[19:25] <kyrofa> cprov, _check_dev_agreement_and_namespace_statuses I think
[19:25] <zyga> mvo: I get dinner now
[19:25] <kyrofa> We check to ensure the account is properly setup with the dev namespace and whatnot
[19:25] <cprov> kyrofa: ah, I see
[19:27] <mvo> zyga: nice
[19:27] <mvo> zyga: I'm also waiting for tests
[19:36] <mup> PR snapcraft#2072 opened:  lifecycle: handle missing version correctly  <bug> <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2072>
[19:45] <jdstrand> popey: you said 'hold on'. I don't know if electron builder pins snapcraft in any way, but I can say the store queue is empty
[19:45] <jdstrand> popey: make sure that schroot, lxd containers, your snaps, your debs, your git checkout, etc is using the latest snapcraft
[19:46] <jdstrand> popey: if you refresh the review-tools snap, you can do 'snap-review /path/to/snap' and it will have better error reporting on the issue and can let you know if -no-fragments was used or not, or something else
[19:48] <jdstrand> popey: you mentioned electron-- if you are using 'allow-sandbox: true' with a setuid chome sandbox, that will also cause it to fail this test (it will also fail *other* tests though too)
[19:48] <sergiusens> any ideas about: cannot change profile for the next exec call: No such file or directory
[19:48] <sergiusens> snap-update-ns failed with code 1: No such file or directory
[19:49] <sergiusens> jdstrand, zyga: ^
[19:49] <jdstrand> popey: but ultimately, you didn't give a store url so I can't look into it more
[19:49] <sergiusens> that's when running corebird, previously working
[19:49] <zyga> Yes
[19:49] <zyga> But not debugged
[19:50] <zyga> Watching movie with family now
[19:57] <mvo> zyga: enjoy. still no travis slot for me, I call it a day
[19:58] <zyga> Boo :/ sorry to hear that
[19:58] <zyga> Travis has a weekend mode
[20:07] <jdstrand> popey: please see the forum
[20:31] <mup> PR snapcraft#2068 closed: states: track override scriptlets <Created by kyrofa> <Merged by kyrofa> <https://github.com/snapcore/snapcraft/pull/2068>
[20:34] <popey> jdstrand: ok. i dont know how electron does it
[20:34] <popey> jdstrand: there's none in the queue I guess because it never got to the store
[20:41] <kyrofa> popey, I believe they call mksquashfs themselves
[20:41] <roadmr> kyrofa: ah, nice! if that's true and electron-builder can be poked at, their mksquashfs call can be made to conform to what Jamie suggested
[20:44] <kyrofa> No, I lied: https://forum.snapcraft.io/t/snapcraft-push-error-keyerror-architectures/4056/9
[20:44] <kyrofa> Sounds like they use snapcraft pack
[20:44] <kyrofa> But what version?
[20:45] <roadmr> aha :)
[20:46] <kyrofa> They have a docker image, I bet it hasn't been respun
[20:49] <kyrofa> Though I can't pretend to understand fully how it works
[20:50] <roadmr> their instructions ask to install snapcraft as a deb
[20:51] <roadmr> 2.40 is available for xenial, artful and bionic, and from this I understand it'd use the deb installed on the system... which popey said was 2.40
[20:52] <kyrofa> Interesting
[20:52] <popey> it looks like e-b manually does some unsquashfs / mksquashfs, but I'm not a developer, so I am going by what I think i see on github
[20:52] <popey> https://github.com/electron-userland/electron-builder/blob/7800ce1301154281564a23b5707e9a79bf3aa52d/scripts/snap-template.sh#L8
[20:54] <kyrofa> I saw that too... but it looked more like a test
[20:56] <popey> looks like the node module electron-builder-lib
[20:57] <popey> I don't understand this :(
[20:57] <popey> one for monday.
[20:58] <mup> PR snapcraft#1911 closed: Add support enable configurable runtime version for .NET Core applications <enhancement> <Created by rakeshsinghranchi> <Merged by kyrofa> <https://github.com/snapcore/snapcraft/pull/1911>
[20:59] <cachio> niemeyer, there?
[21:05] <niemeyer> cachio: Here, but still in the conference
[21:06] <cachio> niemeyer, nice, just 1 quick question
[21:07] <cachio> I am working with the amazon image
[21:07] <cachio> it is working but still trying to resize it
[21:07] <cachio> I read it is not possible to shrink xfs filesystems
[21:07] <cachio> and it is using an xfs
[21:07] <cachio> I see 2 alternatives
[21:08] <cachio> I could create a snaller partition and copy the fs to there
[21:09] <cachio> or leave it with the same size and make a change in spread to skip setting the size of the disk when it is defined in the sprad.yaml
[21:09] <cachio> niemeyer, do you see other alternatives?
[21:09] <cachio> what do you prefer?
[21:14] <niemeyer> cachio: What happens if you set the image to exactly 10GB? I hope it won't try to resize it
[21:15] <niemeyer> cachio: We shouldn't change the nature of the image though.. shouldn't be a different filesystem for example
[21:15] <cachio> niemeyer, you mean if I truncate the image to 10GB?
[21:15] <niemeyer> cachio: Worst case we can have a setting in spread to preserve the image size
[21:16] <cachio> niemeyer, yes, that was the other alternative
[21:17] <cachio> I can try to resize the partition but we could loose data
[21:18] <cachio> If you agree I could make a PR in spread to preserve the image size i¿
[21:19] <cachio> niemeyer, in that case we can use the amazon image which it already uploaded
[21:22] <mup> Issue snapcraft#2064 closed: Support for set-grade <Created by kyrofa> <Closed by sergiusens> <https://github.com/snapcore/snapcraft/issue/2064>
[21:22] <mup> PR snapcraft#2067 closed: many: add snapcraftctl set-grade <Created by kyrofa> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2067>
[21:25] <mup> PR snapcraft#2073 opened: meta: validate extracted and scriptlet metadata <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2073>
[22:42] <niemeyer> cachio: Yeah, I think that's fine.. we can use "preserve" as a value, that internally sets the value to zero, and we interpret it as such
[23:16] <niemeyer> cachio: Actually, zero won't work as that's the default in which we want to resize as what most cases need.. preserve could set it to -1 though
[23:16] <cachio> niemeyer, ok
[23:17] <cachio> I'll try it
[23:18] <cachio> thanks