[00:19] <RingtailedFox> thank
[00:19] <RingtailedFox> thanks Eighth_Doctor
[02:09] <RingtailedFox> how do i go about turning that ./spec file into an RPM?
[06:35] <mborzecki> morning
[06:36] <mborzecki> starting around 9, need to run an errand and do groceries before the panicked crowds show up
[07:39] <pedronis> mvo: hi, maybe it was already noticed, but it seems we have a real issue with seccomp unit tests on 20.04: https://api.travis-ci.org/v3/job/662738603/log.txt
[07:41] <mvo> pedronis: yeah, I'm checking that right now
[07:41] <mvo> pedronis: also the postrm test is still flaky, a bit annoying
[07:43] <mup> PR snapd#8262 closed: interfaces/udisks2: also allow Introspection on /org/freedesktop/UDisks/** <Simple 😃> <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8262>
[07:43] <mup> PR snapd#8263 closed: interfaces/udisks2: also allow Introspection on /org/freedesktop/UDisks2/** - 2.44 <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8263>
[07:43] <pedronis> mvo: this test seems also flaky sometimes snap-session-agent-socket-activation
[07:43] <pedronis> seems a timing issue
[07:44] <mvo> pedronis: on all OSes or just specific ones? I noticed it only on ubuntu-core-16
[07:44] <mup> PR snapd#8261 closed: interfaces: work around apparmor_parser slowness affecting uio (2.44) <Created by pedronis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8261>
[07:44] <pedronis> I seen it also on non-core
[07:45] <mvo> pedronis: ok, I see what I can find
[07:46] <pedronis> mvo: maybe jamesh can also help with that one
[07:52] <zyga> good morning
[07:53] <zyga> pedronis: this is known, I reported it on tumbleweed about a week ago
[07:53] <pedronis> zyga: well we have it on 20.04 now
[07:53] <zyga> https://bugs.launchpad.net/snapd/+bug/1866183
[07:53] <mup> Bug #1866183: seccomp tests fail in tumbleweed on 2020-03-05 <snapd:New> <https://launchpad.net/bugs/1866183>
[07:53] <zyga> fun :|
[07:53] <zyga> ok
[07:54] <zyga> I had a terrible night, I'm barely awake
[07:54] <zyga> how can I help?
[07:54] <zyga> should I jump at seccomp, at reproducible maas content bug, at something existing like app tracking?
[07:54] <mvo> zyga: I'm looking at this right now
[07:54] <mvo> zyga: maas content bug sounds worthwhile
[07:55] <mvo> zyga: I can reproduce the seccomp one
[07:55] <pedronis> mvo: is it segfaulting ?
[07:55] <zyga> I'll get to it
[07:55] <mvo> pedronis: not sure yet, seccomp got upgraded from 2.4.2 to 2.4.3
[07:56] <mvo> pedronis: this is triggering it
[07:56] <pedronis> ok, seccomp upgrade seemed the most likely candidate
[07:56] <zyga> mvo: if you do please check if the list in the bug I referenced is the same as in ubuntu now
[07:57] <zyga> I hope it's the same bug
[07:57] <mvo> zyga: yeah, output looks exactly the same
[07:57] <mvo> zyga: I bet the same lib update happend there too
[07:57] <zyga> yeah, that's a good sign
[07:57] <zyga> whatever it is we will need just one solution
[07:58] <pedronis> this is fixed now:  https://bugs.launchpad.net/snapd/+bug/1863807 ?
[07:59] <mup> Bug #1863807: core snap rev 8716 breaks keyboard input in chromium and inkscape <snapd:New> <https://launchpad.net/bugs/1863807>
[07:59] <zyga> pedronis: I think it was
[07:59] <zyga> pedronis: it was a change in input handling in ibus or something alike
[08:00] <zyga> pedronis: jamie fixed it a few weeks ago
[08:00] <pedronis> mvo: they asked you a question here: https://bugs.launchpad.net/snapd/+bug/1863816
[08:00] <mup> Bug #1863816: trusty: snapd deb package does not start systemd <snapd:New> <https://launchpad.net/bugs/1863816>
[08:03] <zyga> https://bugs.launchpad.net/snapd/+bug/1867415 is interesting :|
[08:03] <pstolowski> morning
[08:03] <mup> Bug #1867415: Hardlinking snaps wastes 400 MB tmpfs RAM in live CDs <snapd:New> <https://launchpad.net/bugs/1867415>
[08:03] <zyga> hey pawel
[08:03] <zyga> pstolowski: it's a morning but not particularly good one ;)
[08:04] <pstolowski> zyga: oh, bugs?
[08:04] <zyga> pstolowski: bugs and spiders
[08:05] <zyga> oh
[08:05] <zyga> public service announcement
[08:05] <zyga> if you are running focal
[08:05] <zyga> be careful not to break your system
[08:05] <pedronis> zyga: I assigned this to you: https://bugs.launchpad.net/snapd/+bug/1866855  not sure what's it's overall priority
[08:05] <mup> Bug #1866855: nvidia driver integration is incompatible with Debian <snapd:Triaged by zyga> <https://launchpad.net/bugs/1866855>
[08:05] <zyga> pedronis: ack, I can check on my nvidia box
[08:05] <zyga> thank you
[08:05] <zyga> I'll look at it to evaluate after maas bug
[08:06] <zyga> I suspect it might go away if we update snapd in debian though
[08:06] <zyga> probably a worthy goal anyway
[08:06] <zyga> mvo: debian question, is debian in some sort of freeze? new queue is full of things that are many weeks old
[08:06] <zyga> or is that just shortage of ftp masters?
[08:06] <zyga> re about focal: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1866844
[08:07] <mup> Bug #1866844: package libc6:amd64 2.31-0ubuntu3 failed to install/upgrade: installed libc6:amd64 package post-installation script subprocess returned error exit status
[08:07] <mup> 127 <amd64> <apport-package> <focal> <package-from-proposed> <glibc (Ubuntu):Fix Released> <glibc (Ubuntu Focal):Fix Released> <https://launchpad.net/bugs/1866844>
[08:07] <zyga> I got affected, need to spend some time to fix my laptop so that it boots normally again
[08:07] <zyga> I see it's got fixed now, that's really good
[08:09] <pedronis> mvo: one you have a moment: https://bugs.launchpad.net/snapd/+bug/1867415 is this medium or high?
[08:09] <mup> Bug #1867415: Hardlinking snaps wastes 400 MB tmpfs RAM in live CDs <snapd:Triaged> <https://launchpad.net/bugs/1867415>
[08:09] <zyga> pedronis: note, we can use statfs to detect overlayfs and switch to symlinks
[08:09] <zyga> it sucks we only learned this the hard way so close to release date :/
[08:20] <pedronis> what was the conclusion on this at the sprint: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1865519 ?
[08:20] <mup> Bug #1865519: apparmor depends on python3 <snapd:Triaged> <apparmor (Ubuntu):New> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1865519>
[08:20] <zyga> pedronis: I think jamie said that it's not bad that python3 is a dependency but I don't remember if the people in the room had another consistent opinion
[08:21] <pedronis> anyway is not a snapd bug strictly
[08:24] <pedronis> why is this bug is saying install/update, when it looks like a purge op: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1866932 ?
[08:24] <mup> Bug #1866932: package snapd 2.44~pre1+20.04 failed to install/upgrade: installed snapd package post-removal script subprocess returned error exit status 1 <amd64> <apport-package> <focal> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1866932>
[08:24] <mborzecki> morning
[08:25] <mborzecki> mvo: got logs from the postrm test?
[08:28] <mvo> mborzecki: yes, one sec
[08:28] <mvo> mborzecki: I was initially suspecting a race, i.e. the output not quite in sync
[08:28] <mvo> mborzecki: but my naive (late night) adding a sleep did not help at all so it's probably something else
[08:30] <mvo> mborzecki: https://api.travis-ci.org/v3/job/662738604/log.txt
[08:30] <mvo> mborzecki: + grep -E 'snap\..*\.(service|timer|socket)'
[08:30] <mvo> â— snap.test-snapd-service-stop-timeout.svc.service                                            not-found failed failed    snap.test-snapd-service-stop-timeout.svc.service
[08:30] <mvo> + echo 'found unexpected leftovers'
[08:30] <mvo> found unexpected leftovers
[08:30] <mvo> + exit 1
[08:30] <mborzecki> ha a different service this time
[08:31] <mborzecki> mvo: should be a simple fix
[08:31] <mvo> mborzecki: oh, ok
[08:33] <pedronis> zyga: this is not fixed yet https://bugs.launchpad.net/snapd/+bug/1821302 right?  I'm marking it just high
[08:33] <mup> Bug #1821302: /usr/lib/snapd unmounted in app + core18 + core (tooling) use case; possibly on core18 refresh <snapd:Triaged by zyga> <https://launchpad.net/bugs/1821302>
[08:34] <zyga> pedronis: correct
[08:34] <mvo> zyga: did suse upgrade to libc6 2.31 too? I suspect it's actually the libc6 update that breaks the tests (and I think I have a fix, just making sure I understand it)
[08:34] <mborzecki> mvo: well, maybe not that simple though, it's the same thing as i investigated last week, there's systemd service state leaking between the tests, even though we call systemctl reset-failed in restore :/
[08:34] <zyga> mvo: I don't know, let me check
[08:34] <mvo> mborzecki: yeah, that part confused me
[08:34] <mvo> mborzecki: it looks like we explicitly reset it :(
[08:35] <mvo> mborzecki: hence suspecting the race
[08:35] <mborzecki> mvo: and it's on sid, before the problem was seen on arch
[08:35] <zyga> mvo: but I would expect so, tumbleweed rolls weekly
[08:36] <zyga> booting now
[08:37] <mborzecki> mvo: systemd 245 here on arch, same in sid
[08:37] <mborzecki> zyga: what's the current version of systemd in tw?
[08:37] <zyga> mborzecki: still updating, give me a moment
[08:38] <pedronis> mvo: again, once you have a moment:  should this be won't-fixed also on the package: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1673247
[08:38] <mup> Bug #1673247: package snapd 2.23.1 failed to install/upgrade: trying to overwrite '/etc/apparmor.d/usr.lib.snapd.snap-confine', which is also in package snap-confine 2.23.1 <amd64> <apport-package> <package-conflict> <patch> <xenial> <snapd:Won't Fix> <dpkg (Ubuntu):In Progress> <snapd (Ubuntu):In
[08:38] <mup> Progress by mvo> <dpkg (Ubuntu Trusty):Confirmed> <snapd (Ubuntu Trusty):In Progress> <dpkg (Ubuntu Xenial):Confirmed> <snapd (Ubuntu Xenial):Confirmed> <dpkg (Ubuntu Yakkety):Invalid>
[08:38] <mup> <snapd (Ubuntu Yakkety):Invalid> <dpkg (Ubuntu Zesty):Confirmed> <snapd (Ubuntu Zesty):Confirmed for slavikstar> <https://launchpad.net/bugs/1673247>
[08:41] <mvo> mborzecki: focal has systemd 244, so that's the most recent we have
[08:42] <mvo> pstolowski: I opened it now and once I'm done with the test issue will look in detail
[08:42] <mvo> pedronis: ^-- (and thanks for the pointer)
[08:42] <mvo> pstolowski: ups, sorry
[08:44] <mvo> zyga: according to their website also glibc 2.31, I think that's the one that breaks things for us, fixing now
[08:50] <pedronis> pstolowski: hi, maybe you can review #8264 when you have a moment
[08:50] <mup> PR #8264: many: introduce snapdenv to present common snapd env options <Created by pedronis> <https://github.com/snapcore/snapd/pull/8264>
[08:50] <pstolowski> pedronis: sure
[08:54] <zyga> mborzecki: 244
[08:54] <zyga> zyga@yantra:~> systemctl --version
[08:54] <zyga> systemd 244 (+suse.138.gf8adabc2b1)
[08:54] <zyga> +PAM -AUDIT +SELINUX -IMA +APPARMOR -SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 -IDN +PCRE2 default-hierarchy=hybrid
[08:54] <zyga> mvo: glibc version 2.31-2.1
[08:56] <mvo> zyga: ta
[08:57] <mborzecki> zyga: thx
[09:08] <mup> PR snapd#8265 opened: tests: add regression test for MAAS refresh bug <Created by zyga> <https://github.com/snapcore/snapd/pull/8265>
[09:08] <zyga> so some good news
[09:08] <zyga> if mborzecki can help me with selinux denials
[09:08] <mborzecki> hm?
[09:08] <zyga> I can fix the bug affecting maas by just enabling robust mount ns updates
[09:09] <mborzecki> i shuld hijack my wife's fedora laptop for a couple of days and try to weed out all the denials if there's any left
[09:09] <zyga> https://github.com/snapcore/snapd/pull/8089 is critical
[09:10] <mup> PR #8089: features: enable robust mount ns updates <Created by zyga> <https://github.com/snapcore/snapd/pull/8089>
[09:10] <zyga> as in, fixing those denials will really improve snap experience for people
[09:11] <mborzecki> i investigated one on thursday evening (because there's better to do on a day off), and i think that desktop-helpers were causing a problem by doing cp -a on user-dirs.locale
[09:11] <zyga> mborzecki: not all denials are equal
[09:11] <zyga> if we just get over the hump that allows 8089 to land
[09:11] <zyga> it will mean a lot
[09:13] <mborzecki> omg, reproduced the problem with postrm-purge in a spread run where i forgot to add -debug /o\
[09:13] <zyga> mborzecki: ouch :|
[09:13] <zyga> btw, it failed for me today
[09:14] <pedronis> zyga: btw, I did work on 8242 end of last week
[09:14] <zyga> yeah, I noticed, I didn't review the changes yet but I plan to soon
[09:15] <zyga> thank you :)
[09:15] <zyga> I looked at snapdenv just now and it looks good
[09:15] <zyga> it's a much needed cleanup
[09:17] <pedronis> yea, though it was an indirect reminder that store (and daemon) are still our least loved code
[09:19] <mvo> fwiw, I will work more on 8266 but wanted to unblock you guys
[09:19] <mup> PR snapd#8266 opened: snap-seccomp: allow mprotect() to unblock the tests <Created by mvo5> <https://github.com/snapcore/snapd/pull/8266>
[09:28] <mborzecki> heh, so with new systemd, there's a unicode dot printed in the sssytemctl line
[09:28] <mborzecki> ● snap.test-snapd-service-stop-timeout.svc.service not-found failed failed snap.test-snapd-service-stop-timeout.svc.service
[09:28] <mborzecki> ofc it doesn't match the awk pattern we search
[09:29] <pedronis> fun, otoh we are culpable of those kind of changes to info or list, so can't blame
[09:30] <mborzecki> heh
[09:33] <pedronis> mborzecki: mvo: on the UC20 uboot front, I worked a bit on it and #8228 needs 2nd reviews
[09:33] <mup> PR #8228: boot,image: ARM kernel extract prepare image <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8228>
[09:35] <mvo> pedronis: nice, thank you!
[09:35] <mborzecki> pedronis: we got ourselves a deadlock in the reviews of that PR, looks like each of us pushed a bit there :)
[09:38]  * mvo will look at this
[09:39] <pedronis> mborzecki: well, I think if both mvo and you look at it again, we are ok
[09:44] <pedronis> mborzecki: mvo: it's missing a unit test though, but that can come in a follow up, anyway there's the todo and rename as well
[09:49]  * mvo is in a meeting but will look at the PR after that
[09:49] <mborzecki> hmm reset-failed does not clear the status though, the unit is still in not-found/failed state, wtf?
[09:49] <mborzecki> maybe someone in #systemd has a clue
[09:59] <zyga> mborzecki: sergio said he needs to reset twice
[10:00] <mborzecki> reset what? the service?
[10:00] <zyga> reset failed
[10:00] <zyga> just do it twice
[10:00] <mborzecki> google:debian-sid-64 .../tests/main/postrm-purge# systemctl reset-failed --force snap.test-snapd-service.test-snapd-service-refuses-to-stop.service
[10:00] <mborzecki> Failed to reset failed state of unit snap.test-snapd-service.test-snapd-service-refuses-to-stop.service: Unit snap.test-snapd-service.test-snapd-service-refuses-to-stop.service not loaded.
[10:00] <mborzecki> google:debian-sid-64 .../tests/main/postrm-purge# systemctl reset-failed --force snap.test-snapd-service.test-snapd-service-refuses-to-stop.service
[10:01] <mborzecki> Failed to reset failed state of unit snap.test-snapd-service.test-snapd-service-refuses-to-stop.service: Unit snap.test-snapd-service.test-snapd-service-refuses-to-stop.service not loaded.
[10:01] <mborzecki> zyga: ^^ :)
[10:01] <zyga> not loaded
[10:01] <zyga> but does it list as failed?
[10:01] <zyga> weird
[10:01] <mborzecki> yup
[10:01] <zyga> maybe deamon reload and then?
[10:01] <mborzecki> however, i can do `systemctl reset-failed` (no params, resets all units) and the status is cleared
[10:01] <zyga> huh
[10:01] <zyga> magic
[10:01] <zyga> or
[10:01] <zyga> maybe the unit is really gone
[10:02] <zyga> and it cannot be referred to by name anymore
[10:02] <zyga> and reset-failed can clear "unreachable" units somehow
[10:02] <mborzecki> it is, but we were able to reset-failed a unit that was not found before
[10:07] <zyga> ackk: hey
[10:07] <zyga> ackk: some good news :)
[10:08] <ackk>  hi zyga, I saw :)
[10:08] <zyga> ackk: please try that and let me know if you run into any issues having that enabled
[10:09] <ackk> zyga, how do I enable?
[10:09] <zyga> ackk: like this https://github.com/snapcore/snapd/pull/8265/files#diff-09aaa35036aab4cfb8a7a8ae90869822R14
[10:09] <mup> PR #8265: tests: add regression test for MAAS refresh bug <Created by zyga> <https://github.com/snapcore/snapd/pull/8265>
[10:09] <ackk> oh I see, thanks
[10:10] <ackk> zyga, I'll let you know if I hit the issue again, thanks
[10:10] <zyga> ackk: I just wanted to say I'm very grateful for the bug report with information on how to reproduce
[10:10] <ackk> zyga, is the plan to enable the feature by default in the next release?
[10:10] <zyga> ackk: I know this was plaguing you for many weeks now
[10:11] <zyga> ackk: yes, we just need to adjust the selinux policy because it currently blocks the branch that enables this from landing
[10:11] <ackk> zyga, no problem, happy that it was helpful :)
[10:11] <ackk> zyga, thank you for jumping on it
[10:11] <zyga> my pleasure :) I really like making things robust
[10:13] <ackk> heh
[10:14] <mup> PR snapd#8267 opened: tests/lib/reset: workaround unicode dot in systemctl output <Simple 😃> <Test Robustness> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8267>
[10:16] <mborzecki> now i can't reproduce the reset-failed problem on sid, wth?
[10:16] <mborzecki> anyways, mvo ^^ 8267 shoudl fix the postrm-purge and snap-mgmt tests
[10:16] <mvo> mborzecki: nice, thank you
[10:25] <pedronis> mborzecki: then maybe more than it alone needs to reset-failed
[10:39] <mborzecki> zyga: suse is using /var/lib/snapd/snap too?
[10:39] <zyga> nope
[11:06] <ackk> mvo, hi WRT https://bugs.launchpad.net/maas/+bug/1867571 (ISTR we talked about the issue in the past) looking at snapd core it seems there is an apparmor rule allowing read access to /run/uuidd/request, is the denial because of the "w" ?
[11:07] <mup> Bug #1867571: apparmor denied warning for /run/uuidd/request <MAAS:Triaged> <https://launchpad.net/bugs/1867571>
[11:14] <mvo> ackk: in a meeting right now, will get back to you
[11:15] <ackk> mvo, np, thanks
[11:15] <zyga> ackk: looks like it
[11:15] <zyga> ackk: let me check snapd to be ure
[11:15] <zyga> *sure
[11:16] <ackk> zyga, fwiw it seems you need write access too, as libuuid does write to the socket to get the uuid
[11:16] <ackk> I'm not sure of the protocol
[11:16] <zyga> ackk: you get write access with openvswitch-support
[11:16] <zyga> ackk: everyone else get s read access by default
[11:16] <ackk> zyga, right, but read access is probably useless?
[11:17] <zyga> I don't know
[11:17] <zyga> I suspect not
[11:17] <zyga> it is a socket
[11:17] <zyga> perhaps just reading is enough?
[11:18] <ackk> zyga, https://paste.ubuntu.com/p/kfNs4pkMyS/ is what "uuidgen -t" does
[11:18] <zyga> ackk: as I said, I don't know
[11:18] <zyga> one would have to read the source of uuidd
[11:18] <zyga> and check what's the protocol
[11:25] <mvo> the mprotect() fix failed aparently in non-ubuntu, does anyone have details? if it's something silly like the postrm-purge failure I can merge and override the failure
[11:25] <mvo> but it looks like I can't see the reason for the restart of the testrun unfortunately
[11:26] <zyga> wasn't me
[11:26] <sparkiegeek> appreciate y'alls thoughts on https://forum.snapcraft.io/t/uuidd-apparmor-denial/16013
[11:27] <ackk> zyga, looking at https://github.com/karelzak/util-linux/blob/master/libuuid/src/uuidd.h#L43-L49 you have to write the type of operation you want, so you need write access
[11:28] <ackk> zyga, I suspect the current rule isn't really working (although libuuid will still work as it falls back to local getrandom() if the socket fails)
[11:28] <zyga> ackk: I see
[11:28]  * ackk adds comment to sparkiegeek's post
[11:28] <zyga> pedronis: I reviewed 8264
[11:28] <zyga> ackk, sparkiegeek: thank you
[11:28] <zyga> I suspect it's a jamie topic
[11:28] <zyga> I can change stuff but he needs to ultimately review that anyway
[11:31] <ackk> zyga, can we test locally by hacking apparmor rules for the maas snap? (adding "w" support)
[11:31] <zyga> yes
[11:31] <zyga> always :)
[11:31] <ackk> zyga, how do I do that?
[11:31] <zyga> ackk: you need to edit the appropriate text file
[11:31] <zyga> they are stored in /var/lib/snapd/apparmor/profiles
[11:31] <zyga> and are named after the snap and application name
[11:32] <zyga> the correct one should be easy to locate
[11:32] <zyga> then you need to locate the line that mentions uuidd
[11:32] <zyga> and change "r," to "rw," - mind the comma, they are relevant
[11:32] <zyga> and issue a command to compile and insert the profile into the kernel
[11:32] <zyga> sudo apparmor_parser -r /path/to/that/profile/file
[11:32] <zyga> mind you that snapd will regenerate profiles for many operations
[11:32] <zyga> so your changes may get undone
[11:33] <zyga> let me know if you need ehlp
[11:33] <zyga> *help
[11:35] <ackk> zyga, that seems to fix the issue
[11:35] <ackk> sparkiegeek, ^
[11:35] <sparkiegeek> ackk: ack
[11:40] <mup> PR snapd#8266 closed: snap-seccomp: allow mprotect() to unblock the tests <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8266>
[11:40] <mup> PR snapd#8267 closed: tests/lib/reset: workaround unicode dot in systemctl output <Simple 😃> <Test Robustness> <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8267>
[11:45] <pedronis> zyga: thanks, I tried to answer your questions
[11:46] <zyga> looking
[11:49] <pedronis> zyga: also notice that the description has a list of TODOs for follow-ups already, this PR is already quite big given what is doing
[11:49] <zyga> ok
[11:49] <zyga> ah
[11:49] <zyga> I missedthe PR message somehow
[11:50] <pedronis> the main motivation for this is actually PreseedMode, but yes ideally all SNAPD_/SNAPPY_ vars would be handled like this, buit is quite a bit of work that is hard to justify atm
[11:51] <pedronis> at least we have a place now
[11:51] <zyga> pedronis: LGTM, happy to see progress in this area :)
[12:02] <jdstrand> sparkiegeek: note that 8537ba5b2816ac1a2f77dc03ba709947b11a2531 added it to snap-update-ns profile. it was only in 2049f2c8245da1d97c96fcc76bab871eb0491d23#diff-a34e166c5b3016c122430c5884f41e9b from last week where it was added to the default template.
[12:03] <cachio> mvo, hey
[12:03] <jdstrand> it still only had 'r' though
[12:03] <sparkiegeek> jdstrand: thanks
[12:03] <jdstrand> has*
[12:03] <mup> PR snapd#8268 opened: snap-seccomp: robustness improvements <Created by mvo5> <https://github.com/snapcore/snapd/pull/8268>
[12:04] <cachio> mvo, snapd,failure is failing also on uc18 na uc20, on PR #8252
[12:04] <mup> PR #8252: tests: update test to make snapd snap fixed twice <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/8252>
[12:05] <ijohnson> guten morgen from the quarantine :-)
[12:06] <sparkiegeek> jdstrand: ninja-edit'd post to reference the correct commit
[12:06] <mborzecki> pedronis: added StoreAcount in #8246
[12:07] <mup> PR #8246: client, daemon, overlord/devicestate: structures and stubs for systems API <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8246>
[12:17] <cachio> Saviq, hey
[12:17] <Saviq> cachio: oi!
[12:18] <cachio> I plan to delete fedora 28/9 images from gce, are you using them ?
[12:18] <pedronis> mborzecki: reviewed
[12:18] <cachio> I need to clean up a bit
[12:18] <mborzecki> pedronis: thanks!
[12:18] <Saviq> cachio: fire away https://github.com/MirServer/mir/blob/master/.travis.yml#L47
[12:19] <cachio> nice, thanks
[12:19] <Saviq> or more importantly https://github.com/MirServer/mir/blob/master/spread.yaml#L5
[12:20] <pedronis> ijohnson: hello, I pushed a bit forward your PRs on Friday
[12:20] <ijohnson> hey pedronis
[12:20] <ijohnson> Great, I see some of them were merged
[12:21]  * zyga breaks for coffee
[12:21] <zyga> or actually
[12:22] <zyga> maybe I'll grab lunch and then coffee
[12:22] <zyga> ahead of the standup
[12:26] <mup> PR snapd#8269 opened: apparmor: add rw for uuidd request to default and remove from snap-update-ns <Created by jdstrand> <https://github.com/snapcore/snapd/pull/8269>
[12:27] <jdstrand> sparkiegeek, ackk: fyi, ^
[12:28] <ackk> jdstrand, thanks!
[12:28] <sparkiegeek> jdstrand: thank you
[12:28] <ackk> jdstrand, fwiw the openvswitch interface also grants that, so it could maybe be dropped from there?
[12:29] <pedronis> zyga: ijohnson: #8258 needs your reviews
[12:29] <mup> PR #8258: interfaces/kubernetes-support: allow autobind to journald socket <Created by jdstrand> <https://github.com/snapcore/snapd/pull/8258>
[12:30] <ijohnson> Yes it's on my list thanks pedronis
[12:31] <pedronis> mvo: #8249 needs a master merge?
[12:31] <mup> PR #8249: interfaces: make gpio robust against not-existing gpios in /sys <Reviewed> <Security-High> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8249>
[12:31] <pedronis> pstolowski: #8235 probably also needs master merged in
[12:31] <mup> PR #8235: cmd/snap-preseed: handle --reset flag <Preseeding 🍞> <Squash-merge> <Created by stolowski> <https://github.com/snapcore/snapd/pull/8235>
[12:32] <pstolowski> pedronis: ack, doing
[12:32] <dot-tobias> zyga: hey there 😊 Short ping about https://bugs.launchpad.net/snapd/+bug/1844496/comments/8
[12:32] <mup> Bug #1844496: “Device or resource busy” error during snap refresh when using layout with variable <snapd:Fix Committed by zyga> <https://launchpad.net/bugs/1844496>
[12:34] <popey> mvo is something blocking an update to snapd in debian? I have snaps which cannot be installed because they need 2.43
[12:35] <mborzecki> mvo: can you take a look at 8246?
[12:36] <mvo> popey: nothing is blocking, I wanted to upload 2.44 once out (this week), re-exec should handle most cases though, I can put this on higher priority
[12:36] <popey> Oh, debian has re-exec?
[12:36] <mvo> mborzecki: yes, once the fires are out
[12:36] <mvo> popey: yes, should be
[12:37] <ijohnson> hey mvo!
[12:37] <popey> ah okay. Is it locked to debian via os-release or somesuch?
[12:37] <mvo> hey ijohnson, welcome back
[12:37] <ijohnson> what's on fire today :-)
[12:37] <popey> Because I have seen reports of outdated snapd on debian derivatives, which suggest they don't get re-exec
[12:37] <mvo> popey: iirc yes, I don't remember the details
[12:37] <jdstrand> mvo: this is the same bug we saw on trusty: snapd snap not getting pulled down
[12:37] <popey> hm, ok
[12:37] <mvo> popey: aha, interessting
[12:37] <mvo> jdstrand: aha, ok
[12:37] <popey> Seen it on the forums mentioned a bit
[12:37] <mvo> popey: for some stuff we need an update
[12:38] <mvo> popey: I will do a new upload
[12:38] <popey> <3 thanks
[12:38] <jdstrand> ackk: ok, yes, done
[12:38] <jdstrand> ackk: (updated the PR)
[12:38] <popey> e.g. snapctl is-connected is in 2.43, I would *love* that :D
[12:38] <mvo> ijohnson: tests were failing left and right
[12:38] <popey> I'll leave you alone, thanks mvo
[12:39] <ijohnson> mvo: :-/ anything I should look at right away? I'm currently trying to triage all my notifications/PR requests and things
[12:39] <mvo> ijohnson: that's fine, I think test are good again
[12:39] <ijohnson> cool
[12:59] <mborzecki> zyga: this is what i have https://paste.ubuntu.com/p/jXh8yNzGqQ/
[13:00] <cjwatson> popey: Certainly works for me on Debian stable
[13:00] <cjwatson> dpkg -l snapd says 2.37.4-1+b1, snap version says 2.43.3
[13:06] <mborzecki> zyga: should I push a patch with policy update to your PR?
[13:10] <pedronis> jdstrand: is #8269 meant for 2.45 or 2.44 ?
[13:10] <mup> PR #8269: apparmor: use rw for uuidd request to default and remove from elsewhere <Created by jdstrand> <https://github.com/snapcore/snapd/pull/8269>
[13:11] <zyga> mborzecki: please!
[13:12] <zyga> pedronis: ack, after lunch
[13:12] <zyga> dot-tobias: replied
[13:14] <dot-tobias> zyga: Thanks!
[13:15] <mborzecki> zyga: hmm any clue why snapd would trigger this on remove? list_dirs_pattern(snappy_mount_t, snappy_var_lib_t, snappy_var_lib_t)
[13:15] <mborzecki> list_dirs_pattern(snappy_mount_t, snappy_var_lib_t, snappy_var_lib_t)
[13:15] <mborzecki> pff
[13:15] <mborzecki> type=AVC msg=audit(1584364147.373:402): avc:  denied  { getattr } for  pid=2756 comm="snapd" path="/run/user/1000/snap.snap-store/dconf" dev="tmpfs" ino=63262 scontext=system_u:system_r:snappy_t:s0 tcontext=unconfined_u:object_r:config_home_t:s0 tclass=dir permissive=1
[13:15] <mborzecki> this ^^
[13:15] <mborzecki> what's snapd doing with /run/user/<id>/snap.<snap> ?
[13:16] <zyga> I don't know off hand
[13:16] <zyga> but it is dconf
[13:16] <zyga> perhaps that is managed by something (helpers or interface)
[13:17] <pedronis> mborzecki: zyga: is simply space that we let the snap use afaict
[13:18] <mborzecki> pedronis: that i know, but i'm wondering why we poke anything under /run/user/<uid>/snap.<snap>/ it when removing a snap
[13:18] <mup> PR snapcraft#2975 opened: specifications: experimental features guidelines <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2975>
[13:21] <pedronis> mborzecki: looking, one sec
[13:23] <pedronis> mborzecki: yes, RemoveSnapCommonData removes filepath.Glob(snap.XdgRuntimeDirs()) afaict
[13:24] <mup> PR core20#26 closed: Disable emergency.target & debug-shell, unless kernel cmdline is dangerous <Created by xnox> <Merged by xnox> <https://github.com/snapcore/core20/pull/26>
[13:24] <mborzecki> pedronis: thanks!
[13:25] <mup> PR snapd#8270 opened: store: support for search API v2 <Needs Samuele review> <Created by stolowski> <https://github.com/snapcore/snapd/pull/8270>
[13:26] <mborzecki> ok, so that explains getattr (caused by globbing probbaly) and then unlinking
[13:34] <mborzecki> zyga: couple more tweaks to the policy
[13:34] <zyga> mborzecki: sure, I will check soon
[13:34] <mborzecki> zyga: np, haven't pushed yet, want to make sure it builds on centos7 too
[13:34] <mborzecki> there's some interfaces that may not be available there :/
[13:35] <jdstrand> pedronis: 2.45
[13:35] <jdstrand> pedronis: I mean, it could be 2.44, but I'm trying to actually let you release the thing :)
[13:35] <jdstrand> pedronis: and nothing is actually broken, it is just noise
[13:36] <jdstrand> (since fallback behavior allows things to work ok)
[13:37]  * mvo had the same question
[13:38] <jdstrand> the k8s one I did on friday though would be good to have in 2.44
[13:38] <jdstrand> actually, I think it was thursday, but yeah
[13:39] <mup> PR snapd#8258 closed: interfaces/kubernetes-support: allow autobind to journald socket <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8258>
[13:39] <pedronis> jdstrand: I asked people to review the k8s one
[13:40] <mup> PR snapd#8259 closed: interfaces/kubernetes-support: allow autobind to journald socket - 2.44 <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8259>
[13:42] <jdstrand> pedronis: I saw, thanks! :)
[13:55] <ppd> @zyga: If you need a tester for https://bugs.launchpad.net/snapd/+bug/1866855, I'm in the office @ my nvidia machine
[13:55] <mup> Bug #1866855: nvidia driver integration is incompatible with Debian <snapd:Triaged by zyga> <https://launchpad.net/bugs/1866855>
[13:55] <ijohnson> pedronis: re the unit test you mentioned https://github.com/snapcore/snapd/pull/8228#discussion_r392221605 should I add that before we merge that?
[13:55] <mup> PR #8228: boot,image: ARM kernel extract prepare image <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8228>
[13:56] <pedronis> ijohnson: not particularly if it get green before, anyway I think we should do the todo as well
[13:56] <ijohnson> ok
[13:56] <pedronis> in a follow up
[14:00] <zyga> ppd: thank you, I need to debug the issue first
[14:00] <zyga> ppd: I have nvidia and debian at home
[14:00] <zyga> ppd: I plan to switch to that soon
[14:02] <mup> PR core20#25 closed: hooks/200-console-conf-after.chroot: perform console-conf ordering checks <Created by bboozzoo> <Merged by xnox> <https://github.com/snapcore/core20/pull/25>
[14:02] <mborzecki> xnox: yay
[14:03] <mup> PR snapcraft#2975 closed: specifications: experimental features guidelines <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2975>
[14:14] <mup> PR snapd#8271 opened: interfaces: add hugepages-control <Created by alfonsosanchezbeato> <https://github.com/snapcore/snapd/pull/8271>
[14:43] <zyga> mvo: and now it is stuck on considering re-refresh
[14:43] <zyga> and now it completed
[14:43] <zyga> so on 2nd try it did towrk
[14:43] <zyga> *did work
[14:43] <mvo> zyga: so racy :(
[14:44] <zyga> huh, but the snap change says it worked
[14:44] <zyga> what?
[14:45] <zyga> ah, no
[14:45] <zyga> just lots of changes
[14:45] <zyga> but I see the error
[14:46] <mvo> zyga: could you attach it to the bug with the snapd logs please?
[14:46] <mvo> zyga: both the bad and good case
[14:46] <mvo> zyga: (if you haven't already)
[14:46] <zyga> yeah
[14:46] <zyga> done now
[14:46] <zyga> looking at journal logs for more details
[14:46] <zyga> mborzecki: ^
[14:46] <zyga> gosh why doesn't launchpad do markdown
[14:46] <zyga> and why does it wrap text
[14:47] <zyga> I have 60%+ whitespace on my screen
[14:47] <zyga> because LP wraps everything to 800x600 or something :/
[14:48] <zyga> logs added
[14:48] <zyga> mborzecki:
[14:48] <zyga>  Mar 16 15:37:40 $HOSTNAME systemd[1]: snapd.service: Current command vanished from the unit file, execution of the command list won't be resumed.
[14:49] <zyga> what the :) ?
[14:50] <zyga> I'll switch gears to Debian now
[14:50] <zyga> if you need anything just ping me please
[15:00] <cachio> ijohnson, hey, this is the pr for the snapd.failure serivce #8252
[15:00] <mup> PR #8252: tests: update test to make snapd snap fixed twice <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/8252>
[15:00] <cachio> if you want to add the fix there should be great
[15:08] <zyga> mborzecki: how can I toggle gadget asset writes?
[15:08] <zyga> er trigger
[15:16] <mup> PR snapd#8272 opened: data/selinux, tests/main/selinux: cleanup tmpfs operations in the policy, updates <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8272>
[15:16] <mborzecki> zyga: ^^
[15:16] <zyga> looking
[15:16] <zyga> and *thank you8
[15:16] <zyga> * :)
[15:17] <mborzecki> zyga: can you cherry pick the patch to your PR and see if it's ok now?
[15:17] <zyga> mborzecki: sure
[15:17] <zyga> first or both?
[15:18] <mborzecki> zyga: i've only tried it locally with experimental.robust-mount-namespace-updates=true, the default ran on spread
[15:18] <zyga> ok, I'll try
[15:21]  * cachio lunch
[15:36] <mborzecki> zyga: hm afaict that systemd line is because we replace the snapd service file, so the file on disk is no longer the same as it was when the service was started
[16:00] <pedronis> mborzecki: what does command vanished though?
[16:00] <pedronis> *mean
[16:06] <zyga> I’m helping with kids upstairs. Afk
[16:07] <zyga> First few days are hard
[16:28] <mup> PR snapcraft#2976 opened: cli: formalize developer debug as hidden build option <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/2976>
[16:35] <mborzecki> pedronis: afair from the last time we investigated this, this happens when there's a reload, systemd does serialze/deserialize sequence, and the service unit it serialized does not match the one that was read again from disk
[16:35] <mborzecki> pedronis: though in the case of this bug, the problem appears to be elsewhere imo
[16:35] <ijohnson> mvo: I looked into extracting just the bit from #8169 that drops RemainsAfterExit=true from snapd.failure.service, but that requires a bit of tests changes that would also need to be pulled out
[16:35] <mup> PR #8169: tests/many: don't use StartLimitInterval anymore, unify snapd-failover variants, build snapd snap for UC16 tests <Test Robustness> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8169>
[16:35] <ijohnson> mvo: I can certainly pull those out into a PR if you like, but it may be easier to just review the PR as it is IMHO
[16:35] <mborzecki> pedronis: zyga: it unexpectedly exits with an error when a restat was requested: Mar 16 12:50:29 $HOSTNAME snapd[680]: cannot run daemon: context deadline exceeded
[16:36] <ijohnson> let me know how you want to procee
[16:36] <ijohnson> d
[16:36] <mborzecki> whgat triggers the rollback
[16:36] <mborzecki> pedronis: zyga: afaict this is caused by a graceful shutdown of the snapd.socket ?
[16:36] <mborzecki> 	d.tomb.Kill(d.serve.Shutdown(ctx))
[16:37] <zyga> mborzecki: I don't know w
[16:37] <zyga> I have my laptop upstrairs and will try to resume work
[16:38] <pedronis> mborzecki: what has changed in this area though ?
[16:40] <mborzecki> pedronis: in that particular daemon shutdown code nothing spectacular, last change was in june 2019
[16:41] <zyga> I think maciek had a good hunch
[16:42] <zyga> the device agent is poking snapd api
[16:42] <zyga> so it may be interacting with snapd as it shuts down
[16:42] <zyga> we may need the /etc/nologin equivalent for the API
[16:42] <pedronis> but we close our sockects
[16:42] <pedronis> before getting there
[16:42] <zyga> are we closing the snapd.socket?
[16:42] <zyga> as in
[16:42] <zyga> are we preventing socked-based activation?
[16:42] <zyga> we may shut ourselves down
[16:43] <zyga> but unless we do it in a way that doesn't disable activation
[16:43] <mborzecki> pedronis: so looking at http.Server.Shutdown(), it loops waiting for all connections to become idle
[16:43] <pstolowski> pedronis: #8264 can land
[16:43] <mup> PR #8264: many: introduce snapdenv to present common snapd env options <Created by pedronis> <https://github.com/snapcore/snapd/pull/8264>
[16:43] <zyga> it may be that a constant ping is enough to break frefresh
[16:43] <mup> PR snapcraft#2977 opened: cli: merge build options into provider options <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/2977>
[16:43] <zyga> *refresh
[16:43] <mborzecki> heh, more, the shutdown timeout we use is 25s, the logs are 25s apart :P
[16:43] <mborzecki> Mar 16 12:50:04 $HOSTNAME snapd[680]: daemon.go:539: done waiting for running hooks
[16:43] <mborzecki> Mar 16 12:50:29 $HOSTNAME snapd[680]: cannot run daemon: context deadline exceeded
[16:44] <pedronis> are we seeing snapd restarts in the middle though?
[16:44] <pedronis> anyway at that point we closed the sockets but are still running
[16:44] <pedronis> so systemd shouldn't start a new version of snapd
[16:44] <pedronis> afaiu
[16:45] <mborzecki> pedronis: but there's a failure instead of clean restart
[16:45] <pedronis> mborzecki: this means that some handler is taking very long
[16:45] <pedronis> I think
[16:46] <mborzecki> pedronis: api endpoint handler right?
[16:46] <mborzecki> pedronis: what if a client is slow to receive the response, would it look the same?
[16:46] <pedronis> yes, or something is keeping a connection alive
[16:46] <pedronis> but I think Shutdown deals with that
[16:46] <pedronis> (our old code did)
[16:46] <pedronis> I can double check for the latter
[16:47] <mborzecki> pedronis: we use the stdlib now, and run it with a context.WithTimer()
[16:47] <mborzecki> when it times out, the error becomes tomb status, and the core that runs later returns that as an error
[16:48] <mborzecki> maybe we should just suck it up if it's a timeout and exit
[16:48] <zyga> I switched the machine on
[16:48] <zyga> checking for changes
[16:48] <zyga> there are changes of the sort "Running service command for snap ..."
[16:49] <zyga> and that has tasks that ... install snapd?
[16:49] <zyga> ah, sorry no
[16:50]  * zyga there are commands but they restart a service of another snap
[16:50] <mborzecki> zyga: i think you need SNAPD_DEBUG=1 to see if anything is poking the api
[16:50] <zyga> aha
[16:53] <zyga> mborzecki: done, I will watch the log
[16:53] <zyga> snapd.failure.service failed on the first restart
[16:54] <zyga> cmd_snapd.go:136: restarting snapd socke
[16:54] <zyga> error:
[16:54] <zyga> -----
[16:54] <mup> PR snapd#8273 opened: interfaces/greengrass-support: add new 1.9 access <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8273>
[16:54] <zyga> Job for snapd.socket failed.
[16:54] <zyga> See "systemctl status snapd.socket" and "journalctl -xe" for details.
[16:54] <zyga> fun
[16:55] <pedronis> mborzecki: took a while but, no, the stdlib code doesn't let reuse a connection once in shutdown
[16:55] <zyga> then
[16:55] <zyga> snapd.socket: Socket service snapd.service already active, refusing.
[16:55] <zyga> Failed to listen on Socket activation for snappy daemon.
[16:55] <pedronis> mborzecki: so it would be a handler<->client taking too long
[16:56] <pedronis> mborzecki: ignoring the timeout is what we do with RestartSystem btw
[16:56] <mup> PR snapd#8274 opened: interfaces/greengrass-support: add new 1.9 access (2.44) <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8274>
[16:57] <mborzecki> pedronis: right, but we only refresh snapd here, so a simple restart
[16:57] <pedronis> yea
[16:59] <mborzecki> hm maybe we should ungracefully call d.serve.Close() when timeout occurred
[16:59] <pedronis> mborzecki: it's the reserve we close the socked below the server ourselves
[16:59] <pedronis> before
[16:59] <mup> PR snapd#8264 closed: many: introduce snapdenv to present common snapd env options <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/8264>
[17:05] <pedronis> pstolowski: thanks,  I'll work on snapdenv.Preseeding next (I'll probably rename PreseedMode to Preseeding in places)
[17:06] <pstolowski> pedronis: great, ty
[17:13] <mup> PR snapd#8235 closed: cmd/snap-preseed: handle --reset flag <Preseeding 🍞> <Squash-merge> <Created by stolowski> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/8235>
[17:15] <pedronis> mvo: ^ that was still targeted for 2.44
[17:20] <pedronis> mborzecki: anyway doing what we do for a system restart seems neccesary, otoh it would still be good to understand what is blocking us for 25s
[17:23] <zyga> mvo: I remembered the gpg key :D
[17:26] <mup> PR snapd#8275 opened: daemon: do a forceful serer shutdown if we hit a deadline <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8275>
[17:26] <mborzecki> pedronis: we could try something like this ^^^
[17:27] <mborzecki> anyways, need to go, kids got their homework, since the school is in lockdown it's e-learning now ;)
[17:28] <mvo> zyga: yay!
[17:28] <mvo> pedronis: yeah, 8235 looks sane for 2.44, no?
[17:28] <pedronis> mvo: yes
[17:28] <zyga> I've pushed my keys to debian and ubuntu
[17:28] <zyga> whee
[17:28] <pedronis> mvo: needs to be cherry picked
[17:28] <zyga> I'm so happy
[17:28] <mvo> pedronis: will do in a sec
[17:35] <mvo> pedronis: thanks, cherry picked now
[17:46] <mup> PR snapd#8276 opened: snap: whitelist lzo as support compression for snap pack <Simple 😃> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8276>
[17:58] <mup> PR snapd#8277 opened: asserts,o/devicestate: support model specified alternative serial-authority <Created by pedronis> <https://github.com/snapcore/snapd/pull/8277>
[17:59] <mup> PR snapd#8278 opened: devicestate: disable cloud-init by default on uc20 <Simple 😃> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8278>
[18:07] <pedronis> mvo: left some comments ^
[18:07] <pedronis> it's almost dinner for me
[19:19] <mvo> pedronis: thanks \o/ ! checking
[19:56] <cachio> mvo, I made uc20 tests work on nested vm
[19:57] <cachio> so no hurry with cloud init
[20:00] <mvo> cachio: !!! how did you do that? amazing \o/
[20:01] <ijohnson> cachio: nice
[20:02] <cachio> mvo, small change on how we modify snapd snap to use the current branch
[20:03] <cachio> mvo, cmatsuoka helped a lot on the process
[20:03] <cachio> I'll add some tests to the branch and pushed again
[20:03] <mvo> thanks cachio and cmatsuoka
[20:03] <cachio> mvo, just a clarification, uc20 works but still the secure boot is not ready on the machine
[20:04] <cachio> so the test which checks the secure boot is currently failing
[20:04] <cachio> but we can boot and check uc on a nested vm with all the tpm machinery
[20:04] <cmatsuoka> cachio did all the stuff, I just tried to boot his images here
[20:05] <cachio> when cmatsuoka branch is landed, that test should pass
[20:05] <mvo> cachio: nice!
[20:05] <mvo> cmatsuoka: and nice job of you too
[20:05]  * mvo hugs cachio and cmatsuoka (totally virus safe!)
[20:05] <cachio> hehehe
[20:06] <cachio> virtual hugs
[20:18] <mup> PR snapd#8279 opened: snap: do not hardlink on overlayfs <Created by mvo5> <https://github.com/snapcore/snapd/pull/8279>
[20:22] <mup> PR snapd#8249 closed: interfaces: make gpio robust against not-existing gpios in /sys <Reviewed> <Security-High> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8249>
[20:24] <ijohnson> cachio: cmatsuoka: which branch should I take a look at to enable uc20 nested tests ?
[20:33] <mup> PR snapd#8280 opened: many: introduce snapdenv.Preseeding instead of release.PreseedMode <Created by pedronis> <https://github.com/snapcore/snapd/pull/8280>
[20:37] <mup> PR snapcraft#2978 opened: tests: add microk8s spread test <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2978>
[20:40] <cmatsuoka> ijohnson:  I was using sergio's tests-enable-nested-on-core20 branch
[20:40] <ijohnson> cmatsuoka: is there a PR open for that?
[20:40] <cmatsuoka> cachio: ^^
[20:41] <cmatsuoka> ijohnson: I think sergio opened it last friday
[20:41] <ijohnson> I'll take a look
[20:41] <ijohnson> looks like https://github.com/snapcore/snapd/pull/8260 is the branch
[20:41] <mup> PR #8260: tests: enable nested on core20 and test current branch <⛔ Blocked> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/8260>
[20:41] <ijohnson> err PR
[20:43] <cmatsuoka> I see I have pending comments there
[20:59] <mup> PR snapcraft#2979 opened: tests: add multipass adhoc spread backend <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2979>
[21:29] <mup> PR snapcraft#2980 opened: vcs: add direnv files to .gitignore <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2980>
[21:31] <popey> wgrant i have another snap which seems to be randomly using snapcraft deb rather than snapcraft snap in build.snapcraft.io :S
[21:32] <popey> https://launchpadlibrarian.net/469283263/buildlog_snap_ubuntu_bionic_s390x_f575d29e94f650a871d01c059930838d_BUILDING.txt.gz
[21:33] <popey> sergiusens have you seen this happening in lp? ^
[21:45] <sergiusens> popey: what triggered this build?
[21:46] <sergiusens> popey: using the wrong snapcraft is sort of out of my hands
[21:46] <sergiusens> unless it is remote-build
[21:47] <sergiusens> if it is build.snapcraft.io triggered, we need to ask Lukewh and verify that they are inadvertingly setting the "snap" fields
[21:47] <popey> it says build trigger unknown
[21:47] <popey> we have a few threads on the forum about it now
[21:48] <sergiusens> popey: seems it is build.snapcraft.io https://launchpad.net/~build.snapcraft.io/+snap/f575d29e94f650a871d01c059930838d/
[21:48] <popey> what would cause this though?
[21:49] <popey> wonder if it's the new build button
[21:50] <sergiusens> popey: well if you are using something new, yeah, most likely, specially since it sends a call to the launchpad APIs, this is where you can incorrectly be setting to NOT use the snap
[21:50] <cjwatson> popey: Has anyone filed a bug against LP about this yet?
[21:50] <popey> no because we couldn't figure out where the bug should be
[21:50] <cjwatson> I don't think it can be a BSI bug
[21:51] <cjwatson> We haven't figured out what's going on, but it's currently hard to see how it could be outside Launchpad (much though I'd like that to be the case)
[21:53] <popey> where should I file a bug?
[21:53] <sergiusens> cjwatson: well, the webteam (I just learned) added a new "build" button which might be calling the api and setting the snap fields incorrectly
[21:53] <cjwatson> popey: https://bugs.launchpad.net/launchpad
[21:54] <sergiusens> cjwatson: would be nice to have launchpad show that info in the logs, I can log a bug about just that :-)
[21:54] <cjwatson> sergiusens: On build.snapcraft.io?  I don't see a recent change like that in its repository
[21:54] <cjwatson> sergiusens: It kind of does if you know how to read it :)
[21:54] <sergiusens> popey: where is this new build button
[21:54] <cjwatson> sergiusens: Or maybe I'm misunderstanding what info you're talking about
[21:55] <popey> login to the store, any snap page and you'll see a new tab next to Listing
[21:55] <cjwatson> Oh, that's been rolled out has it?
[21:55] <popey> yes
[21:56] <cjwatson> That would be the obvious recent change then I suppose
[21:57] <sergiusens> cjwatson: in .requestBuild you can pass a "channels" dict or snaps as keys, channels as values. This is what I suspect is incorrectly being set
[21:57] <cjwatson> sergiusens: Yes but LP is supposed to infer that itself in the case of bases
[21:57] <sergiusens> *dict with
[21:57] <cjwatson> I know about the requestBuild(s) API :)
[21:58] <popey> https://bugs.launchpad.net/launchpad/+bug/1867689
[21:58] <mup> Bug #1867689: snap builds via bsi use the deb rather than snap of snapcraft <Launchpad itself:New> <https://launchpad.net/bugs/1867689>
[21:58] <cjwatson> Thanks - we can redirect if that turns out to be the wrong place
[21:58] <sergiusens> cjwatson: I know you do, I was just trying to be specific of what my thought was on why it could have started failing
[21:58] <cjwatson> If nothing else we could use some better debugging information in our own logs
[21:58] <cjwatson> (internally)
[21:58] <cjwatson> sergiusens: Right, I agree it's a candidate place for things to go wrong, I just haven't been able to work out how that could concretely happen from the code flow
[21:59] <cjwatson> I'll have a look at the new web team changes as soon as I can find where the backend bits for it life
[21:59] <cjwatson> *live
[22:00] <cjwatson> Oh
[22:00] <cjwatson> Hahaha
[22:00] <cjwatson> You may be right
[22:00] <cjwatson>         data = {
[22:00] <cjwatson>             "ws.op": "requestBuilds",
[22:00] <cjwatson>             "channels": "snapcraft,apt",
[22:01] <cjwatson> Why
[22:03] <cjwatson> sergiusens: So yeah, sorry, I didn't expect that they'd have specifically gone out of their way to override this wrongly :-/
[22:05] <sergiusens> hah
[22:05] <popey> oopsie
[22:07] <cjwatson> Filing a bug
[22:09] <popey> thanks cjwatson
[22:09] <mup> PR snapd#8228 closed: boot,image: ARM kernel extract prepare image <UC20> <Created by anonymouse64> <Merged by anonymouse64> <https://github.com/snapcore/snapd/pull/8228>
[22:09] <cjwatson> https://github.com/canonical-web-and-design/canonicalwebteam.launchpad/issues/13
[22:23] <cjwatson> Glad to have found that, that was super-puzzling this morning
[22:26] <popey> heh
[22:29] <mup> PR snapcraft#2981 opened: colcon plugin: rewrite poco absolute library paths <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2981>
[22:59] <nottrobin> thanks for the bug cjwatson. I'll chase it up tomorrow.