[01:01] cachio: around? [01:01] yes [01:02] hey, so I figured out why the refresh fundamentals test has been broken all day [01:02] it got broken by the 2.47 snapd in beta channel today [01:02] and to fix it we need a new kernel snap [01:02] so until that lands, I would like to disable the test [01:02] cachio: so could you quick approve https://github.com/snapcore/snapd/pull/9368? [01:02] PR #9368: tests/nested/manual/refresh-revert-fundamentals: disable temporarily <⚠ Critical> [01:02] ijohnson, sure [01:02] thanks [01:03] +1 [01:03] thanks for that [01:04] thanks, I'll merge it in a few minutes and open a PR undoing the change we can use to monitor until the test is passing again with a new kernel [01:04] have a good night! [01:04] ijohnson, nice, thanks === benfrancis4 is now known as benfrancis [01:04] PR snapd#9368 opened: tests/nested/manual/refresh-revert-fundamentals: disable temporarily <⚠ Critical> [01:09] PR snapd#9369 opened: tests/nested/manual/refresh-revert-fundamentals: re-enable test <⛔ Blocked> [01:19] PR snapd#9368 closed: tests/nested/manual/refresh-revert-fundamentals: disable temporarily <⚠ Critical> === chesty_ is now known as chesty [01:21] * cachio EOD === benfrancis6 is now known as benfrancis [05:20] PR snapd#9370 opened: usersession/userd: separate bus name ownership from defining interfaces [05:49] good morning [05:58] hop hop... [06:14] Bug #1896182 opened: Snapd is waking up my screen from sleep regularly [06:17] morning [06:18] mborzecki: hi [06:18] mborzecki: curious bug [06:18] zyga-x240: hey [06:18] hm, what bug? [06:18] https://bugs.launchpad.net/snappy/+bug/1896182 [06:18] Bug #1896182: Snapd is waking up my screen from sleep regularly [06:18] zyga-x240: heh, i was reading https://bugzilla.redhat.com/show_bug.cgi?id=1880261 [06:18] same thing? [06:19] haha [06:19] ye [06:19] any ideas? [06:19] could that be snapd hotplug logic? [06:33] zyga-x240: https://bugzilla.redhat.com/show_bug.cgi?id=1880261#c3 [06:33] hm long shot but maybe there's something else at play here [06:33] I also replied to the launchpad bug [06:34] hahah [06:34] zyga-x240: noone can say we're not responding to bug reports now [06:34] heh :) [06:34] I pushed https://github.com/snapcore/snapd/pull/9345#pullrequestreview-491167950 [06:34] PR #9345: overlord: introduce the export manager, export snapd tools [06:34] let my grab my wife's laptop and i'll try it there [06:35] I'll squash history and remove some typos and stuff from this later today [06:35] zyga-x240: what's this functionality? sorry, i was uc20 only for the last few weeks [06:40] mborzecki: it's a solution to a very old bug [06:40] mborzecki: introduced pretty much since day one [06:40] mborzecki: when snapd or core refreshes and we don't reboot [06:41] mborzecki: certain tools, like snap-exec and others, as seen by snaps from their mount namespace, become stale [06:41] mborzecki: this fixes that problem by changing how those tools are exposed to snaps [06:41] ah [06:41] mborzecki: snapd allows snaps to export content in a specific way [06:41] mborzecki: the content shows up in /var/lib/snapd/export/// [06:42] mborzecki: there's also a new current symlink at /var/lib/snapd/export//current -> [06:42] there's a special case for core and host [06:43] but in general snapd tools are all available from /usr/lib/snapd/ as symlinks to the export three, through the new current symlink there [06:43] and snapd can update the current symlink atomically [06:43] this also supports using host tools [06:43] so for the first time ever, SNAP_REEXEC=0 uses host's snap-exec and other tools now [06:44] and since this is not related to mount changes, it does not require mount namespace re-configuration when core or snapd change [06:44] there's a new export manager that manages putting content on disk [06:44] and a callback to the snap manager to update the new current symlink when we also update the primary current symlink [06:45] that's pretty much it [06:45] it is relevant to core20 as snapd refreshes separately from boot base [06:45] so we reboot less often [06:45] and we see this bug more and more over time [06:45] mhm, maybe i should take a look at the pull requests then [06:46] heh [06:46] it's not ready for proper review but if you want to look at this, please do [06:46] alternatively [06:46] in udev theme [06:46] please look at [06:46] https://github.com/snapcore/snapd/pull/7614 [06:46] PR #7614: cmd/snap-confine: implement snap-device-helper internally [06:46] you reviewed it [06:46] but maybe we can land it somehow [06:46] and there were changes after your review [06:49] lucy is sleeping but I will go AFK when she wakes up [06:49] I'm so tired today [06:49] 1M -> 6AM [06:49] ENOSLEEP [06:58] morning [06:59] hello [07:08] pstolowski: mvo: morning [07:09] mvo: hello [07:09] hey mborzecki, pstolowski and zyga-x240 ! good morning [07:09] mvo: I will be semi absent in the morning, lucy is still sleeping next to me but she will wake up soon [07:09] mvo: and I was up till midnight and need some rest / break from screens [07:09] zyga-x240: ok [07:09] I pushed https://github.com/snapcore/snapd/pull/9345 but I need to squash, clean up some typos and amend the commit message with a better description [07:09] PR #9345: overlord: introduce the export manager, export snapd tools [07:10] also doing a small self-review showed a few extra changes to make [07:10] I wrote a very useful test that shows many things about how snap execution environment works [07:10] and found a few surprises myself [07:11] the test is here: https://github.com/snapcore/snapd/pull/9345/files#diff-e96db38f8622850cdf91281e5344f762 [07:11] PR #9345: overlord: introduce the export manager, export snapd tools [07:13] zyga-x240: oh, nice [07:13] zyga-x240: anything I can help you with? I mean that does not requires samuele reviews and could still land :) [07:14] mvo: not yet, I will open it properly later today, then it needs reviews [07:14] mvo: I mainly wanted to share what I have now and see what fails [07:15] did anyone look at centos / opensuse failures? [07:15] someone pushed wip/export-manager-patch-set branch to snapd upstream? [07:15] ah [07:15] zyga-x240: ? [07:15] sorry [07:16] yeah [07:16] done [07:16] sorry [07:16] zyga-x240: sure thing [07:20] * zyga-x240 goes to take a shower [07:20] mborzecki: more selinux denials [07:21] from export manager [07:21] 2020-09-18T06:52:34.1787196Z type=AVC msg=audit(1600411883.733:7774): avc: denied { mount } for pid=69477 comm="snap-confine" name="/" dev="tmpfs" ino=227354 scontext=system_u:system_r:snappy_confine_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=filesystem permissive=1 [07:21] 2020-09-18T06:52:34.1789352Z type=AVC msg=audit(1600411883.733:7775): avc: denied { write } for pid=69477 comm="snap-confine" name="/" dev="tmpfs" ino=227354 scontext=system_u:system_r:snappy_confine_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir permissive=1 [07:21] 2020-09-18T06:52:34.1793007Z type=AVC msg=audit(1600411883.734:7776): avc: denied { add_name } for pid=69477 comm="snap-confine" name="etelpmoc.sh" scontext=system_u:system_r:snappy_confine_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir permissive=1 [07:21] 2020-09-18T06:52:34.1796343Z type=AVC msg=audit(1600411883.734:7777): avc: denied { create } for pid=69477 comm="snap-confine" name="etelpmoc.sh" scontext=system_u:system_r:snappy_confine_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=lnk_file permissive=1 [07:21] and more [07:21] ehh [07:21] tmpfs_t ? [07:21] /var/lib/snapd/export is tmpfs? [07:21] yes [07:21] we [07:21] no [07:21] sorry [07:22] /usr/lib/snapd is a tmpfs [07:22] /var/lib/snapd/export is not [07:22] also some real failures on 14.04 [07:22] but first shower [07:29] zyga-x240: when mounting /usr/libexec/snapd, it'd be nice to set some label for that, preferrably the same as the default one for that location [08:26] PR snapd#9371 opened: o/snapstate: improve snapshot iteration [08:46] PR snapd#9372 opened: tests: make nested signing helpers less confusing [08:46] PR snapd#9373 opened: snap: add new `snap debug show-recovery-key` command [08:56] PR snapd#9367 closed: tests: misc nested changes [09:02] pstolowski: I reviewed your PR about snapshots, nice job [09:03] pstolowski: each step to untangle this a bit is good :) [09:04] mvo: thanks, yeah ;) [09:56] morning folks [10:00] hey ijohnson ! [10:00] o/ pstolowski [10:07] pstolowski: added some comments to #9371 [10:07] PR #9371: o/snapstate: improve snapshot iteration [10:08] ty [10:08] anyoen looked at #9370? [10:08] PR #9370: usersession/userd: separate bus name ownership from defining interfaces [10:15] re [10:15] man I was sleepy [10:37] so why does livepatch test faik [10:37] *fail [10:38] hmm [10:38] it passed [10:38] * zyga looks at logs [10:48] ha, reproduced [10:48] now to lok [10:48] look [10:48] so bug or no bug? [10:49] yeah [10:49] figuring out why [10:49] content was not exported [10:50] ok [10:50] I understand [10:50] hmmm [10:50] so [10:50] re exec from old snapd [10:50] the init process of the manager is correct [10:50] but needs to handle exporting core/snapd, not just host [10:50] good thing we have those tests! [11:10] ok testing a fix now [11:15] aa [11:15] I switched to dvorak by accident [11:15] felt uber weird in vim [11:15] spread running, now to write an unit test [11:15] zyga: mind quickly reviewing https://github.com/snapcore/snapd/pull/9374 ? [11:15] PR #9374: tests/main/lxd: disable cgroup combination for 16.04 that is failing a lot [11:17] PR snapd#9374 opened: tests/main/lxd: disable cgroup combination for 16.04 that is failing a lot [11:26] * pstolowski lunch [11:27] and now spread passes [11:27] ijohnson: looking [11:27] thanks [11:27] ijohnson: thanks for that! [11:33] ijohnson: https://github.com/snapcore/snapd/pull/9374#pullrequestreview-491384533 [11:33] PR #9374: tests/main/lxd: disable cgroup combination for 16.04 that is failing a lot [11:33] ah fair enough that makes sense [11:36] zyga: fixed [11:36] looking [11:37] done [11:40] cool, yeah this was the combo I've seen fail just today 2 times, but perhaps another combo with xenial also doesn't work [11:41] we'll see [11:46] pawel has network issues [11:46] mborzecki: this keyboard is so cool, no mouse required [11:46] mborzecki: no arrow keys required [11:47] mborzecki: I think I properly got used to it now [11:49] hmm, all of a sudden the whole documentation links from MOTD are gone on a few of my core18 installs [11:50] there were motd changes to the core* snaps recently [11:50] yeah, feels a bit regressed [11:50] woot [11:50] more tests [11:50] ogra@acheron:~$ ssh 192.168.2.85 [11:50] Welcome to Ubuntu Core 18 (GNU/Linux 5.3.0-1033-raspi2 armv7l) [11:50] Last login: Fri Sep 18 11:12:05 2020 from 192.168.2.48 [11:50] ogra@localhost:~$ [11:50] thats really sparse now [11:51] compared to how it should look like: [11:51] https://paste.ubuntu.com/p/r9dcmvqnfG/ [11:51] ogra: probably worth a bug to github.com/snapcore/core18 [11:51] * zyga found snap.R [11:52] bah ... and bluez doesnt work at all on core16 (and i really dont want to move that specific pi to core1😎 [11:52] grr ... emoji invasion ! [11:52] s/core1😎/core18 / [11:55] ok one more unit test and everything is covered [11:56] I like this new manager [11:56] it has potential [12:01] perfect [12:01] ok [12:02] now to go through the todos left [12:17] meh [12:17] I broke my git config and this makes git push go to origin [12:35] it's a whole 1 degree celsius outside [12:36] SU this morning will be a chilly one [12:37] PR snapd#9372 closed: tests: make nested signing helpers less confusing [12:42] ijohnson: impressive! it's +24°C here [12:43] haha yeah it's supposed to be ~20°C tomorrow [12:44] did you move to antarcica ? [12:44] haha nope, still MN [12:44] crazy [12:44] just very cold September this year [12:44] ogra: I think they have +34°C or something [12:44] brrr [12:45] * ogra sits on the terrace (enjoying my vac) ... at +20°C ... [12:45] weather at the south pole today is -48 C [12:46] ijohnson: heh [12:47] ijohnson: I was refering to https://earthobservatory.nasa.gov/images/146322/antarctica-melts-under-its-hottest-days-on-record but I'm apparently a couple of months late [12:48] yeah I think it's still winter in antartica, so a little bit colder than the summer but yes indeed that is crazy news [12:48] yeah, i read that one on german news sites a few months ago [12:49] current -48°C sounds more like I imagine antartica :) [12:50] as I googled that I was very confused why it was like midnight at the south pole right now, but actually you could use whatever TZ you want, but I guess they customarily use New Zealand time [12:51] TIL [12:52] PR snapd#9375 opened: tests/nested/cloud-init-*: simplify tests and unify helpers/seed inputs [12:52] ijohnson: haha, good point [12:54] xnox, motd being broken in core18 ... do i file that to GH or under lp:snappy ? [12:56] (well ... s/core18/UC18/) [12:56] ogra: github pretty sure [12:57] kay [13:02] * ogra files https😕/github.com/snapcore/core18/issues/170 [13:02] GRRR !!! [13:02] * ogra files github.com/snapcore/core18/issues/170 [13:05] Issue core18#170 opened: [Regression] all documentation links and explanatory text vanished from MOTD [13:40] * zyga breaks for errand [13:41] 1-2h === kirkland is now known as Guest82000 [15:03] PR snapd#9374 closed: tests/main/lxd: disable cgroup combination for 16.04 that is failing a lot [15:03] PR snapd#9376 opened: check-pr-title.py * : allow "*" in the first part of the title [15:04] #9376 is really simple and would be nice to have :-) [15:04] PR #9376: check-pr-title.py * : allow "*" in the first part of the title [15:12] mvo, hey [15:12] the delay that I mentioned was caused by this [15:12] [ 14.613707] systemd[1]: Condition check resulted in Create System Users being skipped. [15:12] error: cannot obtain system details: cannot communicate with server: Get http://localhost/v2/system-info: dial unix /run/snapd.socket: connect: connection refused [15:12] Press enter to configure. [15:12] this is the full log [15:13] https://paste.ubuntu.com/p/tn2sK4cDSC/ [15:13] I created the image from edge [15:13] cachio: edge is broken [15:13] cachio: this is the issue I was talking about in SU, we need a new kernel snap [15:13] ijohnson, I should try with beta? [15:13] cachio: no beta is broken too [15:13] hehehe [15:13] cachio: the only thing to do is to rebuild the kernel snap [15:13] (or wait for that to be done and published) [15:14] ijohnson, so, I could make the focal image work with this old kernel [15:14] cachio: yeah that was great news [15:14] works with kvm [15:14] yes [15:14] cachio: I mean really weird [15:14] but still great that you got it working [15:14] how difficult is it to create and update that image in gce ? [15:14] I tried with another kernel, an older one and didn't work [15:15] ijohnson, I have an image ready [15:15] cachio: great [15:15] ijohnson, test-virt-1 [15:15] cachio: might also be worth asking the server team if they expect any issues with a xenial kernel on focal [15:15] this is the name [15:15] cachio: nice [15:16] it uses the last qemu and ovmf for focal [15:16] right [15:16] PR snapcraft#3291 opened: build providers: make mount_project() interface private [15:16] I¿ll create a new pr to enable kvm and test nested [15:16] nice, let me know when it's up and I'll review it [15:17] ijohnson, sure, I'll make it after lunch [15:17] +1 [15:27] * cachio lunch [15:32] cachio: thanks, in meetings. but thanks for letting me know [15:46] re [15:46] back from store [15:47] * zyga reviews failures on 9345 [15:51] aha [15:51] I guess SNAP_REEXEC=0 is not supported on core :D [15:51] * zyga adjusts [15:55] ok now to dive into selinux permissions [15:58] PR snapd#9377 opened: [RFC] snap-repair: minimal uc20 support [16:12] mmm [16:12] my wife just brought me some coffee [16:28] PR snapd#9378 opened: tests/nested/manual: add uc20 grade signed cloud-init test [17:58] PR snapd#9379 opened: [RFC] cmd/s-b/initramfs-mounts: use ConfigureTargetSystem for install, recover modes