[00:54] PR core20#53 opened: Add secureboot-db [02:04] PR snapd#8611 opened: tests: not fail when boot dir cannot be determined [05:20] morning [05:53] Wow. I've got a green tick on PR 5822 [05:53] PR #5822: wrappers: allow user mode systemd daemons <:birthday:> [06:11] wow, that PR is old :P [06:12] zyga's session-tool work has made the tests a lot more reliable [06:13] eagerly waiting for something else to break, at times it really feels like we're only maintaining the ci system [06:32] quick errand, back in 30 [06:42] PR snapd#8610 closed: many: use /run/mnt/data over /run/mnt/ubuntu-data for uc20 (2.45) [06:44] jamesh: woot [06:45] :-) [06:45] zyga: needed a minor change to session-tool to get things working on 16.04 [06:45] Good morning everyone [06:46] Cool, what was it? [06:46] zyga: "session-tool --has-session-systemd-and-dbus" returns false on 16.04, yet it has a working user instance of systemd [06:47] you don't need a systemd controlled session bus to talk to systemd [06:47] Ah, bit that is real [06:47] You indeed a package installed [06:47] Some of our tests do that [06:47] Dubs-user-session or something like that [06:47] Dubs-session-bus maybe [06:48] After you install that package, session to gives you a green light [06:48] zyga: try running "strace --trace=connect systemctl --user status default.target" locally, and find out where it connects to the session bus :-) [06:48] Where does it connect? [06:49] a socket at "$XDG_RUNTIME_DIR/systemd/private", which talks directly to the systemd instance [06:49] Iirc the package ships usr lib systemd user dbus.socket [06:49] yes, but "systemctl --user" doesn't rely on it [06:49] Oh. Interesting [06:50] But we kind of do, you don’t get the environment variable without it [06:50] and it just so happens 16.04 is a system where it makes a difference [06:51] I worked kind of late yesterday [06:51] I’ll wake up and gladly look at changes [06:51] zyga: Here's the change I made: https://github.com/snapcore/snapd/pull/5822/commits/043ba49f8f95d5111875f99e34425e52674c88cd [06:51] PR #5822: wrappers: allow user mode systemd daemons <:birthday:> [06:51] it's sitting at the tail end of my user-daemons PR at the moment [06:52] I can cherry pick it out into an independent PR, but my preference would be to just land user-daemons :-) [06:52] Does it pass on centos-8 [06:52] 7 [06:52] It passed everywhere [06:52] Mmmm [06:53] I'm doing a "systemctl --user is-enabled default.target" call first, which should only fail if we can't talk to systemd [06:53] The reason I ask is that centos 7 disabled systemd —user explicitly [06:53] Ahh [06:53] Perhaps that is the trick then [06:53] Cool [06:54] Looking forward to having this all merged :-) [06:54] if there is a simpler availability check, we can use that [06:54] I need a coffee though [06:54] Few hours of sleep today [06:55] I’m very happy all this work is paying off [06:58] re [07:03] zyga: i'm glad it's seems to be working now, but i'm also amazed how complicated user session setup is [07:03] morning [07:03] makes me wonder whether it really has to be that complicated [07:03] pstolowski: hey [07:07] good morning pstolowski [07:07] and morning to mborzecki and zyga :) [07:11] mborzecki: I can gladly pass the baton [07:13] zyga: nah, thanks :) [07:14] at my desk [07:14] green ticks [07:15] not everyone [07:15] but man :) [07:15] call me eco-friendly ;D [07:15] mvo: I worked to like 2AM [07:15] mvo: I kind of feel tired today [07:15] I will probably just attend meetings [07:15] and not do much meanwhile [07:16] maybe push some small fixes [07:16] and two claudio's branches are green too [07:16] mborzecki: just wait until we try to set up a working user session on a real Ubuntu Core system, rather than just in the test environment... [07:16] mborzecki: rebase / merge master [07:16] :D [07:16] jamesh: btw, did you notice that core 20 seems to have everything required for user services already? [07:17] zyga: more so than core18? [07:17] yes [07:17] check this out [07:17] https://github.com/snapcore/snapd/blob/master/tests/main/session-tool-support/task.yaml [07:18] core 20 ships /lib/systemd/user/dbus.socket [07:19] zyga: ah. I agree that makes a difference for D-Bus services, yes :-) [07:20] user services are working all the way back to core16 [07:20] indeed [07:20] jamesh: one relevant aspect that I didn't explain properly before [07:20] because I was typing from a phone [07:21] was that dbus.socket unit injects DBUS_SESSION_BUS_ADDRESS into session environment [07:21] and .e.g. snap run checks for it [07:21] so it's plausible that other programs also check for it [07:21] and not fall back to other locations [07:21] but in general, having dbus in the session is desired [07:21] definitely [07:21] and maybe in a year we can pretend that it is always true [07:22] once 16.04 goes EOL [07:22] jamesh: well, that 10 year support probably means my kids can gro up to become engineers and support it [07:23] hahah [07:23] (for the record, my son is almost 14 now) [07:24] neither the desktop or snapd were included in 14.04's ESM [07:24] snapd might not be so lucky for 16.04, but you might not need to care about desktop sessions there any more [07:25] jamesh: so about that [07:25] there's some discussion to extend session-tool to 14.04 [07:25] so that we have better coverage [07:25] I _might_ just do it [07:26] there's no user instance of systemd on 14.04, so it will probably look like CentOS 7 [07:26] I was thinking about emulating all the upstart-goodies there [07:26] so not based on systemd [07:26] but stil following PAM [07:27] and doing "something" that is representative [07:29] the agreement on 14.04 is mostly livepatch should install and work [07:29] mmm, I see [07:29] maybe it's just not worth it then [07:30] afaiu no [07:46] PR snapd#8606 closed: progress: fix progress bar with multibyte duration units [07:46] yay [07:52] see you at the code review meetign! [07:52] *meeting [07:57] mvo: I did a pass on #8334 [07:57] PR #8334: many: add core.resiliance.vitality-hint config setting [07:57] pedronis: \o/ thank you [08:06] PR snapd#5822 closed: wrappers: allow user mode systemd daemons <:birthday:> [08:14] PR core20#53 closed: Add secureboot-db [08:46] I identified and fixed the tests that were breaking the pulseaudio test before [08:46] it was the portal and xdg-open tests [08:47] they created ~test/.config as root [08:47] and that was enoguh [08:47] I'll send patches shortly [08:50] hi I'm once again searching for bug which may or may not be a bug.. after Tumbleweed upgrade to GNOME 3.36 on two machines, the other one is missing snap in XDG_DATA_DIRS and also missing snap_bin_path and snap_xdg_path - what should be setting those? and I wonder if there's anything that could be made bullet proof regarding setting them. [08:50] Mirv: hello [08:50] I can obviously workaround by setting the missing parts manually [08:51] those are set by IIRC the environment generator [08:51] (systemd env generator) [08:52] Mirv: do you have ... [08:53] Mirv: /usr/lib/systemd/system-environment-generators/snapd-env-generator [09:00] yes I have on both machines. I'm doing random grepping but not finding differences between the machines yet [09:03] mborzecki: ^ [09:03] * zyga needs to go afk for a moment [09:07] zyga: snapd-env-generator is for services iirc [09:07] what? why? [09:07] pstolowski: is #8533 really just the test? does it need master merge? [09:07] PR #8533: image, tests: core18 early config test [09:08] zyga: for user sessions there's systemd-environment-d-generator [09:08] zyga: and as for why, i have no clue ;) [09:09] Mirv: does /usr/lib/environment.d/990-snapd.conf exist in your system? does `systemctl --user show-environment | grep '^XDG_DATA_DIRS='` list /var/lib/snapd/desktop ? [09:09] pedronis: ah, no, it's also actual image.go change, i will update the desc [09:09] pstolowski: ah, ok [09:13] mborzecki: exists on both, on the affected system however /var/lib/snapd/desktop is missing while on the working system it's there [09:13] (ie 990-snapd.conf exists on both) [09:13] Mirv: any chance you are using zsh? [09:13] bash on both [09:16] pstolowski: maybe the configcore changes should be in their own PR? [09:16] hmm looks like the issue we had on centos and rhel, where it worked in my vm, but didn't work in degville's [09:17] mborzecki: I think I'll try a fresh install and see what happens. [09:18] pedronis: sure, i can extract them [09:18] pstolowski: if it's not too annoying [09:18] pedronis: it shouldn't [09:18] thank you [09:19] anyway, the problem seems a bit more like systemd problem now that it is in environment.d correctly, not snap problem [09:22] erh... I noticed there is also 99-environment.conf symlink to /etc/environment , so out of interest I tried to copy-paste the content of 990-snapd.conf into /etc/environment and rebooted.. upon login, it seemed like I could not launch any apps, and no extensions were loaded. so I removed the lines from /etc/environment and rebooted again and... now, for some reason XDG_DATA_DIRS has the snapd dir! [09:23] maybe it's a systemd bug [09:23] before this I had used the system with multiple reboots since last week [09:24] perhaps some race condition [09:25] Mirv: is there a /usr/lib/systemd/user-environment-generators/30-systemd-environment-d-generator in your system? [09:28] btw. noticed therels /usr/lib/systemd/user-environment-generators/60-flatpak that runs after, but it seems to be handling the XDG_DATA_DIRS path correctly [09:30] Mirv: if you have that 60-flatpak file ^^ can you run `XDG_DATA_DIRS=/var/lib/snapd/desktop /usr/lib/systemd/user-environment-generators/60-flatpak` and make sure that output contains /var/lib/snapd/desktop path? [09:35] mborzecki: yes there is 30-systemd-environment-d-generator on both machines, and yes the output contains snapd/desktop path. but as said, after I made something error out by modifying /etc/environment and then reverting that change, now the /var/lib/snapd/desktop is correctly there also on this system that didn't have it for the past week, so I can't reproduce the problem anymore. [09:37] mborzecki: .. I rebooted again, and now the problem is again there .. still the output containts the desktop path [09:38] it's something causing the systemd not generate the environment correctly, quite consistently on this machine but not the other [09:39] Mirv: are you using gdm? [09:39] mborzecki: yes [09:40] Mirv: ok, can you `cat /usr/share/gdm/env.d/flatpak.env`? [09:40] if it exists [09:42] -> XDG_DATA_DIRS=$HOME/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/ [09:42] Mirv: heh ok, can you mv that file to say $HOME for a bit, and then reboot/relogin? [09:44] jamesh: are you familiar with gdm internals by any chance? [09:45] mborzecki: not deeply, but maybe I can help? [09:46] jamesh: when the user session starts up, are env files from /usr/share/gdm/env.d/ applied, and if so would that happen to run after/before systemd user env generators? [09:46] PR snapd#8612 opened: sysconfig: use new _writable_defaults dir to create cloud config <⛔ Blocked> [09:47] jamesh: from the code i see it's in the worker https://github.com/GNOME/gdm/blob/2155ffa023d94fd269cf01d69d897ebe9a63a386/daemon/gdm-session-worker.c#L1633 but does it run when user logs in or is it only for the actual gdm process? [09:47] PR core20#54 opened: static: switch /etc/cloud from synced to persistent [09:48] mborzecki: I'm not sure [09:48] mborzecki: I've moved that away and the problem is gone for at least two subsequent boots (more than ever before).. so maybe it's a race condition [09:49] anyway, I'll need to go now for a bit [09:51] jamesh: hmm the call chain goes up to do_start_session() https://github.com/GNOME/gdm/blob/2155ffa023d94fd269cf01d69d897ebe9a63a386/daemon/gdm-session-worker.c#L2764 [09:58] I found and fixed a bug affecting all portals tests [09:58] that should remove some more random failures [09:58] two PRs to go though [10:09] wtf is /usr/lib/systemd/user/dbus.service.d/flatpak.conf ? why are all 3 ways of injecting environment used by flatpak? [10:10] lol [10:10] what? [10:11] at least 2 of those just override whatever is set via user env generators [10:11] what's there? [10:11] are you sure it is only 3? [10:11] # /usr/lib/systemd/user/dbus.service.d/flatpak.conf [10:11] [Service] [10:11] Environment=XDG_DATA_DIRS=%h/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/ [10:11] heh [10:11] so kind [10:11] jamesh: skipped pam_env since we know it's off limits :P [10:11] it's not using /etc/profile.d too? [10:11] maybe we should file a bug? [10:12] that file and /usr/share/gdm/env.d/flatpak.env just set XDG_DATA_DIRS [10:12] unilaterally overriding (rather than appending/prepending) is bad manners [10:12] mborzecki: just ship aaa-snapd and zzz-snapd.conf [10:12] and set it again [10:13] also found this comment https://bugzilla.gnome.org/show_bug.cgi?id=766176#c11 there's like 4 possible scenarios how env is injected [10:13] * zyga queues for the best tech solution award ;) [10:13] jokes aside [10:13] it's a bug [10:13] and a bit eivl [10:13] *evil [10:14] we've definitely had problems where the environment injection methods we thought we universal turn out not to be [10:14] they're probably just adding new ones as they find new corner cases [10:15] we are seriously doomed [10:16] PR snapd#8613 opened: o/configcore: pass extra options to FileSystemOnlyApply [10:16] well the linux desktop is xD [10:23] zyga: I did a pass on #7825 [10:23] PR #7825: many: use transient scope for tracking apps and hooks [10:23] pedronis: thank you! [10:24] zyga: I should look at 8566 next, is that right? [10:24] yes please [10:25] ok, lunch first [10:27] pedronis: the main remark about the transient unit code is that test tests will be probably 500+ lines and I just didn't want to do them all here (behind a feature flag) - the main innovation is being able to do the tests in the first place (but that has since landed to master) [10:27] pedronis: I'm happy to do the tests in the same PR if people are ok with that [10:28] zyga: tbh, I think it makes more sense to split out the helper to meaningful packages and do the tests there [10:28] *helpers [10:28] pedronis: sure, just tell me where [10:28] zyga: I made some suggestions in the comments [10:28] pedronis: cmd/dbusutil? dbusutil top level... somewhere else? [10:29] zyga: I think some bits belong to systemd, some bits might belong to a dbusutil package [10:29] and some bits belong to sandbox/cgroup [10:29] the latter point is the more unclear [10:29] I have a comment with a question about it [10:30] so should I split that into bits that can be discussed [10:30] or do you suggest to continue in the branch, just iterating on the parts there [10:31] (the branch has the advantage of having spread tests to check things end to end) [10:32] PR core20#55 opened: hooks: make systemd-modules-load depend on mounts at /usr/lib/{firmware,modules} [10:35] mborzecki: https://github.com/snapcore/snapd/pull/8614 [10:35] PR #8614: tests: don't create root-owned things in ~test [10:35] PR snapd#8614 opened: tests: don't create root-owned things in ~test [10:35] * zyga needs to run an errand for his parents, I will probably miss standup as we need to drive around [10:36] the portal tests are still not reliable, I found one more bug that is fixed locally [10:36] but there is more, I just cannot understand why it fails [10:42] zyga: you can do the work in the branch and then split it, atm it has the problem that is both too big and not big enough (missing coverage, too much code directly in cmd_run.go) [10:43] Ok, sounds good [10:43] And I agree [11:03] mborzecki: mvo: seems #8464 can be merged [11:03] PR #8464: cmd/snap-boostrap, boot: use /run/mnt/data instead of ubuntu-data [11:06] pedronis: thank you, merged [11:06] pedronis: I updated the vitality-score PR [11:06] PR snapd#8464 closed: cmd/snap-boostrap, boot: use /run/mnt/data instead of ubuntu-data [11:06] yay [11:06] thanks folks [11:06] and g'morning [11:07] ijohnson: hey :) [11:07] hey zyga [11:07] I had a look at your session-tool pulseaudio PR that you merged last night, it all looked good to me [11:07] ijohnson: I fixed that bugger and found the cause for the failure [11:07] very nice [11:07] https://github.com/snapcore/snapd/pull/8614 [11:08] PR #8614: tests: don't create root-owned things in ~test [11:10] zyga: nice [11:10] it feels good to see master very green [11:11] it is not over [11:11] I found two more bugs in portal tests [11:11] fixed one [11:11] investigated the other but cannot grok it yet [11:11] race [11:11] but cannot grok why [11:11] but little by little : [11:11] :) [11:12] * mvo hugs zyga [11:12] PR snapd#8615 opened: github: register the problem matcher in a separate step to running spread [11:12] thank you ijohnson and good morning as well :) [11:13] o/ mov [11:13] ah typing is hard [11:13] o/ mvo [11:13] oho [11:13] kid invasion [11:13] dont call ian a kid ! [11:13] I have a sleeping lucy hugging me now [11:14] haha [11:27] mborzecki: just noticed this unit test failure from one of my runs where I merged master: https://pastebin.ubuntu.com/p/RbzqwMKSbP/ [11:27] I'm a bit unclear what went wrong in the regex there, but it looks related to your change [11:28] it came from https://github.com/snapcore/snapd/pull/8559/checks [11:28] PR #8559: boot, bootloader: adjust comments, expand tests [11:28] ijohnson: hmm yeah, 0B/s weird [11:28] maybe something needs tweaking [11:38] https://github.com/snapcore/snapd/pull/8602 is green [11:38] PR #8602: configcore: only reload journald if systemd is new enough [11:38] and +3 [11:38] pstolowski: ^^ [11:39] grea, and it's green finally [11:41] PR snapd#8602 closed: configcore: only reload journald if systemd is new enough [11:42] PR snapd#8616 opened: progress: tweak multibyte label unit test data [11:42] ijohnson: this should fix it ^^ [11:42] (or at least make it more robust) [12:10] cmatsuoka: hi, I wanted to review #8577 but it has conflicts atm [12:11] PR #8577: secboot,cmd/snap-bootstrap: move initramfs-mounts tpm access to secboot [12:11] pedronis: yes, I'm fixing the conflicts but then in a test installation broke for some reason, I'm checking with ian [12:12] pedronis: I'm retesting installation with pure master now [12:17] pedronis: resolved [12:17] thx [12:18] pedronis: there's a handle type there I couldn't come with a good name for, suggestions welcome [12:40] pstolowski, hi [12:40] pstolowski, I see an error in preseed test https://paste.ubuntu.com/p/j5x2pHBH9m/ [12:41] have there been any snapd changes to make anything talk to avahi via dbus recently ? [12:41] pstolowski, is it familiar for you? [12:41] seems oSoMoN and I both see avahi denials for apps that did not have them before ... but funnily we see similar denials for different apps [12:41] msg='apparmor="DENIED" operation="dbus_method_call" bus="system" path="/" interface="org.freedesktop.DBus.Peer" member="Ping" mask="send" name="org.freedesktop.Avahi" pid=431942 label="snap.chromium.chromium" [12:42] (i see it for chromium, olivier for libreoffice ... neither app has used avahi before) [12:42] chrome is pinging avahi [12:42] zyga, well ... here it is [12:43] seems for oSoMoN libreoffice is all of a sudden pinging avahi :) [12:43] hmm, could that be portal integration stuff? [12:43] to be fair I don't know whether it was doing it before, I never noticed but then again I'm not watching denials all day long [12:44] i tend to have a terminal with journalctl -f while trying our new snaps or fixing something ... i dont run my logs regulary though, but every now and then ... and i thinnk i'D have noticed Avahi denials [12:45] s/our/out/ [12:45] for me this is definitely new ... though i have only upgraded to focal end of last week [12:46] perhaps it is focal related [12:56] pedronis: updated https://github.com/snapcore/snapd/pull/8566 [12:56] PR #8566: cmd/cmdutil: add run inhibition operations [12:56] * zyga -> standupo [12:56] standup even [13:02] cachio: looks like deb is old and doesn't ship snap-preseed binary? [13:24] PR snapd#8615 closed: github: register the problem matcher in a separate step to running spread