=== diddledan6 is now known as diddledan [05:51] morning [06:09] hey [06:09] mborzecki: I'm fixing the tumbleweed degraded test now [06:09] zyga: hey, it's failing? [06:09] yeah, some vty thing is complaining [06:09] nothing relevant [06:09] https://github.com/snapcore/snapd/pull/8596 needs a review [06:09] PR #8596: tests: session-tool --restore -u stops user-$UID.slice [06:10] mborzecki: we have a problem with journal test but let's do one thing at a time [06:10] (and the feature, not just the test) [06:10] persistent journal test? [06:10] yes [06:10] it's soaky wet ouside, my dog will not be happy [06:10] mborzecki: look at the test if you are curious [06:10] it has some countermeasures [06:10] but it's a real problem [06:11] and they are insufficinent apparently, it failed again on the unprotected "snap set" code path at the end of the test [06:16] how was the long weekend? [06:22] zyga: it was ok, but felt a bit weird since everyone else had friday off and i swapped for monday ;) [06:24] zyga: btw. our fonts cache handling is broken i think, we'll need to discuss what to do about that, but the current observation is that we should not trust the font cache generated on the host to be compatible [06:24] mborzecki: happy to [06:24] mborzecki: can we stop sharing caches [06:24] caches belong to /var/snap/$BASE/ [06:24] anything else is broken [06:24] hm idk, caches belog to whatever fontconfig is used [06:25] that can be base, gnome/kde/freedesktop runtime, or the snap [06:26] anyways, desktop apps render boxes or straight out segfault on fedora 32 now [06:26] mborzecki: I'm only talking about the fonconfig managed by snapd [06:31] zyga: but none of the bases we have ship fontconfig [06:32] but each of them has an associated gnome [06:32] zyga: idk, we need to think this though, it was a nice idea for an optimization, but it looks like it's misplaced and doing harm to the cross distro story [06:32] and derived fontconfig [06:32] but yeah [06:32] needs thought [06:32] zyga: the snap can ship a different fontconfig, or not not use the gnome runtime at all [06:34] mborzecki: snaps may ship it, we can just disable the shared (base) cache then [06:34] mborzecki: if they do the simple thing, the use the same version that's in the base [06:34] mborzecki: either via bundling or via content to gnome runtime [06:34] mborzecki: base is just the easiest sharing "key" [06:36] zyga: kenvandine had an idea for per snap cache, with fc-cache getting called in the configure hook [06:37] mborzecki: that will work but it it would be *very* costly IMO [06:37] and the cache has non-insignificant size [06:37] in the meantime, i need to think of a way to disable mounting of cache from hsot on arch and fedora, i'd rather have a longer statup than a segfaulting app [06:37] mborzecki: +1 [06:37] mborzecki: that should be easy [06:37] mborzecki: where is the cache again? [06:38] zyga: the configure hook runs during install, so it's just the snap install that takes longer [06:38] mborzecki: that is true, I do like that part [06:38] mborzecki: but does it mean all snaps get a virtual hook [06:38] zyga: /var/cache/fontconfig and ~/.cache/fontconfig [06:38] mborzecki: or do they need to explicitly cooperate? [06:39] zyga: well, if we make it a gnome extention, then snaps need to be rebuilt/cooperate [06:39] mvo: hey! [06:40] mborzecki: good morning [06:40] mvo: hey [06:40] zyga: and good morning to you as well! [06:41] https://github.com/snapcore/snapd/pull/8598 <- last of the always-happening failures [06:41] PR #8598: tests/degrated: ignore failure in systemd-vconsole-setup.service [06:41] zyga: looking [06:41] PR snapd#8598 opened: tests/degrated: ignore failure in systemd-vconsole-setup.service [06:41] mvo, mborzecki <- please look at https://github.com/snapcore/snapd/pull/8596 as well [06:42] PR #8596: tests: session-tool --restore -u stops user-$UID.slice [06:43] ok, I need to take the dog out [06:43] onto the rain :| [06:43] or maybe into the rain actually [06:46] PR snapd#8593 closed: image: stub implementation of image.Prepare for darwin [06:46] PR snapd#8596 closed: tests: session-tool --restore -u stops user-$UID.slice [07:02] zyga: re. your questions about me trying to use session-tool: https://github.com/snapcore/snapd/pull/5822#discussion_r419901621 [07:02] PR #5822: wrappers: allow user mode systemd daemons <:birthday:> [07:02] morning [07:11] jamesh: good morning [07:11] lookijg [07:11] jamesh: I ported user-mounts to session-tool yesterday [07:12] zyga: the problems seem to suggest that sessions are not being cleaned up between tests [07:13] jamesh: I think https://github.com/snapcore/snapd/pull/8596 may address this [07:13] which just landed [07:13] PR #8596: tests: session-tool --restore -u stops user-$UID.slice [07:13] have a look, it's still a bit murky [07:13] I will look at logind changes [07:13] as it definitely behaves differently from one version to another [07:13] zyga: okay. I'll give it another try. [07:14] thanks [07:14] zyga: overall, I like what you've done. [07:15] I think we are not quite fully there yet, with groking various quirks in session handling across time [07:15] but I feel we are much closer to actually running representative tests [07:15] and getting familiar with bugs that may also impact us [07:16] hopefully we don't end up with anything that depends on systemd environment vs. login shell environment... [07:16] jamesh: login shell environment? [07:16] zyga: the environment you get in a login shell, which could be different to the one systemd units are launched in [07:17] oh, it is [07:17] but we use runuser -l [07:17] which gives us /etc/pam.d/runuser-l [07:17] but it's a good point that we should understand that difference better and see what's different still [07:18] due to runuser-l we get a real logind session and we get a few more pam modules [07:18] zyga: it's probably not an issue [07:19] since a test can add anything extra it needs in the env [07:19] looking just now, I see we don't get this pam module: [07:19] session required pam_env.so readenv=1 user_readenv=1 envfile=/etc/default/locale [07:21] surreal, we will have first green PR in a week, I think [07:21] or more perhaps [07:34] mvo: thank you [07:34] mvo: I was waiting for that green tick [07:34] but let's see it on ALL THE BRANCHES now :) [07:34] PR snapd#8598 closed: tests/degrated: ignore failure in systemd-vconsole-setup.service [07:35] zyga: yeah, this looks super nice now [07:35] zyga: I hope we overcame most of the flakiness for now [07:41] checkSnapSuite.TestCheckSnapSystemUsernamesCalls fails if you have daemon users installed [08:01] zyga: slightly tweaked #8580 [08:01] PR #8580: data/completion: add `snap` command completion for zsh [08:10] hehe, booting old work laptop -> log in to old irc channels [08:11] morning all [08:11] Chipaca: morning sir! [08:11] :-) [08:13] zyga: the pulseaudio test is still kinda flaky, https://github.com/snapcore/snapd/pull/8537/checks?check_run_id=645190804 recording failed on 19.10 and pulseaudio iface test failed on 18.04 [08:13] PR #8537: store: handle error-list in fetch-assertions results [08:15] i'm seeing new selinux denials on centos 7 & fedora, not sure if this is my branch onl [08:15] *only [08:16] howdy Chipaca! [08:16] pstolowski: how're things going? [08:18] Chipaca: red color has been dominant recently. otherwise - excellent :) [08:44] Chipaca: did you see my poem? :) [08:44] mborzecki: looking [08:46] mborzecki: oh, it's in the old style with manual masking and all the hacks [08:46] mborzecki: let me fix it [08:46] zyga: yep :) 'twas good. Also wondering what the background you were talking about looked like. [08:46] Chipaca: a week+ of master red [08:47] Chipaca: and mvo (who can override) being at a sprint [08:47] Chipaca: but it was good, vast majority was fixed already [08:53] mborzecki: testing now [08:54] mvo: hi, can the initrd PRs for early config land? [09:00] pstolowski: I think so [09:01] pstolowski: would be nice to get a review from xnox onhttps://github.com/snapcore/core20/pull/46 [09:01] PR core20#46: static: add new handle_writable_defaults() to handle-writable-paths [09:01] pstolowski: but yeah, I think the code is good now [09:04] mvo: looks good, two minor unresolved conversations on it. [09:04] xnox: aha, thank you, I will check what's missing there [09:11] brb [09:39] re [09:39] man, it's surely raining today [09:40] soaky day [09:41] today is a tea day [09:46] mvo: hi, the travis test for core20 in your PR failed [09:46] pedronis: i've added some reviews for the bulk assertions PR, we should probably land the ones that precede #8564 [09:46] PR #8564: asserts: introduce Pool [09:46] pedronis: checking [09:46] pedronis: and hello :) [09:47] mborzecki: yes, trying but red and queueing atm [09:48] mborzecki: thanks for the reviews btw, same pstolowski [09:49] pedronis: yw. i'm keen to review the others but would be nice to land the prereqs, it was kinda confusing to review stacked PRs [09:50] pstolowski: yes, agree [09:50] pedronis: fun "--2020-05-05 09:45:09-- http://cdimage.ubuntu.com/ubuntu-base/daily/current/focal-base-amd64.tar.gz is 404 now", I guess that needs an update first :) [09:51] oh about focal, mvo do you remember that bug where hardlinking on livecd used 400MB? [09:51] we got a comment that apparently the releaes iso shipped with that bug :| [09:51] so maybe something was off [09:52] zyga: oh, I thought we fixed this [09:53] we did but maybe didn't really [09:53] zyga: https://bugs.launchpad.net/snapd/+bug/1867415 [09:53] Bug #1867415: Hardlinking snaps wastes 400 MB tmpfs RAM in live CDs [09:53] mvo: comment #6 [09:53] zyga: yeah, just saw it [10:10] PR core20#51 opened: Makefile: updated now that focal is released [10:11] PR snapd#8599 opened: tests: port interfaces-audio-playback-record to session-tool [10:44] fwiw it seems we hit queuing quite a bit atm with actions, have things queued for many hours now [10:45] pedronis: result of fixing most of master annoyances [10:45] pedronis: I think it will clear in a few hours, let me look at the state [10:46] pedronis: yeah, we have a few branches queued and lots running as well [11:19] mvo: I got a weird email about authorization refresh [11:19] Launchpad failed to refresh the authorization tokens used to upload this snap package to the store: [11:19] (then) Provided email/password is not correct. [11:19] this is from test-snapd-sh-core18 [11:20] zyga: got one too [11:39] zyga: do you remember a change where we checked whwether the system is going down? was that in snap run? [11:39] yes I do, yes it is [11:39] cmd_run.go [11:39] isSystemStopping IIRC [11:39] why? [11:40] zyga: if the logs ehere are to be belived: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1873550/comments/3 we may need to check if system is shutting down in snap-failure [11:40] Bug #1873550: Snappy daemon reaches 1min30s timeout during shutdown process [11:40] ha [11:40] fun [11:40] why does OnFailure trigger? [11:41] zyga: because we hit the stop timeout, there's a wait for api clients to shut down, hits a 25s timeout, then before snapd stopped systemd killed it? [11:42] I see [11:42] hm maybe that client shutdown timeout is too long still [11:42] 25s is pretty log for any api activity [11:44] btw. pretty lame that systemd triggers onfailure when the system is shutting down [11:45] still, why is there a 5s break between snapd logging the warning and being killed by systemd [12:09] mborzecki: I think systemd.exec knows [12:10] zyga: yeah, noticed a log that there's a stop action queued, so it's not going to enqueue start [12:10] anyways, one of the commenters clearly had a newer snapd version [12:12] hmm, nothing useful in the man page [12:13] there's also systemd.kill [12:13] maybe it's one of the defaults [12:34] ijohnson: hey, i've checked your hotplug + greedy plugs issue, replied to your bug report, does it make sense? [12:55] PR snapcraft#3107 opened: fix yarn version url [12:58] PR snapd#8600 opened: tests: run smoke test with different bases [13:50] PR snapd#8580 closed: data/completion: add `snap` command completion for zsh [13:58] * zyga -> errand [14:13] PR snapd#8601 opened: Adds missing paths to apparmor, solves lp:1876804 [14:18] https://github.com/snapcore/snapd/pull/8597 is green :)D [14:19] PR #8597: tests: port user-mounts test to session-tool [14:19] zyga: that's great [14:20] wait, is there a bzr tree of snapd?!? [14:20] https://bugs.launchpad.net/snapd/+bug/1876804/comments/1 [14:20] Bug #1876804: apparmor logs filled when ubuntu store launches [14:20] zyga: I'm still a little confused though, does the old test really have coverage of something we care about on 14.04 [14:20] zyga: haha yeah I was confused too [14:20] ijohnson: I'm slowly inclined to restore 14.04 support by writing a 14.04 version of session-tool [14:20] zyga: I'm fine if you want to do a followup to re-add 14.04 [14:21] zyga: but I don't feel comfortable just dropping coverage for 14.04 yet as afaik the policy is don't break it if we don't have to [14:21] ijohnson: I think it's preferable to be able to fix issues globally by changing session-tool [14:21] zyga: maybe a followup to add TODO to that test to say "re-add 14.04 when session-tool is updated" [14:21] ijohnson: sure, [14:22] ok, I will +1 your PR [14:22] I can bundle that with something else not to clog the pipe today [14:22] with the next -porting-NNN branch [14:22] right [14:32] mvo: #8537, please double check but I think can probably be force landed [14:32] PR #8537: store: handle error-list in fetch-assertions results [14:34] zyga: do you know what was the first good systemd version wrt to persistent journal issue? [14:34] * zyga is in the car [14:34] pstolowski: yes, one sec [14:35] https://github.com/snapcore/snapd/pull/8594#issuecomment-623584798 [14:35] pstolowski: ^ [14:35] PR #8594: tests: work around journald bug in core16 [14:38] zyga: thanks! [14:38] * zyga is in the car, delivering supplies [14:38] and reading about how to send the control bugs to BTS [14:45] whcih package ships translations for snapd? [14:45] in ubuntu [14:46] mborzecki: ubuntu uses a special langpack system [14:46] apt-cache search langpack [14:47] pedronis: 8557 lgtm, thanks for the changes [14:47] mborzecki: translations are extracted from individual packages (in main) [14:47] mborzecki: and bundled into language-specific package [14:48] zyga: ok, installing bits for uk_UA [14:48] haraszo [14:48] haha [14:52] zyga: hmm there does not seem to be a gettext file for snap tho [14:52] ijohnson: thanks [14:53] #8557 needs a 2nd review, it's mostly a test refactoring [14:53] PR #8557: c/snap-bootstrap: check mount states via initramfsMountStates [14:56] zyga: heh, so we ignore LC_ALL and just use LC_MESSAGES and LANG (a bug then?) [15:05] haa [15:05] got it [15:10] PR snapd#8602 opened: configcore: only reload journald if systemd is new enough [15:29] pedronis: in various meeting, will check in a wee bit [15:32] mborzecki: re [15:32] mborzecki: there are two langpacks, base and "graphical" [15:32] not sure which one you checked [15:32] * cachio lunch [15:52] zyga: shall I merge #8599 since it's green (except unstable system) ? [15:52] I just approved it [15:52] PR #8599: tests: port interfaces-audio-playback-record to session-tool [15:52] looking [15:52] ijohnson: oh, there's a small comment [15:52] I guess, yes [15:52] I will follow up in a moment [15:52] thanks! [15:52] * ijohnson just wants to see green PR's get merged [15:52] :-) [15:52] yeah, that's the spirit! [15:53] keep master rolling [15:53] wanna press the button or shall I? [15:53] #8557 is completely green, incredibly, but needs a 2nd review [15:53] PR #8557: c/snap-bootstrap: check mount states via initramfsMountStates [15:53] pedronis: it feels good, right? :) [15:53] I pressed the button [15:53] PR snapd#8599 closed: tests: port interfaces-audio-playback-record to session-tool [15:54] \o/ [15:54] I will read it on my way back [15:54] will be heading back home soon [16:02] pedronis: merged, sorry for the delay [16:03] PR snapd#8537 closed: store: handle error-list in fetch-assertions results [16:03] mvo: thanks, I need to deconflict the next one now [16:06] PR snapd#8597 closed: tests: port user-mounts test to session-tool [16:14] PR pc-amd64-gadget#46 opened: gadget.yaml: bump edition [16:44] mmm should we be installing snap-failure service on systems that only have the core snap ? [16:49] mvo: pedronis: should we have the snapd.failure service when only the core snap (or only the snapd distro pkg) is installed (and thus no snapd snap?) [16:50] seems like `snap-failure snapd` just no-ops if there is no snapd snap installed, so probably yes it seems === ijohnson is now known as ijohnson|lunch [17:38] PR snapcraft#3102 closed: build providers: prevent snap refreshing in build environment [17:41] PR snapcraft#3104 closed: packaging: use git-based versioning for python package === ijohnson|lunch is now known as ijohnson [18:32] cmatsuoka, hey [18:33] cmatsuoka, for testing pi3 with uc20 do we need to use the model signed by canonical? [18:33] or we can use a model signed by me? [18:33] cachio: afaik you can use your model [18:34] ah, ok [18:34] cmatsuoka, I wanted to make sure it was not going to produce an problem with tpm/secboot [18:35] we don't have it in arm yet so you'll install unencrypted [18:35] cmatsuoka, good [18:35] cachio: we also have dangerous models signed by canonical you can use too [18:35] well unless you have an arm board with tpm, but we never tested that [19:00] ijohnson, yes [19:01] I'll test new images signed by me first [19:01] then canonical [19:06] ijohnson: I think I fixed your serial-port issue (see the bug) [19:07] jdstrand: nice! I was going to look at pawel's response and ping you about it but you saw it first, thanks! [19:07] jdstrand: re: including the snapd snap type, I'm not sure, on your system you tested it are you using the snapd snap? [19:08] ijohnson: it isn't installed, no [19:08] I have it installed, let me see if your new declaration works [19:12] jdstrand: it seems to have auto-connected anyways [19:13] I just installed it and am trying [19:14] the slot comes up as snapd:ft232serialuartic [19:14]