[03:33] Bug #1871827 opened: git ubuntu submit fails on focal

[05:58] https://github.com/snapcore/snapd/pull/8581/checks?check_run_id=631328561 <- persistent journal failed [05:58] PR #8581: tests: port pulseaudio test to session-tool [06:08] PR snapd#8582 opened: github: register matchers before running spread [07:09] morning [07:24] hey pstolowski [07:24] mvo: hi! how is the sprint going? [07:30] hey guys [07:30] I'm doing some test fixes [07:30] pstolowski: do you have backscroll? [07:30] pstolowski: can you see my message at 7:58? [07:31] zyga: i don't have it [07:31] https://github.com/snapcore/snapd/pull/8581/checks?check_run_id=631328561 <- persistent journal failed [07:31] PR #8581: tests: port pulseaudio test to session-tool [07:31] the failure is silly but wonder why [07:38] zyga: yeah it consistent with the previous failures, no idea. looks like snapd gets restarted (or crashes) during configure hook. i will think about a debug section for this test. thanks [07:38] is there something in the logs? [07:42] zyga: nothing obvious, "Running task 84 on Do: Run configure hook of "core" snap" is the last task run, and snap set fails with 'error: cannot communicate with server: Put http://localhost/v2/snaps/core/conf: EOF' [07:42] hmm [07:42] what ran before? [07:43] it's just snap set with true/false in this test [07:43] can you report a bug with the failure fragment, the list of tests that ran before and any additional insight [07:43] I mean tests that ran on this machine before this one [07:43] ah, that [07:43] yep, good idea [07:43] main/cgroup-devices is right before it [07:45] add the complete list [07:45] maybe next time we [07:45] maybe next time we'll know more [07:46] * zyga -> breakfast [07:54] done, https://bugs.launchpad.net/snapd/+bug/1876053 [07:54] Bug #1876053: occasional spread test failure on core-persistent-journal [07:54] thanks! [07:55] one day we'll learn and understand :) [07:56] in the initial version of this code i was restarting journal service which had an undesired effect of restarting snapd during configure hook, but now it's misterious as there is no direct interaction with systemd, just removing/creating journal dir [07:59] do you know why restarting journal restarts snapd? [07:59] is it just snapd that is being restarted in that case? [08:09] dunno [08:11] pstolowski: hmm, perhaps we can do a small experiment with a simple service and see what happens [08:11] maybe it's documented [08:11] I think we ought to know, feels wrong to ship this to devices with potential for failure [08:11] offtopic: https://thepihut.com/products/raspberry-pi-high-quality-camera-lens !!! :) [08:12] i'm so getting this [08:12] and setting up a monitoring with a pi :) [08:12] and botland has it [08:12] https://botland.com.pl/pl/szukaj?controller=search&orderby=position&orderway=desc&search_query=Nowa+kamera+do+Raspberry+Pi&submit_search= [08:20] morning [08:23] mborzecki: hey [08:35] pstolowski: when you have a couple of minutes, could you please double check trello doing? just making sure it's a bit more up-to-date [08:35] mvo: sure [08:36] pstolowski: fwiw, the current trello has a bit too many lanes, this will get cleaned in the following days, so don't worry if it's a bit crowded :) [09:12] I need reviews for https://github.com/snapcore/snapd/pull/7825 [09:12] PR #7825: many: use transient scope for tracking apps and hooks [09:12] it's super close and I'd love to land it this week [09:20] zyga: can yuo remind me whether the session-tool stops the session after the command exits? [09:21] no, because --prepare enables linger and --restore disables linger [09:21] what are you seeing? [09:22] so `session-tool -u test --restore` will kill the session then in restore right? [09:22] yes [09:29] it would be good to focus on making master less red [09:29] something broke in debian-sid now, we cannot compile our golang+C tests [09:29] would be worth taking a look, [09:29] I can in ~3-4 hours [09:29] but perhaps someone can fix it faster [09:30] zyga: got log? [09:30] mborzecki: plenty, check any of my latest PRs [09:30] mborzecki: asm/geneeric.h is missing or something like that [09:30] mborzecki: it fails each time [09:30] mborzecki: on tests/unit/go [09:30] master seems to be green [09:30] mborzecki: nope, it's red red red here :) [09:30] mborzecki: or did it magically go away just as it showed up? [09:31] zyga: https://github.com/snapcore/snapd/commits/master looks green [09:31] mborzecki: try debian-sid-64:tests/unit/go [09:31] the tick is confusing [09:31] it's just travis [09:31] w8, what's that tick? [09:31] we don't run spread there [09:31] pfff [09:31] yeah [09:31] yeah [09:32] omg [09:32] ? [09:32] so, the gh actions only runs on PRs [09:32] yes [09:32] we configured that deliberately [09:32] ok, so we do not run spread on master then [09:32] otherwise everything costs x2 more [09:32] we said we should extend that to run on all landings to release branches [09:33] as we land things regularly it's a reasonable tradeoff [09:33] PR snapd#8583 opened: tests: add debug to core-persistent-journal test [09:33] 2020-04-29T17:55:01.8596162Z + su -l -c 'XDG_RUNTIME_DIR="/run/user/12345" DBUS_SESSION_BUS_ADDRESS="" test-snapd-desktop.cmd xdg-open http://www.example.org' test [09:33] 2020-04-29T17:55:01.8596782Z user-open error: exec: "dbus-launch": executable file not found in $PATH [09:33] heh, on sid [09:33] mborzecki: I really want to burn user.sh and the hacks with fire [09:33] all those XDGD_RUNTIME_DIR=... things are so bogus [09:33] mborzecki: I have a patch that moves one more tests over [09:34] maybe I can remove all of them today [09:34] I'm stuck on "serious" reviews anway [09:34] ok, let me see what i can do about that snap-seccomp unit test failing on sid [09:34] thanks! [09:34] one by one [09:34] and we can ask mvo to merge despite other failures [09:34] I'd love to end the week with totally green master [09:34] so that next week is not like a torture :/ [09:38] zyga: can you take a look at https://github.com/snapcore/snapd/pull/8580 ? or even try it out if you have zsh installed [09:39] PR #8580: data/completion: add `snap` command completion for zsh [09:39] mborzecki: sure [09:39] just dropping _snap file into /usr/share/zsh/vendor-completions/ should do the trick [09:39] or site-functions on other non deb distros [09:40] now that it works locally i see i've been missing out on some goodies :P [09:42] btw the zsh documentation is awful [09:42] that doesn't seem to work [09:44] I installed that on my system [09:44] invoked zsh [09:44] and nothing [09:44] in comparison other completers work ok [09:46] reviewed [09:46] hmmm [09:47] also packaging seems to be broken [09:47] (check the various error logs) [09:50] heh, forgot to list the file for opensuse [09:51] check the suse error [09:52] it says something is listed twice [09:52] maybe a drive-by [09:53] zyga: https://asciinema.org/a/wXGEu1Ky2To2osTCJ8Wl1rD8b [09:53] weird [09:53] that's not what I'm seeing [09:54] I'll recheck [09:54] zyga: on ubuntu you need to move it to /usr/share/zsh/vendor-completins/_snap (with _) [09:54] that's where I put it [09:55] I naively tried to debug it by sourcing [09:56] fyke% source _snap [09:56] _setup:37: compstate: assignment to invalid subscript range [09:56] no, it won't work that way :) [09:59] I don't know how to debug it [10:00] zyga: that's on opensuse? [10:00] no, focal [10:00] weird, right [10:01] but I didn't set shell to zsh [10:01] just execed zsh [10:01] maybe that's why? [10:02] idk, maybe you need to enable completion? try `autoload -Uz compinit ; compinit` ? [10:02] it works for everything else in that directory [10:02] all the other commands generate completios [10:02] *completions [10:03] zyga: did you wget/curl the file or pasted it? [10:03] ... [10:03] pasted [10:03] is that relevant? [10:03] Omg :D [10:04] check whether the first line is `#compdef snap` [10:04] it is [10:04] idk, wget https://raw.githubusercontent.com/snapcore/snapd/060556f7adcf748546cd7c3c97f3254f0bd58644/data/completion/_snap ? :) [10:05] same result [10:06] pfff idk then [10:06] can you try? [10:06] you could try an wipe ~/.zshrc if you're not to used to it and start over [10:06] I mean, just spawn focal (multipass) and put that file [10:07] mborzecki: my zshrc is empty [10:07] zyga: that asciinema is on 20.04 [10:07] mborzecki: yes but from spread [10:07] mborzecki: maybe there's a more magic button somewhere [10:07] ah, ok [10:10] zyga: yup, works [10:10] IDK [10:10] let's ship it then ;D [10:11] I tried setting the GO_... variable for completions and that does print stuff [10:11] did you exec zsh or did you do something more elaborate? [10:11] zyga: rm ~/.zshrc and run zsh again, that should suggest using the default config [10:11] tried [10:12] doesn't fix this issue :) [10:12] zyga@fyke ~ % snap inst [10:12] Completing `file' or `corrections' [10:12] I hit at inst [10:14] zyga: gcc-multilib isn't installed in sid images [10:14] zyga: that's why the unit test fails [10:14] cool, ping me for a patch [10:16] btw. i don't understand why test/unit build and install the snapd package [10:17] mborzecki: weird [10:17] probably for no reason at all? [10:17] that's a huge waste of time [10:17] I also wonder where we waste time [10:17] run 10 tests on a single machine [10:17] with -repeat 10 [10:17] that does "true" [10:17] also, building the package runs unit tests /o\ [10:17] it's incredibly slow [10:17] we are probably wasting loads of time doing something silly somewhere [10:20] uff, at least we pass nocheck to deb-bp [10:20] dpkg-buildpackage [10:39] PR core#113 opened: Makefile: conditionally use the "edge" PPA in live-build [10:47] zyga: https://github.com/snapcore/snapd/pull/8584 [10:48] PR #8584: packaging/debian-sid: add gcc-multilib to build deps [10:48] PR snapd#8584 opened: packaging/debian-sid: add gcc-multilib to build deps [10:48] mborzecki: are we running tests on debian? [10:48] as in, unit tests while building the debian sid package? [10:54] zyga: it was the one that failed in this test [10:54] #8537 needs 2nd review [10:54] PR #8537: store: handle error-list in fetch-assertions results [10:54] pstolowski: trying to review that since morning [10:55] zyga: there's only a bunch of systems we don't run unit tests on: [-ubuntu-core-*, -fedora-*, -opensuse-*, -arch-*, -amazon-*, -centos-*] [10:55] mborzecki: my question was different, we had a missing dependency, how come it didn't come up when building snapd in a pristine environment of a buidd? [10:55] mborzecki: are we running unit tests on package build on debian? [10:56] zyga: perhaps gcc-multilib is one of base dependencies already included? [10:57] zyga: which test runs buildd? i don't see one [10:57] mborzecki: perhaps :) [10:57] mborzecki: I mean "make check" style [10:57] go test ./... [10:57] PR snapd#8585 opened: release: 2.44.5 [10:59] zyga: prepare sets nocheck in DEB_BUILD_OPTIONS, there's a nightly test that runs sbuild, but nobody looks at the result? [10:59] heh [10:59] ok [10:59] thanks for fixing htis [10:59] *this [11:01] zyga: where are the nightly test results? [11:01] replied to https://github.com/snapcore/snapd/pull/8581 [11:01] PR #8581: tests: port pulseaudio test to session-tool [11:01] I have no idea [11:01] probably nowhere [11:01] just read the travis runs [11:01] cachio probably knows [11:08] zyga: so mvo pointed me to where the nightly tests are and as suspected sbuild test was failing, but even before it reached the package build step /o\ [11:09] heh [11:09] where are the results? [11:18] * pstolowski lunch + errand [11:19] zyga: https://travis-ci.org/github/snapcore/spread-cron/branches and look for nightly [11:19] mmm [11:19] thanks [11:20] hm, we don't install sbuild and then run sbuild in the test [11:20] perhaps sbuild is no longer in the sid image? [11:20] IIRC we did but perhaps we stopped [11:20] for efficiency? [11:22] idk, let me check that test [11:22] hm sbuild is installed, wth? [11:28] next chromium issue: https://bugs.launchpad.net/snapd/+bug/1876083 [11:28] Bug #1876083: chromium snap from focal fails DNS lookups, or delays them [11:32] doko: in your report you said "that is using IPv6 only" - what do you mean by that? [11:33] hm idk, looks like sbuild-createchroot is broken [11:33] zyga: my provider is doing IPv6 by default [11:33] I see [11:33] doko: what are the consequences of that, that you don't have an ipv4 address? that you cannot resolve ipv4 domains? [11:34] no, I have an ipv4 address [11:37] mborzecki: we should nuke fedora 28, 29 [11:37] and perhaps 30 [11:37] and just have 31 and 32 [11:37] 30 is eol in a mongth? [11:37] I will look after I wrap up this task [11:37] mborzecki: yeah, they go very quickly [11:37] the nature of fedora community is to move on [11:37] zyga: fwiw, the examples in sbuild-createchroot manpage don't work either, pfff [11:37] mborzecki: haha [11:37] I was in that same boat [11:38] mborzecki: with my recent debian maintainer hat [11:38] mborzecki: the wiki is broken [11:38] mborzecki: but also all of the tools behave different if they detect debian vs ubuntu [11:38] which is enormously confusing [11:39] eh, a wall of perl code :* [11:39] :( [11:39] heh [11:39] if that helps I have a magic line that gives you a working cowbuilder [11:39] it's really really great [11:39] and I use it for git-buildpackage [11:39] but most of the documentation is broken [11:40] and there is some overlap between those tools that doesn't help [11:40] btw, I need to thank stgraber and the team [11:40] lxd is really amazing [11:40] it's revolutionized how I think about systems [11:40] and the container registry is fantastic [11:41] I would love to have a vagrant-like tool for lxd [11:41] that bridges a project directory to the container ootb [11:56] zyga, hey [11:56] hey [11:56] I was trying to merge the no recommends branch [11:56] zyga, but I see these errors https://paste.ubuntu.com/p/BhXjghQ6Nh/ [11:56] https://paste.ubuntu.com/p/hNRPxH7n6D/ [11:56] both in the same test [11:57] zyga, I already tried with other dependencies [11:57] any idea what could be the cause? [11:57] lack of /lib/systemd/user/dbus.socket [11:58] I rewrote this test (though I haven't pushed it as I'm working on a prerequisite) [11:58] is that the only failure? [11:58] zyga, yes [11:58] do you have a debug shell? [11:59] no, let me create it [11:59] is dbus-user-session installed? [11:59] no [11:59] but it pass in other systems where it is not installed [12:00] ah, I missed that there are two test failures [12:00] zyga, yes, different [12:00] dbus-launch is a separate problem [12:00] the tests are really written in a way that won't work [12:00] I'm working on porting them over [12:01] zyga, I can disable it for those systems [12:02] and then we fix it [12:02] zyga, does it make sense? [12:02] can you add a comment like "TODO:session-tool: port and re-enable" [12:02] yeah [12:02] cachio: do you know if we are using different instances by any chance? [12:02] we've noticed that tests are much slower today [12:03] zyga, no [12:03] same [12:04] ok [12:05] zyga, I'll check [12:05] mborzecki: ^^ what do you think about the comments to disable the two tests sergio mentioned [12:05] cachio: today I'm going through failing tests and porting to session-tool, to unbreak from random annoying failures [12:05] so at some point I'll go through all the user.sh users and make them more robust [12:06] yes [12:07] zyga, it caused several issues already [12:10] zyga, pushed the change nad added the comment in both tests disabled [12:11] thanks [12:11] just disabled the failinf systems [12:11] not the test [12:11] do you need a review for the test changes? [12:11] cachio: uh, that's a bit heavy, I think we should not disable debian-sid [12:11] it is already apporoved [12:11] 19.10 is a different story and I would not mind dropping it TBH [12:11] cachio: oh? where is it? [12:12] zyga #8468 [12:12] PR #8468: tests: adding option --no-install-recommends option also when install all the deps [12:13] cachio: ah, ok, [12:13] I misunderstood what you said [12:13] mborzecki: https://github.com/snapcore/snapd/pull/8468#pullrequestreview-403460795 [12:14] PR #8468: tests: adding option --no-install-recommends option also when install all the deps [12:34] zyga: yay, Chipaca is presenting ;) [12:35] watching :) [12:35] zyga: thanks for spotting, i left a comment there [12:35] cool! [12:37] jdstrand: thanks, but now I'm facing LP: #1876083 [12:37] Bug #1876083: chromium snap from focal fails DNS lookups, or delays them [12:37] oops, wrong channel [12:40] mvo: can we merge https://github.com/snapcore/snapd/pull/8584 ? we'll need another PR for the xdg-desktop-portal test [12:40] PR #8584: packaging/debian-sid: add gcc-multilib to build deps [12:40] (needs your superpowers) [12:43] sure [12:43] ta [12:43] PR snapd#8584 closed: packaging/debian-sid: add gcc-multilib to build deps [12:43] mvo: thank you! [13:46] small improvement to session-tool, will replace some hand-crafted checks elsewhere: https://github.com/snapcore/snapd/pull/8586 [13:46] PR #8586: tests: add session-tool --has-systemd-and-dbus [13:46] PR snapd#8585 closed: release: 2.44.5 [13:46] PR snapd#8586 opened: tests: add session-tool --has-systemd-and-dbus [13:46]