=== epod is now known as luk3yx [05:49] hello [05:55] morning [05:56] mborzecki: hey :) [05:56] mborzecki: making breakfast, how are you? [05:56] zyga: hey [05:56] zyga: how's your back? [05:57] mborzecki: better, not sure what happened yesterday, wasn't my typical thing [05:58] zyga: glad that it's better [06:14] heh, so debian sid has some really new version of go, and tests/unit/go bails out do to gofmt changes :0 [06:21] mborzecki: yeah, I noticed :| [06:22] we should disable those or figure out if we can pin the format [06:22] mborzecki: I solved the unit test issues that happen when building in sbuild [06:22] let me push a patch against master [06:22] seccomp? [06:23] no, that is still off [06:23] but I will investigate that next [06:23] I need to compile a few programs on sid toolchain and buster toolchain [06:28] it'd be nice to run the unit tests on sid [06:28] yeah, that's still somewhat away [06:28] we'll probably need to add an off switch for gofmt in run-checks [06:28] the key difference was not sid [06:28] just sbuild [06:29] i meant #6406, it's failing due to gofmt from go 1.11 which uses a different indent length when aligning struct field values & comments [06:29] PR #6406: tests: enable debian sid as part of the main suite on travis [06:32] PR snapd#6409 opened: interfaces/apparmor: mock presence of overlayfs root [06:33] mborzecki: I have more patches but those are of the simple kind that disable stuff [06:34] mborzecki: the key thing I achieved yesterday, was to build a process where it is easy and fast to iterate on this === cpaelzer__ is now known as cpaelzer [06:56] afk === phoenix_firebrd is now known as murthy [07:22] back now [07:22] mvo: hello :) [07:22] mvo: how are you doing after the game last night? [07:23] zyga: hey, still feel a bit battered but ready for work === murthy is now known as phoenix_firebrd_ === phoenix_firebrd_ is now known as murthy [07:27] mvo: I pushed one PR already [07:27] it contains the fixes for unit tests that are applicable to master [07:27] mvo: I also, just now, opened another pull request [07:28] that adds a program to release-tools [07:28] PR snapd#6410 opened: release-tools: add debian-package-builder [07:28] to improve the process on working on debian packaging [07:28] right now the package builds but needs review (it's non-trivial) and it needs more love to fix lintian isues [07:28] adt tests also fail but I didn't investigate any more than that [07:28] please have a look at 6410 as a starting point if that's okay [07:30] zyga: sure [07:30] zyga: thanks! [07:30] mvo: you can reproduce my progress with that PR (just grab the script, it's not tied to the rest of the tree) and the "debian" branch from https://salsa.debian.org/zyga-guest/snapd [07:31] zyga: ok [07:32] zyga: if the package builds and works and lintian is not worse than before, can we push it to the archive? or is anything else missing? [07:32] mvo: it has one E so I think at least that must be fixed [07:32] mvo: I suspect I did something silly in my meld-based-merge [07:33] and we can get rid of most of W-s easily [07:33] mvo: my smoke test would be to install the package and run a few snaps quickly [07:33] #6409 needs a 2nd review and can land [07:33] PR #6409: interfaces/apparmor: mock presence of overlayfs root [07:33] mvo: morning [07:33] mborzecki: good morning [07:34] zyga: yeah, that sounds sensible [07:34] mborzecki: did a second review on 6409 - just one tiny question [07:37] mvo: https://github.com/snapcore/snapd/pull/6405 has a suggestion from Chipaca that can be applied via github [07:38] mvo: replied [07:38] PR #6405: run-checks: ensure we use go-1.10 if available [07:39] PR snapd#6402 closed: spread: increase default kill-timeout to 30min [07:39] PR snapd#6409 closed: interfaces/apparmor: mock presence of overlayfs root [07:39] zyga: thanks, also cherry-picked it now [07:39] thank you! [07:40] zyga: thank *you* :) [07:40] zyga: one thing less to worry about [07:40] mvo: yeah [07:40] mvo: after getting to the point where I can run interactive shell inside sbuild inside debian 10 vm [07:41] mvo: and finding my way around the odd way dh-golang arranges the source code [07:41] it was okay [07:41] mvo: all of the patches I remade/added are in https://salsa.debian.org/zyga-guest/snapd/commits/debian-patches [07:42] mvo: I will look at seccomp now, could you look at lintian please? [07:42] and just see if what I made makes any sense [07:52] Olivier Tilloy thanks for making chromium with vaapi working snap. [07:53] zyga: yeah, could you please pastebin the errors again? [07:53] zyga: or point me to it? [08:39] zyga: I have lintian and your tree etc, looking at this now [08:42] zyga: please apply http://paste.ubuntu.com/p/YkfDs69XSq/ (I can do a formal pull rquest if you prefer) [08:42] zyga: test building that now [08:54] mvo: re, sorry, had some things happening at home [08:54] mvo: applying [09:00] zyga: ta - also your changelog needs one extra space after me@zygoon.pl> and before Thu, [09:01] zyga: there must be two spaces there or lintian is unhappy [09:01] zyga: I look at the ones produced by the binary build now [09:02] zyga: fwiw, the way this can be driven via gbp: http://paste.ubuntu.com/p/YYqSsxV2g6/ [09:03] zyga: this builds fine in sbuild under bionic fwiw [09:03] zyga: (i.e. no unit test or other issues) [09:04] mooing [09:06] meow [09:08] zyga: the other warning that is easy to fix is that snap-discard-ns.8 has "section: 5" in the .rst file, trivial to fix [09:08] ah [09:08] I see [09:08] zyga: there are some more warnings that we should probably just lintian override [09:08] zyga: but I think they are ok for now [09:30] hi, given a .snap file, is it possible to know which arch is it built for? "snap info" doesn't provide that info AFAICT [09:34] ackk: you can look at meta/snap.yaml, at the architecture field [09:34] but there's nothing quick and easy I'm afraiid [09:34] *afraid [09:36] zyga, I ses, so I guess I have to mount the snap filesystem for that [09:36] or extract that file [09:37] zyga, ok, thanks [09:37] zyga, maybe it could be an improvement of "snap info" ? :) [09:39] ackk: do we show that with snap info --verbose? [09:39] or -v [09:43] zyga, no [09:45] zyga: yeah, that sounds like a bug [09:45] ackk: ups, sorry, I mean that sounds like a missing feature [09:45] ackk: you can look at the filename [09:45] ackk: but its *very* crude [09:46] pedronis: you around? [09:49] Chipaca: he is off today [09:49] ah i thought that was yesterday [09:51] mvo: zyga: let's add architecture to the info backlog :-) [09:53] Chipaca: +1 [10:02] PR snapd#6407 closed: tests: get test-snapd-dbus-{provider,consumer} from the beta channel [10:31] mvo: (sorry for the lag, this machine is pretty slow) [10:31] so, I built and installed the package [10:32] I think we should make a few more tweaks [10:32] https://www.irccloud.com/pastebin/uLToDeBM/ [10:32] autoimport, core-fixup should probably be removed [10:32] snap-repair and system-shutdown, as well [10:32] snapd.failure.service, not sure about that one [10:33] mvo: I'll do some more exploratory testing [10:33] mvo: we should upload the package today [10:33] mvo: will the upload impact disco in any way? [10:34] zyga: yeah, all these services can go [10:34] zyga: it shouldn't impact disco [10:35] zyga: we will see [10:35] mvo: I pushed to salsa again, just to ensure that if this crusty machine falls apart we won't lose anything :) [10:35] zyga: +1 [10:36] I'm running microk8s now [10:36] I will run a few more snaps as well [10:36] mvo: I think we need a proper plan on how to bring this back to master [10:36] I feel that despite the effort and progress we are still in "we'll see when we release" state [10:39] mvo: after today I would like to work on feature I'm due to produce, I'm happy to make more progress in 2.38 or 2.37.1 [10:39] mvo: but we should discuss with samuele about what to realistically do about long term plan for snapd release readiness [10:44] zyga: there is a PR up for that :) [10:44] mvo: you mean your PR with sbuild? [10:44] zyga: I mean, we have a plan, we just need to merge your latest additions into 6396 [10:44] zyga: and then we are good [10:44] mvo: I think we still need to figure out how to work with patches [10:45] mvo: we either keep those patches and teach people to work with them [10:45] zyga: well, if we break a patch we break the build in 6396 [10:45] mvo: or figure out how to avoid patches [10:45] yes [10:45] zyga: I think that is acceptable for now and we try to reduce the patches [10:45] 1st requires more learning on all the team members, 2nd requires some ideas and more engineering [10:45] mvo: I would like to adjust debian to use my makefile [10:45] it handles things like "this is for core only, don't ship it" [10:45] zyga: yeah, but the patches are very tiny and isolated, we can always help if they get out of sync [10:46] yeah [10:46] zyga: quick HO for faster communication? [10:46] I mean, as I said, this is much better :) [10:46] zyga: yeah, one step at a time [10:46] but we still need to get through a few releases with no issues [10:46] zyga: I mean, I'm not disagreeing with anything :) [10:46] mvo: we can HO but I think we are in agreement :) [10:46] zyga: just 1. we release from salsa 2. we import salsa into our tree 3. we run the debian packaging on debian-sid in spread 4. ... [10:46] zyga: excellent [10:47] zyga: then lets HO later :) [10:47] yeah [10:47] +1 [10:47] zyga: but yeah, there is more work and open questions down the road! [10:47] zyga: and thanks for working on this! [10:48] mvo: "what about Debian Zygmunt?" :D [10:49] playing with microk8s, I think it works ok :) [10:56] zyga: cool [10:58] another build in progress [11:00] mborzecki: while fresh in my head, I'd like to look at https://github.com/snapcore/snapd/pull/6111 next [11:00] PR #6111: packaging/opensuse: move most logic to snapd.mk [11:00] mborzecki: I will merge master and resolve conflicts [11:00] I would like to merge it as-is [11:00] to avoid bitrot in the makefile [11:01] then use this for 2.37 [11:05] PR snapd#6405 closed: run-checks: ensure we use go-1.10 if available [11:07] zyga: AIUI Neal's comments were phrased as strong suggestions but seem to be actual blockers, or did I misunderstand? [11:07] Chipaca: I don't think those are blockers, the package works OK [11:07] more of a "this is the style to follow", at least in my eyes [11:07] zyga: what's the thing about debuginfo subpackages [11:09] nothing new AFAIK, just a desire to be able to tweak go flags [11:11] Chipaca: also looking at that comment, I'm not sure if that's actually something to do with suse or a preference to make it better for eventual use in fedora [11:11] I don't mind improvements, just want to walk before I run :) [11:39] mvo, well actually I get that file via snap download and it's like snapname_rev.snap. but I can look at the meta.yaml file [11:43] I mean meta/snap.yaml [11:46] ackk: aha, ok [11:55] ackk: but you know the architecture before you download, right? [11:55] ackk: or, rather, you can request a particular architecture :-) [11:56] so then you'd know that's the architecture you got [11:56] Chipaca, well yes and no. this is run from a jenkins job, which takes the revision as parameter. so yes the user know it but I wanted to avoid having to pass it. it's not a big deal at the moment as we're only building amd64, so it can be hardcoded. it'd be nice to have that in "snap info" though, it possile [11:57] ackk: what do you then do with the architecture? [11:57] Chipaca, don't ask :) [11:58] ackk: I ask because I wonder if you contemplate "all" vs ["amd64", "i386"] etc [11:58] bah, ["all"] [11:58] Chipaca, it's used to populate the Architecture field of a debian/control [11:58] well, currently it's not, but that's the idea [11:59] ackk: does debian/control support multiple values? [11:59] not sure [11:59] * Chipaca guesses no [11:59] * Chipaca might not be guessing [11:59] yeah I would think no [11:59] I think it's either "all" or a single arch [12:00] ackk: man deb-control, fwiw [12:00] switched back to my main machine [12:00] man that thinkpad has slow cpu ;0 [12:00] ah, TIL [12:01] Chipaca, btw how do you build a snap for more than one arch? [12:01] ackk: wait, you're creating debs from snaps? [12:02] ackk: build-on vs run-on [12:02] ackk: 1 sec, there's an example [12:02] Chipaca, I said don't ask :) [12:03] ackk: https://forum.snapcraft.io/t/architectures/4972?u=chipaca example 6 [12:06] mvo: I'm happy with CLI testing, I'll do one quick pass over graphical snaps on up-to-date debian-10 [12:06] mvo: then I will ask you to have a last look and dput, okay? [12:06] Chipaca: is there any plan to make snap find more readable on a standard 80x25 terminal? [12:07] Chipaca, thanks, that's nice I didn't know about it [12:08] popey: no plan as yet. I'm aware it's not ideal, and have proposed a few solutions that were shot down [12:08] :( [12:08] popey: currently we want (1) output unchanged (other than minor cosmetic changes) whether it's going to a terminal or to a pipe, and (2) a single result per line [12:09] people pipe the output of snap find? [12:09] popey: my proposals broke both of those :-) [12:09] isn't that what an API is for? [12:10] popey: snap find | grep yadda, yes people do [12:10] and that's reasonable, but I'm hoping we can do something like dpkg -l does which breaks (1) but only lightly [12:10] that'll be my next offering [12:10] when i get to it [12:11] (but it's a blue item) [12:11] popey: do you have any particularly egregious example? [12:13] well, the output is barely readable in a stock terminal [12:14] https://usercontent.irccloud-cdn.com/file/Wsj2NdJH/Screenshot%20from%202019-01-22%2012-14-06.png [12:17] popey: (I think we agree) [12:17] popey: we just need a solution [12:17] This has been solved by apt/dpkg years ago :( [12:18] popey: not really [12:18] what is the solution? [12:19] look at the output from dpkg -l or apt search foo [12:20] 'apt-cache search' just prints the name and the summary; 'apt search' is ungreppable; dpkg -l has you guessing as to whether it's the whole package name or not [12:20] Chipaca: in defense of the idea, I believe find should not require grepping [12:21] well, apt does warn you not to use the output as it's not stable, but can be grepped [12:22] apt search firefox | grep ubufox [12:22] apt search firefox | grep -A 1 ubufox [12:22] I mean, you need to use perl -00 -p '/.../' [12:23] or, that [12:23] * Chipaca itches to write the right perl but stops himself [12:24] but we're arguing over non-essentials I think [12:24] I agree with you that it's not ideal, and I've proposed solutions for it, and the discussion of how to address it is ongoing [12:25] snap find, snap list, snap changes, they're all bad in this sense [12:26] 'snap info' and the progress bars are ok afaik [12:26] Ok. I won't bug you then. Just a shame ::( [12:26] as long as we ignore unicode width that is (e.g. 'snap info unifonter') [12:26] snap info word wraps the summary oddly, snap info libreoffice [12:27] seems finie here [12:27] fine* [12:27] popey: how's it odd? [12:27] https://usercontent.irccloud-cdn.com/file/CLXey6IM/Screenshot%20from%202019-01-22%2012-27-36.png [12:27] " slideshows and databases" [12:28] agh [12:28] you did say summary [12:28] i was looking at description [12:28] i'll fix that [12:29] thanks [12:30] W: snapd: systemd-service-file-refers-to-unusual-wantedby-target lib/systemd/system/snapd.seeded.service cloud-final.service [12:30] hmmm [12:41] hmmm [12:41] snapd doesn't start after reboot [12:41] boot hangs [12:41] I guess the package needs more love [12:47] ah, I was following edge [12:47] switched to stable to really use the package we made [12:48] reboot and same behavior [12:48] hmm [12:51] mvo: snapd.service, failedwith result 'timeout' [12:51] (then we get killed with sigterm) [12:51] snapd.service has onfailure=snapd.failure.service, I guess that should be patched out? [12:52] we are type=notify [12:52] hmmmm [12:53] starting it by hand works ok [12:53] no idea, [12:54] let's see what happens with more debugging [12:55] hmm, not much [13:00] off to pick up the kids [13:09] I've added error handling around sd_notify [13:31] pstolowski|afk: hey [13:31] around? [13:40] man, build cycle is ~20+ minutes [13:43] hmmm [13:43] snapd.service times out starting https://www.irccloud.com/pastebin/F1Jh1CRa/ [13:43] mvo: around? [13:44] zyga: can you try that again with debug? [13:44] it *is* with debug [13:44] zyga: or was the first time already debug [13:44] the first run i mean, that timed out, can you repro it? [13:44] looks like we block somewhere early on [13:44] no, only during reboot [13:44] when we start up [13:44] maybe sanity check mount thing? [13:47] gosh I hate patching packaging [13:47] quilt is the monumental suck [13:48] zyga: yes, sorry for the delay [13:48] no worries :) [13:48] I'm adding more debugging [13:48] but ideas welcome [13:48] if you want we can HO [13:49] zyga: pstolowski|afk is not around this week [13:49] i see [13:49] ok [13:50] zyga: I'm looking at the backlog - do you have a short summary hwat the issue is? [13:50] zyga: aha, start operation timed out? [13:50] snapd works fine [13:50] on reboot it just hangs [13:50] switching to a vt magically unblocks it [13:50] adding debug to pinpoint where [13:50] no more ideas [13:50] zyga: on reboot only? or on stop as well? [13:51] on boot up [13:51] not shutdown part of reboot [13:52] zyga: oh, on bootup - how strange [13:52] mvo: I pushed all the patches to salsa [13:52] I can also share a .deb for debugging [13:52] PR snapcraft#2446 opened: Update schema/snapcraft.yaml [13:53] added more and running another build now, ~~ 20 minutes till results show up [13:53] zyga: can you set SNAPD_DEBUG=1 in /etc/environment [13:53] I did [13:53] the log above was with that in place [13:53] zyga: oh [13:53] right? curious [13:53] zyga: so nothing between Starting ... and timeout :/ [13:53] correct [13:53] my hunch is on mount [13:53] deadlock/ [13:54] we'll ee [13:54] see, gosh keyboards [13:54] sty 22 14:41:30 debian-10 systemd[1]: Starting Snappy daemon... [13:54] sty 22 14:42:35 debian-10 snapd[1040]: AppArmor status: apparmor is enabled but some kernel features are missing: dbus, network [13:54] zyga: can you dump "systemd-analyze --plot" somewhere? [13:54] zyga: well, second start takes 1m05 until output [13:54] on it [13:54] zyga: so strange [13:54] mvo: it starts only after I switch vt [13:54] zyga: also systemd-analyze blame [13:55] zyga: strace would be great, but I guess thats hard - or could you ssh into it? [13:55] mvo: I'll try [13:56] sigh [13:56] pastebinit broken on debian [13:56] I hate that it just doesn't work on fedora / debian normally [13:56] zyga: snap it :-D [13:57] zyga: huh, broken even for paste.debian.net? [13:57] mborzecki: echo broken | pastebinit [13:57] it's broken if that doesn't work [13:57] it doesn't work [13:57] damn [13:57] if you have to configure it it is broken [13:57] mborzecki: this being debian, I assume it needs post-install tweaking to work [13:57] https://pastebin.ubuntu.com/p/XDpPTRhsG8/ [13:57] snapd.seeded [13:58] zyga: doesn't snapd have after: snapd.seeded? [13:58] or was snapd.seeded a snapd thing [13:59] zyga: journalctl -u snapd.seeded? [13:59] snapd.seeded pokes snapd [13:59] i cannot paste the .svg file [13:59] because reasons on pastebin [13:59] ah, seeded is after=snapd [13:59] so snapd does not need seded, indeed [14:00] standup [14:00] zyga: copy it to people.c.c (or dump it in your lab :-) ) [14:00] * Chipaca stands up [14:00] and then? [14:00] maybe it's slow key generation? [14:00] ssh is also slow to start [14:00] ssh is slow yes [14:00] no idea [14:02] https://www.irccloud.com/pastebin/rTkZQ1hN/ [14:02] that's for snapd.seeded.service [14:09] zyga: how's your random pool? [14:10] Chipaca: hard to say but that kernel seems to log and hand out randomness that's not really random [14:10] let me reboot and see [14:10] zyga: sysctl kernel.random.entropy_avail [14:11]