[06:19] morning [07:31] mvo: hey [07:32] mborzecki: hey, good morning! [07:32] mborzecki: how is life, anything important happend on friday? [07:33] mvo: iirc nothing super important, pedronis is off today [07:35] mvo: the pulseaudio test seems to be causing issues on 20.04 [07:35] mborzecki: yeah, cool. no bad news is good news [07:35] mborzecki: anything in particular needs reviews? [07:35] * mvo is current going top-bottom [07:35] mvo: i was not albe to reproduce it, but according to standup notes ian maanged to do so calling spread in loop ratehr than with -repeat [07:36] mborzecki: hm, that's annoying, any clues? [07:37] mvo: no, not really, maybe we should consider skipping the test on 20.04 until we figure it out, seems to break travis runs quite often [07:38] mborzecki: yeah, that works for me [07:38] mvo: oh, and 8077 could use a review ;) [07:38] #8077 [07:38] PR #8077: boot: enable base snap updates in bootstate20 [07:41] mvo: and something off with autorefresh https://forum.snapcraft.io/t/snap-is-updating-off-schedule/15323/10 even though the schedule is calcualted correctly [07:45] mborzecki: and it's not because the last update happend ages ago ? (or there was some sort of clock glitch?) [07:46] good morning [07:47] hey zyga [07:47] sorry, I overslept [07:47] zyga: no worries [07:47] but almost on time for 9 am start [07:47] zyga: flexible hours++ [07:47] zyga: :) [07:47] winter holidays started for the kids today [07:47] so no alarm clocks or anything [07:48] zyga: sounds very nice if you ask me :) [07:49] yeah, all quiet [07:49] zyga: hey [07:49] :) [07:49] let me make some coffee quickly [07:50] mvo: no, looking at the log there's an update on Feb 05 12:22:39 and anothr one on Feb 06 09:54:28, while the 'ages ago' thing is 60 days [07:58] mborzecki: meh, that's annoying, thank you [07:59] morning [08:02] pstolowski: hey [08:03] hey pstolowski ! [08:04] quick errand, back in 30 [08:09] hey pawel :) [08:13] PR snapd#8096 closed: tests: retry mounting the udisk2 device due to timing issue [08:29] re [08:33] hmmm i need to do something about task pruning wrt pre-seeded image [09:59] re [10:00] pstolowski: can you tell me more please? [10:01] hmm so i reproduced the problem with pulseaudio, and looks like the pulseaudio daemon is not actually listetning on any socket [10:03] mborzecki: woah [10:04] mvo: i'll try with a custom script for the server [10:06] mborzecki: how is that possible [10:06] mborzecki: as in misbehaving and not listening on any socket [10:07] zyga: on the pre-baked image we have changes & tasks with spawn-times in the past. i wonder if prune mat abort them if the image is booted after >24h from pre-baking [10:07] s/mat/may/ [10:08] zyga: idk, lsof clearly shows it's not listening on anyting, no logs, nothing, any action tries /run/user/12345/pulse/native and gets connection refused (cause nobody's listening) [10:08] pstolowski: ah [10:08] pstolowski: interesting! [10:08] zyga: restarting pulseaudio magically fixes this [10:08] pstolowski: that's tricky, do we need any extra state so that they are not pruned? [10:08] mborzecki: is that easy to reproduce? [10:10] zyga: i wonder if that isn't a general concern (but maybe i'm missing something): what if you have a change in flight, power off in the middle, turn back on after >24h. you may hit readyTime.IsZero() == true and spawnTime.Before(abortLimit) == true afaict, meaning the change gets aborted. this is if you are unlucky and the change does not finish in 10 minutes after boot (this is the prune ticker interval) [10:11] zyga: not really, happened just one so far [10:13] zyga: i don't think we need extra state for pre-seed case, should be enough to check if there is any "become-operational" change and if so, just block pruning for some time based on its ready-time [10:14] i need a test case, this is indeed a bit tricky stuff [10:16] pstolowski: yeah, I think we don't model suspend/off time in our system [10:41] E: [pulseaudio] socket-server.c: bind(): Address already in use [10:41] E: [pulseaudio] module.c: Failed to load module "module-native-protocol-unix" (argument: ""): initialization failed. [10:41] heh [10:46] mborzecki: that's interesting [10:46] mborzecki: do we have any pulseaudio fakes? [10:46] mborzecki: that might clobber the unix socket [10:46] mborzecki: or perhaps it's something more silly, like stale socket being left in place [10:46] mborzecki: I don't recall what happens if a socket is in the FS and you try to create a new one there [10:47] zyga: added rm -f /run/user/12345/pulse/native just in case it's needed [10:47] mborzecki: do we remove /run/user/12345 anywhere? [10:51] mborzecki: I think the code that handles the test user [10:51] mborzecki: as in, uses it [10:51] mborzecki: could remove /run/user/12345 after each test [10:56] zyga: and probably kill all processes of test user too [10:56] mborzecki: yeah, that's a good idea [10:56] mborzecki: too bad that doesn't scale to "root" :-) [10:56] mborzecki: it would be easier for us [11:00] PR snapd#8108 opened: tests/main/interfaces-pulseaudio: use custom pulseaudio script, set kill timeout [11:00] mvo: zyga: ^^ [11:00] reading [11:00] still running some tests, will force push as needed [11:02] reviewed [11:02] hmm why does paplay hang when it cannot connect after the plug was disconnected? [11:03] mborzecki: paplay is simple, maybe it assumes too much [11:03] mborzecki: but really, no idea [11:03] mborzecki: does it spin or hang? [11:44] mborzecki: can you look at https://github.com/snapcore/snapd/pull/7980 [11:44] PR #7980: packaging,snap-confine: stop being setgid root [11:44] zyga: sure [11:44] mborzecki: I've handled the review from jamie and apart from one small tweak it is ready for re-review [12:21] mborzecki requested a review on https://github.com/snapcore/snapd/pull/8108 [12:21] PR #8108: tests/main/interfaces-pulseaudio: use custom pulseaudio script, set kill timeout [12:21] er [12:21] not on this one [12:21] I meant 7925 [12:22] * zyga switches to https://github.com/snapcore/snapd/pull/7825 [12:22] PR #7825: many: use transient scope for tracking apps and hooks [13:00] zyga: some comments in https://github.com/snapcore/snapd/pull/7980 [13:01] PR #7980: packaging,snap-confine: stop being setgid root [13:01] aha [13:01] looking [13:01] thank you for reviewing! [13:07] mborzecki: updated [13:07] mborzecki: please look again :) [13:08] I need to take Bit out [13:10] zyga: thanks! [13:26] PR snapd#8107 closed: tests/lib/prepare: use a local copy of uc20 initramfs skeleton [13:26] #8081 needs a 2nd review, zyga maybe you can take a look since you did a pass before? [13:27] PR #8081: tests/main/user-session-env: add test verifying environment variables inside the user session [13:27] mborzecki: yes [13:27] mborzecki: looking [13:27] thanks! [13:28] mborzecki: do you remember why [[ vs [ ? [13:28] wow [13:28] zyga: [[ is bash builtin, [ is /bin/test alias, but bash does a builtin anyway? [13:28] tunderstorm [13:28] thunder hit nearby [13:28] wow, that was LOUD [13:29] wonder if any glasses broken [13:29] or windows or whatever [13:29] zyga: had hailstorm here today :P [13:29] and +9C :P [13:29] mvo: how is your weather? do you experience the big storm that sweeps across EU now? [13:29] mborzecki: same here +9 [13:29] it's so warm for such a windy day [13:30] mborzecki: as for [[ vs [, I would use [ unless [[ is required, [13:31] zyga: i can push a patch htere [13:31] because it's non obvious why [[ is used here and we use [ nearly everywhere unless we require extra semantics from [[ [13:31] mborzecki: I can apply a inline suggestion, just wanted to understand why first [13:31] zyga: inline is fine, saves the checkout/commit/push cycle === ricab is now known as ricab|lunch [13:33] yep [13:33] thanks [13:34] mborzecki: no permission to push [13:34] zyga: weird [13:34] let me do it locally [13:34] try applying my suggestion [13:34] maybe it's that [13:42] happy monday folks [13:48] mborzecki: were you able to sort out the weirdness with pulseaudio ? [13:48] I have some thoughts from my investigation on Friday that I didn't get around to putting in the SU doc [13:49] ijohnson: yeah, sort of, plase look at https://github.com/snapcore/snapd/pull/8108 [13:49] PR #8108: tests/main/interfaces-pulseaudio: use custom pulseaudio script, set kill timeout [13:49] ack looking now [13:49] perhaps not all of it is needed, but i suppose it won't hurt [13:51] wow, thunderstorm here [13:51] ahhhh okay your last line at the end explains one of the things that was very confusing to me [13:52] I was constantly seeing it fail when I had restarted it in a way that it stuck around, but having your last statement kill it would make the execute not fail [13:52] it was very confusing because I even put a `exit 0` at the end of the execute section and it still would say the run failed [13:56] ijohnson: yeah, i believe the pulseuadio process is keeping the pty allocated by ssh still open, so either nohup with that pulseaudio start line, or su -P (?), or just kill it before leaving [13:57] ijohnson: that daemon start script is superfluous, but it makes the setup simpler, eg. pulseaudio doesn't try to use bluez and so on [13:57] mborzecki: great! yeah I think your PR looks good, did you try it running spread in a loop? [13:58] ijohnson: i was able to get it to block on each run eventually :P [13:59] btw https://news.ycombinator.com/item?id=22279585 (cc zyga) [14:37] mvo: let me review the netlink code now [14:43] cachio: are "fedora-rawhide-64" images something you build? we're having trouble with GPG keys on those https://github.com/MirServer/mir/issues/1260 [14:43] Saviq, yes [14:44] well I just update this [14:44] apply the latest changes [14:44] Saviq, I'll take a look [14:51] aaand no power, running on battery [14:58] mborzeck1: oh no hope your power is restored soon [15:01] mvo: I'll grab lunch/dinner and sit on that netlink PR [15:02] mvo: I'm not super happy about it, it's very messy still (I know there are constraints) [15:02] mvo: I'd like to make it simpler and more robust [15:05] PR snapcraft#2924 opened: cli: explicitly ensure type 'snapd' is non-legacy === ricab|lunch is now known as ricab === msalvatore_ is now known as msalvatore [15:16] * zyga eats [15:19] ijohnson: posted some comments in #8077 [15:19] PR #8077: boot: enable base snap updates in bootstate20 [15:19] mborzeck1: thanks will look after meeting [15:19] and power is backing, switching systems [15:19] s/backing/back/ [15:26] zyga: in a meeting right now [15:26] no rush [15:26] zyga: but yeah, thank you [15:26] zyga: simple++ === Eleventh_Doctor is now known as Pharaoh_Atem [15:39] * cachio lunch [16:04] mvo: I had to add one more commit to https://github.com/snapcore/snapd/pull/8106, do you think I can merge if it goes green now? [16:04] PR #8106: tests: add "core" suite for UC specific tests [16:13] ijohnson: +1 [16:13] zyga: back now [16:13] thanks mvo [16:13] ijohnson: thank you :) [16:14] also it seems mborzecki's pulseaudio changes aren't enough, I will try to merge my stuff with what he has and if it works I will push to his branch [16:16] oh fun [16:23] * ijohnson goes to buy another goat to sacrifice to the pulseaudio spread test gods [16:29] PR snapd#8109 opened: [RFC] o/devicestate: don't prune tasks immediediately [16:33] pstolowski: some instant suggestions on comments [16:34] zyga: thanks, i think i'm gonna change it, just had an enlightement [16:35] should I refrain from reviewing the rest? [16:36] zyga: yep, please hold on, thanks! [16:36] ok [16:44] jdstrand, I would appreciate your input on bug #1859643 [16:44] Bug #1859643: [snap] cannot use shared NSS db [16:49] oSoMoN: I believe jamie is off today [16:49] zyga, ah, thanks, I'll try again tomorrow then [16:50] zyga: I updated 8063 now [16:51] aha [16:52] let me read it quickly [16:53] mvo: done [16:53] mvo: it looks great, there are a few comment that were not responded to though [16:53] let me update those that were quickly [16:54] done [16:54] mvo: can you scan through those quickly [16:54] I think a quick line in remove-user/task.yaml will solve the first one [16:54] the [[ can probably just become [ [16:55] the wording on snap remove user is something we can talk about (not blocking) but I think the original error is not comprehensible much [16:55] zyga: aha, github did hide some from me, sorry for this [16:55] mvo: the api docs can be "not documented" [16:55] zyga: I see them now [16:55] zyga: let me look at those [16:55] mvo: but I wasn't sure [16:55] mvo: I guess we ought to document this [16:55] mvo: but not super critical [16:55] mvo: if you rush this [16:55] mvo: lastly since this deals with users I would love to see even a post-factum review from security [16:56] mvo: and perhaps the test I suggested [16:56] as it could be a nasty CVE if that were to work [16:56] mvo: (this doesn't capture that the changes you made look great) [16:56] * zyga hugs mvo [16:57] zyga: thank you! I address the other bits now [16:57] * mvo hugs zyga === ricab is now known as ricab__ [17:07] ijohnson: hey, will you find some time to look at #8046? [17:07] PR #8046: many, tests: integrate all preseed bits and add spread tests [17:07] pstolowski: yes I will try to get to it today or tomorrow [17:08] ijohnson: thanks! [17:18] * zyga breaks, family is back === heather1 is now known as hellsworth [17:28] PR snapd#8110 opened: store: add support for resume in DownloadStream [17:43] mvo, https://paste.ubuntu.com/p/Vns7w2Jq8v/ [17:58] PR core20#21 opened: Add bash-completion support [18:00] PR snapd#8111 opened: tests: Adding new backend which includes images with tpm support [18:02] * cachio afk [18:03] I'll be out 30 minutes [18:06] PR core18#147 opened: Add bash-completion support [18:12] mvo: I'm making progress on netlink code [18:12] mvo: I will have it tonight [18:33] ah, maciek is off already [18:42] zyga: nice [18:43] cachio: nice! [18:43] cachio: cool to see this [19:06] PR snapd#8106 closed: tests: add "core" suite for UC specific tests [19:45] I sent mvo the epoll based netlink stuff [19:45] and will wrap up for the day [19:46] ijohnson: I can telegram you a zip if you want to see [19:46] zyga: if I want to see what? [19:46] ijohnson: epoll based netlink listener in go [19:46] oh, nah I have other things to review/look at this afternoon :-) [19:47] in other news I think I have the pulseaudio spread test under control [19:47] :) [19:47] sure [19:47] haven't hit the failure yet in 2 consecutive runs :-) [19:47] have fun with pulseaudio there, I'd love to know what's the root problem in the end [19:56] don't get your hopes up too much, my solution is just to start it, kill it violently, and restart it, but there's now at least some more information in the logs if someone wanted to file a bug upstream with pulseaudio folks [20:02] PR snapd#8112 opened: tests: move main/ubuntu-core-* tests to core/ suite [20:21] ijohnson: can I assign dimitri's snapcraft build bug to you, just as a reminder to unblock the PR once the prerequisite bugs are fixed? [20:21] cmatsuoka: sure, I just updated the PR today, so it's ready as soon as snapcraft is ready [20:22] ijohnson: thanks, hopefully we'll have this fixed soon [20:23] ondra: wrote regression test, fix coming up tomorrow [20:36] Issue core20#20 closed: Unpublish the core20 snap for i386 [20:56] Has anyone tested on Fedora 31 recently? It appears that it's broken and has been for a while [21:00] https://forum.snapcraft.io/t/error-system-does-not-fully-support-snapd/14911/14 [21:00] PR snapcraft#2920 closed: meta: initialize Snap at once in from_dict() [21:01] Snapd, that is [21:19] Saviq, I left a message in the issue [21:19] with that the problem should be fixed [21:23] cachio: thanks! [21:23] Saviq, yaw, [21:30] cachio: while we could do this, shouldn't an image rebuild fix this? It feels wrong to ignore gpg errors? [21:33] PR snapcraft#2919 closed: sanity_checks: fix sanity check on Windows [21:34] Saviq, we have these error often on rolling systems [21:35] in opensuse we already and usign the same option [21:35] on tumbleweed [21:36] Saviq, I can try tomorrow to fix that [22:00] PR snapcraft#2925 opened: project loader: use get_build_base() to assign plugin handler base [22:06] ondra: https://github.com/snapcore/snapd/pull/8113 [22:06] PR #8113: cmd/snap-confine: bring /var/lib/dhcp from host, if present [22:06] PR snapd#8113 opened: cmd/snap-confine: bring /var/lib/dhcp from host, if present [23:10] PR snapcraft#2924 closed: cli: explicitly ensure type 'snapd' is non-legacy