[06:24] morning [07:51] hey mborzecki [07:51] mborzecki: who's in today? [07:52] zyga: hey, happy blue monday ;) [07:52] oh is that today? [07:52] I'm feeling pretty happy :) [07:53] more branch iteration [07:53] need to double check roadmap work [07:53] zyga: mvo, samuele and Chipaca are out, maybe cmatsuoke too (?) he mentioned something during the standup didn he? [07:53] mborzecki: chachio is out too AFAIR [07:53] zyga: oh, right, summer time ;) [07:54] zyga: oh, and ian is out today too? [07:54] haha really? [07:54] in that case we can just chat in #snappy-pl ;) [07:54] hahah [07:56] zyga: have you read about the drama in rust community? [07:56] zyga: https://words.steveklabnik.com/a-sad-day-for-rust [07:57] no, did someone die? [07:57] hmmm [08:01] I will call my programs unsafe.c ;) [08:02] this whole thread re-affirms my perception that most social networks are poison [08:02] and reddit is particularly bad [08:02] it's one to argue with someone you know [08:02] it's another thing to play the game "endless waves of people talking you down" [08:02] zyga: idk, imo twitter takes the crown [08:03] as the worst social network [08:03] I disagree for a single reason [08:03] twitter is more of a rain [08:03] you can get hit by drops and stuff [08:03] but it's not a place where you are under a open pipe [08:03] reddit has sub-reddits that amplify this [08:03] reddit has "better" thread tools that amplify efficiency of negative feedback [08:04] it doesn't negate twitter being junk in other forms but I haven't seen as much crap there, specifically on FOSS topics [08:04] hey pawel [08:04] pstolowski: hey [08:04] heyas [08:04] dzindybry [08:05] zyga: yeah, readdit is quite effective at spreading negative feedback, though imo still twitter is where most drama takes place, and it's a short ride from twtitter to shitty news outlets everywhere [08:05] mborzecki: for general news yes [08:06] mborzecki: e.g. trump [08:06] mborzecki: but my entire foss/twitter interaction was way better than reddit [08:06] to change the topic slightly [08:06] I was playing with C code coverage over weekend [08:06] specifically using clang on macos [08:06] and I learned a few things [08:07] zyga: old dog, new tricks? :) [08:07] more like alt toolchain I rarely use [08:08] my toy library has better code coverage now [08:08] it is just all these fancy new languages you all use ... rust ... go ... nobody complains about shell on reddit ;) [08:08] not at 100% yet but much closer [08:08] also wrote a few new manual pages [08:08] I'd say 1/3 of the API has manual pages now [08:08] ogra: oh [08:08] zyga: about C, did you use that pvs studio open source license and tried to analyze out C bits? [08:08] I found a cool thing for shell [08:08] one sec [08:08] mborzecki: yes I did [08:09] * ogra sits in awe .... new things ! [08:09] ogra: a pep-8 like thing for shell [08:09] oh gosh, how was it called... [08:09] shellcheck on steroids ? [08:09] smellcheck probably :) [08:09] *sniff* *sniff* [08:11] oh well, I cannot remember [08:11] maybe I'll recall it [08:12] i'ts fine ... usually my code doesnt survive long enough to be worth checking it for pep-8 before someone rewrites it in a fancy lang ;) [08:12] zyga: https://github.com/openstack/bashate ? [08:12] mborzecki: no, it had a more polite name [08:12] I should have kept the tab [08:14] btw, i finally managed to get gnome-shell up on core with mir-kiosk on the weekend (only very rudimentary yet) ... [08:15] my snapcraft.yaml only has ~100 layout overlays yet (with more to come i guess) [08:15] layouts really rock ! [08:16] (until you decide to move an overlayed dir to single files with an upgrade and cant uninstall the sanp anymore at least :P ) [08:16] uh [08:18] what would be a really helpful feature would be "overaly every file thats in the snap but not in core in one go" [08:18] *overlay [08:19] i.e. simply diffing the dir structure and binding all non-existent files ... [08:21] i was surprised how many bits and pieces are still hardcoded in gnome and not overridable by env vars ... (search paths for data in /usr/share etc ... ) [08:23] ... and finding out that even gnome-terminal doesnt run without dbus, systemd and logind anymore was a rather shocking moment ... half of it is javascript nowadays !! [08:59] PR snapd#8020 closed: interfaces/apparmor: fix doc-comments, unnecessary code [09:01] mborzecki: do you think you could look at https://github.com/snapcore/snapd/pull/7980 [09:01] PR #7980: packaging,snap-confine: stop being setgid root [09:01] zyga: sure [09:33] * zyga has enormous headache :/ [09:33] the air today is terrible [09:35] zyga: so is atmospheric pressure, was suppsoed to be 1050hPa [09:35] uh, that's high [09:35] PR snapd#7995 closed: debian: check embedded keys for snap-{bootstrap,preseed} too [09:35] hey mvo [09:35] :) [09:35] hey zyga [09:35] mvo: hey! [09:36] any news? [09:36] hey mborzecki [09:36] hi mvo ! [09:36] mborzecki, zyga not much yet [09:36] hey pstolowski [09:37] zyga: though the pressure sensor in my phone is showing 1024hPa [09:38] but it looks like we need to revert the do-not-show-writable PR, I think john is on this. or are there more news? [09:38] mvo: do not show writable PR? [09:38] mvo: are you referring to /writable unmounting we do? [09:38] zyga: yeah, let me try to give you more context [09:39] zyga: looks like statvfs("/writable") was used and that is no longer working [09:39] mvo: yeah, I know this from last week - did we find out why they are doing it? [09:39] zyga: monitor diskspace [09:39] mmm [09:40] it's not really reliable [09:40] but I see [09:40] zyga: because we do not work well when we run out of diskspace [09:40] zyga: yeah, we need to fix the out-of-diskspace problem [09:40] I mean the check itself is also unreliable [09:40] PR snapd#8018 closed: spread.yaml: fix ubuntu 19.10 and 20.04 names [09:40] because partitions [09:40] zyga: in what way? [09:40] zyga: can you expand that a bit please? [09:40] because /writable is just one directory [09:41] maybe /writable/user-data is another [09:41] it's not fixed in stone [09:41] it's just a way to observe current implementation on some systems [09:41] zyga: aha, I see. but for their devices it's all on a single partition. but in general yes, it may not be [09:41] zyga: yeah, thanks, I get what you mean now [09:42] mvo: I'm okay to revert this for now but I'd like not to be hostage to /writable [09:42] as in, that it is a single partition and that you can stat it to know disk space [09:42] btw. this isn't reported in any way in /sys right? [09:42] zyga: yeah, I think we may need to short-term revert, and then ensure we deprecate this mount and then remove it again in 1-2 releases [09:42] mvo: agreed [09:42] mborzecki: it might be [09:42] zyga: thanks! [09:42] mborzecki: a good question, that would be interessting [09:44] mvo: /sys/ext4/ constains some files but none that seems to carry size/free info [09:44] * mvo nods [09:44] mborzecki: ironically [09:44] mvo: stat -f $HOME is an equally good check [09:45] zyga: heh, yeah, but looks like i can trigger fs errors :P [09:45] mvo: maybe we should not revert [09:45] mvo: but instead ask to tweak the one-liner stat check [09:45] mvo: AFAIK it's one customer and one bit in their software [09:45] zyga: yeah, I will explore if that is possible [09:45] mvo: thank you [09:45] mvo: would certainly be more in line with what would work long term IMO [09:45] zyga: and you need a mounted fs to check free space too? [09:45] mborzecki: haha - really? [09:46] mborzecki: yes [09:46] zyga: will this work inside the snap too? [09:46] mvo: yes, stat is not mediated [09:46] mvo: nor is statfs [09:46] mvo: but I can double check if you want me to [09:46] * zyga checks on a pi next [09:46] zyga: if it's not too much hassle would be nice [09:47] zyga: a good alternative will make the conversation easier [09:49] mvo: it works [09:49] https://www.irccloud.com/pastebin/r6fnRvXK/ [09:49] mvo: this is on 2.43.1 [09:49] mvo: so I'd vote +1 on customer change rather than our change unless they strongly oppose [09:50] mvo: but even if they oppose we should have a proper interface for disk space [09:50] mvo: statfs on a random picked directory is not it [09:50] zyga: does "snapd-hacker-toolbelt.busybox stat -f $HOME" also work? [09:50] zyga: +1 for proper interface [09:50] zyga: also we should just be able to handle low/no diskspace better [09:50] mvo: yes because $HOME is expanded in unconfined shell [09:51] mvo: and it is unmediated as far as apparmor is concerned [09:51] mvo: it is also working with snap rewritten home [09:51] https://www.irccloud.com/pastebin/wBLAE3nq/ [09:51] zyga: nice, I will see if we can get away with this then [09:51] cool :) [09:52] I didn't think of this last week [09:52] but I think it's a clean solution out of this problem [09:52] I need to grab painkillers, as infrequently as I use them I cannot work with constant headache :? [09:52] mvo: high-pressure wave across .pl [09:52] brb [09:56] zyga: uh, good luck! [10:00] PR snapd#8022 opened: Revert #7421 [10:01] back [10:03] <__chip__> zyga: 👋 [10:03] hey __chip__ :) [10:04] <__chip__> zyga: can you take a look at #8022 ? there was a conflict in the revert so I hand-edited a couple of files, but I'm not sure they came out OK. I ran the spread test but the conflicts were in 1604 test files that don't seem to be exercised ((?)) [10:04] PR #8022: Revert #7421 [10:04] <__chip__> concretely tests/main/mount-ns/google.ubuntu-core-16-64/PER-USER-16.expected.txt and tests/main/mount-ns/google.ubuntu-core-16-64/PER-SNAP-16.expected.txt [10:04] <__chip__> were the ones with conflicts [10:05] ok [10:05] <__chip__> (I'm not sure they're exercised as that spread test only runs in 1804) [10:05] __chip__: FYI check out the conversation with mvo just above [10:05] <__chip__> zyga: I've only just logged onto irc so i'll have to wait for ubuntulog2 for that :) [10:06] __chip__: idea to use stat -f $HOME instead of the revert [10:06] __chip__: and ask the customer to use that instead [10:06] <__chip__> zyga: $HOME, or /home ? [10:06] __chip__: due to how statfs works they can check /var/snap, /var/lib/snapd and other places [10:06] <__chip__> $HOME is probably /root FWIW [10:06] __chip__: /home would work as well [10:07] that works too btw [10:07] <__chip__> yep [10:07] <__chip__> zyga: I think we can ask the customer to make any amount of changes, the issue is the timeline [10:07] __chip__: I was concerned about cementing /writable as an interface [10:07] __chip__: changing stat on their side is a one liner [10:07] <__chip__> agreed, and note i targeted the release branch with the pr, not master [10:08] and it works in past snapd versions as well [10:08] __chip__: sure, I just wanted to recap what was said above [10:08] <__chip__> zyga: i wouldn't presume to know the timeline of them releasing a one liner [10:08] <__chip__> but yeah if they can change it quicker than we can revert it, +2 [10:08] yeah [10:09] I don't know either [10:09] <__chip__> in any case that's async [10:09] <__chip__> we can have this pr ready to go if they come back with '2 months to get this rolled out to prod' [10:09] I'll adjust your PR to pass [10:09] <__chip__> ah, the PR title is probably wrong :-) thanks [10:13] <__chip__> zyga: i pinged you on telegram before coming on here because sprint-mode [10:13] <__chip__> zyga: you can ignore that now :-D [10:14] <__chip__> also it might've gone to your spanish phone # [10:14] I see it now :) [10:14] <__chip__> :) [10:14] it's my proper telegram number btw [10:14] <__chip__> i see you seeing it [10:15] cjwatson: hi, do you have access to the machine where https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1860324 occurred? [10:15] Bug #1860324: snapd crashed and can't start: "Re-refresh task has 1 tasks waiting for it"

[10:15] * __chip__ out [10:28] mborzecki: yes, it's my normal development laptop [10:29] cjwatson: can you post the output `snap changes` command? or if snapd cannot start, then make sure it's stopped and try `snap debug state /var/lib/snapd/state.json` and post that [10:32] mborzecki: added to the bug [10:33] cjwatson: thanks! [10:37] hmmm, so switch-snap channel got added, and everything is ordered before it, incl rerefresh [10:41] mborzecki: ? [10:42] zyga: see the bug report ^^ [10:43] heh, Switch snap "gitlptools" from channel "latest/stable" to "stable" [10:43] uh [10:43] that's not good [10:43] wish chipaca was around ;) [10:44] hm potentially same thing with toggle-snap-flags task [10:46] snap debug state is lovely [10:46] but that argument could be optional (state file path) [10:54] zyga: it would be super useful to teach it more things [10:54] pstolowski: now's the chance to merge ;-) [10:55] zyga: haha [11:50] PR # closed: core#38, core#107, core#108, core#110 [11:51] PR # opened: core#38, core#107, core#108, core#110 [12:18] PR snapcraft#2886 opened: Use ensure_dir_exists instead of mkdir -p [12:29] cjwatson: would you be able to attach the output of `sudo cat /var/lib/snapd/state.json| jq '.data.snaps.gitlptools'` too? [12:30] mborzecki: done [12:30] cjwatson: cool, thanks [12:36] mborzecki: any closer? [12:40] zyga: no, not really, added a managers level test, but can't seem to reproduce it [12:42] mborzecki, zyga it's pretty scary we have logger.Panicf("Re-refresh task has %d tasks waiting for it.", numHaltTasks) [12:42] mborzecki: do you have a theory what is going on? [12:42] pstolowski: are we hitting that? [12:43] * zyga is out of sync with that bug [12:43] zyga: yep, it's in the desc of the PR [12:43] Jan 20 09:54:38 niejwein snapd[9861]: panic: Re-refresh task has 1 tasks [12:43] uh [12:43] I see [12:43] some assumption doesn't hold [12:43] is this code in the candidate branch? [12:44] yeah.. root cause is something else, but we shouldn't do this.. [12:44] can we degrade that to an error [12:44] so that some refresh goes on [12:44] I really think we should not panic :/ [12:44] yeah, not in the task handler that is bound to be re-run [12:45] brb, [12:47] mborzecki: so yeah, as you said above (sorry, just catching with the backlog on that bug), it's because of adding switch-snap-channel [12:49] re [12:57] mborzecki, zyga so i think (afair from talking to pedronis long time ago) the basic invariant has always been for check-refresh to be the last task. UpdateWithDeviceContext seems to have bug as it can append to tasks from doUpdate [12:58] pstolowski: i think it's related to having channel: "stable" in the state, and the code around that does a simple lexical comparison [12:59] mborzecki: that may be another problem then.. but as is, I think nothing prevents UpdateDeviceContext from appending task after check-rerefresh, in which case check-rerefresh will panic [13:01] pstolowski: so we have possible aliases update, a task gets added, then rerefresh (why is the alises update done at all?), then level up the call stack, switch channel is added because snapstate channel is "stable", while we currently resolve "stable" (that came from the command line) to "latest/stable" [13:03] pstolowski: oh, and another problem, now snapd is stuck and panics, how we get out of that ;) [13:03] (other than editing the state) [13:03] mborzecki: i see. i suppose it would end up badly also with a switch from stable, to say latest/edge? [13:05] mborzecki: bummer.. [13:05] we shouldn't panic in task handlers :( [13:05] first time to use recovery mode? [13:10] For myself I'm perfectly happy to edit the state if given instructions on how to do that correctly, but I assume this might affect others [13:20] hmm edited my state in a way that should break it, but no luck there === ricab is now known as ricab|lunch [14:04] PR snapcraft#2885 closed: lifecycle: only warn when a default provider snap is missing [14:04] PR snapcraft#2886 closed: Use ensure_dir_exists instead of mkdir -p [14:17] PR snapcraft#2887 opened: 3.9 cherrypicks [14:26] net issues? [14:26] yeah, I have net issues at home [14:29] sil2100: https://github.com/snapcore/pi-gadget/issues/13 did you see? [14:36] PR snapd#8023 opened: devicestate: encryption regardless of grade [14:59] re [15:03] mborzecki: thanks for the PR to etrace, the options there aren't well documented :-( [15:03] ijohnson: figured it's a WIP :) [15:04] mborzecki were you using it to measure a cli program or did the window name stuff not work? [15:04] Yes still a WIP === ricab|lunch is now known as ricab [15:04] ijohnson: started with `etrace run ls` [15:05] Ah yes that probably just got stuck [15:09] pstolowski: haha, got `panic: Re-refresh task has 1 tasks waiting for it.` [15:11] mborzecki: re stable vs latest/stable, it seems we have a bug in that we don't normalize --stable and then stable != latest/stable in snapstate's Update [15:11] mborzecki: uhh, what did you do? how? [15:11] pstolowski: let me open a PR [15:12] however, i still ahve no idea how we ended up tracking the 'stable' channel expliclitly in the state [15:12] mborzecki: good work L) [15:12] mborzecki: i've a theory on that [15:12] :) [15:14] mborzecki: in patch 6_3 we only normalize tracking channel, but not the channel sequence (maybe that's fine, i don't know). perhaps that revision was installed with older snapd so didn't get normalized. revisions installed with new snapd get the full channel inside sequence [15:18] PR snapd#8024 opened: overlord/snapstate: add reproducer for LP#1860324 scenario [15:19] pstolowski: ^^ [15:19] mborzecki: as for aliases you're right, the number of aliases changed, and this resulted in check-rerefresh task (because reportUpdated is non zero) [15:19] pstolowski: yeah, and snap declarations are refreshed on their own [15:39] jdstrand, hey, I would welcome your opinion on bug #1859643 [15:39] Bug #1859643: [snap] cannot use shared NSS db

[16:03] zyga I need your input on https://forum.snapcraft.io/t/software-based-presentation-remote-or-need-an-interface-to-talk-uinput/13951 [16:03] mmm [16:03] looking [16:05] responded [16:17] PR snapcraft#2887 closed: 3.9 cherrypicks [16:22] Thanks ^ I replied ;-) [16:22] om26er: reading [16:23] om26er: can you please run that and attach a strace to the thread [16:23] om26er: ideally strace -e ... that limits to the relevant parts [16:23] (probably open + ioctl or open + write) [16:23] om26er: thank you! [16:23] Sure, I will do that [16:23] om26er: btw, is the rest of what I said sensiblke? [16:23] om26er: it could be handled by the user session daemon, perhaps [16:24] om26er: also, do you know how this differs in wayland? [16:24] does wayland have a proper event injection API? [16:24] zyga I think the X11 interface allows access to /dev/uinput hence it works fine in that environment. Not sure if the wayland interface allows that [16:25] om26er: apart from interfaces in snapd [16:25] om26er: I mean, x has an explicit message to inject events [16:25] om26er: so you can either push them back from linux side [16:25] om26er: or talk to X and tell it "here's an event" [16:25] om26er: do you know how _this_ aspect looks like in Wayland? [16:26] no, not aware about internals of wayland at all :/ [16:27] Will need to look if wayland has a "higher level" of input injection API of sorts [16:27] om26er: ok [16:28] that discussion might be relevant https://lists.freedesktop.org/archives/wayland-devel/2017-March/033514.html [16:32] interestingly Jamie had some feedback on a related topic a while back https://bugs.launchpad.net/snappy/+bug/1622639 [16:32] Bug #1622639: Snap access to /dev/uinput [17:07] re [17:11] switching gears to one more PR [17:11] but I'll EOD in about an hour [17:25] ijohnson, maybe you know, how something like 'Dec 12 21:56:18 evt2-integration-3 snapd[1225]: stateengine.go:150: state ensure error: Get https://api.snapcraft.io/api/v1/snaps/sections: dial tcp: lookup api.snapcraft.io: no such host' could happen? [17:25] abeato: looks like dns resolution failure [17:26] ijohnson, yeah, but they seem to have access... weird [17:27] abeato: how do you know they have access ? was this a temporary failure perhaps ? [17:27] it does not look like that - maybe they use a proxy though [17:27] I'll ask them back [17:28] could be proxy issue [18:11] * zyga takes a break === msalvatore_ is now known as msalvatore [19:23] PR snapd#8025 opened: overlord/snapstate: add reproducer for LP#1860324 scenario (2.43)