=== jamesh_ is now known as jamesh [06:35] morning [06:40] damn isp === mborzeck2 is now known as mborzecki [07:33] PR snapd#7747 closed: gadget: skip fakeroot if not needed [07:38] hey [07:38] zyga: hey [07:38] mborzecki: check out "Sourcetrail" [07:38] it's FOSS just now [07:38] pretty neat IMO [07:39] zyga: hmmm 'something went wrong' [07:39] mborzecki: did you finish all of the HR training? [07:39] zyga: yup [07:39] https://github.com/CoatiSoftware/Sourcetrail [07:39] did you have any issues? [07:39] zyga: with the hr training? [07:39] I had to re-do one from scratch because the in progress version was stuck on last page without ability to "sign" the result [07:40] yes [07:40] I finished everything but more like at 1-2AM [07:42] zyga: so sourcetrail is like a better gnu global web mode? [07:44] I have no idea what that is [07:44] hey mvo [07:44] how did you sleep? [07:50] hey zyga ! much better. did stay up longer and had a relatively good sleep [07:50] zyga: and you? [07:50] mvo: hey [07:51] mvo: I managed to sleep at 2:30 - 3:00 AM [07:51] better than yesterday for sure [07:52] mvo: I shared this with Maciek earlier: https://github.com/CoatiSoftware/Sourcetrail [07:52] it's neat, I played with it a little [07:53] their website is slashdotted now [07:53] but should recover eventually [07:53] zyga: 2:30am - 3:00 am ? that is still not great :/ [07:53] hey mborzecki [07:53] zyga: oh, nice === pstolowski|afk is now known as pstolowski [08:03] morning [08:05] hey pstolowski ! good morning [08:06] good morning pawel [08:07] pstolowski: hey [08:32] mvo: I think I should take a swap day today [08:32] mvo: yesterday was okay to "burn" the day on HR and some random reviews [08:32] but today I feel so tired and sleepy, in the morning, that I really think I need to just take it easy and sleep/rest all day [08:32] and not pretend to work effectively [08:33] zyga: sure, thats fine [09:16] if there's someone who is not really badly sleep deprived and has access to openSUSE Tumbleweed, please update to 20191116 and check if you can reproduce what I have bug #1853110 [09:16] Bug #1853110: On up-to-date openSUSE Tumbleweed all snaps show "broken" [09:16] j,, [09:16] hmm [09:17] Mirv: hey! yes i'm actually looking at it right now [09:18] pstolowski: thank you [09:18] (it's my bug triaging day today) [09:18] zyga: you are not counted into that someone according to log above :D take care. [09:18] pstolowski: ok, thanks! [09:18] thank you so much guys [09:18] * zyga returns to cleaning the office [09:21] apparmor was updated but not sure what was changed https://lists.opensuse.org/opensuse-factory/2019-11/msg00264.html - that said, disabling apparmor does not seem to help me [09:21] Mirv: i'm installing tumbleweed atm as i don't have it handy [09:22] pstolowski: it eats gobs of space [09:22] pstolowski: using btrfs and snapshots [09:22] not very vm friendly [09:22] you may want to disable snapper (snapshot thing) [09:23] ..or just select ext4 for everything in advanced setup instead of btrfs [09:33] zyga, Mirv good to know, thanks, yeah, too late to change fs, will look at disabling later [09:49] a second review for 7745 would be great [09:50] should be simple and will unblock 7746 [09:50] mvo: let me do it [09:52] pstolowski: so for example I've rocketchat-desktop installed, and with sudo mount /var/lib/snapd/snaps/rocketchat-desktop_97.snap /mnt I get: mount: /mnt: mount failed: Operation not permitted. [09:52] I wish there was something meaningful in some log, but I can't find any [09:54] Mirv: look at audit log [09:54] Mirv: /var/log/audit/audit.log [09:56] I tried, but I did not find other than "ype=AVC msg=audit(1574157270.914:590): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap.signal-desktop.signal-desktop" pid=7789 comm="apparmor_parser"" type of messages [09:58] that's odd [09:59] Mirv, zyga: i cannot reproduce the exact mount issue, but something is definately broken. all snaps get mounted, also after reboots. but i can get a snap to run only after i install it, and until reboot. after reboot i get "cannot change profile for the next exec call: No such file or directory [09:59] snap-update-ns failed with code 1: File exists" for all snaps [09:59] pstolowski: did you enable snapd.apparmor.service? [09:59] profiles seem to be in place, snaps mounted [09:59] it's part of the install docs [10:00] ah, i missed that [10:00] mvo: https://github.com/snapcore/snapd/pull/7745#pullrequestreview-318910411 + 1 [10:00] PR #7745: snap-bootstrap: add go-flags cmdline parsing and tests [10:01] PR snapd#7745 closed: snap-bootstrap: add go-flags cmdline parsing and tests [10:02] zyga: thank you! [10:04] zyga: right that was it [10:05] also 7712 needs a second review (a bit more work but really important to move forward with uc20) [10:06] Mirv: ok, cannot reproduce on fresh install, wonder what when wrong on your box. do you have everything updated, no held dependencies etc? [10:06] I do see "missing file /snap/.../meta/snap.yaml" type of messages but that's obvious since nothing is mounted, it's just annoying that regarding the "Operation not permitted" I'm not finding anything, and sudo systemctl stop apparmor + sudo systemctl stop snapd.apparmor do not help so it does not seem to be apparmor that's denying the permission (also as nothing is in audit.log) [10:07] Mirv: the fact you cannot even mount .snap manually is a key, that's pretty weird [10:07] pstolowski: :( everything is updated, nothing seems to be held [10:08] pstolowski: yes it is [10:08] squashfs is installed, that's one thing that I thought about [10:08] Mirv: that's not even a snapd issue but something more fundamental [10:08] Mirv: is it a standard suse kernel? [10:09] pstolowski: standard yes. but I have another TW laptop at 20191112, I'll update that one too and see if it gets broken or not [10:10] Mirv: also, can you try mount -v /var/lib/snapd/snaps/rocketchat-desktop_97.snap /mnt (-v for verbose) [10:11] pstolowski: not anymore verbose with that [10:12] pstolowski: ah, it's loop devices broken, somehow.. I can't mount an iso either [10:13] but yes this is not snapd anymore [10:13] Bug #1853122 opened: Facing requires classic confinement error [10:13] Mirv: losetup -a? [10:13] maybe you ran out of loopback devices [10:14] zyga: empty [10:14] can you try: [10:15] Mirv: losetup -f /path/to/a/squashfs/file/ [10:15] (as root) [10:15] perhaps also with -r for --read-only [10:15] hmm, losetup: cannot find an unused loop device [10:15] so that's the problem [10:16] can you strace it? [10:17] yep, and from there I can see that there's nothing ls /dev/loop* [10:17] do you have loop-control? [10:17] not even that [10:17] I don't know what makes it [10:17] on my system that is 10:237 [10:23] Mirv: thanks for updating the bug report! [10:24] pstolowski: sorry for taking your time :( it very effectively seemed like "snapd broken" for a long time.. [10:25] so currently I've filed openSUSE bug report, but then again my other laptop did not break. mknod /dev/loop-control c 10 237 helps. I'll follow up on the more relevant channels, maybe of course snapd current report a bit more if loop devices are broken. [10:25] Bug #1853122 changed: Facing requires classic confinement error [10:25] pstolowski: consider adding a check that /dev/loop-control exits and has major/mior [10:26] sanity check, that is [10:26] zyga: exactly [10:26] Mirv: no worries. we do some sanity checks, and i think this problem highlights one other potential problem that we may want to check [10:39] Chipaca: mvo: https://raw.githubusercontent.com/snapcore/pc-amd64-gadget/20/grub.cfg-recovery note improvements 1) variables select the default entry by id 2) all recovery systems are enumerated 3) entries are created per-mode / per-system 4) no idea what duplicate hotkeys will do 5) added menuentry to go back to UEFI setup, as my VM is too quick [10:39] it's kind of evolution of previous globgging, with type entries. [10:40] and the cmdline is now correct. [10:41] i was thinking that load_env --file /systems/$snapd_recovery/grubenv could e.g. declare the modes that are supported, and then generate entries for modes that a given recovery system can do? [10:41] xnox: I too was thinking of an env per system [10:41] xnox: sounds good! should we handle this stuff via PRs or do you prefer to commit directly to master? [10:42] it was a rewrite, thus i just pushed it to master [10:42] xnox: i'm not sure the logic for default= is sane there though [10:43] xnox: shouldn't that be set after the loop? [10:43] Chipaca: mvo: we were meant to have design for the boot menu right....? because it matters what order things are shown, what the text / hotkeys are, what they do, how they are grouped. [10:44] Chipaca: it is set if there is a default provided in the global env (i.e. boot that next); otherwise it is set each time a new better system is found. [10:45] mvo: Chipaca: and we need to like search for ubuntu-boot and add entry to chainload "normal mode" grub right? [10:46] * xnox also dislikes run/recover. I would prefer "normal/recover/install" such that hotkeys are unique: n r i [10:47] xnox: I pushed https://github.com/snapcore/core20/pull/13 with a very first draft of the separation of writable-path from initramfs. its the old code almost untouched. I will look to add tests and then we can start refactoring I think [10:47] PR core20#13: handle-writable-paths: extract the writable-path handling [10:47] xnox: but its what you had in mind, rightß [10:47] ? [10:48] niemeyer: it would be great if https://github.com/snapcore/core20/ pull requests could be added to mup [10:48] * mvo needs to step out for ~10min [10:48] mvo: Heya [10:48] mvo: Sure, will do [10:50] mvo: yes, that is the right approach, and a good starting point. It shouldn't live in /etc though, and can always iterate that once we create it as an entry point. [11:05] Chipaca: hey, why was this assigned to some other team? https://bugs.launchpad.net/snapd/+bug/1851480 [11:05] Bug #1851480: Hooks are not included in slot/plug label expressions [11:08] pstolowski: community [11:11] zyga: someone specific is going to fix it? [11:11] xnox: yeah, do you prefer /usr/lib/something/handle-writable-path? [11:12] pstolowski: that's dot-tobias [11:12] Chipaca: ah, ok [11:12] mvo: /usr/lib/core/handle-writable-paths yeah [11:13] pstolowski: they said on 2019-11-07 they'd get to it "next week", and that they'd report progress [11:13] pstolowski: so maybe it's time to jiggle the plug [11:14] xnox: cool, will move it over [11:16] pstolowski: yes, though please reassign if you want to fix it sooner [12:09] PR snapd#7749 opened: tests: restart the snapd service in the snapd-failover test [12:13] mvo hey [12:14] #7749 is to fix the issue I raised about that test on pi3? [12:14] PR #7749: tests: restart the snapd service in the snapd-failover test [12:16] cachio: it *could* be related but it was the outcome of a discussion with mborzecki about the snapd failover robustness and we wanted to make sure our tests are complete (i.e. testing that we didn't just restart into the right snapd but also ensuring that its stable accross the restart) [12:16] cachio: you need to remind me about this pi3 issue, I don't recall :/ [12:17] mvo, sure [12:17] I see this error https://paste.ubuntu.com/p/xBhPYrjfVf/ [12:18] PR snapd#7750 opened: sanity: check /dev/loop-control device as part of sanity checks [12:18] mvo, It happens after we install the broken snapd [12:20] PR snapd#7712 closed: seed: support in Core 20 seeds local unasserted snaps for model snaps [12:21] cachio: thanks, thats interessting. is this happening all the time or just sometimes? [12:21] mvo, all the time [12:23] cachio: do you know when this started to happen? [12:24] cachio: or is it happening since forever? [12:24] cachio: also I assume this happens on all the Pis, not just pi3, pi2 as well? [12:25] mvo, checking logs [12:26] cachio: thank you, no rush, I get lunch now but that is interessting, I wonder if there is a race with that path [12:28] mvo, 2.42 execution [12:28] first time I saw that [12:28] I raised the issue at that moment [12:29] and then I did a manual execution to get more info [12:29] last week [12:30] mvo, I mostly reproduced that on pi3 [12:31] on pi2 not sure [12:32] just once on pi2, it was during 2.42.1 [12:37] mvo, it is not happening all the time on pi3 [12:37] mvo, this is an execution during last week https://trello-attachments.s3.amazonaws.com/5da8bc830df86851446e9d4e/5dcf78cb5984618fb21cd5d7/1a9b05bfb0491c8faad045fc9f6143d3/snapd_2.42.1%2Bgit1140.g9173632_(5459)_pi3-refresh-18_arm32.log [12:38] this is all stable + snapd from edge [12:38] and it works well [12:50] cachio: is this still the case https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1852635 ? [12:50] Bug #1852635: mount-ns test failing with latest bionic updates [12:52] pstolowski, yes [12:52] pstolowski, this is the last log https://travis-ci.org/snapcore/spread-cron/builds/613333438 [12:53] from yesterday [12:53] cachio: ok, thanks, will investigate [12:53] pstolowski, thanks [13:48] cachio: thank you! interessting find, I wonder what is going on there, our code did not change in a while, maybe systemd did change somehow === ricab is now known as ricab|bbl [14:09] pstolowski: https://devfest.withgoogle.com/#devfest-map try poland [14:09] mborzecki: thanks [14:22] PR snapd#7748 closed: overlord: fix the get command help message [14:24] cachio: fwtw i run spread -debug google:ubuntu-18.04-64:tests/main/mount-ns and it didn't fail [14:31] cachio: pstolowski: do you have the log? [14:33] mborzecki: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1852635 [14:33] Bug #1852635: mount-ns test failing with latest bionic updates [14:37] PR snapd#7751 opened: cmd: fix the get command help message [14:37] mborzecki: did it over in the right message: https://github.com/snapcore/snapd/pull/7751 [14:37] PR #7751: cmd: fix the get command help message [14:38] pstolowski, are yo uusing the image test-bionic-1 ?= [14:40] mborzecki, this is the last error https://travis-ci.org/snapcore/spread-cron/builds/613333438#L1850 [14:41] cmatsuoka: thanks, will take a look [14:42] cachio: no, how do you mean? [14:43] mborzecki: thanks for noticing that, I grepped for the faulty message and I think I didn't expect more than one instance [14:45] cachio: ah, you mentioned it in the beginning of output in the bug report [14:46] cachio: how do i get it? [14:48] mborzecki: regarding epochs on #7431, I don't think we should do anything for that, AFAICT it should just work with epochs, my gut feeling is that if there is something special that should happen with disabled services and epochs the "transitioning" snap between epochs should take care of that in post-refresh, pre-refresh hooks IMHO [14:48] PR #7431: overlord/snapstate: don't re-enable and start disabled services on refresh, etc [14:48] anyways do you think I'm ok to land that PR now since it's green? (CC mvo) [14:49] mvo: also should we get a +1 from jdstrand on #7731 before merging? [14:49] PR #7731: usersession/userd: add "apt" to the white list of URL schemes handled by xdg-open (LP: #1776873) [14:50] ijohnson: yeah, a review for 7731 from jdstrand would be great, should be fine but best to double check with him [14:50] ack [14:51] ijohnson: let's merge 7431, it got two +1 and conceptually samuele had looked at it so AFAIK we should not block on him [14:52] pstolowski, I created it by updating and upgrading bionic and then rebooting it [14:52] \o/ I'm on it [14:52] ijohnson: great, thanks for this one! [14:53] PR snapd#7431 closed: overlord/snapstate: don't re-enable and start disabled services on refresh, etc [14:53] * ijohnson can't wait for pstolowski to rewrite all of that code for preseeding and review it all again [14:53] PR snapcraft#2809 opened: project-loader: address errors from mypy uprev [14:54] ijohnson: yay [15:14] pstolowski: I'm not quite done going through my backlog of forum/email/github notifications, did you submit a PR adding 'k' to the serial-port apparmor rules? If not I can do that [15:14] looks like jdstrand gave a +1 to adding 'k' to the rule [15:14] https://forum.snapcraft.io/t/not-able-to-start-configure-custom-built-ubuntu-core/14142/23?u=ijohnson [15:16] ijohnson: he did, but he also said he was going to do it afaiu [15:17] ah I see his other comment now, nvm :) [15:18] * cachio lunch [15:19] PR snapd#7752 opened: tests: enable degraded test on arch linux after latest image updates [15:19] ijohnson: yeah, i was wondering about the snap service naming, it's totally up to whoever builds the snap, so you can have a service named A, then the user disables it, the publisher renames A to B, the user proceeds to disable B, then A returns as something completely new and gets disabled because we carry the history of the disabled service names, but it a corner case so meh ;) [15:21] well the question I would have is when you say "A returns as something completely new", how do you qualify something completely new? :-) it has the same name, so the publisher probably intended some kind of connection between the first service and it's reincarnation [15:21] but yes that is a corner case === ricab|bbl is now known as ricab [16:35] PR snapcraft#2810 opened: build-providers: address errors from mypy uprev [16:40] Chipaca: I think the snapd REST API wiki docs need an update, see https://forum.snapcraft.io/t/facing-requires-classic-confinement-error/14163 [16:40] hmm [16:40] * Chipaca looks [16:41] ijohnson: oh, doc seems to be wrong, not outdated [16:42] wrong == outdated ? [16:44] ijohnson: /v2/snaps never took options [16:44] right got it [16:44] so all those options descriptions should be on /v2/snaps/ === pstolowski is now known as pstolowski|afk [17:14] augh [17:14] * Chipaca takes a break [17:21] degville, when you get a chance can you take a look at making the docs changes for https://bugs.launchpad.net/snapd/+bug/1612440 ? [17:21] Bug #1612440: Full systemd socket activation support [17:23] ijohnson: will do, thanks for letting me know. [17:29] thanks! [17:41] Hello! I am beginning an upgrade of the snapcraft forums. The forum will be in read-only mode until this is completed. ETA: less than one hour. [17:53] EOD for me it is [17:53] πŸ‘‹ [18:32] forum.snapcraft.io has been upgraded. Maintenance is complete. [18:32] PR snapcraft#2811 opened: extractors: address errors from mypy uprev [18:39] thanks jk0ne! [18:46] hmm [18:46] jk0ne, not sure if that was your intent ... https://imgur.com/a/es4hRK9 [18:47] (has anyone else lost all CSS from the forum ?) [18:48] bah ... ok, now it is back but i'm hard logged out ... [18:48] looks OK here [18:48] (and cant remember my credentials since the silly forum doesnt use SSO) [19:03] ogra: I think you might have hit it during the actual reload. Let me know if you are seeing issues still. [19:15] jk0ne, nope, seems ok now [19:17] glad to hear [23:24] PR snapcraft#2812 opened: store: address errors from mypy uprev [23:39] PR snapcraft#2813 opened: pluginhandler: address errors from mypy uprev