[05:20] morning === chihchun is now known as chihchun_afk [05:45] PR snapd#6686 closed: testutil: fix MockCmd for shellcheck 0.5 [05:47] Hey Maciek [05:47] * zyga is totally sick [05:48] zyga: hey, take a day off, get some rest [05:52] I will :/ [06:15] mvo: morning [06:15] hey mborzecki [06:17] mvo: i was poking the fixed in #6685, looks like the current code works by accident [06:17] PR #6685: image: prefer local for snapd/core snaps [06:35] mborzecki: yeah, its a bit puzzling, when I wrote the test things were a bit strange [06:35] mborzecki: I haven't looked deeper, it got a bit late [06:38] mborzecki: aha, nice find [06:39] mvo: kept wondering why i did not hit this problem before :P [06:39] mborzecki: yeah, it looks like this code needs a closer look, maybe we can simplify and get rid of all the PreferLocal ones [06:40] mvo: fwiw, noticed something yesterday, when i did 'snap set system refresh.hold=' on a ssytem that did not refresh yet at all (eg. pristing image) it triggered a refresh [06:40] need to check that with the tests, maybe i did something wrong [06:55] mborzecki: that can be ok, if the image is really old snapd will detect no refresh for more than 60 days and force a refresh [06:55] mvo: right, want to double check that, we're using seed-time as reference [06:57] Store being silly about the description. https://usercontent.irccloud-cdn.com/file/u2vd6WZ6/Screenshot%202019-04-04%20at%2008.56.56.png [06:59] Store being silly about license duplicates https://usercontent.irccloud-cdn.com/file/uIzKQ0Zb/Screenshot%202019-04-04%20at%2008.59.36.png [07:00] Store being silly about case sensitive search for GPL https://usercontent.irccloud-cdn.com/file/EUWodfbQ/Screenshot%202019-04-04%20at%2009.00.13.png [07:01] zyga: Known issue [07:02] https://github.com/canonical-websites/snapcraft.io/issues/1329 [07:02] Store being silly when selecting the 2nd of the "identical" GPL v2 only licenses https://usercontent.irccloud-cdn.com/file/Mzh8Y4Q0/Screenshot%202019-04-04%20at%2009.01.41.png [07:02] https://github.com/canonical-websites/snapcraft.io/issues/1585 [07:02] brlin: thank you! [07:02] zyga: You're welcome! === pstolowski|afk is now known as pstolowski [07:03] morning [07:06] pstolowski: good morning [07:07] mborzecki: not sure if seed time is the right reference, if you e.g. get the current stable image from cdimage.u.c it will be 8 month old already :/ [07:08] mvo: we set it ourselves once seeding is done [07:08] mborzecki: right [07:08] mborzecki: what I mean is we need to think, if the snaps are 8month old we probably want to update soon [07:09] zyga > Store being silly about case sensitive search for GPL [07:09] Not reproducible at my end though [07:09] brlin: if you look for GPL instead of gpl, does it find anything? [07:09] zyga: https://usercontent.irccloud-cdn.com/file/waciCfQn/Screenshot_20190404_150931.png [07:09] fun [07:09] then it is a client side search bug [07:10] Firefox seems to be fine [07:15] thanks sil2100 ! [07:15] PR core18#121 closed: Make the version number date-based [07:18] mvo: https://github.com/snapcore/snapd/pull/6583 is an easy win [07:18] PR #6583: cmd/snap-confine: move ubuntu-core fallback checks [07:23] * mvo looks [07:26] PR snapd#6583 closed: cmd/snap-confine: move ubuntu-core fallback checks [07:29] woot thank you! [07:29] mvo: so now shall I add a core16 fallback? [07:30] I wonder what technical difficulties are blocking the `core16` base? [07:31] brlin: I think it is mainly time [07:31] brlin: we want core16 to be different from core [07:31] brlin: it should behave like core18 [07:31] brlin: but have the contents of core [07:32] brlin: the plan is to allow "core" to be used instead of core16 [07:32] (so no new traffic) [07:32] but using core18 semantics [07:33] where semantics is really about how it behaves on a system using "core" as boot base [07:33] zyga: Thanks for the explaination [07:34] zyga: mvo: I think it should be possible to tweak mvo PR about that, I alos still wonder whether we can compute the base root path early [07:34] pedronis: ah, right, this all started with an initial PR [07:34] mvo: hi, I added a card about CommandFromCore [07:34] pedronis: we can but it should not block the work right now, I will add a patch for that soon, just trying to wrap up some of the existing fixes [07:35] mvo: assigned to you but feel free to give it to somebody else if that works better [07:36] mborzecki: image has been refactored many times, it might be that some bits are not needed anymore, I wouldn't jump to conclusions to fast either tough, it's pretty delicate/subtle, though there are tests [07:36] pedronis: just checking, did you see https://bugzilla.suse.com/show_bug.cgi?id=1127366#c14 ? [07:37] no [07:37] there are some interesting bits there [07:41] pedronis: sure, i was just trying to find out why i did not run into this before [07:42] * zyga considers breakfast a good thing and goes [07:53] PR snapd#6687 opened: snap-confine: set rootfs_dir in sc_invocation struct [07:53] pedronis: I create a PR to calculate the base root path (cc zyga) [07:53] pedronis: will look at comand-from-core, thanks for this [07:54] zyga: I can tweak my PR about core->core16 fallback, should be easy now thanks to your work on this :) [07:54] mborzecki: so I think it's correct that we don't mostly need the PreferLocal anymore, the only things that goes wrong is the message Copying vs Fetching [07:54] which could be moved inside acquire itself [07:56] the ListContains is still not quite right though [07:56] it should use local.hasName [08:03] mvo: mborzecki: I'll pick up that PR myself today or tomorrow [08:04] pedronis: thank you [08:10] zyga, pedronis just fyi - I updated the old 6418 (core16->core fallback) to use the new place for this [08:15] mvo: it looks right to me, but I haven't relooked at the whole PR [08:17] pedronis: no worries, I'm looking at adding one more test that zyga suggested [08:22] drwxr-xr-x 6 root root 4096 Apr 4 07:46 /root [08:22] why is root so weirdly open on core? [08:23] it does't seem to be open in real core [08:23] just in our tests [08:23] Hi everyone [08:24] perhaps /root is from writable somewhere [08:24] but has wrong permissions [08:25] zyga: wrong how? [08:26] Chipaca: it's too open [08:26] ls -ld /root [08:26] and compare [08:27] ah, /root, me was reading /root as / (the brain is funny) [08:27] :D [08:27] yes some overloaded meanings abound [08:29] dot-tobias: Hello [08:30] * zyga looks at a real core device [08:31] zyga: drwx------ 4 root root 4096 Apr 4 07:04 root [08:31] is that from core16? [08:31] 18 [08:31] ok [08:31] so real core devices are probably ok [08:31] so our tests are broken somehow [08:36] Found it [08:36] fixing [08:46] Short question re: cloud.conf in gadget snaps: Reading https://forum.snapcraft.io/t/how-to-preconfigure-custom-image/4154/15 and https://forum.snapcraft.io/t/cloud-init-with-netplan/1301/5 , it seems to me this does not work (properly / yet). The docs for gadget snaps just mention “optional cloud.conf” (https://docs.snapcraft.io/the-gadget-snap/696) Can someone confirm or point me to further resources? ping ogra 😊 [08:48] hmmm [08:49] it seems that we don't have some modules and ubuntu core cannot be used in vmware [08:50] it's quite possible [08:50] pstolowski: degville: hi, I did a quick read over the hot plug docs, they look overall [08:50] *look good [08:51] I left a question/comment [08:53] using IDE disk fixes this [08:53] so that's good [08:54] and I can boot core in vmware [08:54] so useful :) [08:55] pedronis: ty [08:56] zyga@core16:~$ sudo snap refresh [08:56] error: cannot refresh []: cannot refresh snap-declaration for "snapweb": parsing assertion headers: expected 4 chars nesting prefix after multiline introduction "plugs:": " network:" [08:56] how do we update from vanilla core systems again? [08:56] (this is a freshly installed core system as obtained from cdimage) [08:56] zyga: there's a bug for that [08:56] right, I recall this discussion [08:57] that's not great :/ I thought the store had fixed the response [08:57] they haven't yet :/ [08:57] so we should regenerate our core images [08:57] pedronis: thanks for the review! I'll work on those two comments now. [08:57] https://bugs.launchpad.net/snapstore/+bug/1802773 [08:57] they are useless [08:57] Bug #1802773: Cannot refresh from ubuntu-core 2.15.2 on raspberry pi "ubuntu core 16" image - snap-declaration for "snapweb": parsing assertion headers: expected 4 chars nesting prefix after multiline introduction "plugs:": " network:" Store:New for wgrant> [08:58] pedronis: how can we escalate this? [09:00] I will install a core18 system in the meantime [09:00] but it feels like the image should be refreshed to current core16 [09:00] and we should remove snapweb [09:02] pstolowski: after I've made those edits, would you like me to move the Hotplug support doc over the forum, or would you prefer to do it? [09:03] the qemu images are whooping 4GB [09:03] they are not sparse [09:03] compression "helps" but this looks like an oversight [09:03] pedronis: who can I poke about that? [09:04] zyga: I think mvo needs to decide what he wants to do, I poked a couple of times already in the past [09:05] mvo: ^ [09:06] degville: please do, thank you! [09:06] pstolowski: np! [09:08] sil2100: we cannot set hostname on a core18 system [09:08] zyga@localhost:~$ sudo hostnamectl set-hostname core18 [09:08] Could not set property: Failed to set static hostname: Read-only file system [09:09] on core18 the system-shutdown helper is not working [09:09] I just got a failure about that when rebooting [09:09] Chipaca: ^ [09:09] zyga: yes we know [09:10] oh [09:10] not the last two [09:10] zyga: what failure did you get? can i see? [09:10] a flash on the vmware screen, I'm looking at logs now [09:10] zyga: if you do halt instead of reboot you should see them [09:10] I added persistent journal now [09:10] zyga: some failures are expected (that's why the script exists) [09:11] zyga: the interesting messages are post-journal [09:11] i mean, systemd *execs* the system-shutdown helper [09:11] ha [09:11] tricky beast [09:11] system-shutdown is pid 1 [09:11] yes [09:11] :-) [09:11] * Chipaca could tweak it a bit so it starts snapd and takes over the world from there [09:12] zyga: hm, I thought we had this before and fixed it [09:12] sil2100: does the fix require a new core image [09:12] sil2100: I just installed core18 from cdimage [09:13] Chipaca: I have the logs now, let me try to go through them [09:13] zyga: if you have logs they'll say they couldn't unmount writable [09:14] https://pastebin.ubuntu.com/p/WmkCSTjgnD/ [09:15] Apr 04 09:10:48 localhost kernel: systemd-shutdow: 67 output lines suppressed due to ratelimiting [09:15] hahah [09:15] let me fix that [09:15] zyga: that's not the helper [09:15] zyga: that's systemd-shutdown [09:16] zyga: to see the helper's logs, use halt [09:16] and screenshot [09:16] https://usercontent.irccloud-cdn.com/file/bcoUzJVf/Screenshot%202019-04-04%20at%2011.16.38.png [09:17] the /dev/loop0 is interesting [09:17] but apart from that it looks less bad [09:17] just the failures are ugly to look at [09:17] zyga: wrt /dev/loop0, new enough kernels auto-disassociate [09:17] so it's fine [09:18] zyga: that's a successful run of the helper [09:18] mvo: core has 2.38 in stable but snapd is 2.37.4 [09:18] mvo: did we forget to push snapd snap to stable? [09:19] reminder that #6577 needs a 2nd review [09:19] PR #6577: many: make Remodel() download everything first before installing [09:20] pedronis: I will gladly look [09:21] https://github.com/snapcore/snapd/pull/6642/commits/07cd4fa0911006f2bbcc6c6bebf8f1470e093765 <- this was breaking /root permissions [09:21] PR #6642: cmd/snap-confine: prevent cwd restore permission bypass [09:21] mkdir on the following line [09:21] pstolowski: btw were my brief comment on #6660 understandable? what is my concern there? [09:21] PR #6660: cmd/debug: integrate new task timings with "snap debug timings" [09:21] * zyga goes to make coffee and goes back to do reviews [09:24] pedronis: yes; i played yesterday with timings.Get function & a filter predicate and not making *JSON public but couldn't find a good solution [09:26] pstolowski: do we need to chat ? [09:28] pedronis: if you have time then yes, that would speed things up [09:28] mborzecki: can you do a 2nd review for https://github.com/snapcore/snapd/pull/6642 [09:28] PR #6642: cmd/snap-confine: prevent cwd restore permission bypass [09:28] mborzecki: I will get a review from jamie as well [09:28] zyga: sura [09:29] mborzecki: I will be working on fixing the core16 image permissions today, I will not merge before that lands [09:29] mborzecki: FYI: core18 was fixed and merged (with tests) yesterday [09:30] zyga: btw. can you take a look at https://github.com/snapcore/snapd/pull/6661 later on? it's the last one [09:30] PR #6661: data/selinux, tests/main/selinux-clean: fine tune the policy, make sure that no denials are raised [09:31] zyga: thats a question for cachio, we need to check where the validation stands [09:33] mvo: is snapd.snap validated as a part of core18? [09:33] mborzecki: sure [09:33] zyga: re "I think mvo needs to decide what he wants to do, I poked a couple of times already in the past"> can you give me some more context? I read backlog but its not obvious, sorry [09:33] zyga: yes [09:33] zyga: see the trello card on releasing 2.38 [09:34] zyga: the usual flow is that QA validates core (uc16) and core18+snapd (uc18) [09:34] mvo: stale core16 images [09:34] mvo: our core16 images are 100% useless now [09:34] mvo: because of a store bug that prevents old snapd from refreshing [09:34] zyga: and if the core18 image is fine we promote that too [09:34] mvo: we should regenerate them to have core instead of ubuntu-core and to have more recent snapd [09:34] zyga: what is this bug? that sounds serious? [09:34] mvo: we should also drop snapweb [09:35] mvo: https://launchpad.net/bugs/1802773 [09:35] mvo: it's trivial to reproduce, I hit this a moment ago [09:35] Bug #1802773: Cannot refresh from ubuntu-core 2.15.2 on raspberry pi "ubuntu core 16" image - snap-declaration for "snapweb": parsing assertion headers: expected 4 chars nesting prefix after multiline introduction "plugs:": " network:" Store:Confirmed for wgrant> [09:35] zyga: fwiw, the core images are created by foundations, I raised the topic of stale images in the meeting with them on monday and they are on it (cc pedronis) - not sure what I need to make up my mind about but happy to do so (once I learn more :) [09:35] zyga: "ubuntu-core" ?!?! [09:35] zyga: we have an image that is still on ubuntu-core for download? [09:36] mvo: yes [09:36] that's the reference image we have [09:36] that's great, right? [09:36] zyga: that sounds incredible wrong [09:36] indeed [09:36] zyga: and that is not something I was aware of, we need to fix this. where is that image linked? [09:37] mvo: http://cdimage.ubuntu.com/ubuntu-snappy/16.04/current/ [09:37] perhaps the whole directory should go [09:37] I bet the remaining images are equally broken [09:38] zyga: the site is a bit of a mess - the right link is "ubuntu-core" and that has more current images. but we need to remove this dir [09:38] +1 [09:40] mvo: there are two issues, one is the images, the other is that bug, old snapd still using the old assertions end point get assertion formats they cannot parse [09:41] pstolowski: can you caht now? otherwise it needs to be this afternoon or tomorrow [09:42] pedronis: now is fine [09:43] pstolowski: I'm in the standup [09:43] pedronis: not disputing the bug :) was just curious what I need to make my mind about. also the fact that we have a "ubuntu-snappy" link on cdimage is highly confusing so that needs fixing too [09:44] 6641 is ready for a re-review [09:46] mvo: I will look, going through remodel first [09:54] zyga: ta [10:06] pedronis: https://forum.snapcraft.io/t/building-with-build-snaps-bottle-necking-on-fuse/10747 fwiw [10:07] pedronis: fuse bottlenecks on lxd, because snapfuse is apparently single-threaded [10:08] pedronis: setting the flag to unsquash on install doesn't work because fuse detection gets in the way [10:08] pedronis: disabling fuse detection breaks the sanity check [10:08] pedronis: :-( [10:08] Chipaca: is it urgent? [10:09] pedronis: for who :-) [10:10] Chipaca: sorry, I'm asking why you raise it now instead of standup I suppose [10:10] pedronis: ah, because i want to forget about it and get back to what i'm supposed to be doing :-) [10:10] but i don't want to strand sitter with no response beyond "yeah sucks" [10:10] Chipaca: you should have poked me from in the forum [10:10] I think [10:10] pedronis: ok, fair [10:10] for such a case [10:11] pedronis: i was chatting with sitter in #snapcraft, dumping to the forum for posterity [10:11] but yeah [10:11] #snapcraft is not logged :-( [10:12] anyway I don't think we recommend a _TEST thing even if it worked for building [10:12] we would need to make that more first class [10:12] but how much unclear [10:12] (independetly that it chokes atm) [10:14] pedronis: agreed [10:14] pedronis: anyway, pinged you in the forum so it can carry on async'ly [10:16] pedronis: ~3 months until they need it in production [10:16] * Chipaca gets back to work^Wcoffee [10:26] PR snapd#6688 opened: gadget: add validation of cross structure overlap and offset writes [10:26] mvo: ^^ if you have some time for reviews [10:27] back to zyga's PR [10:27] mborzecki: heh, this sounds complicated :) [10:35] PR core18#120 closed: Backport wpa_supplicant.service.d/snap.conf from core [10:38] stepping away for a bit, bbl [10:55] zyga: quick questions about opensuse, snapd is there in a separate project atm right? [10:55] yes [10:55] pending inclusion after changes raised through the security review [10:55] perhaps 2.39 actually .. [10:55] if I'm fast enough [10:56] zyga: wdyt about extending tests/upgrade/basic to add the repo and install the snapd package from there? [10:57] yeah, it makes sense, thank you for raising it [11:13] is there a way for snapcraft push to try a couple of times without shell scripts hacking around it? [11:13] got a 500 this morning in the CI [11:18] zyga: some comments under #6642 [11:18] PR #6642: cmd/snap-confine: prevent cwd restore permission bypass [11:18] mborzecki: thank you [11:18] mvo: reviewed the remodelling branch [11:21] mborzecki: replied [11:24] zyga: i think I"m missing something about O_PATH, https://paste.ubuntu.com/p/KNcd5RXDdw/ [11:25] zyga: there's still permissions on the prefix that are needewd [11:26] mborzecki: mmmm [11:26] * zyga checks [11:28] ah, indeed [11:28] that's a nice catch [11:28] about O_PATH: https://www.irccloud.com/pastebin/8TPWkUbn/ [11:28] I will adjust the code [11:29] zyga: ok [11:42] pstolowski: I +1ed 6665 with a small comment [11:43] ty [11:54] mvo: https://github.com/snapcore/snapd/pull/6641#pullrequestreview-222712382 [11:54] PR #6641: snap-gdb-shim: switch to the SUDO_UID when available [11:56] mborzecki: looking at https://github.com/snapcore/snapd/pull/6661/files now [11:56] PR #6661: data/selinux, tests/main/selinux-clean: fine tune the policy, make sure that no denials are raised [11:58] zyga: thanks! [12:16] zyga: I made a comment in #6673 [12:16] PR #6673: cmd,tests: forcibly discard mount namespace when bases change [12:17] pedronis: looking [12:18] pedronis: that's a fair comment, for now using os-release should work but I agree ideally we'd know better. I will think about what to do best [12:37] pedronis: we can actually use /run for that, it's always a tmpfs that we get for free [12:38] pedronis: the only problem is that this doesn't tell us the name of the base in absence of that file, which may require a fallback to something else (or switch to probing) [12:38] zyga: that is always bound mount from the host, no? [12:38] yes [12:38] is not sharing back though? [12:38] I don't understand what you mean [12:38] we don't need to mount anything, just leave a trace what we did [12:39] like we write to /run/snapd/ns/ [12:39] we can also store meta-data file for each mount namespace there [12:41] zyga: I think I need to understand a bit more what you plan to use, we need to find the info that correspond to the namespace [12:42] pedronis: we can either determine the base snap name without any lookaside storage [12:42] pedronis: that may be possible [12:42] pedronis: or we can simply write the name of the base [12:42] pedronis: just like we do with mount profiles [12:43] zyga: sorry, we need the name of the base that the namespace was made with [12:43] not just the name of the current revision name [12:43] pedronis: I know that, that's what I meant [12:44] pedronis: we may be able to infer that from the mount namespace alone [12:44] pedronis: but if that's hard or we cannot do that for whatever reason we can just write that down when we first create it [12:44] zyga: I agree, I'm still not sure where you plan to write it down [12:44] pedronis: if we write it down it can go to /run/snapd/ns/$SNAP_INSTANCE_NAME.info [12:45] zyga: and we remove it and rewrite when we discard the namespace ? [12:45] yes [12:45] pedronis: just like the mount profiles [12:45] pedronis: we might even include it in the mount profile itself as a comment but I think that'd be a stretch [12:46] that's were I get lost, I though the mount profile is written by snapd [12:46] pedronis: there are two files [12:46] pedronis: snapd writes "what I want" [12:46] pedronis: snap-update-ns writes "what I did" [12:46] ok, so a sibling to the latter [12:46] pedronis: snap-confine could write "what I started with" until that logic migrates to snap-update-ns [12:46] mvo: need to step out and skip the standup, i'll send a note in the forum [12:47] (I'm working on moving most of the initialization to go but that's a slow process that I don't spend much time on) [12:47] pedronis: but yes, a sibling to what the lower layers write [12:48] zyga: I think I got lost because for a while it sounded like our only option was to read something from inside the namespace [12:48] but that's not true [12:48] I don't think we need to be clever here, if blunt and simple works I'm all for it [12:48] pedronis: that's correct, we can do both [12:48] pedronis: reading something from inside the namespace has one advantage [12:48] pedronis: migration is easier [12:48] pedronis: no new data is required [12:48] pedronis: I agree [12:48] pedronis: less complexity is good [12:49] pstolowski: your changes to #3/4 LGTM [12:49] pedronis: perhaps base snaps could have one extra file [12:49] pedronis: /meta/snap.info with simple key=value pairs :) [12:49] zyga: no, that is not simple [12:49] pedronis: for things we wish we could get out of the yaml [12:49] is simple for us [12:49] but not overall [12:49] pstolowski: there's still the bit where you have a day constant that i suspect isn't tested :-) [12:50] pedronis: perhaps, I think it'd be nice though, it's not like it's a complex cost for anyone [12:50] zyga: we have meta/snap.yaml [12:50] it has a name too [12:50] yes but parsing a simpler format with limited data would be beneficial [12:50] I understand but that just weird [12:50] parsing yaml is pretty hard [12:50] Chipaca: ty! did you notice the change to doForget? [12:51] yeah, I'm not seriously proposing it for this use case [12:51] we can write the simple stuff ourselves [12:51] somewhere [12:51] I think it'd be nice but it's not required for sure [12:51] pedronis: that brings me back to one idea [12:51] the key=value facts file [12:51] we could stop trusting environment [12:51] I would love if we did that [12:51] I don't think is nice to be clear [12:51] for the record [12:51] pstolowski: I did! you could check it's auto before trying to remove it, no? [12:51] it's a pending security hole if someone finds a way to abuse it by confusing snap-confine [12:52] zyga: confuse what? [12:52] confuse snap-confine by setting variables we depend on and provide additional command line options [12:52] Chipaca: yes, indeed! will add [12:52] we could write information that a snap has classic confinement and not depend on --classic [12:52] pstolowski: no biggie as it's a reasonably fast op [12:52] pstolowski: but, ¯\_(ツ)_/¯ [12:52] we could write information about the base snap and not depend on --base-snap-name or whatever the argument is [12:53] we could write snap instance name and not depend on the environment variable [12:53] that would be much stronger guarantee than what we have now where all of this is untrusted input [12:53] that's sounds an ok plan, but sound still different than current [12:53] that we need to carefully parse and cross check [12:53] problem [12:53] yes [12:53] that's about current snap revision [12:53] because it would be snapd writing that [12:53] not current actual namespace [12:53] and here it would be snap-confine [12:54] Chipaca: nb, i noticed that doForget test(s) don't set set-id on the test task, not sure it's a an issue or not, we probably are not testing doForget as througly as we could (but since all these bits are well tested separately, than maybe that's ok) [12:54] but if we agree on some basic snap-confine provided "here is what I did" file in /run that helps me a lot [12:54] because I will also immediately use it for nvidia migration later on [12:54] e.g. I could write: base_snap_name=foo and nvidia_mode=legacy-mount [12:54] pstolowski: patches welcome :-þ [12:55] (or legacy-symlinks) [12:55] pedronis: so if you agree on storing simple meta-data like that I'm super happy to pursue that [12:55] pedronis: in addition I can do one special optimization [12:55] :) [12:55] pedronis: if base snap name is "core" I will not write it [12:55] then migration is free :) [12:56] you can migrate from core to core18 and from core18 to core [12:56] and from old snapd that didn't know about this to new snapd that does [12:56] Chipaca: yep, i can re-visit this soonish, didn't touch it now as it's too easy to get side-tracked ;) [12:57] pstolowski: *applause* [12:57] i think doForget should error-out if set id is not provided, since client requires it afair [13:03] PR snapd#6665 closed: overlord/ifacestate: implement String() method of HotplugDeviceInfo for better logs/messages [13:03] dot-tobias, i dodnt touch cloud-init, specifically not on core ... but ondra has some experience with it === ricab is now known as ricab|lunch [13:16] ogra Thanks, will ping him with a forum question [13:17] he is here as well but might not watch IRC all the time === cpaelzer_ is now known as cpaelzer [13:40] mborzecki: can you have a quick look at https://github.com/snapcore/snapd/pull/6642 -- I fixed the issue with O_PATH [13:40] PR #6642: cmd/snap-confine: prevent cwd restore permission bypass [13:48] mvo, pedronis, I'll be offline for a bit while I relocate back; this was a bad idea ;-) [13:48] Chipaca: ok [13:48] cmatsuoka: I'll review the delete-user pr when i'm back [13:48] mborzecki: I started going through the selinux change but it's not easy [13:48] if that's ok [13:48] (otherwise i could probably sneak it in now while t'internet still works here ...) [13:48] it's hard to map a type to a comment with a path that justifies it [13:49] Chipaca: no worries, see you [13:53] Chipaca: no hurry, I'll work on that after finishing a snapcraft assignment that should take the entire afternoon [13:54] * cmatsuoka vs patchelf [13:59] re [13:59] zyga: thanks! ;) [13:59] zyga: ping me if you have questions [13:59] ack [14:01] PR snapcraft#2522 opened: build providers: add API for friendly instance type names === kali___ is now known as kaliy === ricab|lunch is now known as ricab === chrisccoulson_ is now known as chrisccoulson [14:52] PR snapcraft#2521 closed: cli: cleanup environment detection [15:17] https://github.com/snapcore/snapd/pull/6684 needs a 2nd review [15:17] PR #6684: overlord,tests: perform soft refresh check in doInstall [15:54] * cachio lunch [15:57] Strange warning in Solus 4 https://usercontent.irccloud-cdn.com/file/vnB4i37M/Screenshot_20190404_235706.png [16:05] Is this normal? I assume this is the problem that causes the `execv failed: No such file or directory` error when running core18 snaps. https://usercontent.irccloud-cdn.com/file/nxT13Vc8/Screenshot_20190405_000148.png [16:10] re [16:10] brlin: looking [16:10] <3 [16:11] brlin: so I'm confused as to what you are referring to [16:11] in the first image I see that sudo resets PATH [16:11] Solus appears to not set /snap/bin as one of the PATHs in the sudoers policy [16:11] brlin: run sudo env | grep PATH [16:12] in the second case [16:12] I don't see any errors at all, unless I'm missing something [16:12] just regular debug output [16:13] zyga: Is the "current mount profile (after applying changes)" line expected to be (none)? [16:13] zyga: ``` [16:13] $ sudo env | grep PATH [16:13] PATH=/usr/sbin:/usr/bin:/sbin:/bin [16:13] ``` [16:13] yes, because user mount profiles are not preserved yet [16:14] there's a long effort that is close to being do to change that === pstolowski is now known as pstolowski|afk [16:15] brlin: in addition it is a special mount entry that we only apply if the source and destination exist [16:16] brlin: it is a mount entry for the document portal [16:16] Oh, that makes sense now, thanks for the help. [16:18] brlin: thanks for checking, sorry that this is not documented more clearly [16:18] perhaps we should log something when the error is ENOENT [16:19] Well it is a debug output, I expect no more what it already is. [16:19] I'm working on the next batch of refactoring for that area [16:19] the whole interaction will be much easier to follow and extend [16:20] I strongly want to finish that to allow me to open a new feature [16:20] where the ~/snap directory is gone :) [16:20] but ... complexity and layering [16:21] mvo: wanna do a 2nd review for https://github.com/snapcore/snapd/pull/6684 ? :) [16:21] PR #6684: overlord,tests: perform soft refresh check in doInstall [16:21] mvo: refresh app awareness awaits :) [16:23] niemeyer, hey, I have another PR to review https://github.com/snapcore/spread/pull/70 [16:23] PR spread#70: Close ssh connection when reboot is stuck [16:23] it is an old one which is still needed [16:25] niemeyer, thanks [16:37] It seems that Solus have solved the core18 snaps' launching problem: https://dev.getsol.us/R3609:5fd02a7c03eca0f53aa389a701385558cf7fd423 [16:37] mvo: if still around: https://github.com/snapcore/snapd/pull/6687/files#r272266095 [16:37] PR #6687: snap-confine: set rootfs_dir in sc_invocation struct [16:37] PR snapd#6667 closed: tests: enable tests that write /etc/{hostname,timezone} on core18 [16:38] zyga: thanks ,I check it [16:38] 6595 needs a review (should be easy) [16:40] mvo: cachio asked for the timeout to be 5m [16:40] that hasn't been applied afaics [16:41] mvo: reviewed [16:41] pedronis: uh, thanks, will fix [16:41] ah, indeed [16:41] mvo: wait [16:41] I can change that in a suggestion [16:41] and you can just click on "commit" :D [16:42] mvo: look [16:42] mvo: refresh and click on what you like :) [16:44] jdstrand: hey [16:44] jdstrand: gentle ping for https://github.com/snapcore/snapd/pull/6605 [16:44] PR #6605: cmd/libsnap,osutil: fix parsing of mountinfo [16:44] zyga: re: the hostname problem (sorry for looking into it only now) - what image were you using? [16:45] sil2100: hey [16:45] sil2100: I got the image following the usual links from the ubuntu download website [16:45] sil2100: I debugged it a little and it's curious [16:45] it *gets* set [16:45] but there is still an error displayed [16:45] zyga: ah, yeah, ok, so it's fixed in the current one, as I thought this was the thing we were fixing in systemd AFAIK [16:45] let me show you [16:45] current one? [16:45] zyga: so if you fetch the latest daily, it seems to work [16:45] PR snapd#6689 opened: tests: run create-user on core devices [16:46] hmm [16:46] daily image or daily snap? [16:46] sorry, please be specific [16:46] zyga: yes, I have that and 3 other PR reviews. Is this super-urgent and blocking a whole bunch of things? I'm still working through a rather large todo list and PR reviews are high on it. hoping to get to it today otherwise tomorrow [16:46] zyga: http://cdimage.ubuntu.com/ubuntu-core/18/20190404/ <- try using this one [16:46] I just booted the vm and hostname is reset [16:46] jdstrand: no, not urgent [16:46] zyga: and tell me if it's still busted there [16:46] mvo, pedronis yes, we need 5 minutes of timeout because otherwise it fails on the boards [16:47] sil2100: can I refresh to a different channel? [16:47] zyga: (since maybe my test-case is wrongish, it's been a long day) [16:47] ok, then I will proceed with the priorities I outlined earlier in the week and will get to it [16:47] jdstrand: +1, thank you [16:47] sil2100: I can try a fresh vm tomorrow, since it is marginally more work, [16:48] zyga: guess you can try the core18 from edge/beta too [16:48] Since I suppose you reproduced it on the stable one, right? [16:48] in the past we updated all the short timeout to 5 minutes because of that [16:48] sil2100: or I can refresh to new channel to see [16:48] I can try that [16:48] correct [16:49] sil2100: trying core18 from edge [16:49] Fingers crossed! [16:50] mvo, I can make that change if you want [16:50] sil2100: edge works [16:51] sil2100: even after reboot [16:51] sil2100: thanks! [16:51] zyga: thank mvo for that! He prepped the systemd changes to make it work o/ [16:51] mvo: thank you then :-) [16:52] zyga: but phew, for a moment thought the systemd changes weren't enough, and I just released those to -updates [16:52] haha [16:52] some cold sweat :) [16:52] it's great, thanks [16:53] hmmm [16:53] are core18 builds going? [16:54] cachio: thank you [16:54] one of my tests failed two hours ago because core18 didn't have /var/lib/snapd/snap [16:54] er [16:54] I meant /var/lib/snapd/void [16:55] mvo, change done, now waiting for tests results [16:55] cachio: ta [16:55] mvo, yaw [16:58] * zyga EODs [16:59] mvo: if you can review / approve 6684 I would love to end the day with that [17:02] zyga: sure [17:05] pedronis: pstolowski|afk: those docs are now published - https://docs.snapcraft.io/developing-hotplug-interfaces and https://docs.snapcraft.io/hotplug-support. I still need to add some 'hardware interface' clarifications to the relevant interface docs. [17:05] I wonder what's the usage of `/var/lib/snapd/void` ? [17:08] zyga: it should have, since we had a new core18 package yesterday, an hour ago and even a few minutes ago! [17:12] degville: thx [17:13] zyga: Thanks for checking out the Solus issue! [17:13] degville: let me know if you need help to find out which ones those are [17:14] pedronis: thanks, will do! [17:20] brlin: AFK [17:21] brlin: literally holding my daughter and typing with one finger [17:21] zyga: Take your time, then ;) === JanC is now known as Guest19220 === JanC_ is now known as JanC [17:50] drat [17:51] ++ stat -c %a /var/lib/snapd/void [17:51] stat: cannot stat '/var/lib/snapd/void': No such file or directory [17:52] earlier in the log I see [17:52] + rsync -av --delete /home/gopath/src/github.com/snapcore/snapd/tests/snapd-state/snapd-lib/snaps /var/lib/snapd [17:52] I bet the state is corrupt somehow and something removes writable directories [17:52] I'll run that test in isolation to see if it passes [17:54] brlin: the void directory is a fallback location [17:54] brlin: when snap-confine starts a program and cannot represent the original working directory inside the changed filesystem [17:54] brlin: it uses /var/lib/snapd/void [18:00] PR snapd#6684 closed: overlord,tests: perform soft refresh check in doInstall [18:04] mvo: around? [18:15] mvo: what writes /writable/system-data/? [18:15] is that ubuntu-image? [18:15] mvo: I just realised that all of the work to make core18 have skeleton tree was 100% useless [18:17] zyga: ? [18:17] pedronis: /var/lib/snapd and /var/cache/snapd are bind-mount to system-data/ [18:17] and the content in the boot base is irrelevant [18:17] the real change has to be done elsewhere [18:18] zyga: does it mean the opengl stuff also never worked on core? [18:18] pedronis: yes, it was never supported on core to begin with [18:19] pedronis: unless the snap ships all the stuff internally [18:19] pedronis: I need to untangle this tomorrow [18:19] it's too late today [18:19] zyga: anyway we need to discuss because making it ubuntu-image problem sounds kind of wrong [18:20] pedronis: I think we need to start by saying what we want to have [18:20] comparing it to what we have now across bases and history cruft [18:20] I'm just puzzled by how this works [18:20] this is also compounded by writable paths and general complexity associated with how that operates [18:21] it's fine, we can discuss tomorrow/next week [18:21] and what is the configuration of the initial mount namespace on core devices [18:21] yes, I'm just puzzled by how complex this turned out to be, it was supposed to be a bugfix for a low-priority CVE [18:22] I thought we were already using void [18:22] for some things [18:23] yes, but nothing tested it on core18 [18:23] it doesn't work [18:23] otoh we know that core 16 mount setup with core != core 18 core18 [18:23] yes, exactly [18:23] on core16 it does work [18:23] because what the core snap carries does matter :) [18:24] that is still confusing to me [18:24] the host has still writable parts [18:24] that come from /writable [18:25] did we simplify writable path config too much on core18 ? [18:25] pedronis: I don't know what the config for core18 is [18:26] pedronis: we should sit down and discuss this for core20 context [18:26] pedronis: at some point in the past I wanted to have a mini-version of snap-confine that does just the mount namespace setup for the initial mount namespace to replace the shell logic currently in the initrd [18:26] heh, sounds like a super tangent [18:26] because keeping that in the core-xxx repo on the side not helping with understanding what is going in [18:26] *on [18:27] yes, to some extent yes [18:27] but the point was to bring the magic closer to snapd repo [18:27] so that we see what's the state on boot [18:27] right now that is locked away in an untested shell script in an initrd somewhere [18:27] it's not exactly transparent :) [18:28] in some way it also relates to snapd.snap and bases [18:28] it would be good to have snapd in the loop of the initial mount namespace construction [18:28] right now we cannot evolve it easily [18:33] pedronis: while I have you: how long should I allow running app to inhibit refresh? [18:39] I need to think a bit, we have a policy for servers, but not sure it applies here [18:40] I will set one week as a strawman for now [18:43] zyga: so yes writable-path config for /var/lib/snapd for core 18 and core 16 is different [18:43] I remember that mvo wanted to simplify it [18:44] we'll need to chat with him [18:45] zyga, pedronis I'm sort of here but would like to EOD - anything urgent? [18:45] I did simplify wirtable-path, does it mean anyhting is missing? [18:46] also remember core18 is smaller than core so some things just did not made any sense anymore [18:46] mvo: yes, /var/lib/snapd is missing stuff in core 18 [18:46] apparently [18:46] mvo: I think it's a conversation for tomorrow [18:46] but is not urgent [18:46] mvo: we are untangling the layers of why things are broken :) [18:46] and will probably be complicated to untangle at this point [18:46] anyway [18:46] pedronis, zyga right - its missing stuff because in core we installed snapd so all the right dirs were there [18:46] in core18 we don't anymore [18:46] zyga: oh, i see [18:47] maybe [18:47] zyga: its /var/lib/snapd in writable-path, correct? [18:47] but the writable-path config is also different [18:47] it was synced , now is persistent [18:47] pedronis: I think we can make it transition [18:47] that should fix things [18:47] this means whatever was there before will be surfaced but only once [18:48] anyway I need to learn what those things mean :) [18:48] synced is a terrible setting [18:48] and is not a topic for tonight at this point [18:48] we should not use this [18:48] yeah, lets sync on this tomorrow [18:48] mvo: I think the issue is that what we assumed was the case, directories being present in core16 because of mount namespace layout, are not true in core18 because the real set is constructed by ubuntu-image [18:48] zyga: setting it to transition is worth a shoot [18:48] zyga: lets chat about this in the morning [18:50] mvo: it's already set to transition [18:54] anyway it's likely actually that the answer is more stuff that the snapd snap needs to do on setup [19:02] re, sorry, small accident [19:24] heh [19:24] so [19:24] go time sucks [19:25] time is not restored across serialization [19:25] not perfectly [19:25] this upsets our conflict checker [19:36] PR snapd#6690 opened: overlord/snapstate: inhibit refresh for up to a week [19:38] * zyga EODs again, for real