| sergiusens | cachio: that is a test that recently broke, joc can put you up to speed on that. | 00:27 |
|---|---|---|
| cachio | sergiusens, great, thanks | 00:58 |
| mup | PR snapcraft#2339 closed: lifecycle: switch to multipass by default <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2339> | 01:40 |
| mup | PR snapcraft#2344 opened: schema: remove the deprecated snap keyword for bases <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2344> | 02:35 |
| mup | PR snapcraft#2345 opened: build providers: remove the qemu implementation <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2345> | 02:47 |
| === chihchun_afk is now known as chihchun | ||
| mup | PR snapd#5972 opened: tests: initial setup for testing current branch on nested vm and hotplug management <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5972> | 03:35 |
| zyga | good morning | 05:23 |
| === chihchun is now known as chihchun_afk | ||
| === chihchun_afk is now known as chihchun | ||
| mvo | good morning! 5867 needs a second review | 06:32 |
| mborzecki | morning | 06:51 |
| mvo | hey mborzecki | 06:54 |
| mvo | mborzecki: if you have a moment, a second review for 5867 would be great | 06:54 |
| mborzecki | mvo: looking | 06:56 |
| mborzecki | mvo: left a comment there | 07:05 |
| === pstolowski|afk is now known as pstolowski | ||
| pstolowski | heyas | 07:05 |
| mborzecki | pstolowski: hey | 07:05 |
| mvo | hey pstolowski ! good morning | 07:08 |
| mup | PR snapd#5913 closed: daemon: expose snapshots to the API <Snapshots 📸> <Created by chipaca> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/5913> | 07:44 |
| mup | PR snapd#5879 closed: vendor, cmd/snap: refactor to accommodate the new less buggy go-flags <Created by chipaca> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/5879> | 07:45 |
| Chipaca | morning | 07:46 |
| Chipaca | who was it that said something about everything being green, yesterday | 07:47 |
| Chipaca | you've doomed us all | 07:47 |
| mborzecki | heh ;) | 07:48 |
| zyga | Hey | 07:49 |
| zyga | (again) | 07:49 |
| zyga | Sorry I sent kids to school and overslept again if that is a thing | 07:50 |
| zyga | I’ll be down shortly | 07:50 |
| Chipaca | WRT "Sorry I sent the kids to school", homeschooling sucks tho | 07:50 |
| lool | ogra: how long to land the kernel patch in source and a kernel deb and a kernel snap? :-) | 07:59 |
| pstolowski | mvo: hey, wrt to hooks optimization from yesterday, hijacking seems to be the only painpoint. the problem is this: in ifacestate.Connect() i know which hooks exist based on snapinfo.Hooks (it has all hooks defined explicitely and also those not defined but present on the disk in meta/hooks). but hijacked ones (core - configure only?) are exceptional as (afaiu) they don't have to be listed in snap yaml (and they have no | 07:59 |
| pstolowski | file on the disk). ideally i'd just ask hookmgr if a (snap, hook) IsHijacked, but we don't have access to hookmgr there. any thoughts on this? there would be no issue if we required hijacked hook to be explicitely listed in the snap yaml. | 07:59 |
| Chipaca | ohhh, nice, there's a spread test that hasn't been working for who-knows-how-long and we didn't know because pipefail :-( | 08:05 |
| mvo | Chipaca: *gar* we should simply kill pipefail again | 08:05 |
| mvo | Chipaca: I think it trades one horror for another | 08:05 |
| * Chipaca looks at mvo | 08:05 | |
| Chipaca | mvo: yes, yes it does | 08:06 |
| Chipaca | mvo: my looking at you was because I was looking at 'git log -p' for the test in case | 08:06 |
| Chipaca | mvo: you should feel guilty | 08:06 |
| mborzecki | Chipaca: which one? | 08:06 |
| Chipaca | nothing serious :) | 08:06 |
| mvo | pstolowski: hm - I need to look but if adding the one hijack we have to meta/snap.yaml for core that seems ok | 08:06 |
| mvo | Chipaca: which oneß | 08:06 |
| Chipaca | still, let me guilt some cake out of mvo | 08:06 |
| mvo | Chipaca: ? | 08:06 |
| Chipaca | mvo: tests/main/help/task.yaml | 08:06 |
| mvo | Chipaca: guilt is my default emotion | 08:07 |
| Chipaca | mvo: grep -e and grep -E are *not the same thing* | 08:07 |
| pstolowski | mvo: but how about reverts? | 08:07 |
| mvo | Chipaca: *cough* indeed - this looks like you reviewed this though ;) I usually write '(foo|bar)' | 08:08 |
| Chipaca | mvo: and in particular, grep -evFx 'foo|bar' will always fail because the pattern is now vFx and 'foo|bar' the file to search for | 08:08 |
| Chipaca | mvo: yep, i missed this too | 08:08 |
| Chipaca | mvo: but what's the fun in blaming myself when I can blame somebody kilometers away | 08:08 |
| mvo | Chipaca: :( what an annoying bug | 08:08 |
| mvo | Chipaca: haha | 08:08 |
| Chipaca | anyway, nothing serious, i'll just be removing the test entirely soon :-) | 08:09 |
| Chipaca | (because I know how to do this in static tests now :-) ) | 08:09 |
| mvo | Chipaca: shame that its review friday or we could just push a pr that removes pipefail again | 08:09 |
| Chipaca | mvo: i think we've already removed pipefail, otherwise this would've failed? | 08:10 |
| mvo | Chipaca: oh, ok | 08:10 |
| Chipaca | but, yeah no, pipefail is just choosing one bag of bugs over another | 08:10 |
| mvo | Chipaca: I thought you discovered while looking at it | 08:10 |
| mvo | Chipaca: anyway, I get back to core18 | 08:10 |
| Chipaca | nah, while looking at running it 'by hand' for a static check | 08:11 |
| Chipaca | mvo: enjoy your eighteen cores | 08:11 |
| Chipaca | sergiusens: do you have tests for packing things with weird exclusion lists? | 08:12 |
| dot-tobias | I'm having trouble installing snapcraft or git through apt in the classic snap on Core stable. “E: Sub-process /usr/bin/dpkg returned an error code (1)” → Does someone have a hint how I can fix that? | 08:15 |
| mup | PR core18#82 opened: hooks: use hkp:// to fetch the key from the keyserver <Created by mvo5> <https://github.com/snapcore/core18/pull/82> | 08:15 |
| ogra | lool, thats a ppisati question, he said he'd put it nxt on his TOOD | 08:19 |
| ogra | *next | 08:20 |
| lool | ogra: Hmm ok, should we put a workaround in place? | 08:20 |
| lool | I realize we have been without a 3b+ compatible image since Jan/Feb, but I'm keen to wrap up this effort, especially since UC18 is coming soon | 08:21 |
| pedronis | mvo: pstolowski: hijacking exists for core configure only, pstolowski are you doing your change in hookstate, and just for interface hooks? | 08:21 |
| pedronis | *and not just | 08:21 |
| dot-tobias | (re my own question five minutes ago): seems like this error line: “update-alternatives: error: error creating symbolic link '/etc/alternatives/jsonschema.dpkg-tmp': No such file or directory” is the root cause; once I've created /etc/alternatives everything works fine | 08:21 |
| pedronis | pstolowski: we run configure even before we have the core | 08:22 |
| pedronis | so we cannot depend on it being there in the meta yaml | 08:22 |
| pedronis | by definition | 08:22 |
| sil2100 | mvo: thanks for the PRs for console-conf! I'll release them into the PPA in some moments | 08:22 |
| mvo | sil2100: I have one more pending | 08:22 |
| pedronis | pstolowski: we need to do snap set system before we have core | 08:22 |
| pstolowski | pedronis: no, i'm doing it in ifacestate.Connect only for interface hooks. i know hijacking is for core configure only, but it's kinda 'open-ended' and made extensible | 08:22 |
| mvo | sil2100: testing it right now | 08:22 |
| pedronis | pstolowski: we cannot make it mandatory to have it in meta.yaml anyway | 08:23 |
| pedronis | pstolowski: it's designed exactly to work even before the snap there | 08:23 |
| pstolowski | pedronis: i see | 08:23 |
| pedronis | pstolowski: honestly if you are changing ifacestate, I would ignore the problem for now | 08:23 |
| mvo | mborzecki: hm, I just go "error: internal error: unsupported download with instance name "/home/mvo/devel/snaps/core18/core18_18_amd64.snap"" from ubuntu image - that looks like a recent change, no? | 08:24 |
| pstolowski | pedronis: i could naively make an exception for core configure in ifacestate, but that would circumvent hijacking (which in the future can have more hijacked stuff, at least in theoyr) | 08:24 |
| ogra | lool, well, honestly .... i wouldnt have held back on this issue but the test lab wil not be able to manage when their IPs change, i dont think its a biggie and it will be easily fixed with a kernel update | 08:24 |
| ogra | and image with a known issue is still better than no image at all ... any workarounds would be rather hackish | 08:24 |
| ogra | *and an | 08:24 |
| pedronis | pstolowski: I don't understand how configure relates to ifacestate | 08:24 |
| pedronis | I'm getting a bit lost | 08:24 |
| mborzecki | mvo: what was the command? | 08:25 |
| mborzecki | mvo: ah it was from ubuntu image | 08:25 |
| lool | ogra: I was thinking u-boot var to override mac address, perhaps via cmdline/initrd | 08:25 |
| pstolowski | pedronis: ifacestate.Connect() with my change will only create HookTask if it sees given hook. and if it doesn't create task for core configure, then hijacking will not even have a chance to run | 08:25 |
| lool | but yeah, I agree it's hackish | 08:25 |
| mvo | mborzecki: yes, via --extra-snaps | 08:25 |
| lool | ogra: alright, let's fix it cleanly and take the time | 08:26 |
| ogra | lool, well, that first needs an initrd hack to make use of that uboot var | 08:26 |
| Chipaca | mvo: ogra: who "owns" the classic snap? | 08:26 |
| pedronis | pstolowski: what's the relation between ifacestate.Connect and core configure ? | 08:26 |
| pedronis | core configure is not an interface hooks | 08:26 |
| pedronis | pstolowski: I'm missing something here | 08:26 |
| ogra | Chipaca, theoretically mvo and me i guess | 08:26 |
| mborzecki | mvo: that's weird, it's like it's calling Download(/home/mvo/....core18_18_amd64.snap) | 08:26 |
| pstolowski | pedronis: aaah, you're right, silly me | 08:27 |
| pstolowski | of course | 08:27 |
| Chipaca | ogra: mvo: dunno if you read dot-tobias's report above, but it looks like we've broken it recently with /etc/alternatives shenanigans | 08:27 |
| mvo | Chipaca: uh | 08:27 |
| ogra | Chipaca, i used it yeterday ... i that core 16 ? | 08:27 |
| pstolowski | pedronis: so yeah, it's theoretical problem anyway in case of future hijacked hooks | 08:27 |
| ogra | +s | 08:27 |
| Chipaca | ogra: did you use it to install something that tried to write to /etc/alternatives? | 08:28 |
| pedronis | pstolowski: I don't think we need to solve it now, anyway as I said given the nature of hijack | 08:28 |
| pstolowski | (i got carried away with configure hook because of that) | 08:28 |
| pedronis | expecting it to be in the yaml | 08:28 |
| ogra | nah, just to build a snap with snapcraft | 08:28 |
| mvo | dot-tobias: thanks for the report! we will fix it soon | 08:28 |
| pedronis | would not be a solution | 08:28 |
| ogra | did we change alternatives handling in core 16 ? i thought that was an 18 thing | 08:29 |
| mvo | mborzecki: aha - nevermind, its not validation its just a stupid error message | 08:29 |
| mvo | mborzecki: typo in the path | 08:29 |
| mvo | mborzecki: so it does stat() fails -> tries to download | 08:29 |
| mvo | mborzecki: which is silly, if it looks like a path it should give a useful error. sorry for the noise | 08:30 |
| mborzecki | mvo: right, we could produce more useful error in image.go:acquireSnap() | 08:30 |
| mborzecki | mvo: in case the snap ends with .snap | 08:30 |
| mvo | mborzecki: yeah, not high priority but definitely a good improvement | 08:30 |
| mborzecki | mvo: also funny cause it would actually poke the store to get that snap? :) | 08:31 |
| mborzecki | (if it wasn't for that instance name check) | 08:31 |
| mvo | mborzecki: yeah | 08:31 |
| dot-tobias | mvo, ogra: I just double-checked, the error occurs on a raspi 3B with core 16-2.35.2 stable (rev 5546) and classic snap 16.04 rev 26 (edge) | 08:33 |
| dot-tobias | (/etc/alternatives thing with apt install) | 08:33 |
| Chipaca | dot-tobias: and this was trying to install what? | 08:37 |
| dot-tobias | Chipaca: Tried installing git and snapcraft (independently) | 08:38 |
| mup | PR snapd#5944 closed: cmd/snap: speed up unit tests <Created by chipaca> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/5944> | 08:38 |
| ogra | weird, i did the same just yesterday | 08:38 |
| ogra | on a dragoboard though but that shouldnt matter | 08:39 |
| mborzecki | Chipaca: left some comments in #5955 and #5957 | 08:40 |
| mup | PR #5955: cmd/snap, tests: snapshots for all <Snapshots 📸> <⛔ Blocked> <Created by chipaca> <https://github.com/snapcore/snapd/pull/5955> | 08:40 |
| mup | PR #5957: overlord/snapshotstate/backend: fall back on sudo when no runuser <Snapshots 📸> <Created by chipaca> <https://github.com/snapcore/snapd/pull/5957> | 08:40 |
| Chipaca | mborzecki: yep, just reading | 08:40 |
| Chipaca | mborzecki: when you say "list" i'm assuming you menan "list-snapshots" or some such? | 08:40 |
| mborzecki | yes | 08:41 |
| Chipaca | mborzecki: note "saved" was not my decision, but was arrived at after much discussion | 08:41 |
| mborzecki | ahh ok, so then it | 08:41 |
| Chipaca | same for all of the commands in there, actually | 08:41 |
| mborzecki | 's ok for me | 08:41 |
| Chipaca | onlu one I winged was check | 08:41 |
| Chipaca | only* | 08:41 |
| mborzecki | Chipaca: maybe we could make that check help message more friendly | 08:42 |
| Chipaca | mborzecki: «check-snapshot tells you if your 📸 is actually a 🍆»? | 08:43 |
| * Chipaca runs | 08:44 | |
| mborzecki | eggplant? | 08:44 |
| Chipaca | vegetable | 08:45 |
| mborzecki | haha | 08:45 |
| Chipaca | mborzecki: https://www.dictionary.com/e/emoji/eggplant-emoji/ | 08:46 |
| mborzecki | well, hashsums feels brutalist :) | 08:46 |
| mborzecki | wow, that's one interesting emoji | 08:46 |
| Chipaca | hehe | 08:47 |
| mborzecki | mvo: the logs are here https://forum.snapcraft.io/t/snapd-not-responding/7867/6 | 08:52 |
| mborzecki | hm should failure in catalog refresh be fatal? | 08:55 |
| mborzecki | nvm, the errors are only logged, the machine is rebooting though | 09:01 |
| mvo | mborzecki: hm, hm, the logs are not exactly telling much :( | 09:07 |
| mvo | mborzecki: I guess we need grub-editenv list next to see if something strange in our boot config is happening | 09:07 |
| mvo | mborzecki: but it might be something external rebooting, like a broken watchdog or something | 09:08 |
| * mvo scratches head | 09:08 | |
| zyga | mvo: systemd knows why the system rebooted | 09:14 |
| * zyga tires to remember how | 09:14 | |
| mborzecki | zyga: journalctl -b -1 ? | 09:17 |
| zyga | I think there was something deeper | 09:26 |
| zyga | perhaps it was not systemd but UEFI | 09:26 |
| zyga | AFAIK the boot reason is stored for user space to know | 09:26 |
| zyga | but starting with the last boot log would be useful | 09:26 |
| mvo | zyga: did scott get back to you last night about the overlay issue btw? | 09:26 |
| mvo | zyga: I wonder what the status here is and if we need some last minute fixes | 09:27 |
| zyga | mvo: I read the backlog | 09:27 |
| zyga | and I have the data to reproduce this | 09:27 |
| zyga | I think it is problematic, we were not expecting this use case | 09:27 |
| zyga | give me a moment | 09:27 |
| zyga | woah! | 09:30 |
| zyga | I managed to install a snap | 09:30 |
| zyga | after installing core | 09:30 |
| zyga | but then core install failed | 09:30 |
| zyga | and I got a snap without core | 09:30 |
| * zyga collects changes | 09:30 | |
| === cpaelzer_ is now known as cpaelzer | ||
| zyga | http://paste.ubuntu.com/p/q7pNPtJKNH/ | 09:32 |
| zyga | mvo: the system is using overlayfs | 09:33 |
| zyga | mvo: AppArmor service is active (despite some logic that disables it when / is overlayfs, the logic doesn't trigger in this case because the backing directory is different, | 09:34 |
| zyga | snapd detects the overlay and injects a special profile for snap-confine | 09:35 |
| zyga | but that profile is probably not active, we did not reload the profile of the system-loaded snap-confine (the one in /etc, not from the core reexec) | 09:35 |
| pedronis | mvo: one of the conclusion was that they don't actually need snapd to work | 09:36 |
| pedronis | in that context | 09:36 |
| pedronis | indeeed is a bit of waste of snapd to seed there | 09:36 |
| zyga | I think there is a real bug still | 09:37 |
| pedronis | but they have use case with overlayfs that need to work to some extent | 09:37 |
| pedronis | zyga: it's ok, I just expected them to then lament that we made seeding work but then boot there is slow :) | 09:37 |
| zyga | heh :) | 09:38 |
| pedronis | I mean fixing bug is orthogonal to turning off snapd there anyway | 09:38 |
| mvo | pedronis: aha, thank you | 09:38 |
| zyga | interestingly, in the scenario today we do not reexec | 09:39 |
| zyga | because 18.10 is ahead of stable | 09:39 |
| zyga | that probably compounds the bug I mentioned | 09:39 |
| zyga | but something more is fishy, hold on | 09:39 |
| zyga | aha | 09:40 |
| pedronis | mvo: I think last words sounded like they could probably work around things on their side by disbaling snapd.service completely there | 09:40 |
| pedronis | that sounded what steve was proposing | 09:40 |
| zyga | we grant permission: "/media/root-rw/overlay/{,**/}" r, | 09:40 |
| zyga | and the denial is: "/overlay/" r, | 09:41 |
| mup | PR snapd#5973 opened: image: improve validation of extra snaps <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5973> | 09:41 |
| zyga | I suspect the paths differ because of something that was done in intrd | 09:41 |
| mborzecki | mvo: this should catch the typos and invalid names ^^ | 09:41 |
| mvo | mborzecki: nice | 09:41 |
| zyga | mvo: we can detect and add a hack | 09:42 |
| zyga | with that changed I can install and run a simple snap | 09:42 |
| zyga | trying to install lxd now | 09:43 |
| zyga | trying to use lxd for whatever | 09:45 |
| zyga | mvo: lxd doesn't work because it cannot setup networking | 09:47 |
| zyga | failed to list ipv4 rules for LXD network lxdbr0 (table filter) | 09:47 |
| zyga | there are no more denials | 09:47 |
| zyga | so perhaps missing module or something like that | 09:48 |
| zyga | anyway, I think that we can fix the immediate issue they saw with a one liner hack | 09:48 |
| zyga | but it's not something I'm very proud of | 09:48 |
| zyga | mvo: let me prepare a patch, you can decide | 09:49 |
| mvo | zyga: ta | 09:54 |
| mvo | and 5971 needs a review | 09:57 |
| Chipaca | mvo: i retried 5971 ~5 times last night | 09:57 |
| Chipaca | fyi | 09:58 |
| mvo | Chipaca: meh, that sounds nasty | 09:59 |
| mvo | Chipaca: failures all over the place? or some pattern? | 10:00 |
| mvo | things look very red in general right now :/ | 10:00 |
| Chipaca | mvo: no discernible pattern | 10:00 |
| zyga | mvo: what is the magic LP:# syntax that is picked up by gnome-terminal | 10:00 |
| zyga | for launchpad bug references | 10:00 |
| Chipaca | wat | 10:00 |
| mvo | zyga: in changelogs we use LP:#1234 | 10:01 |
| mup | PR #1234: debian: make snapd get its environ from /etc/environment <Created by chipaca> <Merged by niemeyer> <https://github.com/snapcore/snapd/pull/1234> | 10:01 |
| zyga | thanks! | 10:01 |
| cjwatson | missing a space there | 10:01 |
| mvo | zyga: *but* gnome-termainal I'm not sure | 10:01 |
| mvo | cjwatson: aha, thanks (^- zyga) | 10:01 |
| cjwatson | LP: #1234 | 10:01 |
| cjwatson | https://help.launchpad.net/Code/Git#Linking_to_bugs has the regex | 10:01 |
| mvo | ta! | 10:01 |
| zyga | hmm | 10:06 |
| * zyga found more weirdness | 10:06 | |
| zyga | but anyway | 10:06 |
| zyga | patch incoming | 10:06 |
| zyga | mvo: I think there will be two patches, one is ready, the other one may be needed to trigger regular snap-confine profile reload | 10:08 |
| zyga | we don't have code for that, do we? | 10:08 |
| zyga | plus this will go south if someone notices and fixes the apparmor.service unit not to start | 10:08 |
| mvo | zyga: hm, hm | 10:09 |
| mup | PR snapd#5974 opened: osutil: workaround overlayfs on ubuntu 18.10 <Created by zyga> <https://github.com/snapcore/snapd/pull/5974> | 10:15 |
| zyga | mvo: ^ | 10:15 |
| zyga | mvo: I also updated the bug report | 10:15 |
| zyga | I will check the other part of this issue now | 10:15 |
| mvo | ta | 10:16 |
| zyga | mvo: ok, I looked | 10:17 |
| mup | PR snapd#5975 opened: ifacestate/hooks: only create interface hook tasks if hooks exist <Created by stolowski> <https://github.com/snapcore/snapd/pull/5975> | 10:17 |
| zyga | we are good, we do reload profiles | 10:17 |
| pstolowski | mvo, zyga, pedronis ^ | 10:17 |
| mvo | pstolowski: thanks | 10:20 |
| mvo | ha! 5971 is green - now it just needs reviews :) | 10:20 |
| pstolowski | mvo: i'm curious how it improves situation | 10:20 |
| niemeyer | mvo: Minor suggestion for the message on #5971 and LGTM | 10:22 |
| mup | PR #5971: interfaces: include invalid type in Attr error <Simple 😃> <Created by mvo5> <https://github.com/snapcore/snapd/pull/5971> | 10:22 |
| mvo | niemeyer: thank you! | 10:22 |
| mup | PR snapd#5739 closed: interfaces/default: don't scrub with change_profile with classic <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5739> | 10:32 |
| mup | PR snapd#5867 closed: many: enable layouts by default <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5867> | 10:32 |
| zyga | \o/ | 10:32 |
| Chipaca | mvo: question for you sir: is there anything using .snapignore that you know of? | 10:33 |
| mup | PR snapd#5971 closed: interfaces: include invalid type in Attr error <Simple 😃> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5971> | 10:33 |
| mvo | Chipaca: I don't know | 10:33 |
| Chipaca | :+1: | 10:34 |
| Chipaca | :) | 10:34 |
| mvo | Chipaca: is it supported by snapcraft too? | 10:34 |
| mvo | Chipaca: or just by us? | 10:34 |
| Chipaca | mvo: no | 10:34 |
| mvo | Chipaca: ok, then I think its fine to kill it | 10:34 |
| Chipaca | mvo: it's from 2015 | 10:34 |
| mvo | Chipaca: its cargo cult from click | 10:34 |
| Chipaca | yep | 10:34 |
| Chipaca | mvo: and what about the exclusion of DEBIAN/? | 10:34 |
| Chipaca | sounds weird also | 10:34 |
| mvo | Chipaca: same I would say | 10:34 |
| Chipaca | mvo: maybe we don't need exclusions at all :-) | 10:35 |
| mvo | Chipaca: killemall | 10:35 |
| mvo | Chipaca: haha | 10:35 |
| mvo | Chipaca: really? | 10:35 |
| mvo | Chipaca: are those the only patterns? we are an inclusive bunch | 10:35 |
| Chipaca | mvo: well, snapcraft very carefully builds the thing | 10:35 |
| Chipaca | mvo: nah, we have a bunhc | 10:35 |
| Chipaca | bunch | 10:35 |
| mvo | good point | 10:35 |
| mup | PR snapcraft#2345 closed: build providers: remove the qemu implementation <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2345> | 10:35 |
| mup | PR core18#82 closed: hooks: use hkp:// to fetch the key from the keyserver <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/core18/pull/82> | 10:41 |
| zyga | pstolowski: reviewed the hooks PR | 10:45 |
| mborzecki | zyga: wonder if we could improve detection of snap mount dir in snapd, say what if we could ask snap-confine for the right path (i.e. the path it was configured with)? | 10:46 |
| zyga | mborzecki: I would actually do it backwards | 10:47 |
| zyga | mborzecki: have snapd tell snap-confine | 10:47 |
| pstolowski | zyga: thanks! looking | 10:47 |
| zyga | and have the entire logic in dirs.go in one place | 10:47 |
| zyga | mborzecki: it's not supposed to be a new PR Friday but I have a PR that removes the directory configure option from snap-confine that I mentioned a few times | 10:47 |
| mborzecki | zyga: yeah, one source of truth would be nice | 10:48 |
| zyga | one source of facts ;-) | 10:48 |
| mborzecki | zyga: as for no-new-PRs, I opened one :P although a simple one | 10:48 |
| zyga | mborzecki: let's explore this on monday | 10:48 |
| zyga | and devote this day to important fixes and reviews | 10:49 |
| zyga | mvo: have you seen the overlay PR:? | 10:49 |
| ppisati | ogra: we already have facilities to set the mac address from DT: https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/xenial/commit/?h=raspi2&id=d093067d07ffba9f50d38a2e572f7ddbd36daf5a | 10:49 |
| ppisati | ogra: check drivers/of/of_net.c::of_get_mac_address | 10:50 |
| ppisati | ogra: it's the same stuff as the other patch | 10:50 |
| ppisati | ogra: oh, same for LEDs | 10:51 |
| ppisati | ogra: https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/xenial/commit/?h=raspi2&id=1ad73f268cedf4a7999fabf9aecb77e0f0cd3c00 | 10:51 |
| pstolowski | zyga: replied | 11:05 |
| zyga | yep, I see | 11:05 |
| jdstrand | mvo: hi! as you probably saw in the email, I didn't finish the k8s interfaces, but I could today. I just have a couple of open questions I can discuss with niemeyer to clean things up for the review | 11:42 |
| zyga | jdstrand: hey | 11:44 |
| jdstrand | niemeyer: hi! do you have a few minutes to discuss some kubernetes interfaces? | 11:44 |
| jdstrand | that I'm working on | 11:44 |
| jdstrand | niemeyer: I have 3 questions | 11:45 |
| jdstrand | hey zyga :) | 11:45 |
| zyga | jdstrand: can you look at https://github.com/snapcore/snapd/pull/5974 | 11:45 |
| mup | PR #5974: osutil: workaround overlayfs on ubuntu 18.10 <Created by zyga> <https://github.com/snapcore/snapd/pull/5974> | 11:45 |
| zyga | it's very short, just tell me what you thibnk | 11:45 |
| jdstrand | zyga: what does mountinfo say about this system? | 11:52 |
| zyga | jdstrand: the test I added shows the mountinfo data exactly from the affected system | 11:53 |
| zyga | and I still have it booted if you have any questions | 11:53 |
| jdstrand | zyga: that shows a single entry, yes, but are there multiple? | 11:53 |
| zyga | multiple overlayfs-es? | 11:54 |
| jdstrand | eg, maybe there are layered overlayfs | 11:54 |
| zyga | no, there's only one | 11:54 |
| zyga | let me pastebin the whole file | 11:54 |
| zyga | http://paste.ubuntu.com/p/6kgSyPjJGG/ | 11:55 |
| zyga | I have to "paste" the URL by hand because $reasons | 11:55 |
| mborzecki | when a snap has sockets or timers, snapctl stop <snap> only stops the service, but not the sockets nor timers, then #5777 addresses that, but when snapctl start is issued it starts sockets, timers and the service :/ | 11:56 |
| mup | PR #5777: wrappers/services.go: don't start disabled services <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/5777> | 11:56 |
| zyga | mborzecki: I'm more re-affirmed that we should expose the full set of services, sockets and timers inside the snap and not outside | 11:56 |
| zyga | mborzecki: and on the outside, only expose "knobs" that are arbitrary subsets of the real set | 11:57 |
| zyga | mborzecki: that are defined by the snap developer | 11:57 |
| mborzecki | zyga: well, atm you cant really disable a socket or a timer from inside the snap | 11:57 |
| mborzecki | idk feels like we're missing something or trying to hide too much | 11:58 |
| zyga | I agree | 11:58 |
| jdstrand | zyga: you are supposed to end up with this rule: '"###UPPERDIR###/{,**/}" r,' which becomes '"/overlay/{,**/}" r,' | 11:59 |
| jdstrand | zyga: which should cover '/overlay/' | 11:59 |
| zyga | right | 11:59 |
| zyga | jdstrand: am I missing something/ | 12:00 |
| jdstrand | zyga: oh, you are saying that you fixed *that* | 12:00 |
| zyga | yes | 12:00 |
| zyga | :D | 12:00 |
| jdstrand | I thought you were saying that still remained | 12:00 |
| zyga | no no, it's working with this patch | 12:00 |
| zyga | but I cannot explain the behaviour of the kernel | 12:00 |
| jdstrand | ok, well the overlayroot thing is doing a pivot_root or something | 12:00 |
| zyga | hmmm, I'm not sure it does | 12:01 |
| zyga | but it may | 12:01 |
| zyga | but ... | 12:01 |
| zyga | the upper and lower paths are still available | 12:01 |
| zyga | I mean, /media/... all of that exists and shows the real stuff | 12:01 |
| zyga | btw | 12:01 |
| zyga | I noticed something odd, look at the path of the workdir | 12:01 |
| zyga | yes, it ends with _ | 12:01 |
| zyga | why? | 12:01 |
| ogra | ppisati, well, then i dont get why it does not work ... does the code actually use "local-mac-address" from the DT ? | 12:01 |
| jdstrand | I was wondering about that too. but that is setup by whatever did the mount | 12:01 |
| jdstrand | well, typically | 12:02 |
| zyga | yep | 12:02 |
| ogra | ppisati, because thats where it is in the DT | 12:02 |
| jdstrand | maybe the kernel changed | 12:02 |
| jdstrand | zyga: what package is used to setup the mount? | 12:02 |
| zyga | I don't know but there is /etc/overlayroot.local.conf | 12:02 |
| zyga | let me look | 12:02 |
| zyga | looks like overlayroot | 12:03 |
| mvo | jdstrand: yeah, saw it but no worries, if its isolted we can cherry pick it later | 12:03 |
| zyga | jdstrand: cloud-initramfs-tools | 12:03 |
| ijohnson | mborzecki: what do you think I should do with #5777 ? I haven't gotten a chance to add tests, but from the forum it seemed unclear that everyone was on board with `snap stop <service>` always also stopping the associated sockets/timers, which is what I implemented | 12:03 |
| mup | PR #5777: wrappers/services.go: don't start disabled services <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/5777> | 12:03 |
| jdstrand | ah, cloud-initramfs-tools | 12:03 |
| zyga | jdstrand: reading that now | 12:04 |
| zyga | god how I hate shell | 12:05 |
| jdstrand | # PERSIST_DIR will persist after pivot-root | 12:05 |
| zyga | miles of shell in initrd | 12:05 |
| jdstrand | so, something is doing that | 12:05 |
| zyga | yeah | 12:05 |
| zyga | but it is set to /run/initramfs | 12:05 |
| jdstrand | that's good enough for me to explain the directory | 12:05 |
| jdstrand | that dir, yes | 12:06 |
| jdstrand | the point is, its talking about the initramfs pivot, etc | 12:07 |
| Chipaca | zyga: is your lab & mac available for some poking? | 12:07 |
| zyga | Chipaca: sure | 12:07 |
| Chipaca | zyga: ok thanks | 12:07 |
| Chipaca | zyga: ssh: Could not resolve hostname imac-pro-zygmunt.local: Name or service not known | 12:07 |
| zyga | ah, I think I tried to fix the hostname | 12:08 |
| zyga | one sec | 12:08 |
| Chipaca | heh | 12:08 |
| zyga | moved from wired to wifi | 12:08 |
| zyga | try ipro-zygmunt.local please | 12:08 |
| Chipaca | better | 12:08 |
| zyga | I renamed it because while I had both connections macOS would rename the hostname to unclash it with itself | 12:08 |
| zyga | jdstrand: so how do you feel about the PR | 12:09 |
| zyga | I think nothing short of teaching apparmor about pivot root will break this | 12:10 |
| zyga | but if that happens all hell will break loose | 12:10 |
| zyga | (we'll notice) | 12:10 |
| jdstrand | ah, the man page talks about overlayroot doing a chroot | 12:11 |
| jdstrand | zyga: I like the pr fine. I don't like why we don't know why the path is what it is | 12:11 |
| zyga | brb, the dog is telling me it's time to go | 12:16 |
| zyga | jdstrand: grepping for pivot_root in the package yields nothing | 12:16 |
| zyga | jdstrand: but chroot *is* used | 12:17 |
| zyga | jdstrand: btw, I looked at /proc/self/root and it was / | 12:17 |
| zyga | is that expected if a chroot happened? | 12:17 |
| zyga | (I'll reply from the phone) | 12:17 |
| jdstrand | yes, and with chroot, you'll see the beginning of the path for the chroot (ie, the chroot directory is known to apparmor) | 12:17 |
| zyga | Another question is why this happens here and not on the live cd | 12:19 |
| zyga | Is it just a coincidence that it works on the live cd? | 12:19 |
| zyga | About chroot, so are you saying that since I saw / and not some longer path then chroot did not happen? | 12:20 |
| jdstrand | zyga: I'm saying that a) I didn't read or trace the whole source to figure out exactly what is happening but that b) we know it is using a combination of overlay and chroot (as opposed to overlay and pivot_root) | 12:25 |
| jdstrand | zyga: so I responded to the PR with that in mind | 12:26 |
| jdstrand | zyga: in summary, I approved it, but please adjust the comment | 12:27 |
| zyga | Ack | 12:27 |
| zyga | Thank you | 12:27 |
| mup | PR snapd#5976 opened: interfaces: improve Attr error further <Created by mvo5> <https://github.com/snapcore/snapd/pull/5976> | 12:27 |
| jdstrand | niemeyer: can you ping me when you have a couple of minutes, I don't want to just fire of the questions with no context | 12:27 |
| jdstrand | off* | 12:27 |
| mup | PR snapd#5973 closed: image: improve validation of extra snaps <Simple 😃> <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5973> | 12:31 |
| niemeyer | jdstrand: Heya | 12:41 |
| niemeyer | jdstrand: I have a few meetings back to back now, but later in the afternoon it's open.. happy to chat | 12:42 |
| jdstrand | niemeyer: hi! | 12:42 |
| jdstrand | niemeyer: I think this should be quick | 12:42 |
| jdstrand | niemeyer: the context is that we want interfaces to support k8s worker nodes | 12:43 |
| jdstrand | niemeyer: and this is defined as kubelet and kubeproxy | 12:43 |
| jdstrand | niemeyer: which are two different daemons | 12:43 |
| jdstrand | niemeyer: so I have 2 interfaces, currently named k8s-kubelet and k8s-kubeproxy. I think that should change | 12:44 |
| jdstrand | niemeyer: to use -support at least. eg, k8s-kubelet-support or just kubelet-support | 12:44 |
| jdstrand | probably the latter | 12:45 |
| jdstrand | thoughts? | 12:45 |
| jdstrand | (and of course, kubeproxy-support (or whatever)) | 12:45 |
| jdstrand | niemeyer: these are similar to other -support interfaces in that we are granting special things that only these need | 12:46 |
| mup | PR snapd#5977 opened: release: 2.36~pre2 <Created by mvo5> <https://github.com/snapcore/snapd/pull/5977> | 12:48 |
| mup | PR snapd#5947 closed: many: cleanup remaining parallel installs TODOs <Parallel installs ⛓> <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/5947> | 12:53 |
| popey | On a clean core install, if you snap refresh it reboots twice. One for core, one for kernel. Why doesn't it bundle them in one boot? | 13:12 |
| ogra | popey, never heppened to me ... weird | 13:36 |
| ogra | *happened | 13:37 |
| popey | might be pilot error | 13:39 |
| ppisati | ogra: uhm, ok | 13:39 |
| ppisati | ogra: let me dig in | 13:39 |
| ogra | ppisati, i definitely see the MAC changing in the running system but i also see the MAC fixed in the DT ... so something isnt properly picked up | 13:43 |
| ppisati | ogra: do you get a random MAC at every reboot? | 13:45 |
| ogra | ppisati, yes ... ifconfig shows a different one every boot ... | 13:47 |
| ogra | the devicetree has the same one in "local-mac-address" though | 13:47 |
| ogra | (the same non-changing one that is ... differing from ifconfig output) | 13:48 |
| ppisati | ogra: ok | 13:49 |
| smoser | zyga: i think i root-cause diagnosed that bug. commented on your pull request. | 13:49 |
| zyga | smoser: looking | 13:49 |
| smoser | sorry for the bikeshed conversation with jdstrand (i hadn't noticed he'd commented there) on the code comments :) | 13:51 |
| smoser | but i do think it better to mention the 'overlayroot' package than "Ubuntu server 18.10" | 13:51 |
| * pstolowski late lunch | 13:53 | |
| roadmr | zyga: time to try that snap upload that was giving you trouble - the required version of reviewer tools is now in the store | 13:56 |
| jdstrand | smoser: the overlayroot man page and scripts specically say they used chroot. if pivot_root was used, I would expect the directory to be '/', not '/overlay/' in the apparmor denial. regardless. the comment can be further generalized. the code was never meant to be completely generalized for any overlay situation | 13:56 |
| zyga | roadmr: thanks, I will try now | 13:57 |
| zyga | I tried earlier today without success | 13:57 |
| zyga | smoser: looking at the source of the package I was unable to find the use of pivot_root but I did see chroot | 13:57 |
| jdstrand | smoser: but rather only meant to handle specific situations where we know snaps are being run in a overlayfs situation. we need better apparmor/overlay support in the kernel to be better | 13:57 |
| mvo | cachio: I triggered the 2.36~pre2 core build, should be in edge ~1h | 13:58 |
| smoser | jdstrand: overlayroot is a "normal" initramfs hook. it sets up some mounts and then lets initramfs do its normal pivot and exec of /sbin/init | 13:58 |
| mvo | (max) | 13:58 |
| zyga | smoser: ah, I see, this explains it then | 13:59 |
| zyga | and is in sync with the pivot_root behaviour of apparmor | 13:59 |
| jdstrand | well except that upperdir is showing up as /overlay/ somehow with apparmor | 13:59 |
| jdstrand | I don't know the overlyroot and initramfs stuff well enough to explain why it is happening that way | 14:00 |
| smoser | jdstrand: did you read my comment ? | 14:00 |
| jdstrand | I did | 14:00 |
| niemeyer | jdstrand: +1 to dropping the k8s- prefix | 14:00 |
| smoser | the code just should not assume that the first field '/cow' in casper | 14:00 |
| smoser | means anything | 14:00 |
| niemeyer | jdstrand: Are their needs different enough to justify the two interfaces, or would it be worth exploring the potential for a single one? | 14:00 |
| jdstrand | smoser: as mentioned, this code can't be generalized without better apparmor/overlay support in the kernel | 14:01 |
| smoser | rather it needs to parse /proc/mounts in reverse order to find the second field that matches. | 14:01 |
| smoser | it can be generalized for at least these 2 cases. | 14:01 |
| smoser | but yes, i do understand that apparmor and overlayfs aren't the best of friends. | 14:02 |
| jdstrand | niemeyer: ok, well, there is an existing kubernetes-support interface, but it is ancient and nothing uses it | 14:02 |
| jdstrand | niemeyer: jcastro (yes, that ancient) was super jazzed about k8s as a snap so I through something together for comment and then never heard anything back | 14:03 |
| niemeyer | jdstrand: But are they mostly the same? | 14:03 |
| jdstrand | niemeyer: they is significant overlap, yes. the old kubernetes-support has overlap with docker-support too though | 14:04 |
| jdstrand | s/is// | 14:05 |
| cachio | mvo, nice | 14:05 |
| jdstrand | niemeyer: see https://github.com/snapcore/snapd/compare/master...jdstrand:k8s-worker-interfaces?expand=1 | 14:05 |
| jdstrand | niemeyer: but kubelet needs more than kubeproxy | 14:05 |
| jdstrand | niemeyer: and kubelet needs a lot | 14:05 |
| jdstrand | niemeyer: specifically, kubelet needs to do mounts into the container, but kubeproxy does not | 14:06 |
| niemeyer | I see | 14:06 |
| jdstrand | niemeyer: so it would be nice if kubeproxy didn't have those accesses | 14:06 |
| jdstrand | niemeyer: we could have attributes for flavors of a single interface | 14:06 |
| jdstrand | which achieves the desired security properties, but isn't as transparent to the user of the system | 14:07 |
| niemeyer | jdstrand: +1 | 14:07 |
| jdstrand | ok, I'll refact to use attributes | 14:08 |
| jdstrand | refactor | 14:08 |
| niemeyer | jdstrand: If we have nothing using kubernetes-support, we should probably kill it | 14:08 |
| jdstrand | niemeyer: well, or, I could just use it with attributes :) | 14:08 |
| jdstrand | pedronis: can you do a store search to see if anything is using the kubernetes-support interface? | 14:09 |
| jdstrand | niemeyer: I'll work through those details, but I have another interesting question | 14:09 |
| pedronis | jdstrand: in a meeting, I can in a bit | 14:09 |
| jdstrand | niemeyer: so, you may recall the probelsm with ptrace | 14:09 |
| jdstrand | pedronis: thanks | 14:10 |
| niemeyer | jdstrand: Meeting rolling too, but feel free to go ahead and I'll reply once I'm off | 14:10 |
| jdstrand | niemeyer: specifically, the kernel triggers a ptrace (trace) denial in situations when an application is not actually trying to attach and read its memory/etc | 14:10 |
| jdstrand | niemeyer: eg, certain invocations of 'ps' will cause ptrace denials | 14:11 |
| jdstrand | niemeyer: that use of ptrace is harmless enough, but we can't tease out the difference in policy between that and a full on gdb-style "I'm taking you over" ptrace | 14:12 |
| jdstrand | niemeyer: because the kernel doesn't give us the info to do that (it is kinda similar to how SYS_ADMIN is a catchall for a lot of things. 'trace' groups unrelated things together | 14:13 |
| jdstrand | ) | 14:13 |
| jdstrand | niemeyer: ok, so, we need to allow 'ps' in, say, system-observe (that is the point of the interface), but we don't ptrace (trace) of unconfined, etc, etc because that would constitute a sandbox escape | 14:14 |
| jdstrand | niemeyer: all of the above is context for the fact that we have 'deny ptrace (trace)' rules sprinkled in a couple interfaces to cut down on ridiculously verbose logged denials, but allow the use of something like 'ps' (ps will trigger the denial but work fine otherwise) | 14:15 |
| mup | PR snapd#5777 closed: wrappers/services.go: don't start disabled services <Created by anonymouse64> <Closed by anonymouse64> <https://github.com/snapcore/snapd/pull/5777> | 14:16 |
| jdstrand | niemeyer: *but* k8s actually needs 'ptrace (trace)' to function. because you can't undo a deny rule, the k8s worker can't (currently) specify 'plugs: [ system-observe ]' | 14:17 |
| mup | PR snapd#5970 closed: snapstate: tweak GetFeatureFlagBool() to have a default argument <Created by mvo5> <Merged by stolowski> <https://github.com/snapcore/snapd/pull/5970> | 14:18 |
| jdstrand | niemeyer: which brings me to my point: I'd like for the worker to user standard interfaces like system-observe, network-control, etc rather than copying stuff from those into the k8s interface to avoid the deny rule | 14:19 |
| Chipaca | mvo: I remembered what I was going to say in the standup and then forgot | 14:21 |
| jdstrand | niemeyer: but wanted you opinion on how to do it. the option I'm currently liking is that system-observe, network-control, etc have the same default behavior, but we add an interface attribute (eg, 'omit-ptrace-deny: true') which, when specified, simply skips adding those rules to the policy | 14:21 |
| Chipaca | mvo: cachio: starting with 2.36 I'd like to add a new step to the release checklist; how do I go about doing that? | 14:22 |
| jdstrand | niemeyer: in these specific cases, that's ok security wise, because we are default deny and the access is still block by the interface. but it allows another interface to be used in conjunction with it | 14:22 |
| jdstrand | niemeyer: another route is not expose it to the snap developer and instead simply reuse the rules unconditionally in the k8s interface | 14:23 |
| jdstrand | niemeyer: but don't pull in the ptrace rules. eg, system-observe internally has systemObserveConnectedPlugAppArmor, so we might break that into systemObserveConnectedPlugAppArmor and systemObserveConnectedPlugAppArmorDenyPtrace. then in kubernetes-support it adds systemObserveConnectedPlugAppArmor to its policy | 14:26 |
| jdstrand | niemeyer: so it would get all of it by simply using 'plugs: [ kubernetes-support ]' | 14:26 |
| jdstrand | niemeyer: a 3rd option is trying to work out a new 'ptrace' or two, but I don't quite know what that would look like | 14:27 |
| jdstrand | niemeyer: a new 'ptrace' interface* | 14:28 |
| jdstrand | niemeyer: I don't really want to pursue that last option now-- it is roadmapped (not product roadmapped, mind you) as something to look into in the future | 14:28 |
| jdstrand | niemeyer: .. | 14:28 |
| zyga | smoser: about parsing proc/mounts, we do parse it and in the right order too | 14:30 |
| zyga | smoser: it's just that according to the kernel the upperdir is different | 14:30 |
| zyga | smoser: this is probably because of pivot_root | 14:30 |
| zyga | smoser: so unless I misunderstood something there is nothing that we can do better | 14:30 |
| zyga | smoser: please do correct me if I'm wrong, I can show you what we do today in the code if that helps | 14:30 |
| zyga | (actually, we don't iterate over the mount entries in reverse order, that's an omission on my part) | 14:31 |
| zyga | smoser: the logic is on https://github.com/snapcore/snapd/blob/master/osutil/overlay_linux.go#L27 | 14:31 |
| rharper | zyga: why does apparmor not see the same mount path as present in the mountinfo ? | 14:34 |
| zyga | rharper I believe this is a kernel question that only jjohansen can answer | 14:34 |
| zyga | rharper: but in general apparmor doesn't "see" pivot root | 14:35 |
| zyga | so if you setup a profile with some paths | 14:35 |
| zyga | then pivot root | 14:35 |
| zyga | the profile is not changed | 14:35 |
| niemeyer | jdstrand: I see.. hmmmm | 14:35 |
| rharper | zyga: interesting | 14:35 |
| zyga | and applies to the paths as they are | 14:35 |
| zyga | rharper: what is more interesting IMO | 14:35 |
| zyga | is that mountinfo doesn't reflect this either | 14:35 |
| zyga | so if you mount something like overlayfs | 14:35 |
| zyga | and pivot_root | 14:35 |
| zyga | the paths lie | 14:35 |
| niemeyer | jdstrand: The attribute feels too much like a hack for very difficult to understand reasons, for a person that isn't developing snapd | 14:36 |
| zyga | they may or may not be representable in the new root | 14:36 |
| zyga | but the kernel just says "those are the paths" | 14:36 |
| zyga | where in other places it actually hints at the fact they are no longer there | 14:36 |
| zyga | just not in this case | 14:36 |
| zyga | rharper: in the ephemeral Maas image case the kernel really says "overlayfs is mounted there and those are the options" | 14:36 |
| zyga | but it doesn't render the mount options with the relation of the current root | 14:37 |
| zyga | it just seems to say what they were at the time mount happened | 14:37 |
| rharper | zyga: it seems fine for pivot_root to close of some mounts if they're not available under the new root; but why can other things "See" the old paths ? | 14:37 |
| zyga | rharper: this is unusual in the sense that other parts of mountinfo are always rendered from the perspective of the observer | 14:37 |
| jdstrand | niemeyer: yes, it is weird. also, one could argue that kubelet can't function without system-observe, so why have it as a separate interface that could be manually disconnected | 14:38 |
| niemeyer | jdstrand: We might come up with something like an AllowPtrace() method in the intenral AppArmor specification, and when the profile is being generated, check if AllowPtrace is on, otherwise deny it | 14:38 |
| zyga | rharper: as I said above :) it's a jj question | 14:38 |
| rharper | zyga: =) | 14:38 |
| niemeyer | jdstrand: This is relatively clean, and prevents exposure of implementation difficulties | 14:38 |
| jdstrand | niemeyer: iiuc, you are saying that in k8s we would set AllowPtrace such that in system-observe it can ask if AllowPtrace is set and make a decision. correct? | 14:39 |
| niemeyer | jdstrand: Almost.. Imagine a method AllowPtrace() which sets a private attribute allowPtrace=true in the apparmor Specification | 14:41 |
| niemeyer | jdstrand: This would generally be false | 14:41 |
| jdstrand | oh | 14:42 |
| niemeyer | jdstrand: When the apparmor is being generated, we always deny it, unless one of the interfaces allowed it | 14:43 |
| niemeyer | s/apparmor/apparmor profile/ | 14:43 |
| jdstrand | right. at the very, very end, before writing it out, we can see if it is set | 14:43 |
| jdstrand | and if it is, do something | 14:43 |
| jdstrand | I like that | 14:43 |
| niemeyer | Right, exactly.. if nothing enables it, we deny it | 14:43 |
| niemeyer | Then the interfaces that want to enable it would call the method instead of putting it directly into the profile | 14:44 |
| jdstrand | it yes, that is very nice indeed | 14:45 |
| jdstrand | I think I can use this idea with a problem I had with a conflicting x modifier between home and docker-support | 14:46 |
| jdstrand | cool | 14:46 |
| jdstrand | niemeyer: ok, my 3rd question was what to do with the existing kubernetes-support. if pedronis tells me nothing is using it, i'll just take it over | 14:47 |
| jdstrand | niemeyer: thank you! :) | 14:47 |
| * jdstrand really like the spec variable idea | 14:47 | |
| jdstrand | likes* | 14:47 |
| pstolowski | mvo: do you want to take a look at https://github.com/snapcore/snapd/pull/5975 or can i land when green? | 14:47 |
| mup | PR #5975: ifacestate/hooks: only create interface hook tasks if hooks exist <Created by stolowski> <https://github.com/snapcore/snapd/pull/5975> | 14:48 |
| pstolowski | (it has 2 reviews) | 14:48 |
| mvo | pstolowski: yes, I had a quick peek already and it looked good but I did not do a line by line proper review | 14:48 |
| mvo | pstolowski: let me look again | 14:48 |
| mvo | pstolowski: and thanks for doing this so quickly! | 14:48 |
| mvo | Chipaca: what step? | 14:49 |
| mvo | Chipaca: aha, right | 14:49 |
| mvo | Chipaca: I think sergio is maintaing this checklist now | 14:49 |
| pstolowski | mvo: yw, happy to move to something else other than hotplug ;) | 14:49 |
| Chipaca | mvo: cachio: updating the brew formula | 14:49 |
| niemeyer | jdstrand: Glad it sounds reasonable.. let's see if it works out | 14:50 |
| mvo | pstolowski: *g* | 14:50 |
| Chipaca | mvo: cachio: I'll do 2.36, and then I'd like to hand it off (should be just chaning a version number and updating a hash) | 14:50 |
| Chipaca | changing* | 14:50 |
| smoser | ok. i'm back. and reading | 14:51 |
| smoser | zyga, rharper pivot_root issues aside, snapd's logic there is broken | 14:52 |
| smoser | it is trusting the first field of /etc/fstab to mean something | 14:52 |
| smoser | it does not mean anything | 14:52 |
| zyga | smoser: ? can you be more precise please | 14:52 |
| smoser | overlayroot uses 'overlayroot' as the 'fs_spec' for the mount it does | 14:52 |
| smoser | casper uses '/cow' | 14:52 |
| smoser | the kernel ignores it | 14:53 |
| smoser | as overlayroot reads the relavent values from 'fs_file' (second field) and upperdir=,workdir= | 14:53 |
| zyga | we do not read /etc/fstab | 14:53 |
| smoser | sorry | 14:53 |
| zyga | we read /proc/self/mountinfo | 14:53 |
| smoser | not fstab. proc/mounts | 14:53 |
| smoser | or mountinfo | 14:53 |
| smoser | same thing really | 14:53 |
| mvo | cachio: 2.36~pre2 is in beta now | 14:54 |
| zyga | so what are we doing wrong? | 14:54 |
| zyga | we look at the filesystem type | 14:54 |
| zyga | smoser: we are not looking at the backing device | 14:55 |
| zyga | which as you say can be anything | 14:55 |
| zyga | we are looking at the filesystem type "overlay" and the mount point "/" | 14:55 |
| smoser | ok. re-reading as i assumed /proc/mounts let me see | 14:56 |
| rharper | smoser: https://github.com/snapcore/snapd/blob/master/osutil/mountinfo_linux.go | 14:56 |
| zyga | we don't use proc mounts because it has fewer things exposed | 14:57 |
| zyga | smoser: and to be clear, our logic did trigger in the image I tested | 14:57 |
| zyga | it just used the wrong path, as instructed to by the kernel | 14:57 |
| zyga | all I did in the fix was to re-map it manually because we know what happens in this particular case | 14:58 |
| zyga | (where "knows" is perhaps too strong but you know what I mean) | 14:58 |
| rharper | https://bugs.launchpad.net/apparmor/+bug/1703674 | 14:58 |
| mup | Bug #1703674: inconsistent required directory rules needed with overlayfs <aa-kernel> <AppArmor:New> <https://launchpad.net/bugs/1703674> | 14:58 |
| rharper | zyga: is that the bug that tracks the "odd" requirement for the path ? | 14:58 |
| smoser | ok. zyga this is casper /proc/1/mountinfo | 14:59 |
| smoser | http://paste.ubuntu.com/p/ftMNjsv8tx/ | 14:59 |
| zyga | rharper: I don't know - perhaps | 14:59 |
| smoser | and this is overlayroot | 14:59 |
| smoser | http://paste.ubuntu.com/p/b4PZXYhCF7/ | 14:59 |
| smoser | I *think* that you are reading the 10th field of line 9 (/cow) in casper to mean something | 14:59 |
| zyga | 28 0 0:24 / / rw,relatime shared:1 - overlay overlayroot rw,lowerdir=/media/root-ro,upperdir=/media/root-rw/overlay,workdir=/media/root-rw/overlay-workdir/_ | 15:00 |
| zyga | I don't think we do | 15:00 |
| zyga | we read the fstype filed | 15:00 |
| zyga | not the backing store | 15:00 |
| zyga | https://github.com/snapcore/snapd/blob/master/osutil/mountinfo_linux.go#L183 | 15:00 |
| Chipaca | I'm going afk for a few hours. Have a great weekend, y'all! | 15:00 |
| zyga | Chipaca: enjoy :) | 15:00 |
| zyga | smoser: the first field after the - | 15:01 |
| Chipaca | zyga: i'll be asking for logs of this conversation later, to catch up -- super interesting | 15:01 |
| * Chipaca off | 15:01 | |
| smoser | zyga: i'm reading https://github.com/snapcore/snapd/blob/master/osutil/overlay_linux.go#L58 | 15:01 |
| smoser | "if mount source is path, strip it from dir" | 15:02 |
| zyga | smoser: as ultimate proof I can show you this https://github.com/snapcore/snapd/pull/5974/files#diff-989405fde2087a02f99e11b0c532699dR88 | 15:02 |
| mup | PR #5974: osutil: workaround overlayfs on ubuntu 18.10 <Created by zyga> <https://github.com/snapcore/snapd/pull/5974> | 15:02 |
| smoser | mount source does not mean anything | 15:02 |
| pedronis | jdstrand: no snap has been granted use of kubernetes-support | 15:02 |
| cjwatson | diddledan: hi, you've typoed "build-on" as "build on" in your audacity snapcraft.yaml | 15:02 |
| zyga | smoser: ah, I missed that aspect | 15:02 |
| zyga | I was focusing on the triggering | 15:03 |
| smoser | i pointed to that line in my comments! | 15:03 |
| smoser | :) | 15:03 |
| zyga | let me think, I think this came from jdstrand | 15:03 |
| diddledan | oh fudge | 15:03 |
| diddledan | thanks cjwatson | 15:03 |
| zyga | sorry, I missed that | 15:03 |
| cjwatson | (noticed because LP emailed me an oops report) | 15:03 |
| diddledan | yey | 15:03 |
| smoser | well, zyga you were derailed by my 'fstab' comments. which i understand were confusing. | 15:03 |
| zyga | hmm | 15:03 |
| zyga | but if dir, ok := entry.SuperOptions["upperdir"]; ok { | 15:03 |
| zyga | dir is the upper path | 15:03 |
| smoser | that part is right. | 15:04 |
| cjwatson | it should go somewhere more useful, and arguably shouldn't oops, but ...) | 15:04 |
| smoser | well, right for this :) | 15:04 |
| pedronis | jdstrand: but maybe you were asking if anything has kubernetes-support plug, that is a trickier question | 15:04 |
| zyga | jdstrand: can you explain that part please: https://github.com/snapcore/snapd/blob/master/osutil/overlay_linux.go#L58 | 15:04 |
| zyga | smoser: I never saw mount source being a path | 15:05 |
| zyga | I'm curious as well now | 15:05 |
| zyga | smoser: thank you for persisting! I totally missed that | 15:05 |
| cjwatson | diddledan: same for gnome-twitch, but I see you noticed that too :) | 15:05 |
| jdstrand | pedronis: that is good enough. it needs an installation constraint | 15:05 |
| diddledan | yup, both fixeded | 15:05 |
| cjwatson | ta | 15:05 |
| diddledan | thanks for the alert :-) | 15:05 |
| jdstrand | pedronis: so it would have to be granted the use for anyone to use a snap with it | 15:05 |
| jdstrand | zyga: what is the question? all of this is very casper specific. we knew it would have to be updated if the livecd pattern changed | 15:06 |
| diddledan | (some people might say I'm a "lert") | 15:06 |
| zyga | jdstrand: the question is why do we interpret mount source in overlayfs detector | 15:06 |
| zyga | jdstrand: since mount source is unused arbitrary string in the cases I'm familiar with at least | 15:07 |
| jdstrand | because of the way casper set it up, it was something we could look at | 15:07 |
| jdstrand | yes | 15:07 |
| zyga | jdstrand: that's interesting, thanks! | 15:07 |
| zyga | smoser: so we should look at casper | 15:07 |
| jdstrand | it's not 'dependable' | 15:07 |
| zyga | but I think the path is (probably) an accident more than a feature, I wonder if overlayfs itself uses it | 15:08 |
| zyga | I will finish writing a test for something I have and return to this | 15:08 |
| jdstrand | https://github.com/snapcore/snapd/commit/0f40f37c416587bd2e3b51db66601262869d2dc7 | 15:08 |
| diddledan | am I correct with my belief that snaps don't expose url handlers currently? i.e. vscode in a snap cannot hook the vscode:// url scheme from the system (I'm triggering it with a non-snap process, so it's not the confinement out of another snap that's a problem): https://github.com/Microsoft/vscode-pull-request-github/issues/573 | 15:09 |
| zyga | diddledan: they do | 15:10 |
| zyga | diddledan: grep for vscode in generated desktop files in /var/lib/snapd/desktop | 15:10 |
| diddledan | what am I looking for? | 15:11 |
| jdstrand | zyga: see the above commit. the detection is completely arbitrary based on what we observed. it was always known that if something changed or we wanted to support something else, we'd need to do things differently | 15:11 |
| zyga | diddledan: desktop files can declare URL handlers | 15:11 |
| zyga | jdstrand: thank you, I see | 15:11 |
| zyga | I will dig into that shortly after the other bug | 15:11 |
| zyga | diddledan: let me grep my system | 15:11 |
| popey | zyga: isnt the list hard wired? | 15:11 |
| jdstrand | so, right now it is looking for '/cow', cause that is what we know casper does | 15:11 |
| zyga | popey: which list? | 15:12 |
| popey | mailto: http etc | 15:12 |
| popey | we discussed this in belgium | 15:12 |
| zyga | that's in the snap handler | 15:12 |
| jdstrand | so the upperdir is /cow/upper, but because of the pivit root htat becomes simply /upper | 15:12 |
| zyga | not in what is published to the system | 15:12 |
| jdstrand | pivot* | 15:12 |
| jdstrand | so yes, now I see smoser's point fully | 15:13 |
| zyga | jdstrand: right I think we were not discussing the pivot aspect, just that overlayfs mount source (which is "overlay" or "overlay root" or "whatever unused") was being used by the logic | 15:13 |
| zyga | right | 15:13 |
| zyga | +1 | 15:13 |
| zyga | popey: vscode is not defining any | 15:14 |
| zyga | popey: compare to telegram desktop for example: | 15:15 |
| zyga | telegram-desktop_telegramdesktop.desktop:MimeType=x-scheme-handler/tg; | 15:15 |
| popey | but even if it did, it woudln't work, surely, because they're hard wired in snapd | 15:15 |
| popey | yes, that one doesnt work | 15:15 |
| popey | because it's not listed in snapd | 15:15 |
| jdstrand | zyga (cc smoser): ok right, it has all come back to me. in a general fashion, we can't tell that something is going to pivot_root and to what from simply the mountinfo entry. so for the livecd, we try to discover if we think it is a livecd mountinfo entry, and then do stuff (assuming it will be consistent) | 15:15 |
| zyga | what is hard wired? | 15:15 |
| popey | the list of handlers | 15:15 |
| zyga | popey: if the handler is registered the opening that URL from the desktop does work | 15:15 |
| popey | we discussed this in belgium, talking about extending the list | 15:15 |
| zyga | snaps cannot open such links though | 15:15 |
| kyrofa | zyga, I'm trying to use snap try in 2.35.2 and I'm getting "cannot snap-exec: cannot read info for "my-snap-name": cannot find installed snap "my-snap-name" at revision x1: missing file /var/lib/snapd/snaps/my-snap-name_x1.snap" whenever I try to run the app. Any ideas? | 15:16 |
| diddledan | the new window action has the wrong Exec line, too: https://www.irccloud.com/pastebin/TA7qUMzu/ | 15:16 |
| zyga | kyrofa: are you using encrypted FS? | 15:16 |
| jdstrand | zyga (cc smoser): if we have this: overlayroot / overlay rw,relatime,lowerdir=/media/root-ro,upperdir=/media/root-rw/overlay,workdir=/media/root-rw/overlay-workdir/_ 0 0 | 15:16 |
| kyrofa | zyga, full, yeah. Used to work | 15:16 |
| zyga | diddledan: indeed (ouch) | 15:16 |
| zyga | mvo: ^ | 15:17 |
| jdstrand | zyga (cc smoser): then, in theory, we could look for 'overlayroot' instead of '/cow', and then try to do something with upperdir | 15:17 |
| jdstrand | zyga (cc smoser): for the overlroot case | 15:17 |
| zyga | yeah | 15:17 |
| kyrofa | zyga, not home dir encryption | 15:17 |
| zyga | we could hold a map of the use cases we saw and apply a transofrmation | 15:17 |
| jdstrand | ie, '/' + basename(upperdir) | 15:17 |
| zyga | kyrofa: where is your snap? | 15:18 |
| zyga | kyrofa: as in in the filesystem | 15:19 |
| zyga | popey: extending the list is something we can discuss and arrange for, I'm not opposing that | 15:19 |
| kyrofa | zyga, /var/lib/snapd/snaps | 15:19 |
| zyga | wait, you had that snap there? | 15:20 |
| kyrofa | zyga, oh wait, sorry, misunderstood | 15:20 |
| zyga | h | 15:20 |
| zyga | so where was it when you tried it? | 15:20 |
| kyrofa | zyga, in my home dir somewhere | 15:20 |
| zyga | did you remove the prime/ directory? | 15:20 |
| zyga | and built it again? | 15:20 |
| zyga | can you please paste /proc/self/mountinfo, grep for the snap to make the output shorter | 15:20 |
| kyrofa | No, just ran `snap try prime` and then tried running the command immediately | 15:20 |
| zyga | (ideally one line) | 15:20 |
| kyrofa | zyga, oh wait, actually, it was in /tmp. I bet that's the issue huh | 15:21 |
| * cachio lunch | 15:21 | |
| zyga | mmm, perhaps but I want to see the mountinfo anyway | 15:21 |
| zyga | but most likely so | 15:21 |
| zyga | because snap-exec cannot handle that | 15:21 |
| zyga | well | 15:21 |
| zyga | actually, ignore me | 15:21 |
| zyga | it should not matter | 15:21 |
| zyga | at that stage it should be mounted in /snap/... | 15:21 |
| zyga | do you see your snap mounted there? | 15:21 |
| pstolowski | cachio: the nested vm PR seems to have failed misteriously ("null"), is it too expensive to build ubuntu image maybe? | 15:22 |
| pstolowski | cachio: btw, added a few comments, this is going to be great, thanks for working on this | 15:22 |
| kyrofa | zyga, https://paste.ubuntu.com/p/sgdjp5v3tg/ | 15:22 |
| kyrofa | zyga, yeah, it's mounted in /snap/my-snap-name/x1 | 15:22 |
| zyga | kyrofa: this looks correct | 15:23 |
| zyga | mmm | 15:23 |
| zyga | hold on, I need to break for dinner now | 15:23 |
| zyga | kyrofa: please hold this bug :) | 15:23 |
| mvo | zyga: in a meeting right now, would love to get a tl;dr summary in a bit | 15:26 |
| zyga | k | 15:26 |
| mup | PR snapd#5977 closed: release: 2.36~pre2 <Created by mvo5> <Merged by zyga> <https://github.com/snapcore/snapd/pull/5977> | 15:26 |
| mup | PR snapd#5978 opened: interfaces: don't allow snaps to write to $HOME/bin <Created by zyga> <https://github.com/snapcore/snapd/pull/5978> | 15:27 |
| zyga | jdstrand: ^ | 15:27 |
| mup | PR snapd#5976 closed: interfaces: improve Attr error further <Created by mvo5> <Merged by stolowski> <https://github.com/snapcore/snapd/pull/5976> | 15:30 |
| mborzecki | pstolowski: hey, can you take a look at https://github.com/snapcore/snapd/pull/5948 ? | 15:33 |
| mup | PR #5948: asserts, image: ensure kernel, gadget, base and required-snaps use valid snap names <Parallel installs ⛓> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5948> | 15:33 |
| pstolowski | mborzecki: sure | 15:33 |
| mborzecki | pstolowski: thanks! | 15:33 |
| mborzecki | pstolowski: responded :) | 15:45 |
| zyga | re | 15:45 |
| pstolowski | mborzecki: ok, i'm still on it | 15:47 |
| smoser | zyga, jdstrand just as an fyi. http://paste.ubuntu.com/p/pzDk8bZMjb/ | 15:49 |
| smoser | if i change overlayroot like ^ and update initramfs then it works | 15:49 |
| smoser | as basically this puts '/media/root-rw' in the source (fs_spec) option fed to 'mount' for my overlay mount | 15:50 |
| smoser | and that makes it happy. | 15:50 |
| zyga | smoser: ack | 15:51 |
| smoser | but that will break 'overlayroot-chroot' command | 15:51 |
| zyga | smoser: I'm not familiar with that command but I think with the workaround I proposed for snapd we don't need to change anything else | 15:52 |
| smoser | which relies on finding 'overlayroot' in /proc/mounts | 15:52 |
| smoser | well, that command is probably not used by anyone other than me. | 15:52 |
| smoser | but it allows you to easily remount rw and jump into the "real root" to make a change | 15:52 |
| mup | PR core18#83 opened: add swapfile support (ported from ubuntu-core-config) <Created by mvo5> <https://github.com/snapcore/core18/pull/83> | 15:53 |
| === chihchun is now known as chihchun_afk | ||
| === pstolowski is now known as pstolowski|afk | ||
| zyga | roadmr: trying now | 16:07 |
| zyga | roadmr: no change | 16:07 |
| zyga | same error as last time | 16:07 |
| roadmr | mm | 16:08 |
| roadmr | zyga: did the snap change? | 16:08 |
| zyga | I rebuilt the snap but I believe it didn't change at all | 16:08 |
| zyga | the error is : | 16:09 |
| zyga | - __all__: The upload does not appear to be a valid package. | 16:09 |
| mup | PR core18#84 opened: hooks: port classic mode generation to UC18 <Created by mvo5> <https://github.com/snapcore/core18/pull/84> | 16:09 |
| roadmr | zyga: let me check the snap I have against the tools | 16:09 |
| zyga | thank you | 16:10 |
| zyga | mvo: man, you are killing it on Friday :) | 16:10 |
| zyga | mvo: how about locale? | 16:10 |
| zyga | mvo: can you please review https://github.com/snapcore/snapd/pull/5974 | 16:10 |
| mup | PR #5974: osutil: workaround overlayfs on ubuntu 18.10 <Created by zyga> <https://github.com/snapcore/snapd/pull/5974> | 16:11 |
| roadmr | zyga: the tools work fine with the snap build I have, so perhaps your rebuild did introduce more evil. Wormhole? | 16:13 |
| zyga | sure | 16:13 |
| mvo | zyga: whats the status for 5974 ? do we need it? | 16:17 |
| mvo | zyga: if so, for 2.35? 2.36? | 16:17 |
| mvo | zyga: I can review after dinner in a bit | 16:18 |
| zyga | mvo: I think we need it | 16:18 |
| zyga | for whatever ships | 16:18 |
| mvo | zyga: for whatever ships would be 2.35.5 | 16:19 |
| mvo | zyga: do we need it like *now* i.e. tonight/before the weekend? | 16:19 |
| zyga | mvo: I'm not sure what to answer | 16:20 |
| zyga | smoser: ^ | 16:20 |
| zyga | smoser: so we need it like now, tonight, before the weekend? | 16:20 |
| smoser | I think that submitting as "zero day SRU" is fine. | 16:23 |
| smoser | all users of overlayroot and our images are broken | 16:23 |
| smoser | which is clearly bad, but | 16:23 |
| smoser | the two users that i'm aware of are | 16:24 |
| smoser | a.) open-iscsi autopackage test http://autopkgtest.ubuntu.com/packages/o/open-iscsi/cosmic/amd64 | 16:25 |
| smoser | b.) curtin vmtest (we just put in a workaround to disable snapd) | 16:25 |
| smoser | c.) MAAS installation. this works fine because they disable apparmour | 16:25 |
| smoser | so I personally dnot think there is reason for fire-drill upload at this point since we have it well understood. | 16:26 |
| smoser | but... if there was an upload going to happen anyway | 16:28 |
| smoser | then i think that that pull request should be taken | 16:28 |
| smoser | even in its simple form it fixes default use-case | 16:28 |
| smoser | zyga: ^ | 16:28 |
| smoser | mvo: ^ | 16:28 |
| zyga | ack thanks | 16:29 |
| zyga | I think we will still release but for another reason | 16:29 |
| zyga | and we can piggy back this patch | 16:29 |
| plars | ogra: I can't recall, is bluetooth expected to work on our rpi images? | 16:29 |
| plars | ogra: nm, I found https://bugs.launchpad.net/snappy/+bug/1674509 - I thought there was something like that but that it had been resolved already | 16:34 |
| mup | Bug #1674509: Unable to find bluetooth device on RPi3 running Ubuntu Core 16 <snapd-interface> <Snappy:Confirmed> <linux-raspi2 (Ubuntu):Invalid by p-pisati> <https://launchpad.net/bugs/1674509> | 16:34 |
| zyga | roadmr: any finings? | 16:38 |
| zyga | findings :) | 16:38 |
| roadmr | zyga: still fighting kibana for the logs | 16:38 |
| zyga | thank you | 16:38 |
| smoser | zyga: for now the solution is just to special case '/media/root-rw/overlay' as you did ? | 16:44 |
| zyga | yes | 16:44 |
| zyga | next week we can explore better options | 16:44 |
| smoser | i think thats appropriate for now. | 16:44 |
| smoser | thank you | 16:44 |
| zyga | without time pressure | 16:44 |
| zyga | thank you! and double so for pointing out the other aspect of the current solution :) | 16:44 |
| roadmr | zyga: does git-lp still exist/work? | 16:50 |
| zyga | roadmr: I don't know, I'm not using it | 16:50 |
| roadmr | ok :) | 16:50 |
| roadmr | zyga: hey any chance you could retry pushing the snap? log spew is making it very hard to find the exact entry I need | 17:00 |
| roadmr | having a shorter time window owuld help :) | 17:00 |
| zyga | doing now | 17:01 |
| zyga | done | 17:01 |
| roadmr | thanks! | 17:02 |
| jjohansen | rharper, zyga: its a limitation in the LSM atm that we are working on fixing by extending the hooks available | 17:05 |
| zyga | jjohansen: thank you for confirming this, so our prediction about pivot_root being the cause are correct? | 17:05 |
| zyga | jjohansen: would you consider it a separate bug that mountinfo doesn't show correct paths after pivot_root? | 17:06 |
| zyga | (in the mount options field) | 17:06 |
| jjohansen | yes, we see the pivot_root and can mediate it but we can't properly update/track it because its a permission hook, not a stateful hook which means the pivot_root might actually not happen, and if we track at that point, and it fails we are completely messed up | 17:07 |
| jjohansen | well, no more messed up than now I suppose, its mess up one way or the other | 17:08 |
| zyga | I mean, regardless of apparmor in this case | 17:08 |
| jjohansen | atm namespaces in general are a problem because we don't have hooks to perform permission checks or track unshare or setns | 17:09 |
| jjohansen | right, it messes up apparmor not the kernel | 17:09 |
| zyga | jjohansen: right but I was specifically asking about just plain /proc/self/mountinfo | 17:14 |
| zyga | part of the problem is that apart from the apparmor aspect we got confused by the path reported by the kernel there | 17:15 |
| zyga | it seems like a bug in overlayfs | 17:15 |
| zyga | or mountinfo rendering of itself | 17:15 |
| mup | PR snapcraft#2346 opened: ruby plugin: add support for base <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2346> | 17:15 |
| jjohansen | overlayfs is another issue | 17:15 |
| jjohansen | you have two mounts | 17:15 |
| smoser | i never expected mountinfo to be updated. | 17:15 |
| jjohansen | will potentialy more | 17:15 |
| jjohansen | s/will/well/ | 17:15 |
| smoser | id have ben surprised if two processes (one with an NS change and one without) saw different values for 'upperdir=' | 17:16 |
| zyga | smoser: mountinfo is updated when you pivot normally | 17:16 |
| zyga | for all the other paths | 17:16 |
| jjohansen | yep | 17:16 |
| zyga | smoser: but the upperdir (being a mount option string) is probably just printed verbatim | 17:16 |
| jjohansen | pivot_root is magic | 17:17 |
| zyga | it just seems that using system paths in mount options is not that common | 17:17 |
| smoser | right. | 17:17 |
| smoser | but essentially you're just feeding '-o' as a string to the filesystem | 17:18 |
| zyga | right | 17:19 |
| smoser | so anything could go in there and is only properly understood by that specific fs | 17:19 |
| zyga | right | 17:19 |
| zyga | but it could be :) | 17:19 |
| roadmr | zyga: hey do you have the exact filename for the snap you tried to upload? pm if you like | 17:20 |
| zyga | yes, same as I gave you but .. | 17:21 |
| roadmr | zyga: I'm not finding anything in logs, makes me suspect it's barfing even before being able to determine the snap id | 17:21 |
| roadmr | because I'm searching with snap id and getting nothing | 17:21 |
| smoser | oh. hm.. zyga so in http://paste.ubuntu.com/p/P2pdtCDn2v/ | 17:21 |
| smoser | whihc is a "post-pivot" /proc/1/mountinfo | 17:21 |
| zyga | roadmr: fedora29.2018.10.12.x86_64.snap | 17:21 |
| smoser | did the kernel remove '/cow' from some string above line 9 ? | 17:22 |
| roadmr | thanks zyga | 17:22 |
| smoser | i couldnt understand how 'upperdir=/cow/upper' was a thing at the point of that mount | 17:22 |
| zyga | 30 1 0:24 / / rw,relatime shared:1 - overlay /cow rw,lowerdir=//installer.squashfs://filesystem.squashfs,upperdir=/cow/upper,workdir=/cow/work | 17:22 |
| zyga | probably it existed in the initrd | 17:22 |
| zyga | mkdir /cow | 17:22 |
| zyga | ... | 17:22 |
| zyga | pivot | 17:22 |
| zyga | I played with pivot and overlayfs almost all of last week | 17:23 |
| smoser | bah. duh | 17:23 |
| smoser | i guess i was just expecting to see *some* '/' mount | 17:23 |
| zyga | trying to write a spread test for the live CD fix | 17:23 |
| zyga | mm | 17:23 |
| smoser | but the initramfs / doesn't get represented anywhere | 17:23 |
| zyga | you have / mount | 17:23 |
| zyga | overlay is there :) | 17:24 |
| zyga | nothing else | 17:24 |
| smoser | yeah. but i was expecting to see something above that line. | 17:24 |
| zyga | the fact that overlay still holds onto a mounted filesystem | 17:24 |
| zyga | that we cannot see | 17:24 |
| zyga | is not visible | 17:24 |
| smoser | right. | 17:24 |
| zyga | if you had a process living in the initrd namespace | 17:24 |
| zyga | you would see that | 17:24 |
| zyga | it's actually easier to reason about | 17:24 |
| zyga | just make a mount namespace | 17:24 |
| smoser | :) | 17:24 |
| zyga | do a overlayfs + pivot there | 17:25 |
| zyga | and see what you see in mountinfo | 17:25 |
| zyga | you can get to one-liner overlatyfs + /proc | 17:25 |
| zyga | *overlayfs | 17:25 |
| zyga | but you can see the backing store in the initial mount namespace | 17:25 |
| zyga | roadmr: perhaps the permissions on the snap directory tree are unusual? | 17:26 |
| smoser | ok. yeah, well that is enough for me today. | 17:26 |
| zyga | or something else is making it fail | 17:26 |
| smoser | i really should be looking at other things. | 17:26 |
| zyga | :-) | 17:26 |
| smoser | thank you | 17:26 |
| zyga | thank you smoser :) | 17:26 |
| ogra | plars, sadly not, i pinged koza on the forum about it today as well ... | 17:28 |
| ogra | the bluez snap delivers everything we need but it needs manual intervention to actually initialize the BT stack | 17:29 |
| plars | ack | 17:29 |
| mup | PR snapd#5979 opened: install: don't start disabled services <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/5979> | 18:45 |
| mup | PR snapcraft#2343 closed: schema, meta: add "full" app adapter <Created by kyrofa> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2343> | 19:36 |
| mup | PR snapcraft#2346 closed: ruby plugin: add support for base <Created by kyrofa> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2346> | 19:39 |
| mup | PR snapcraft#2347 opened: extensions: support adding root properties <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2347> | 19:45 |
| kyrofa | Any update on when 2.36 will be stable? | 19:47 |
| roadmr | zyga: still around? | 20:00 |
| mup | PR snapd#5978 closed: interfaces/home: don't allow snaps to write to $HOME/bin <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/5978> | 20:06 |
| mup | PR snapcraft#2344 closed: schema: remove the deprecated snap keyword for bases <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2344> | 20:24 |
| mup | PR snapcraft#2347 closed: extensions: support adding root properties <Created by kyrofa> <Merged by kyrofa> <https://github.com/snapcore/snapcraft/pull/2347> | 21:30 |
| mup | PR snapcraft#2348 opened: extensions: remove root extensions <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2348> | 21:36 |
| mup | PR snapd#5980 opened: interfaces/apparmor: conditionally add explicit deny rules for ptrace <Created by jdstrand> <https://github.com/snapcore/snapd/pull/5980> | 22:20 |
| mup | PR snapcraft#2349 opened: extensions: use extension docstring <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2349> | 22:51 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!