/srv/irclogs.ubuntu.com/2019/12/09/#snappy.txt

=== Wouter01004 is now known as Wouter0100
mborzeckimorning06:39
zygamborzecki: hey :)07:02
mborzeckizyga: hey hey07:03
zygahow was your weekend?07:03
zygamborzecki: easy landing https://github.com/snapcore/snapd/pull/786707:03
mupPR #7867: tests: fix fwupd version regular expression <Simple πŸ˜ƒ> <⚠ Critical> <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/7867>07:04
zygamborzecki: https://github.com/snapcore/snapd/pull/7864 needs love07:05
mupPR #7864: cmd/snap-mgmt, packaging/postrm: stop and remove socket units when purging <Simple πŸ˜ƒ> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7864>07:05
mupPR snapd#7867 closed: tests: fix fwupd version regular expression <Simple πŸ˜ƒ> <⚠ Critical> <Created by cmatsuoka> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/7867>07:06
mborzeckizyga: do you know the backstory of https://github.com/snapcore/snapd/pull/7865 ?07:07
mupPR #7865: travis-ci: add go import path <Created by sd-hd> <https://github.com/snapcore/snapd/pull/7865>07:07
zygamborzecki: no idea but probably allows one to travis a fork07:08
mborzeckizyga: https://golang.org/cmd/go/#hdr-Remote_import_paths07:09
mborzeckihm, why didn't that fail on < 19.0407:12
mupPR snapd#7854 closed: snap-confine: raise egid before calling setup_private_mount() <Created by jdstrand> <Merged by zyga> <https://github.com/snapcore/snapd/pull/7854>07:25
sdhd-saschahello :-)07:25
zygahey sdhd-sascha07:37
mupPR snapd#7840 closed: tests: use test-snapd-sh snap instead of test-snapd-tools - Part 2 <Created by sergiocazzolato> <Merged by zyga> <https://github.com/snapcore/snapd/pull/7840>07:37
sdhd-saschazyga: I'm not sure if it made sense to split the travis jobs. Maybe they were summarized because of the total build-time. And it is difficult to get a free build slot on the online travis-platform...07:42
zygasdhd-sascha: yeah, it's a weird limitation; I don't know if your PR should be merged or not yet. I think sergio should review it07:43
zygamborzecki: can you please review https://github.com/snapcore/snapd/pull/712907:44
mupPR #7129: userd: allow setting default-url-scheme-handler <Created by jwheare> <https://github.com/snapcore/snapd/pull/7129>07:44
sdhd-saschazyga: but the gopath import would be nice. Because with it, it's possible to build on own github/travis account, before PR07:44
zygait's pretty old and I think nobody looks at it07:44
zygamborzecki: thinking about https://github.com/snapcore/snapd/pull/720507:52
mupPR #7205: rfc: introduce confinement options failsafe flag <Created by zyga> <https://github.com/snapcore/snapd/pull/7205>07:52
zygawhat can we do to make it landable?07:52
mborzeckizyga: hmm so the snap-mgm test fails bc there's lxd socket unit, turns out the socket units are there base image07:54
zygaok, let's filter those out then07:55
pstolowskimorning08:00
zygahey :)08:00
mupPR snapd#7866 closed: travis-ci: split integration tests into parts (>4MB log limit) <Created by sd-hd> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7866>08:18
mupPR snapd#7868 opened: Revert "travis-ci: split integration tests into parts (>4MB log limit)" <Created by mvo5> <https://github.com/snapcore/snapd/pull/7868>08:19
mborzeckipstolowski: mvo: hey08:21
mvohey mborzecki08:25
mupPR snapd#7868 closed: Revert "travis-ci: split integration tests into parts (>4MB log limit)" <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/7868>08:26
mupPR snapd#7862 closed: release: 2.42.5 <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7862>08:27
mborzeckizyga: heh, updated #786408:33
zygammm, looking08:33
mupPR #7864: cmd/snap-mgmt, packaging/postrm: stop and remove socket units when purging <Simple πŸ˜ƒ> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7864>08:33
zygaaha08:33
zygasmart08:33
zyganice08:33
pstolowskimborzecki: going to review your seccomp branch this morning08:35
mborzeckipstolowski: cool, thank you!08:35
zygapstolowski: can you do a pass over https://github.com/snapcore/snapd/pull/786308:35
mupPR #7863: interfaces/builtin: add uio interface <Created by zyga> <https://github.com/snapcore/snapd/pull/7863>08:35
pstolowskizyga: yes, sure08:36
mvosdhd-sascha: hey, re 7866 - I accidently merged it but it was not reviewed yet so I reverted it again. please just reopen (or I can reopen). sorry for this! fwiw, we only hit the 4mb limit under certain circumstances of failing tests. not sure we want to split generally - the downside of a general split is that we only have a fixed amount of travis slots aiui so it may slow things down08:36
mborzeckiha, wireguard has landed in mainline08:36
zygacoffee and 1-2-1 prep08:38
zygattyl08:38
sdhd-saschamvo: no problem.08:39
mvosdhd-sascha: thank you!08:39
sdhd-saschamvo: could you test the PR with the go import path? with them it was possible to build snapd on my own repo and travis08:40
mvosdhd-sascha: could you please merge it on master or rebase it on master - there was a bug in master that caused the integration tests to fail which got fixed this morning. this should make the test green.08:41
sdhd-saschamvo: so anybody can test building before any PR.08:42
mvosdhd-sascha: also could you please elaborate a little bit on this works? it sounds very cool!08:42
sdhd-saschamvo: what should i merge?08:42
mvosdhd-sascha: just fetch the lastest upstream from snapd and merge master into your PR. I'm happy to do this for you if you prefer08:42
sdhd-saschamvo: sure, but i'm also new on travis ;-) i'm just googling for a solution ...08:43
mvosdhd-sascha: I see :)08:43
sdhd-saschamvo: ok08:43
mvosdhd-sascha: so with 7865, what would I have to do to run my forked branch? what steps to follow etc :) ? (does my question make sense?)08:44
mborzeckimvo: as i understand, you'll be able to run the travis job on your private fork08:50
sdhd-saschamvo: with 7865. just add the go_import_path, and add your repo under "MyRepositories" on travis. Then manually build on travis or wait on the next git push08:50
sdhd-saschamborzecki: right08:50
zygare08:53
sdhd-saschaI will now merge master on my pull-request. Can somebody please cancel my travis-jobs ? I wait until it works my private repo first.08:55
zygasdhd-sascha: jobs auto-cancel on push08:55
sdhd-sascha...on my private repo08:55
sdhd-saschazyga: ?08:55
zygano, on your private repo you can cancel them08:55
zygaI was talking about the snapd repo08:55
sdhd-saschazyga: I'm too. I won't waste your build time08:56
sdhd-saschadon't want08:56
Eighth_Doctormborzecki, zyga : do either of you know what's going on here? https://bugzilla.redhat.com/show_bug.cgi?id=178071208:59
Eighth_Doctorit seems like snapd doesn't know how to read the path properly...08:59
mborzeckiEighth_Doctor:  it's sudo dropping env09:00
zygaEighth_Doctor: the keyword is sudo09:00
Eighth_Doctorblech09:00
Eighth_DoctorI was afraid of that09:00
zygatry sudo env09:00
zygaI need to jump to a meeting09:00
mborzeckiEighth_Doctor: https://forum.snapcraft.io/t/centos-7-error-system-does-not-fully-support-snapd-cannot-read-the-value-of-fs-may-detach-mounts-kernel-parameter/14392/509:00
sdhd-saschawould "sudo -E" help ?09:02
sdhd-sascha"""-E, --preserve-env            preserve user environment when running command"""09:02
mborzeckisdhd-sascha: iirc the security policy will still drop your current $PATH09:03
mborzeckisudo -i should work though09:03
sdhd-saschamvo: is there a reopen button on github for 7866, or should i opened a new pr09:32
zygare09:36
mvosdhd-sascha: I think you need to open a new PR09:36
mvosdhd-sascha, mborzecki thanks for explaining how 7865 works09:37
zygaEighth_Doctor: do you think the proposed solution in https://bugzilla.redhat.com/show_bug.cgi?id=1691996 is workable?09:48
zygaEighth_Doctor: btw, I need to thank you for excellent maintenance of snapd in Fedora. I've been using Fedora for a few weeks on my daily driver and it's nothing short of excellent09:48
zygaEighth_Doctor: it really helps me to develop cgroupv2 features09:49
mupPR snapd#7869 opened: travis-ci: split integration tests into parts (>4MB log limit) <Created by sd-hd> <https://github.com/snapcore/snapd/pull/7869>09:50
sdhd-saschamvo: i had to reapply the commit, because of the revert09:51
mvosdhd-sascha: yeah, again, sorry for the trouble :( won't happen again09:52
mupPR snapd#7853 closed: [RFC] boot,bootloader: setup the snapd_recovery_kernel grubenv <UC20> <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/7853>09:54
mborzeckizyga: will you be pushing fixes to 7129?09:56
zygachecking09:56
zygawhat kind of fixes?09:57
mborzeckizyga: just added a bunch of comments, but i can push the fixes too if you're busy with somehting else09:57
zygamborzecki: you forgot to click send09:57
zygathere are no new comments09:57
mborzeckizyga: try refreshing09:57
zygamborzecki: did09:57
zygaagain09:58
mborzeckizyga: https://github.com/snapcore/snapd/pull/7129#pullrequestreview-32879416009:58
mupPR #7129: userd: allow setting default-url-scheme-handler <Created by jwheare> <https://github.com/snapcore/snapd/pull/7129>09:58
zygawow, now I see them09:58
mborzeckihaha ;)09:58
zygaman, eventually consistent my ass09:58
zygayeah, go for it09:58
zygathank you for looking09:58
mborzeckiok09:58
mborzeckitrivials anyway ;)09:58
zygafeels like something close09:58
zygaI'll jump back into https://github.com/snapcore/snapd/pull/762509:58
zygato get it unblocked09:59
mupPR #7625: cmd/snap-confine: stop being setgid root <Created by zyga> <https://github.com/snapcore/snapd/pull/7625>09:59
mborzeckibut really, the desktop stuff is like duct tape and glue all the way09:59
pedronismborzecki: notice that 7129 is blocked on jdstrand review09:59
zygamborzecki: yeah09:59
mborzeckipedronis: right, i can ping jdstrand_ in case he missed it (though i'm sure it's in his queue)10:00
pedronisit's in his queue10:00
mupPR snapd#7768 closed: interfaces: add raw-volume interface for access to partitions <β›” Blocked> <Created by jdstrand> <Closed by pedronis> <https://github.com/snapcore/snapd/pull/7768>10:13
mupPR snapd#7768 opened: interfaces: add raw-volume interface for access to partitions <Created by jdstrand> <https://github.com/snapcore/snapd/pull/7768>10:14
ijohnsonmorning folks10:37
zygahey ijohnson10:38
ijohnsonhey zyga10:38
mvohey ijohnson10:39
ijohnsonmvo: did you see my comment on the SU doc? I didn't make it very far in updating the grub.cfg.normal for the UC20 pc gadget, also I had it in my notes that grub.cfg.recovery was the ubuntu-seed partition but reading through it, I think grub.cfg.recovery is for ubuntu-boot no?10:39
ijohnsonreading the grub.cfg specifically10:40
pedronisijohnson: recovery goes to ubuntu-seed and *not* recovery goes to ubuntu-boot10:42
ijohnsonpedronis: Hmm that's not how it's written now though, the grub.cfg.recovery is currently doing the chainloading to the recovery system grub IIUC10:43
pedronisijohnson: sorry, we are talking different things10:43
ijohnsonsure, it is like 4 am here so not surprised I have something confused :-)10:44
pedronisijohnson: that should happen only for mode != run10:44
pedronisnot saying that is what happens right now, neither are definitive10:45
pedronisor close to that10:45
mvoijohnson: I have not yet, will look at it next, sorry!10:45
pedroniswhich is part of the problem10:45
pedronisijohnson: I mean in run mode we should chain from ubuntu-seed recovery grub into ubuntu-boot grub10:45
ijohnsonmvo: no problem just wanted to make sure you were aware if you needed someone else to work on this before Wednesday10:46
mvoijohnson: yeah, that's fine, I figured :)10:46
pedronisthe recovery systems should really play a role only if mode != run10:46
mvoijohnson: hope the trip was fine?10:46
ijohnsonpedronis: yes I understand that part, I'm just a bit confused which grub file in the repo corresponds to which partition10:46
pedronisijohnson: recovery should correspond to seed, and the other to boot10:47
pedronisif it's the other way around right, things are more messed up than I thought10:47
ijohnsonmvo: actually I found a flight this morning so I'm at the airport waiting for my flight (hence being up at 4 am)... Hopefully the trip is fine it's currently snowing and we're supposed to get like 10 cm+ of snow this morning10:48
mvoijohnson: uh! good luck!10:49
ijohnsonpedronis: it's possible I am wrong about what it's currently doing but the recovery grub seems to be doing the logic to look for a recovery system to boot into10:49
mvoijohnson: also - 4am sounds brual10:49
mvoijohnson: brutal :)10:49
mvoijohnson: was wonder about the early time :)10:49
pedronisijohnson: that's exactly what I would expect for seed10:49
pedronisseed == recovery10:50
ijohnsonpedronis: right that's why I was confused as well10:50
ijohnsonErr wait sorry I misread10:50
pedronisit looks correct, maybe there is no run mode logic there yet10:51
pedronisI don't know10:51
pedronisit's all part of the problem10:51
mvoI think there is no run mode logic in the recovery grub yet10:52
ijohnsonpedronis: why would the seed recovery system grub need to look for all the other recovery systems? Is that by design that you can switch between recovery systems using grub? I guess that kinda makes sense now thinking about it10:52
ijohnsonmvo: yeah it's a _bit_ early :-)10:52
pedronismvo: ah10:54
mvopedronis: it's something to discuss with xnox today10:55
zygabrb, cold, going to make tea11:26
mborzeckipstolowski: thanks for the comments, updated #782111:34
mupPR #7821: interfaces/seccomp: parallelize seccomp backend setup <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7821>11:34
mupPR snapd#7864 closed: cmd/snap-mgmt, packaging/postrm: stop and remove socket units when purging <Simple πŸ˜ƒ> <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/7864>11:34
pstolowskiyw, thanks for the PR!11:36
zygare11:37
mupPR snapd#7870 opened: boot,bootlaoder: setup the snap recovery system bootenv <UC20> <Created by mvo5> <https://github.com/snapcore/snapd/pull/7870>11:39
zygapstolowski: good catch, mborzecki https://github.com/snapcore/snapd/pull/7821#discussion_r35540548811:47
mupPR #7821: interfaces/seccomp: parallelize seccomp backend setup <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7821>11:47
zygasdhd-sascha: https://github.com/snapcore/snapd/pull/7865#pullrequestreview-32887038412:00
mupPR #7865: travis-ci: add go import path <Created by sd-hd> <https://github.com/snapcore/snapd/pull/7865>12:00
mborzeckizyga: already been addressed12:09
zygamborzecki: thanks12:09
pstolowskizyga, pedronis did you want jdstrand_ 's review on https://github.com/snapcore/snapd/pull/7830 or can it land?12:09
mupPR #7830: interfaces: include hooks in plug/slot apparmor label <Bug> <Created by stolowski> <https://github.com/snapcore/snapd/pull/7830>12:09
zyganot really but I think it's short and would be good to keep him in the loop12:10
pstolowskizyga: sure12:11
sdhd-saschazyga: just started a new build on my repo12:19
zygasdhd-sascha: you don't need to use repo builds you know12:19
sdhd-saschazyga: could you see, this build from me ? : https://travis-ci.org/sd-hd/snapd/builds/62198951112:19
zygathere are faster ways to iterate12:19
zygayou can just use spread on a local system12:19
sdhd-saschazyga: kitchen was on my plan12:19
sdhd-saschazyga: kitchen is local travis12:19
sdhd-saschazyga: the build above, from weekend. has this patch and only debian sid is failed. all others are green12:20
mupPR snapd#7850 closed: apparmor: allow 'r' /sys/kernel/mm/transparent_hugepage/hpage_pmd_size <Simple πŸ˜ƒ> <Created by jdstrand> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/7850>12:21
pedroniszyga: any blockers to merge #7228 ?12:23
mupPR #7228: interfaces: add audio-playback/record and pulseaudio spread tests <Created by jdstrand> <https://github.com/snapcore/snapd/pull/7228>12:23
zygapedronis: checking12:23
sdhd-saschazyga: i do more tests, i will report12:25
zygapedronis: I think it's good12:26
mupPR snapd#7871 opened: tests: add spread test for hook permissions <Created by stolowski> <https://github.com/snapcore/snapd/pull/7871>12:27
mupPR snapd#7228 closed: interfaces: add audio-playback/record and pulseaudio spread tests <Created by jdstrand> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/7228>12:28
vidal72[m]zyga: this possibly will be in linux 5.6 https://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git/commit/?h=work.openat2&id=b767b87044c4b15c4f222daf6948a7a43f802f7e12:33
zygayeah, I noticed on twitter yesterday12:33
zygaalong with the new mount APIs it's great to see those12:33
zygaI don't have a clear vision of how to adopt such new APIs in snapd12:34
zygaas we will likely support the old APIs for foreseeable future12:34
pedroniszyga: I pushed something to #7768, let me know whether it is what you expected12:41
mupPR #7768: interfaces: add raw-volume interface for access to partitions <Created by jdstrand> <https://github.com/snapcore/snapd/pull/7768>12:41
zygalooking12:41
zygapedronis: looks great, thanks12:43
pedroniszyga: I am a bit confused by https://github.com/snapcore/snapd/pull/7768/files#r355413333, that's a go regexp, not something we feed apparmor12:45
mupPR #7768: interfaces: add raw-volume interface for access to partitions <Created by jdstrand> <https://github.com/snapcore/snapd/pull/7768>12:45
zygaoh, did I misunderstand that12:45
zygauff12:45
zygasorry12:45
zygaI didn't notice that12:46
zygaI just noticed composition and was worried12:46
zygacommented12:46
pedronisgiven that hotplug is for follow up (and we don't have an immediate use case), I think it can land if it gets green12:47
zygaI agree12:47
zygahotplug is should be more careful, it's not as easy as uio as it is fairly dynamic in nature12:47
zygaand covers a wide variety of cases12:48
zygabut I think it's a natural case to attempt12:48
* zyga small break13:04
* Chipaca lunch13:08
mupPR snapd#7872 opened: doc: HACKING.md change autopkgtest-trusty-amd64.img name <Created by sd-hd> <https://github.com/snapcore/snapd/pull/7872>13:10
sdhd-saschaCan anybody stop my Travis-Build for modifying "HACKING.md" ?13:11
pedronispstolowski: what's the status of this: https://bugs.launchpad.net/snapd/+bug/184956413:19
mupBug #1849564: snap connections doesn't work as expected when interface attributes change <snapd:Confirmed for stolowski> <https://launchpad.net/bugs/1849564>13:20
=== ricab is now known as ricab|lunch
pedronismvo: zyga: I think we have finally fixed this: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1814141 ?13:30
mupBug #1814141: fail to run any snap after snapd refresh, reinstalling snapd from the archive is a temporary fix <snapd (Ubuntu):Confirmed> <https://launchpad.net/bugs/1814141>13:31
pedronismvo: I suppose this should be closed somehow at this point: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/173949413:32
mupBug #1739494: snapd 2.29.4.2 is not installable on powerpc <regression-update> <snapd (Ubuntu):Triaged> <https://launchpad.net/bugs/1739494>13:32
mvopedronis: totally13:34
pstolowskipedronis: it's probably another case of  https://bugs.launchpad.net/bugs/1848516 (as suggested in the report) but i'd need to double check if it's fixed too. note that in the bug report the plug name was changed, that's why it's the same problem (i think). if it was just attributes change, then i think we would hit the (known) problem of stale attributes on refresh that we only fixed for content interface13:41
mupBug #1848516: snap connections/interfaces shows dropped interfaces as connected after refresh <snapd:Fix Committed by stolowski> <https://launchpad.net/bugs/1848516>13:41
mupPR snapd#7873 opened: image: set recovery system label when creating the image <UC20> <Created by mvo5> <https://github.com/snapcore/snapd/pull/7873>13:51
pedronismvo: is it won't fix or fix released?14:12
stgrabermvo: how are things going with getting 2.42.5 rolled out?14:30
mvostgraber: looking good, we are waiting for the cert team right now to give us a +1 for moving to candidate, cachio  has the details on where we stand here14:37
jdstrand_pedronis: please don't land 7768 yet. some of the changes that went in I want to change14:38
=== jdstrand_ is now known as jdstrand
pedronisjdstrand: you'll have to tell me more14:39
pedronisjdstrand: which of the changes?14:39
pedronisjdstrand: the test changes or the path changes14:39
jdstrandpedronis: in ijohnson's comment, there needed to be a change, but not to rip out the mmc and vda114:40
pedronisjdstrand: they aren't used14:40
=== ricab|lunch is now known as ricab
jdstrandpedronis: I know. that is what needed to change, to use them, not to remove them14:40
pedronisjdstrand: the code doesn't care where the slot come from14:40
pedronisso not sure what difference it would make14:40
pedronisjdstrand: I mean, I follow up sounds ok, I don't think anything fondumental is untested at this point14:41
jdstrandI picked those 3 paths specifically so make sure it worked in the normal case, the requested mmc case and the i2o case14:41
jdstrandthe tests aren't done yet so I'd like to add it back14:42
jdstrandplus, I don't understand the s#vda1#vda1/# change14:42
jdstrandI only am just looking at it, but I want to understand why that was changed before committing14:43
pedronisjdstrand: that tests the Clean14:43
pedronisit wasn't tested14:43
pedronisand zyga asked about it14:43
pedroniswe are actually likely missing tests about that for other interfaces that also have a Clean fwiw14:44
pedronisjdstrand: anyway I don't see any fundamental blockers, but of course more testing is alway appreciated14:47
jdstrandpedronis: now I'm rethinking that clean is there at all. the regex is very precise14:48
jdstrandthat clean should be there*14:48
pedronisjdstrand: as I said to zyga all the other interfaces like this one do that14:48
pedronisthe Clean14:48
pedronisso it would be incosistent not to have it at this point14:48
pedronisjdstrand: see serial-port or spi14:49
pedronisetc14:49
pedronisI don't know what it was decided to do it that way14:49
pedroniss/what/why/14:49
jdstrandpedronis: yeah, but maybe that is wrong. why would we allow /dev/././././vda1////////14:50
pedronisjdstrand: we cannot simply change it, you would need to start with warning in review-tools14:50
pedronisfor the old ones14:50
jdstrandpedronis: well, I'm talking about this one14:50
pedronisit's a well defined process, clean and then check14:51
pedronisand consistency is important14:51
jdstrandbut consitently wrong?14:51
pedronisI didn't make the initial decision14:51
jdstrandit is not a bad pattern in general14:52
jdstrandjust conceptually14:52
jdstrandbut seeing /dev/vda1/ - that is not what we want to allow14:52
pedronisjdstrand: but we do allow it14:52
jdstrandthere are things, like i2o, that are a subdir14:52
jdstrand/dev/i2o/14:52
pedronisthat's the code you wrote14:53
pedronisI just refactored it14:53
jdstrandpedronis: yes, and I am rethinking that after seeing the test for /dev/vda1/14:53
pedronisjdstrand: ok, but then you need to give a list of other interfaces you think we ideally should remove the Clean14:54
jdstrandof course, /dev/i2o/ going to /dev/i2o doesn't match the regex, but still14:54
pedronisbecause is everywhere atm14:54
pedroniswhich is probably why you also put it in the first place14:54
jdstrandit is, I copied an existing over as a template14:55
Chipacanice 500s from the forum14:55
pedronisjdstrand: what I'm saying, you either convince me it's exactly a problem only for the new one, or which it is, and  a path forward14:55
jdstrandpedronis: we should've been doing if path != filepath.Clean(path) err14:55
pedronisjust changing it for this one feels arbitrary14:56
jdstrandwell, bugs happen. sometimes you only notice after a while14:56
jdstrandI just noticed this14:56
jdstrandI'm not sure consistency is a reason to add another one14:57
pedronisthat's ok, but we still need a path forward for everything14:57
pedronisalso because otherwise we'll get there again14:57
pedronisjdstrand: 1 thing doing it one way, and 5 another is not a pattern14:58
jdstrandpedronis: I'm not saying that none of them can or can't change, but my point of 5 historical and everything else going forward as correct is a pattern. the 5 historical can be documented in the code14:59
Chipacawhat's the problem with 'path != filepath.Clean(path)'?14:59
pedronisChipaca: that all the previous interfaces don't do that15:00
jdstrandChipaca: the potential issue is that we might break existing snaps15:00
Chipacaah15:00
Chipacak15:00
jdstrand(if we change the existing ones)15:00
pedronisChipaca: we cannot just change it for old interfaces15:00
pedroniswithout going to some review-tools long warning phase15:00
pedronisor something15:00
jdstrandfyi, common files uses if p != np {15:01
jdstrandso does content15:02
jdstrandbool does not15:03
zygare15:03
jdstrandneither does hidraw, i2c, iio, serial-port or spi15:04
pedronisto be clear this has path, so the closest are the ones useing a path attribute15:04
zygajdstrand: did snapd solve p != np?15:04
zyga;D15:04
zygaI'll grab lunch and be back15:05
zygado I need to read the backlog?15:05
jdstrandso, content, personal-files and system-files use !=, and bool, hidraw, i2c, iip, serial-port and spi just feed the cleaned path through the regex15:05
pedronisjdstrand: as I said from attribute used (path), this belong to the latter set15:07
jdstrandthe first three don't have an attribute named 'path'15:08
pedronisjdstrand: which 3 ?15:10
jdstrandpedronis: now that I've seen it. I'm uncomfortable adding this to raw-volume. I disagree we should for consistency. I see your point about content, personal-files and system-files (ie, the first three) not having a "path" attribute, but they are effectively the same an imo should be considered15:11
jdstrandpedronis: I don't see how to move existing over in a very acceptable manner. it is your prerogative to say no to me15:12
jdstrandpedronis: what I could do is fix raw-volume and then add to all the others a code comment explaining that the pattern is historical15:13
jdstrandpedronis: that is probably the best I can do barring a long drawn out migration15:13
pedronisjdstrand: ok, we probably want to think how to change those but is not trivial15:14
pedronisthey have been like that since a long time15:14
jdstrandyeah15:15
jdstrandpedronis: I'm not sure where we stand. I think you said fix raw-volume to use != (and while you didn't say it, I'll do a followup pr to add a code comment to the others and a test verifying the current behavior). if that isn't what you meant, please let me know what you want15:18
pedronisjdstrand: what I would prefer is also that we make the path function a helper in utils, so there's a chance that we can use it again for similar new cases15:19
pedronisjdstrand: also probably all the comments in the old code needs to point to raw-volume and or this helper15:20
jdstrandpedronis: something like where the api is pathHelper(path, regex) (name tbd)?15:20
pedroniswell it can take the slot etc15:21
pedronisbut whatever works15:21
pedronisjdstrand: look at the helper as it exist now15:21
jdstrandpedronis: gotcha. ok, I'll do that in a followup PR15:21
* zyga -> afk15:30
ijohnsonpedronis: jdstrand: could you do the store check thing to see a full listing of all the paths that folks are using for these slots? If nobody's using a non-clean path then you could make the change more safely15:31
jdstrandijohnson: that is true15:36
jdstrandijohnson: btw, did you solve the rofs issue?15:37
ijohnsonWell haven't tested solving it yet, but the first thing I did was patch containerd to always put containers under a specified label if an env var is set15:37
jdstrandpedronis: btw, when 2.43 goes to candidate, we need to do the final round of pulseaudio grandfathering. we could do a round now(ish) to get a jump start on it15:38
ijohnsonjdstrand: But I also got strace of the mounts that containerd does and the mount ns of the containerd and I can't see why the rootfs would be ro15:38
jdstrandpedronis: what you gave me last time is sufficient for my needs15:38
pedronisjdstrand: ok15:39
jdstrandijohnson: weird. it occurred to me that perhaps temporarily an overlayfs could be used to workaround it. that wouldn't be a long term fix of course15:40
sborovkovHey guys, I am trying to build snap with QtWebEngine for rpi eglfs. So it builds fine. Qt apps work. But when trying to run apps that actually use webengine I am getting a crash in webengine process (failed to execvp...). When running with LD_DEBUG I get this - /snap/screenly-client/x40/bin/screenly-client: error: symbol lookup error: undefined symbol: nspr_use_zone_allocator (fatal)15:40
sborovkovany idea what that can be realted with? I am building latest Qt, using system libs as dependencies15:40
jdstrandpedronis: just whenever you have a chance. if you don't have a chance until candidate, that is ok too15:40
pedronisjdstrand: yea, we'll see, I'm also not sure when we'll branch 2.4315:41
ijohnsonjdstrand: If you have a little bit of time maybe you could look at https://pastebin.canonical.com/p/yh8M7G6qpk/ and https://www.irccloud.com/pastebin/IhtxTEjG/15:43
ijohnsonI can't see anything obvious there about why the rootfs in the container would be read-only15:44
ijohnsonNote that in the strace log, there are multiple containers being created and we really only care about the last one15:44
* cachio lunch15:46
jdstrandijohnson: sorry, someone asked me something. line 369 of https://pastebin.canonical.com/p/yh8M7G6qpk/15:59
* zyga back from lunch, hacking more16:01
jdstrandijohnson: that is doing what I thought it might be. it isn't necessarily an issue, but a pivot_root did happen before16:01
ijohnsonjdstrand: no worries, so that mount call is making "/" which at this point is an overlayfs mount, a slave mount, but that shouldn't make it read-only right?16:02
ijohnsonjdstrand: yeah I don't think pivot_root is as big of a deal as I thought it was16:02
ijohnsonBecause apparently our dockerd patches don't even prevent docker from doing pivot_root anymore and it still works because it runs inside a different apparmor label16:03
mborzeckisdhd-sascha: left a comment https://github.com/snapcore/snapd/pull/7845#issuecomment-563306248 imo there's an opportunity to fix in in systemd upstream too16:03
mupPR #7845: cmd: add prefix to SYSTEMD_SYSTEM_GENERATOR_DIR <Simple πŸ˜ƒ> <Created by sd-hd> <https://github.com/snapcore/snapd/pull/7845>16:03
ijohnsons/it runs/the container runs/16:03
jdstrandijohnson: no, but doesn't line 285 look weird? what is going on with lowerdir?16:04
ijohnsonjdstrand: what's weird about that overlayfs lowerdir? It just has multiple lower dirs which last I checked is perfectly valid and well supported16:05
zygaijohnson, jdstrand: what are you chasing?16:06
ijohnsonzyga: the dream16:06
zygadon't we all? :)16:06
ijohnson(of kubernetes as a snap working nicely)16:06
ijohnson:-)16:06
ijohnsonMaybe zyga you could take a look at the links I posted a bit above in the backlog, essentially the issue is that a container is started and it has "/" as read-only which is weird because it should have its own "/" built from overlayfs mounts16:07
zygalooking16:08
sdhd-saschamborzecki: is there a bug in systemd too ?16:09
ijohnsonzyga: note to only look at the last set of mounts, etc. Because the first few are unrelated containers16:09
mborzeckisdhd-sascha: hm not a bug really, they don't have that path in sytemd.pc16:09
cmatsuokacachio: is this something you're aware of? https://api.travis-ci.org/v3/job/622591349/log.txt16:09
sdhd-saschamborzecki: ah, ok. i could do that ?16:10
jdstrandijohnson: ok, I haven't seen that before but see in https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt that it is supported.16:11
zygaijohnson: stat -f /var/snap/kubernetes-worker/common/containerd/var/lib/io.containerd.snapshotter.v1.overlayfs/snapshots/91/fs16:11
zygajdstrand: yeah, that's the thing I mentioned in Vancouver :/16:12
zygajust many lowerdirs16:12
ijohnsonzyga: issue is that this is an ephemeral container that is launched immediately and dies before we can really do anything16:12
zygaok16:12
zygaijohnson: so what's the symptom, that / is read-only?16:13
ijohnsonzyga: yes / is read-only is the symptom16:13
zygaijohnson: how is this determined?16:13
zygaijohnson: can I reproduce this?16:13
cachiocmatsuoka, thanks for the report, I'll ake a look16:14
ijohnsonwell not easily reproducible locally due to many levels of $CONTAINER_TECHNOLOGIES and $CLOUD_TECHNOLOGIES16:14
cachiocmatsuoka, this test has been updated recently16:14
zygaijohnson: I don't see why it would be read only otherwise16:15
ijohnsonzyga: the container in it's entry point tries to create `/host/dir`16:15
zygaaha16:15
zygaand you get EROFS?16:15
ijohnsonYes16:15
zygado you get a write error or a specific errno?16:15
ijohnsonUhh I don't have the specific error in front of me16:15
zygaijohnson: I can help but I would love to get my hands on this16:15
ijohnsonjoedborg: are you around? Do you have the specific error when you get that read only issue?16:15
zygaijohnson: otherwise it looks like it's not true, something is read only for real16:16
zygathe mountinfo dump, where is that from16:16
zygafrom the container?16:16
joedborgijohnson: i'm here16:16
zygacan we get a shell at the moment that mountinfo was collected?16:16
joedborgijohnson: which did you want?16:16
ijohnsonjoedborg: can you pastebin the error you see when launching the container with the AWS script?16:17
joedborgijohnson: sure16:17
ijohnsonzyga: kinda it's tough because the container pops up for very short amount of time then exits16:17
jdstrandwhat is going on with line 173: [pid  3012] mount("/", "/", 0xc0001c60a0, MS_RDONLY|MS_REMOUNT|MS_BIND|MS_REC, NULL) = 016:17
zygaijohnson: how do you get that shell trace then?16:18
ijohnsonzyga: so to get that mountinfo we had an `nsenter -all -t $(until pgrep install-aws.sh)`16:18
zygaijohnson: looks like it makes the bind mount read only without remounting the filesystem16:18
zygaer16:18
zygajdstrand: ^16:18
ijohnson(sorry can only type so fast on a plane from my phone :-) )16:18
zygaand recursively so, though I'm not quite sure that is really respected in the kernel16:18
zygaijohnson: oh16:18
zygaijohnson: man, you're good :D16:18
jdstrandzyga: I more meant, hey, ijohnson, does this look like it is doing the right thing?16:19
zygaI see16:19
* ijohnson thinks 16:19
zygajdstrand: but note that the 1st line of mountinfo shows it is writable, both the filesystem and the mount point - you can see "rw" there twice16:19
ijohnsonjdstrand: I saw that too I think it's a red herring16:19
ijohnsonThere are multiple containers run and the first one is just a bug checking container for the kernel I think16:20
jdstrandijohnson: so, I bet you could get some of the info zyga wants by using a container that works (ie, one that doesn't try to create /host/..., run a shell, and then try to touch /host/foo or something to see if you get the same error16:20
ijohnsonI think containerd creates a dummy container on startup to see if it has to work around kernel bugs or not16:20
zygajdstrand: that's a good idea16:20
ijohnsonwell yes that would be good but is complicated due to $CLOUD_TECHNOLOGIES and $CONTAINER_TECHNOLOGIES16:21
ijohnsonEssentially we don't know the full config of the container being created so we could reproduce16:21
joedborghttps://www.irccloud.com/pastebin/dna5cAMu/16:22
joedborgijohnson: zyga: ^16:22
ijohnsonIf we did yes it would be easy to reproduce but for example a simple container created with the cli tool ctr for containerd doesn't use anywhere near the same config as this cloud provider does16:22
jdstrandijohnson: well, I know that joedborg was able to get some containers to run with what is available today. I'm just saying, maybe shell into one of those pods and see if the /host issue exists there too. if it does, perhaps it is the same issue was are seeing with eks16:23
zygajoedborg: that's EROFS allright16:23
zygajoedborg: how hard would it be to run some custom shell script instead?16:23
jdstrandijohnson: that would at least rule in if it is eks only or if it is a generic issue16:23
joedborgzyga: it’s pretty easy to open a shell, just not within that container as that comes straight from Amazon.  Can it be a new container and be useful?16:24
zygayeah16:24
zygaanything that is similar enough to reproduce the problem16:24
zygaI'm mainly interested in:16:24
jdstrandjoedborg: that is the question. it is probably useful if we can reproduce the same rofs error in that container16:25
zygajoedborg: /proc/self/mountinfo from the point of view of the failure16:25
jdstrands/same/same type of/16:25
zygajoedborg: stat -f /16:25
zygajoedborg: along with any logs from the host that may contain relevant stuff16:25
zygajdstrand: it would be quite evil to craft a seccomp profile that returns EROFS instead of EPERM ;)16:26
ijohnsonI'm about to land so I'll be AFK for a bit but yeah if we can get a similarly configured container shell that would be helpful16:26
zygaijohnson: safe travel16:26
jdstrandhah, it would16:26
zygaI'll be around for a while16:26
zygaif I'm gone just leave me a message please16:26
joedborgnow working on getting a shell16:31
zygathank you16:32
joedborgthe issue is, because it's the networking pod that's failing, interacting with the cluster is a pain16:36
zygajoedborg: are you racing with the container failure?16:37
zygajoedborg: is this .... (winks) pod racing? :D16:37
joedborgzyga: hahahaha, there are as many crashes16:38
ijohnsonHahaha yes this is exactly pod racing, this demo is just like Anakin's racer :-)16:57
ogratwo thin strings holding it all together ?16:58
mupPR snapcraft#2825 closed: remote-build: remove need to specify user <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2825>16:58
joedborgzyga: ijohnson: jdstrand: just wondering if any of you could take a look over this https://pastebin.canonical.com/p/yxh2bnq6ms/, can anyone see if the URI specified at the end is inside a pivot root?  I don't think it is, but I think the Permission Denied is related to what we're seeing17:18
zygalooking17:18
zygajoedborg: is this from snap run --strace?17:19
zygajoedborg: if so _all_ of is it after snap-induced pivot17:19
joedborgzyga: thanks!  yep, i'm running ctr within the snap17:20
joedborgzyga: when i say within, i mean via17:20
zygahow do you mean?17:20
* ijohnson looks 17:20
zygapivot affects everything in the mount namespace17:20
zygaall paths are affected17:20
zygaI see some permission failures on epoll17:21
joedborgzyga: from what i understand, this socket is meant to be opened to transport the tty, but it's failing to so I can't get a shell17:22
ijohnsonzyga, epoll EPERM is expected I think IIRC17:22
zygajoedborg: on the host do you see any denials?17:22
zygajoedborg: to answer your question, yes this is "inside" a pivot root17:22
ijohnsonjoedborg: do you have all interfaces connected17:23
zygathe permission denial needs to be overcome first17:23
zygathere are ways to do that17:23
zygaconnect existing interfaces17:23
zygaor switch the entire profile to complain mode17:23
joedborgijohnson: yeah i do this time :)17:23
joedborgzyga: there aren't any denails17:23
joedborg*denials17:23
zygaare you running this as root? perhaps those are DAC failures17:23
joedborgzyga: i am yeah17:24
jdstrand'abstract unix socket'17:27
zygadog17:27
zygadoh17:27
jdstrandthat won't be affected by pivot_root, assuming the error is accurate17:27
zygaI didn't notice that17:27
zygayeah17:27
joedborgjdstrand: yeah it is /not/ documented well17:27
zygajdstrand: sharp eyes!17:27
joedborgso i have no idea where it's meant to be written17:27
mupPR snapd#7874 opened: seed: support ModeSnaps(mode) for mode != "run" <UC20> <Created by pedronis> <https://github.com/snapcore/snapd/pull/7874>17:27
joedborgjdstrand: zyga: ijohnson: I've got to run to an appointment, i'll have patchy comms, but will try to answer anything17:28
jdstrandjoedborg: the policy doesn't allow that path17:29
jdstrandwe have:17:29
jdstrandunix (bind,listen) type=stream addr="@/containerd-shim/moby/*/shim.sock\x00",17:29
jdstrandunix (bind,listen) type=stream addr="@/containerd-shim/k8s.io/*/shim.sock\x00",17:29
jdstrandwe need unix (bind,listen) type=stream addr="@/containerd-shim/default/*/shim.sock\x00",17:29
zygajdstrand: it's interesting that joedborg said there are no denials on the host17:29
joedborgjdstrand: i think it used to be the case, but now has changed17:29
jdstrandjoedborg: did you disable kernel rate limiting?17:29
joedborgzyga: that might be because i have a custom snapd17:29
joedborgjdstrand: yeah17:30
jdstrandjoedborg: I suggest while trying to get this up and running to stay on a particular tag/commit/something for containerd17:30
joedborgjdstrand: ah i meant between when that policy was written for docker and now, i have always stayed on 1 commiut17:31
jdstrandjoedborg: can you add to /var/lib/snapd/apparmor/profiles/snap.kubernetes-worker.containerd the above rule?17:31
jdstrandjoedborg: and reload the policy with apparmor_parser -r /var/lib/... and then restart containerd and see if you get farther?17:31
joedborgjdstrand: ah that works for getting a containerd shell17:32
joedborgthanks17:32
jdstrandcool17:32
jdstrandjoedborg: i've made a note of that17:33
joedborgjdstrand: thanks!17:34
* zyga EODs17:36
joedborgokay, so this shell allows me to mkdir /host and write to it, so it's either an issue specifically via k8s or the aws container image17:36
joedborgthanks for your help zyga17:37
zygagood luck17:37
ijohnsonjoedborg: so with that rule you still have issues with the eks container image?17:38
jdstrandjoedborg: that is useful. you said "it's either an issue specifically via k8s or the aws container image". is the shell you just used not via k8s (ie, the MVP from last cycle)?17:39
joedborgjdstrand: no so i can't get a shell via k8s atm because this aws container that won't start controlls the netwroking in and out of the node17:40
joedborgjdstrand: with out own control plane, it works fine17:40
jdstrandjoedborg: is this a regression from the mvp, this never worked or this wasn't tested before?17:41
joedborgjdstrand: it's an aws specific thing17:41
jdstrandjoedborg: ok, so we've seen it work with OTHER_CONTROL_PLANE_TECHNOLOGY17:42
jdstrandijohnson: ;)17:42
zygahaha17:42
joedborgjdstrand: we have, but this issue will surface with any workload that doesn't conform with standard unix root directories17:42
joedborgjdstrand: i agree they shouldn't do this, but people like to use /myapp when in containers17:43
ijohnsonjdstrand: :-) if only we weren't going to try and demo this with $SPECIFIC_CONTROL_PLANE vendor17:43
zygawe can whitelist any number of /myapp variants ;)17:43
jdstrandjoedborg: sure, I'm just trying to understand if this is a regression or not17:43
zygamaybe they get bored eventualyl17:43
* zyga really EODs17:44
zygagood luck17:44
joedborgguys, really sorry, i have to run for this appointment (normally i wouldn't, but it's not rescheduable)17:44
jdstrandijohnson: you are interested in understanding if the added unix rule allows for a shell with aws?17:44
ijohnsonjoedborg: no problem17:44
jdstrandjoedborg: np. I'll give you a deb with the unix rule17:45
ijohnsonjdstrand: sorry don't quite follow the question, yes I am interested in getting AWS eks to work with the snap17:45
jdstrandijohnson: you asked "joedborg: so with that rule you still have issues with the eks container image?" - I was just making sure that question wasn't lost17:46
joedborgjdstrand: thanks, maybe where it says β€œdefault”, β€œk8s.io” should probably be generic17:46
ijohnsonOh right yes I did17:46
joedborgBecause that’s just the namespace sep by the container17:46
jdstrandjoedborg: I already did that17:47
jdstrandjoedborg: thinkikng the same thing17:47
joedborgAwesome thanks17:47
jdstrandjoedborg: ok, uploaded a new deb17:52
cmatsuokacachio: got this one too:18:04
cmatsuokarsync: write failed on "/mnt/user-data/gopath/src/github.com/snapcore/snapd/share/locale/mnw/LC_MESSAGES/snappy.mo": No space left on device (28)18:04
cachiocmatsuoka, about the previous one I could reproduce the issue, the problem is in another test which is leaving the environment corrupted18:05
cmatsuokacachio: ah interesting18:06
cmatsuokacachio: do you know how to fix it?18:06
cachioso you know in which system is happening hte "no space error"?18:06
cmatsuokacachio: ubuntu-core-18-6418:07
cmatsuokaon snap-set-core-config18:08
cachiocmatsuoka, not yet because for some reason the snap test-snapd-service-watchdog is leaving garbage after it is deleted18:08
cachiocmatsuoka, I left some links in the stanup doc18:08
cachiocmatsuoka, https://paste.ubuntu.com/p/FgV9nr9rDg/18:08
cachioso then when we run the test snap-set-core-config it fails18:09
cachiobut is it because systemd is degraded18:09
cachiocmatsuoka, and if I run the snap-set-core-config in a loop it works perfect18:09
cachioso I presume there is some test which is executed before and it is causing this18:10
cachiocmatsuoka, I am now trying to see which is that test18:10
mupPR snapd#7875 opened: interfaces: refactor path() from raw-volume into utils with comments for old <Created by jdstrand> <https://github.com/snapcore/snapd/pull/7875>18:16
mupPR snapd#7876 opened: seed: fix seed location of local but asserted snaps <:birthday:> <Created by pedronis> <https://github.com/snapcore/snapd/pull/7876>18:39
* cachio afk19:06
mupPR snapd#7848 closed: cmd/snap-bootstrap: mount ubuntu-data tmpfs, in one go with kernel & base <UC20> <Created by xnox> <Merged by cmatsuoka> <https://github.com/snapcore/snapd/pull/7848>20:15
cmatsuokacachio: that test is passing now, did you do anything?20:19
=== pedronis_ is now known as pedronis
cachiocmatsuoka, no, it dependes on the tests order20:24
cachiothe problem is on the test ubuntu-core-gadget-config-defaults20:25
cachioit is leaving a wrong systemd state20:25
cachioI am testing a fix for this20:25
cmatsuokacachio: ah nice, thank you20:25
cachiocmatsuoka, thanks for showing me the issue20:26

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!