/srv/irclogs.ubuntu.com/2020/08/20/#snappy.txt

dstathisn 20.04 many services (in particular kubelet) are packaged as snaps00:09
dstathisDoes anyone know how to correctly interact with services running as a snap? There doesn't seem to be a systemd unit00:09
dstathisfor example 'systemctl status kubelet' (or docker or whatever) fails with a not found error00:10
jameshdstathis: systemd units owned by snaps will have names like "snap.$snapname.$app.service".  You should see them in the "systemctl list-units" output.01:29
ishI've installed a few Snaps on Fedora 32, and those with tray icons have garbled menus...  Anything obvious I'm missing?03:05
oerheksFor font issue, libpango can be a reason03:09
oerheksor libfreetype03:09
mupPR snapd#8301 closed: interfaces/many: deny arbitrary desktop files and misc from /usr/share <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8301>05:15
mupPR snapd#9183 closed: tests: use "set -ex" in prep-snapd-in-lxd.sh <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9183>05:15
mborzeckimorning05:54
mvogood morning mborzecki05:57
mborzeckimvo: hey05:57
zygagood morning06:38
zygaI will be around shortly06:38
zygajust need to send some paperwork for the insurance06:38
zygamvo if you can, please merge https://github.com/snapcore/snapd/pull/917506:41
mupPR #9175: tests: find -ignore_readdir_race when scanning cgroups <Simple 😃> <Test Robustness> <Created by zyga> <https://github.com/snapcore/snapd/pull/9175>06:41
mborzeckizyga: hey06:43
zygahi06:45
mvozyga: looking06:48
zygajust random test failure fix06:49
mvozyga: thanks, what did you change in the force push, unfortunately GH does not show me a diff for that :/06:51
zygamvo I moved the -ignore_readdir_race after the path06:51
zygaoriginally it was the first argument but old find didn't like that06:51
zygacompare https://github.com/snapcore/snapd/commit/cd5b94776066bbe76359e912c960c9d4258abc9c and https://github.com/snapcore/snapd/commit/8c10ddfc32cf8f909ba73bfcfe691f174917c2e406:52
mvozyga: ta06:53
mupPR snapd#9175 closed: tests: find -ignore_readdir_race when scanning cgroups <Simple 😃> <Test Robustness> <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9175>06:55
zygathank you06:56
zygauh, it's a meeting day06:56
zygajust meetings and meetings06:56
pstolowskimorning!07:03
mborzeckipstolowski: hey07:05
pstolowskihey mborzecki, welcome back!07:06
mvogood morning pstolowski07:14
mvofwiw, I'm working on the lxd test failures currently07:43
zygamvo thank you07:43
mvoyw - it's very painful as each iteration takes forever07:44
mvobut I hope I have a handle on it now (but I thought this the two previous runs too :(07:44
zygamvo 300+ MB download is not free07:44
pstolowskimvo: thank you, i'm trying something as well on top of your existing PR but i'm not clear what root problem is and yes it is super slow07:44
mvopstolowski: http://paste.ubuntu.com/p/p6pFKnRxrb/ is my best attempt so far, test is running07:45
* mvo also wonders why wait_for_service seems to be not using the retr-tool07:46
jameshmvo: I noticed the desktop code review meeting isn't on my calendar for this week.  Did you cancel it, or is that a glitch?07:59
mvojamesh: I canceled it because we lack people but if you want to do it I can be available for you08:00
mvojamesh: you also should have goten a mail about this by the calendar :/08:00
jameshmvo: I don't see any email about the cancellation, which is why I asked.  That's fine.08:01
mborzeckimvo: https://github.com/snapcore/snapd/pull/9150 can land too08:02
mupPR #9150: gadget,kernel: add new kernel.{Info,Asset} struct and helpers (1/N) <Created by mvo5> <https://github.com/snapcore/snapd/pull/9150>08:02
mborzeckimup: and https://github.com/snapcore/snapd/pull/915608:02
mupmborzecki: In-com-pre-hen-si-ble-ness.08:02
mborzeckimvo: and https://github.com/snapcore/snapd/pull/915608:03
mupPR #9156: boot: copy boot assets cache to new root <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9156>08:03
jameshmvo: I think https://github.com/snapcore/snapd/pull/9168 is good to merge, but is failing on ubuntu-20.04-64 for some tests/main/lxd:snapd_cgroup_* tests08:06
mupPR #9168: o/hookstate/ctlcmd: make is-connected check whether the plug or slot exists <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/9168>08:06
pstolowskijamesh: yes these tests have been investigated since yesterday08:07
jameshpstolowski: is it best just to wait til they get fixed then?08:07
mvojamesh: if the failure is just on lxd I can force-merge08:08
mvojamesh: merged08:09
jameshmvo: cheers!08:09
mvomborzecki: landed 9150 now too08:09
mvomborzecki: and 915608:10
mborzeckimvo: thanks!08:10
mupPR snapd#9150 closed: gadget,kernel: add new kernel.{Info,Asset} struct and helpers (1/N) <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9150>08:10
mupPR snapd#9156 closed: boot: copy boot assets cache to new root <UC20> <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9156>08:10
mupPR snapd#9168 closed: o/hookstate/ctlcmd: make is-connected check whether the plug or slot exists <Created by jhenstridge> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9168>08:10
mvofwiw, 9184 passed locally now for me on 20.04, let's see if it survies a full spread run08:11
* mvo takes a short break while waiting for this08:11
zyga-x240nice work08:18
pstolowski\o/09:02
mvoso looks like 20.04 is now working but 16.04 is still failing :( *oh well* but with a very different error09:23
zyga-x240mvo: what does 16.04 say?09:28
mvozyga-x240: "Failed to execute operation: Connection timed out"09:28
mvozyga-x240: in a meeting right now, I can paste the full error late but it's also in the CI i think09:29
zyga-x240hmmm09:30
zyga-x240ijohnson: https://github.com/snapcore/snapd/pull/9187#discussion_r47382078009:41
mupPR #9187: tests/lib/nested.sh: use more robust code for finding what loop dev we mounted <Run nested> <Test Robustness> <âš  Critical> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9187>09:41
zyga-x240hmmm09:52
mborzeckizyga-x240: hmm trouble getting self-hosted runners? https://github.com/snapcore/snapd/runs/100699230709:55
zyga-x240hmm checking09:57
zyga-x240the host is up09:57
zyga-x240workers are up09:57
zyga-x240I think I know what's going on09:58
zyga-x240it seems that cla checks are hitting a time-out to grab a worker09:59
zyga-x240I don't recall seeing that before, we have queueing for a reason after all09:59
zyga-x240I restarted that and it passed instantly09:59
zyga-x240so... no idea/09:59
mborzeckihahah09:59
mborzeckiwell, maybe it's a one off occurrence10:00
zyga-x240I hope so but I fear we will learn in time10:05
zyga-x240time for coffee, I'm falling asleep here10:05
zyga-x240maybe pressure is low or something10:06
zyga-x240mvo: I looked and it looks like systemd is not responding, maybe the socket is gone somehow?10:06
zyga-x240but no idea why10:06
mvozyga-x240: so the nested lxd shows me a gazillion "permission denied", e.g. /bin/mount for / exited 1, mount: permission denied etc10:14
mvoon 16.0410:14
zyga-x240mvo: seems like lxd apparmor10:14
zyga-x240mount is really disallowed10:15
zyga-x240only bind may be allowed10:15
mvoyeah, strange that it's inside the nested though10:15
zyga-x240maybe nesting is broken somehow10:15
zyga-x240I wonder if we can stop doing something and get nested working without snapd10:15
zyga-x240and then see what part breaks it10:15
mborzeckizyga-x240: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1889318 is it because when run by lxc there's no apparmor namespace setup like lxd does?10:16
mupBug #1889318: install chromium in lxc container for 20.04 fails <amd64> <apport-bug> <focal> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1889318>10:16
mvozyga-x240: it fails even before snpad it seems10:16
mvozyga-x240: hm, well, maybe or maybe not10:16
zyga-x240mborzecki: looking10:16
zyga-x240I ... don't know10:17
mvohm, "interessting" - it fails in systemctl daemon-reload but everyting else seems to work10:17
zyga-x240well10:17
zyga-x240when systemd is not responding10:17
zyga-x240it's not really a place where things work10:17
mvothe funny thing is - systemctl restart snapd works10:18
mvosystemctl --list works10:18
mvoafaict all the things I tried work10:18
zyga-x240hmm10:18
mvojust not the daemon-reload10:18
mvoit's a bit strange10:18
zyga-x240what does daemon-reload say?10:18
zyga-x240I mean, there is journal10:19
mvoand I think this is broken since forever10:19
mvowe never saw it because that script did not have set -e10:19
zyga-x240ohhhh10:19
zyga-x240that's interesting10:19
zyga-x240so it would fail on stable releases as well10:19
zyga-x240mvo: do you have 5 minutes for a quick call?10:19
jdstrandmvo: hi! thanks for committing PR 8301! have you decided to marge master into release/2.46? if you aren't, I need to prepare a PR for 8301 (that would include 9167) for 2.46 (which is fine, just need to know that I should do it :)10:24
mupPR #8301: interfaces/many: deny arbitrary desktop files and misc from /usr/share <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8301>10:24
jdstrandmvo: all that is left for new PRs that small k8s-support one that I need to investigate. then I'll be doing 'needs security review' reviews10:26
mvojdstrand: thank you10:29
jdstrandmvo: (also, PR 8920 needs final reviews)10:34
mupPR #8920: interfaces: update cups-control and add cups for providing snaps <Needs Samuele review> <Created by jdstrand> <https://github.com/snapcore/snapd/pull/8920>10:34
jdstrandmvo: do note that I intentially did not milestore PR 9186 for 2.46. that needs discussion. if there happened to be a 2.46 point release after that is merged, we could consider it, but it floating into 2.47 would be fine too10:36
mupPR #9186: interfaces: add vcio interface <Created by jdstrand> <https://github.com/snapcore/snapd/pull/9186>10:36
mvook10:38
jdstrandmvo: I'm looking at https://github.com/snapcore/snapd/pulls?q=is%3Aopen+is%3Apr+label%3A%22Needs+security+review%22. nothing is milestoned for 2.46. is there anything in there that should be that you would like me to prioritize?10:42
jdstrandif not, I have a good idea of the priority10:43
zygaI keep getting vendor.json changes10:49
zygaI purged cache10:49
zygapurged the tree10:49
zygait keeps changing10:49
jdstrandzyga: me too! I'd love to see that resolved. (my dev container is on bionic still. wonder if it is a focal vs bionic thing)10:52
pstolowskimvo: why is #9171 blocked?10:53
mupPR #9171: [RFC] config: "virtual" configuration for timezone <Needs Samuele review> <Run nested> <â›” Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9171>10:53
mborzeckizyga: jdstrand: perhapos a different version of govendor was used when vendor.json was last updated in the tree10:54
zygahmm10:54
mborzeckithe difference is a checksum only10:54
mborzecki(at least here)10:54
zygaperhaps but what's the version and who has it is interesting10:54
mborzeckii have 1.0.910:55
zygaI have 1.0.9 in /usr/bin and 1.0.8 in GOPATH10:56
mborzeckibtw. i guess we all should be using the latest master govendor, since we're go getting it in run-checks10:56
zygaand get-deps uses that10:56
zygamborzecki: totally offtopic: https://github.com/snapcore/snapd/pull/918911:01
mupPR #9189: snap/snapenv: set SNAP_REAL_HOME <Created by zyga> <https://github.com/snapcore/snapd/pull/9189>11:01
mupPR snapd#9189 opened: snap/snapenv: set SNAP_REAL_HOME <Created by zyga> <https://github.com/snapcore/snapd/pull/9189>11:01
zygamborzecki: running 1.0.9 modifies my vendor.json11:02
mborzeckizyga: there's a couple of recent commits to vendor.json, mine was on 3.08 (and i'm using 1.0.9), the later commits were by claudio, samuele and jamesh11:04
zygahmmmm11:04
zygathe change comes from 7a9cb154a0c (Claudio Matsuoka     2020-08-13 09:08:15 -0300 115)                        "revision": "68200eea7bdcb97e27fe8e5ff443776383908637",11:05
zygaso maybe claudio has older version11:05
mborzeckilet's ping him when he's online11:06
zygayeah11:06
mvopstolowski: I think 9171 really needs samuele approval, I think it's okay otherwise, if the design gets approval I would like to tweak it a bit more with some helpers11:10
pstolowskimvo: i made a few comments, not sure what was already discussed and agreed, so maybe my comments make no sense11:22
mvopstolowski: cool, happy for any feedback at this point11:22
mvopstolowski: hm, great point about snap get -d11:24
mvopstolowski: I think you are right, we should probably not bypass this for that11:24
* mvo scratches head11:24
pstolowskimvo: yes i think we are breaking some high level assumptions here. and i fear it's going to be annoying to handle :/11:25
mvopstolowski: yeah, this needs more discussion for sure11:25
mvopstolowski: your comment about the transations is also interessting, maybe we need to hook into commit() here instead11:26
mvolxd tests passed locally in 918411:27
* mvo vanishes for lunch11:27
zygaenjoy11:27
pstolowskimvo: what if we do store in state but synchronize config state with system setup? or is it too terrible?11:27
* pstolowski lunch as well11:29
zygapstolowski: who wins?11:29
zygapstolowski: if system was modified when snapd was down?11:29
pstolowskizyga: system always wins. we update system on snap set.11:30
pstolowskibut maybe it's naive11:30
pstolowskijust throwing ideas11:30
zygapstolowski: so when would we use the value from state?11:31
pstolowskizyga: we would always update state from system before reading. that could simplify transaction logic without special-casing. but just brainstorming at this point11:33
pstolowskianyway, time for lunch11:33
zygapstolowski: I see11:33
zygaI don't know either, just trying to understand your point better11:34
cachiozyga, hi11:41
cachiozyga, do you have any idea about what could be causing that when I test beta image and do "systemctl --user daemon-reload" as root I get "Failed to connect to bus: No such file or directory11:42
cachio"11:42
cachioIf I do that as ubuntu user I dont see any error, but as root I see that error11:42
cachioand it makes fail the snapd-failover test11:42
zygacachio: hi11:43
zygacachio: yes, I explained that a few days ago11:43
zygacachio: when we reload systemd-logind.service on older versions of systemd11:43
zygacachio: we cause it to forget about the session of the root user11:43
zygacachio: then subsequent test that prepares a session for the root user11:43
zygacachio: enables linger, which starts services and so on11:44
zygacachio: but then the restore disables linger11:44
zygacachio: because systemd logind is no longer tracking the incoming ssh session of the root user11:44
zygacachio: it shuts down systemd --user for root11:44
cachiozyga, do you know which is the difference between the test we run in github and the one I run in beta validation to explain why it works in github and fails with the beta image?11:46
zygacachio: I don't know what version beta was forked form11:46
zygacachio: please check if it contains changes to prepare-restore.sh11:47
zygatalking exactly about this issue in the commit message11:47
cachiozyga, I see the change11:51
cachioI need to see how to make it for external backend11:51
cachiothanks for the explanation11:51
zygacachio: the fix I made is generic11:54
zygacachio: if you have it and the bug persists then there's something more broken11:54
cachiozyga, yes11:54
cachiobut in case of external backend we exit before that code11:54
zygacachio: output from loginctl list-sessions would help11:54
zygacachio: when do we exit then?12:00
cachiozyga, most of the prepare_project is not done for external backend12:00
cachioI am trying to move the linger configuration to see if it works12:01
zygacachio: I see12:02
zygacachio: yeah, you need to apply the fixes to systemd-logind12:02
zygacachio: those are one-off12:02
zygacachio: do you perform package upgrades? what's the target?12:02
zygais that core16?12:03
cachioyes12:03
zygacore16 needs a special workaround12:04
zygain essence, I repackage core12:04
zygathough recently ijohnson applied a fix to core so maybe that is no longer required12:04
zygafollowing that /var/lib/systemd/linger is bind-mounted to writable using a mount unit12:05
zygalook at the patches I apply to replicate that12:05
zygaI wasn't aware the external target skips all of that12:05
cachiozyga, no problem, I'll try to extend that to external12:05
zygayou may only need the 1) mount unit 2) change to logind configuration followed up by REBOOT12:05
ijohnsonzyga: the core fix has not landed yet, PR is still open12:37
ijohnsonalso morning folks12:37
ijohnsonhey mborzecki welcome back12:37
cachiopstolowski, hey, any idea about thie error https://paste.ubuntu.com/p/SRTGRMHx45/12:45
cachioit is happening in Core20-early-config test12:45
cachioI see this gadget.yaml parse error: Duplicate key: system12:46
cachionot sure if it could be the raeson12:46
pstolowskilooking12:48
zygaijohnson: I see12:49
* zyga was having pierogis for lunch12:49
ijohnsoncachio: that was what I fixed in my PR12:49
pstolowskicachio: yeah, definately12:49
zygacachio: there's a duplicate key :)12:49
ijohnsoncachio: that is fixed by #918712:49
mupPR #9187: tests/lib/nested.sh: use more robust code for finding what loop dev we mounted <Run nested> <Test Robustness> <âš  Critical> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9187>12:49
pstolowskiright, i was just going to say that :), thanks ijohnson12:50
cachioijohnson, ah, thanks!!!12:50
mvoijohnson: should I force merge 9187? there are still failures in nested12:58
cachiomvo, the errors in nested are fixed in other pr12:59
ijohnsonmvo: let me look quickly12:59
cachioin mine12:59
ijohnsonmvo: the tests are still failing12:59
ijohnsonalso SU time now12:59
cachiopr: #909812:59
mupPR #9098: tests: new organization for nested tests <Run nested> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9098>12:59
ijohnsonmvo: sorry I meant the tests still seem to be running?13:00
cachiobut mine fails sometimes because the erorr which is fixed on 918713:00
cachioijohnson, I retriggered the tests13:00
mupPR snapd#9081 closed:  secboot,cmd/snap-bootstrap: cross-check partitions before unlocking, mounting <Run nested> <UC20> <Created by anonymouse64> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9081>13:16
zyga-mbpoh, and I'm making progress on unit testing bash, https://listed.zygoon.pl/ has the details13:37
cachiozyga-mbp, so I see we do https://github.com/snapcore/snapd/blob/master/tests/lib/prepare.sh#L66213:40
zyga-mbpright13:40
cachiowe need that for external as well?13:40
zyga-mbpyou want that and the configuration change immediately below that13:40
zyga-mbpyes13:40
cachiook, that plus the change https://github.com/snapcore/snapd/blob/master/tests/lib/prepare-restore.sh#L44613:41
zyga-mbpcachio note that this is the static part, the dynamic part is where we decide systemd-logind needs reloading and REBOOT13:41
zyga-mbpyes13:42
cachioperfect13:42
zyga-mbpwe try to enable linger for the test user, if that fails we know we need to restart13:42
zyga-mbpnote that this does essentially the same change (configuration file edit) so make sure to just reboot in that case as the config file is already in place, just inactive13:42
cachiook13:43
zyga-mbpI hope this works :)13:43
zyga-mbpcachio I left a small comment on v13:43
zyga-mbphttps://github.com/snapcore/snapd/pull/8986/files13:43
mupPR #8986: tests: new snaps-state command - part1 <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/8986>13:43
cachiozyga-mbp, thanks!!13:44
zygacachio: any luck?14:26
cachiozyga, no yes, trying in 5 minutes14:26
zygaok14:26
cachiostill making some changes14:26
mupPR snapd#9188 closed: interfaces: misc policy updates xlvi <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9188>14:42
ijohnsonmvo: 9184 is super super green :-)14:46
mupPR snapd#9190 opened: [RFC] cmd/s-b/initramfs-mounts: make recover -> run mode transition automatic <Run nested> <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9190>14:47
cachiozyga-mbp, Could not enable linger: No such file or directory14:48
cachioI  see that14:48
zyga-mbpcachio, ok do you have a shell14:49
cachiowhen I do loginctl enable-linger test14:49
cachioyes14:49
zyga-mbpis the logind.conf file modified?14:49
cachioyes14:49
cachioStateDirectory=systemd/linger14:49
zyga-mbpis /var/lib/systemd/linger a mount point to the corresponding directory in writable?14:49
cachiowith this config14:49
cachioI created that in writable14:49
zyga-mbpthat's not all, is a mount in place?14:50
cachioI cant write to /var/lib/systemd/linger14:50
zyga-mbpis there a mount unit that makes /var/lib/systemd/linger a bind mount to /writable/system-data/....14:50
cachioit is protected14:50
zyga-mbpyou need the linger directory to exist14:51
zyga-mbpand it must be a mount point as I've explained14:51
zyga-mbpno way around that14:51
zyga-mbpTHEN you can reboot to reconfigure logind14:51
zyga-mbp(via REBOOT)14:51
cachioI cant create /var/lib/systemd/linger14:51
zyga-mbpand then it will work14:51
zyga-mbpcachio so merge the core change and rebuild the snap14:51
cachiowhich is the change to merge?14:52
zyga-mbpian proposed a PR for core14:52
cachiozyga-mbp, ok, so no workaround until the chagne in core is merged14:53
zyga-mbpcorrect14:53
zyga-mbpunless you can repackage core14:53
cachiozyga-mbp, agree but on beta validation we don't repack core14:54
cachioso I'll push the change done by ian14:54
cachioijohnson, is this the change? https://github.com/snapcore/core/pull/11614:55
mupPR core#116: extra-files/writable-paths: make all /var/lib/systemd writable <Created by anonymouse64> <https://github.com/snapcore/core/pull/116>14:55
ijohnsonyes14:55
cachiowaiting for a second review?14:56
cachioijohnson,14:56
zyga-mbpijohnson what happens when we remove entries?14:56
ijohnsoncachio: I can actually merge it, not sure if I should wait though14:57
ijohnsonzyga-mbp: what do you mean ?14:57
zyga-mbpijohnson this looks good but may need follow ups for hacks in prepare-restore14:57
zyga-mbpijohnson the removal of /var/lib/systemd/rfkill and random-seed lines14:57
ijohnsonhmm14:57
zyga-mbpI'd +1 without those14:58
zyga-mbpbut now that I see them14:58
zyga-mbpI would not keep them14:58
zyga-mbpI just want to understand what happens in practice14:58
ijohnsonI need to think about this, I don't remember how refreshes work, but I think it's just the initramfs that modifies the etc/fstab according to the writable-paths14:58
zyga-mbpon existing systems that shipped with those lines14:58
zyga-mbpcachio can you test that upgrade14:58
ijohnsonso after a refresh to the revision here it should work fine14:58
zyga-mbpI think it's very important for correctness14:58
ijohnsonthere have been other situations where we drop things there though that have landed without controversy however I think14:59
ijohnsonone moment14:59
zyga-mbpijohnson we are trigger happy sometimes14:59
pstolowskimvo: +1 for #918415:00
mupPR #9184: packaging: umount /snap on purge in containers <âš  Critical> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9184>15:00
ijohnsonzyga-mbp: see https://github.com/snapcore/core18/pull/12715:02
mupPR core18#127: Make /var/cache writable and disable dynamic motd <Created by woodrow-shen> <Merged by vorlonofportland> <https://github.com/snapcore/core18/pull/127>15:02
zyga-mbpijohnson in another call15:02
cachiozyga-mbp, ijohnson sorry, which upgrade?15:05
ijohnsoncachio: nvm I can test the upgrade, it is about the writable-paths changes15:05
cachioijohnson, ah, ok, thanks15:06
zyga-mbpre15:13
zyga-mbpcachio check if we can upgrade core from what it looks like now to what is proposed in that PR 12715:13
mupPR #127: Use cap instead of cap1 where only one capability is being manipulated <Created by zyga> <Merged by elopio> <https://github.com/snapcore/snapd/pull/127>15:13
zyga-mbpcachio to see if anything explodes15:13
zyga-mbpI mean15:13
zyga-mbpwe'll know if we land it15:13
zyga-mbpand run tests15:13
zyga-mbpbut a quick run might be helpful as well15:14
zyga-mbpmvo ^ do you know what happens if we remove existing entries from writable paths?15:14
ijohnsonugh building the core snap is so annoying15:16
ijohnsonI wish we could update the core snap to build with modern snapcraft15:16
cachioijohnson, yes15:16
cachiotomorrow I can test that we new core is in edge15:17
ijohnsoncachio: but the PR hasn't been merged though is the thing15:17
cachioI really don't know how to build core15:17
ijohnsonI just built it locally though15:17
ijohnsoncachio: do you use wormhole? I can send it to you in a few moments15:17
cachioI can test that if you built core?15:17
cachioijohnson, yes15:17
cachioplease send me it15:18
mvozyga-mbp: if we remove writable-path they stop being writable which may break things - what in particular are you asking?15:18
zyga-mbpmvo please look at https://github.com/snapcore/core/pull/116/files15:18
mupPR core#116: extra-files/writable-paths: make all /var/lib/systemd writable <Created by anonymouse64> <https://github.com/snapcore/core/pull/116>15:18
zyga-mbpare those changes safe in your eyes15:18
ijohnsonzyga-mbp: fwiw he already approved it15:19
zyga-mbpI know but not sure if he really looked ;D (sorry mvo)15:19
mvozyga-mbp: oh, this should be fine15:19
zyga-mbpmvo in that case let's land it15:19
zyga-mbpand rebuild core15:19
mvozyga-mbp: removing subpath but making more available is fine15:19
zyga-mbpwe can always revert15:19
zyga-mbpand this will help cachio15:19
mvozyga-mbp: please approve first15:20
ijohnsonwell I have it built now, testing what happens on refresh of a core16 vm15:20
ijohnsoncachio: I don't think you need to test what I built here anymore15:20
cachiook15:21
mupPR snapcraft#3250 closed: cli: implement list-tracks <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3250>15:23
mupPR snapcraft#3252 closed: snapcraft: use system certificates by default for https requests <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3252>15:23
ijohnsonjdstrand: is this a typo or intentional? https://github.com/snapcore/snapd/pull/9188/files#r47406683915:23
mupPR #9188: interfaces: misc policy updates xlvi <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9188>15:23
* zyga-mbp needs to break soon15:24
mupPR snapcraft#3255 closed: remote-build: use requests.get() instead of urlopen() <bug> <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3255>15:28
mvoijohnson: if you test is successful please let me know and I will merge15:32
ijohnsonmvo: yes sorry got distracted it is rebooting now15:32
mvoijohnson: no worries15:32
ijohnsonmvo: mmm the VM didn't come back after the refresh15:38
ijohnsonmvo: need to debug, perhaps there is a problem15:38
mvota15:39
* cachio lunch15:43
mvoijohnson: uh, I accidently merged 116 :/ let's hope it's not a  problem with that change or I have to revert15:44
ijohnsonmvo: my hope is that this is a multipass problem, I am recreating a uc16 VM with plain qemu :-/15:45
mupPR core#116 closed: extra-files/writable-paths: make all /var/lib/systemd writable <Created by anonymouse64> <Merged by mvo5> <https://github.com/snapcore/core/pull/116>15:46
* zyga runs away from meetings and works for a while in quiet15:46
zygamvo: shall we merge https://github.com/snapcore/snapd/pull/918415:47
mupPR #9184: packaging: umount /snap on purge in containers <âš  Critical> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9184>15:47
zygajust to make some progress and see how it behaves over time15:48
ijohnsonI'd say so15:48
zygayeah15:49
zygame too15:49
dstathisHas anyone here used kubelet installed via snap? If so do you know how to get it to use the systemd cgroup driver?15:51
ijohnsonjoedborg: can you answer dstathis question ?15:52
ijohnsonmmm mvo, better revert that core PR, I can reproduce the problem in qemu, I need to debug this, but it somehow broke ssh15:56
ijohnsonso sorry :-/15:56
joedborgHey dstathis.  We’re currently working on this under strict confinement for both microk8s and a fully working kubernetes node (I.e. kubelet, proxy, containerd etc).  This isn’t complete yet though as there are lots of working parts that need working through15:56
dstathisjoeborg: The snap of kubelet is not complete yet? I'm asking because kubelet is no longer available via apt. Only snap15:58
dstathis(in Ubuntu 20.04)15:59
mvoijohnson: my bad, I should not have merged prematurely16:01
ijohnsonno, it's my bad I didn't test an actual refresh of it16:02
zygamvo: https://github.com/snapcore/snapd/pull/9184#pullrequestreview-471764868 can I merge?16:02
mupPR #9184: packaging: umount /snap on purge in containers <âš  Critical> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9184>16:02
mvozyga: yes, I'm still trying to create a reproudce for the lxd team about the systemctl daemon-reload16:03
joedborgdstathis: that is usually used by our kubernetes distribution Charmed Kubernetes.  It is a classic snap, so shouldn’t need any interfaces connected.  However it’s not really documented outside of CK context16:03
mvozyga: but merging will make things work again16:03
zygamvo: ok16:03
zygamerging16:03
zygait's ok to revert if it explodes in new ways16:03
dstathisjoedborg: is there another recommended way to install kubelet on 20.04? Or do you know how to configure the cgroup diver using the snap?16:04
mupPR snapd#9184 closed: packaging: umount /snap on purge in containers <âš  Critical> <Created by mvo5> <Merged by zyga> <https://github.com/snapcore/snapd/pull/9184>16:07
joedborgdstathis: I don’t know for sure, but I’m reasonably confident you do `snap set kubelet [kubelet arg]=[value]16:09
joedborgYou need to swap hyphens for underscores i.e `api-server` becomes `api_server`16:10
ijohnsonhow does a change to /var/lib/systemd/private cause `sshd[1151]: Missing privilege separation directory: /var/run/sshd` :-(16:10
dstathisjoedborg: is there a way to ask the snap which configuration options it will recognize? I know there is a command line argument that can be passed in this case16:12
joedborgdstathis: sadly not as snap config is key value without any firm validation16:12
mvoijohnson: pushed the revert now16:29
* mvo gets dinner16:29
ijohnsonthanks, I am very puzzled how this is broken16:29
mvoyeah16:31
zygawhat is broken?16:31
zygaI heard qemu16:31
ijohnsonzyga: my writable-paths PR somehow breaks ssh16:32
zygaijohnson: aha16:32
zygamaybe it only breaks ssh in testing16:32
ijohnsonso you were right :-O16:32
zygawhat are the symptoms?16:32
zygaijohnson: ha16:32
dstathisjoedborg: 'sudo snap set kubelet cgroup-driver=systemd' worked. Thanks16:32
zygaijohnson: heh, I didn't mean to16:32
ijohnsonzyga: it makes the ssh server service fail to start like this:16:33
ijohnsonhttps://www.irccloud.com/pastebin/NbSCym9q/16:33
zygainteresting16:35
zygabut how is that related to what you did?16:35
ijohnsonI have no idea16:35
zygamaybe we neutered the whole directory16:35
zygathere used to be more state in that systemd tree16:35
zyganow it's all an empty directory16:35
zygamaybe that's why?16:35
ijohnsonzyga: no the other files are still there16:36
zygaoh16:36
zygaI'll look at ssh source16:36
ijohnsonzyga: the files are /var/lib/systemd/... and those are still there in the core snap after I made this change16:36
ijohnsonunless systemd is doing something weird with /var/lib/private16:36
zygahah16:36
zygado we have /var/empty?16:37
zygathis is what this is complaining about16:37
zygaah, no16:37
zygasorry16:37
zygaconfflags += --with-privsep-path=/run/sshd16:37
ijohnsonzyga: /var is not empty16:38
zygaif /run/sshd doesn't exist I'd love to log in interactively16:38
zygaand see the status16:38
zygaof all the services16:38
zygaI bet something else failed16:38
ijohnsonmmm actually there is a tmpfiles failure earlier16:38
zygaijohnson: can we retry with just added linger16:38
zygaand then circle back with more oomph16:38
ijohnsonhttps://www.irccloud.com/pastebin/4GceHjg4/16:39
zygaunsafe symlinks? never heard that one before16:39
zygachecking16:39
ijohnsonyeah very odd16:40
zygaI cannot find that in current systemd16:42
zygatrying older16:42
zygaijohnson: I think this explains it16:44
zygahttps://github.com/systemd/systemd/issues/14226#issuecomment-56046710716:44
ijohnsonmmmm interesting let me see16:45
zygaijohnson: I'm too tired to build core and boot it in qemu16:45
zygaI wish there was a fire-and-forget for that16:45
ijohnsonzyga: no worries thank you for looking!16:46
ijohnsonI am actively debugging this16:46
ijohnsonha actually I'm a fool I didn't even build the right branch, so this core snap that I built is just master16:48
zygaijohnson: it would be amazing if we could test core against snapd test suite16:48
zygajust build it there and clone snapd in an action16:48
zygaand set some env in a spread run16:48
zygaso it gets that core16:48
zygaI think it's doable now16:49
ijohnsonit would be amazing if I didn't need all this chroot and vm nonsense with a legacy snapcraft to build the core snap16:49
ijohnsonand I could just do `snapcraft --use-lxd`16:49
zygaijohnson: we could use one action to build it and store an artefact16:49
zygaijohnson: and then another action to test it16:49
ijohnsonzyga: sorry what do you mean? for the snapd repo ?16:50
zygarun all of snapd tests against a core built from a branch of the core repo16:51
zygaright?16:51
zygaso we get way better coverage16:51
ijohnsonmmm that would be nice16:52
zygathe checkout action can checkout any repo16:53
zygaso we have 90% of what we want in the snapd repo already16:53
zygawe just need to teach snapd test suite to use a core that's wgettable from somewhere16:54
zygait can still rebuild / repack snapd16:54
zygabut the base would be from the PR changing the core snap16:54
zygawe could even write a test that runs only in this mode16:54
zygawhere a vanilla core can refresh to the new core16:55
zygaand boot16:55
zygaanyway16:55
zygayou get the point16:55
ijohnsonalso :-( I think the whole reason this failed is because I forgot to do this:16:55
ijohnson  - sudo sed -i '/^deb/s/$/ universe/' $CHROOT/etc/apt/sources.list16:55
ijohnsonso the snap I built built from the wrong packages16:55
ijohnson*sighes16:56
=== ijohnson is now known as ijohnson|lunch
zygaoh?16:58
zygaoh well16:58
* zyga suspends and takes a break16:58
ijohnson|lunchhttps://github.com/snapcore/core/blob/master/.travis.yml is how I build all my core snaps16:58
ijohnson|lunchbut just doing that locally16:58
ijohnson|lunchand I missed a step16:58
cachiozyga-mbp, this test is failing as well17:29
cachiohttps://paste.ubuntu.com/p/GXdqXc4YWg/17:29
cachiocould be related to the revious issue ?17:30
zyga-mbpit's the same failure17:30
zyga-mbpyeah17:30
zyga-mbpit's expected17:30
cachioor you think it is something different17:30
cachiozyga-mbp, nice, thanks, I thought that but just wanted to make sure17:30
cachiothanks17:30
zyga-mbpcool17:31
=== King_InuYasha is now known as Conan_Kudo
=== Conan_Kudo is now known as King_InuYasha
=== King_InuYasha is now known as Conan_Kudo
=== Conan_Kudo is now known as King_InuYasha
=== ijohnson|lunch is now known as ijohnson
* cachio afk17:57
mupPR snapd#9189 closed: snap/snapenv: set SNAP_REAL_HOME <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/9189>18:02
joedborgdstathis: ah thanks for letting me know18:40
cachiocmatsuoka, https://github.com/snapcore/snapd/pull/9098/checks?check_run_id=1008878037#step:5:741219:28
mupPR #9098: tests: new organization for nested tests <Run nested> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9098>19:28
cmatsuokacachio: let's see...19:28
cachionow without any error during reboot19:29
cachioduring boot19:29
cachiono reboots involved19:29
cmatsuokaah nice!19:29
cmatsuokaexcellent19:29
cmatsuokaah wait19:29
cmatsuokabut it failed?19:29
cmatsuokaugh19:30
cachiocouldnt login after that19:30
cmatsuokaand this is qemu without kvm?19:30
cachioyes19:30
cmatsuokalet me see this log in detail19:31
cachioijohnson, I merged master and still see19:32
cachio2020-08-20T18:18:54.2486726Z + mount /dev/mapper/ /tmp/tmp.wFbPmXutiu19:32
cachio2020-08-20T18:18:54.2486835Z mount: /tmp/tmp.wFbPmXutiu: /dev/mapper is not a block device.19:32
cachiosorry, thought that #9187 was merged19:33
mupPR #9187: tests/lib/nested.sh: use more robust code for finding what loop dev we mounted <Run nested> <Test Robustness> <âš  Critical> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9187>19:33
ijohnsoncachio yeah it's not merged, needs mvo to merge it though maybe now If I merge master to that PR it will be green19:34
cachioijohnson, I see lxd tests failing there https://github.com/snapcore/snapd/pull/9187/checks?check_run_id=1007716427#step:5:700319:34
mupPR #9187: tests/lib/nested.sh: use more robust code for finding what loop dev we mounted <Run nested> <Test Robustness> <âš  Critical> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9187>19:34
ijohnsoncachio yeah let me merge master to this pr so we have some chance of being all green19:40
cachioijohnson, nice, thanks19:42
ijohnsoncachio: running now, let's see how it does19:43
cachiocool19:43
cmatsuokacachio: I think I found the problem19:43
cachiocmatsuoka, awesome19:44
cachiowhich one?19:44
cachiowhich is?19:44
cmatsuokaI think there's a call missing there, I'm checking the change history to see if this is was lost at some refactoring, or it was never there19:47
cachiocmatsuoka, nice19:47
cmatsuokacachio: I'm checking a possible fix with chris20:11
cachiocmatsuoka, nice, thanks20:16
cachioplease ping me if you need to test it20:16
cmatsuokacachio: thanks, I think we'll need to test it a lot since it seems that it happens infrequently20:17
cachiocmatsuoka, yes20:20
mupPR snapd#9191 opened: interfaces/{docker,kubernetes}-support: load overlay and support systemd cgroup driver <Created by jdstrand> <https://github.com/snapcore/snapd/pull/9191>20:23
mupPR snapd#9192 opened: interfaces/{docker,kubernetes}-support: load overlay and support systemd cgroup driver - 2.46 <Created by jdstrand> <https://github.com/snapcore/snapd/pull/9192>20:28
mupPR snapcraft#3251 closed: build providers: honour http proxy settings for snapd <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3251>21:03
mupPR snapd#9193 opened: interfaces/many: deny arbitrary desktop files and misc from /usr/share - 2.46 <Created by jdstrand> <https://github.com/snapcore/snapd/pull/9193>21:08
mupPR snapd#9194 opened: Interface cups slot 2.46 <Created by jdstrand> <https://github.com/snapcore/snapd/pull/9194>21:13
mupPR snapd#9195 opened: interfaces: misc policy updates xlvi - 2.46 <Created by jdstrand> <https://github.com/snapcore/snapd/pull/9195>21:23
mupPR snapd#9159 closed: cmd/snap-update-ns: detach all bind-mounted file <Bug> <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/9159>21:28
mupPR snapd#9196 opened: osutil: add OpenExistingLockForReading <Created by zyga> <https://github.com/snapcore/snapd/pull/9196>21:33
mupPR snapd#9187 closed: tests/lib/nested.sh: use more robust code for finding what loop dev we mounted <Run nested> <Test Robustness> <âš  Critical> <Created by anonymouse64> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9187>21:53

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!