| === diddledan0 is now known as diddledan | ||
| mup | PR core20#55 closed: hooks: make systemd-modules-load depend on mounts at /usr/lib/{firmware,modules} <Created by anonymouse64> <Closed by anonymouse64> <https://github.com/snapcore/core20/pull/55> | 00:32 |
|---|---|---|
| mup | PR snapd#8623 opened: tests/lib/prepare.sh: delete patching of the initrd <UC20> <⚠ Critical> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8623> | 00:57 |
| mborzecki | morning | 05:49 |
| mup | PR snapd#8622 closed: cmd/snap-bootstrap/initramfs-mounts: add sudoers to dirs to copy as well <UC20> <Created by anonymouse64> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/8622> | 05:57 |
| mborzecki | mvo: hey | 06:30 |
| mborzecki | mvo: we can land https://github.com/snapcore/snapd/pull/8623 | 06:31 |
| mup | PR #8623: tests/lib/prepare.sh: delete patching of the initrd <UC20> <⚠ Critical> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8623> | 06:31 |
| mvo | mborzecki: cool | 06:31 |
| mvo | mborzecki: done | 06:32 |
| mborzecki | mvo: thanks | 06:32 |
| mup | PR snapd#8623 closed: tests/lib/prepare.sh: delete patching of the initrd <UC20> <⚠ Critical> <Created by anonymouse64> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8623> | 06:33 |
| mborzecki | seems like that go 1.9 failure in wrappers is random after all | 06:33 |
| zyga | good morning | 06:51 |
| zyga | eventful evening? | 06:51 |
| mvo | zyga: good morning - we did all the bits missing for beta I think | 06:52 |
| mvo | zyga: snapd snap just building, that was the last piece, then we hopefully have a working beta | 06:52 |
| zyga | that's impressive | 06:52 |
| zyga | mvo: I'll do what claudio mentioned | 06:53 |
| zyga | boot it and leave it running | 06:53 |
| zyga | an aging test | 06:53 |
| mvo | zyga: nice | 06:53 |
| mborzecki | zyga: hey | 06:53 |
| zyga | :-) | 06:54 |
| zyga | guys, if you have daughters | 06:54 |
| mborzecki | hm? | 06:54 |
| zyga | then spend 30zł / < 10 euro | 06:54 |
| zyga | and play it with them | 06:55 |
| zyga | https://www.gog.com/game/foxtail | 06:55 |
| zyga | this game is incredibly beautiful and fun | 06:55 |
| zyga | localized, with click-to-advance in dialogue, so kids can read at their own pace | 06:55 |
| mborzecki | zyga: hmm looks like an old school adventure game? | 06:55 |
| zyga | it is | 06:55 |
| mborzecki | zyga: do you remember teenagent? :P | 06:56 |
| zyga | and the mood and looks are just so nicely crafted | 06:56 |
| zyga | mborzecki: yes but this is way better :) | 06:56 |
| zyga | (I bought teenagent as a teen) | 06:56 |
| mborzecki | no way xD | 06:56 |
| zyga | yeah :D | 06:56 |
| mborzecki | too bad gog doesn't support their client on linux | 06:57 |
| zyga | there's no DRM and it supports linux as well | 06:57 |
| zyga | mborzecki: yeah but the game is just a zip to download and run | 06:57 |
| mborzecki | zyga: well, it is, but it's clearly a work in progress title, so on linux you'll have trouble keeping up with the updates, none of the open source clients i tried seem to support updates nicely and there's some issue with gog updates api that apaprently makes it hard to integrate | 06:59 |
| zyga | mborzecki: this game is updated roughly once a year | 06:59 |
| zyga | mborzecki: anyway, just consider it | 06:59 |
| zyga | it really is worth the money | 07:00 |
| zyga | mborzecki: each update brings the next episode (3 out of 7 are available now) | 07:00 |
| zyga | it is made by a tiny studio | 07:00 |
| pstolowski | morning | 07:03 |
| zyga | good morning pawel! | 07:03 |
| mborzecki | anyone able to reproduce the unit test timeout in wrappers package? | 07:13 |
| zyga | mborzecki: on master? | 07:13 |
| zyga | mborzecki: I started go test -count 1000 ./wrappers/ | 07:15 |
| zyga | I'll let you know if it triggers | 07:15 |
| mborzecki | pedronis: hi, were you able to reproduce the unit test timeout in wrappers by any chance? | 07:16 |
| pedronis | no | 07:16 |
| mborzecki | hm intersting, also the PRs that ijohnson opened yesterday were green today | 07:17 |
| pedronis | it's definitely weird | 07:17 |
| pedronis | also it might be that something else failed and the panic covers it because it doesn't get printed | 07:17 |
| mborzecki | yeah, that's true | 07:18 |
| pedronis | mvo: hi, it's not needed for the beta but this also needs to be backported https://github.com/snapcore/snapd/pull/8602 I don't see it in 2.45 yet | 07:18 |
| mup | PR #8602: configcore: only reload journald if systemd is new enough <Squash-merge> <Created by stolowski> <Merged by stolowski> <https://github.com/snapcore/snapd/pull/8602> | 07:18 |
| pedronis | mvo: this also wasn't ported https://github.com/snapcore/snapd/pull/8622/ ? | 07:21 |
| mup | PR #8622: cmd/snap-bootstrap/initramfs-mounts: add sudoers to dirs to copy as well <UC20> <Created by anonymouse64> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/8622> | 07:21 |
| mvo | pedronis: yes, that is right, let me fix that | 07:30 |
| zyga | pedronis: I moved https://github.com/snapcore/snapd/pull/8566 to snaplock | 07:34 |
| mup | PR #8566: cmd/cmdutil: add run inhibition operations <Created by zyga> <https://github.com/snapcore/snapd/pull/8566> | 07:34 |
| zyga | once this is +1 I will adjust the dependencies | 07:34 |
| pedronis | thx | 07:34 |
| pedronis | need to do some other things before getting back to reviews | 07:35 |
| zyga | sure | 07:35 |
| zyga | I have plenty of things to write so not blocked | 07:35 |
| mup | PR snapd#8624 opened: cmd/snap: fix the order of positional parameters in help output <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8624> | 07:35 |
| mborzecki | trivial fix ^^ | 07:37 |
| zyga | mborzecki: I reproduced the panic | 07:38 |
| zyga | https://www.irccloud.com/pastebin/vmigrXPM/ | 07:38 |
| zyga | mborzecki: so | 07:46 |
| zyga | mborzecki: this trace is funny | 07:46 |
| zyga | mborzecki: we clearly exec something | 07:46 |
| zyga | yet the SetUpTest says | 07:46 |
| zyga | s.systemctlRestorer = systemd.MockSystemctl(func(cmd ...string) ([]byte, error) { | 07:46 |
| zyga | s.sysdLog = append(s.sysdLog, cmd) | 07:46 |
| zyga | return []byte("ActiveState=inactive\n"), nil | 07:46 |
| zyga | }) | 07:46 |
| zyga | which seems to suggest this mock is one thing | 07:46 |
| zyga | but calling "systemctl" executable is another | 07:47 |
| mborzecki | zyga: which go version is that? | 07:47 |
| zyga | 1.13.8 | 07:47 |
| zyga | brb | 07:47 |
| mborzecki | ok, so not go specific, probably just borked setup | 07:48 |
| mborzecki | or test | 07:48 |
| zyga | bugs, bugs, bugs | 08:35 |
| zyga | but also progres | 08:35 |
| pedronis | #8564 got a +1 from pstolowski and now I applied his suggestions and is ready for 2nd reviews | 08:40 |
| mup | PR #8564: asserts: introduce Pool <Bulk assert refresh :scroll::scroll::scroll:> <Created by pedronis> <https://github.com/snapcore/snapd/pull/8564> | 08:40 |
| zyga | pedronis: what is the 'h' assertion header for? | 08:53 |
| ackk | mvo, hi, I canceled today sync as there isn't really anything to talk about | 08:53 |
| mvo | ackk: ok, I see there is a bug mentioned, I have a look at that | 08:53 |
| ackk | mvo, ah yeah, not sure if it's really relevant for snapd, but if you want we can talk about it. it's just me today | 08:54 |
| ackk | mvo, mostly, I added it there to ask if there was a best practice for knowing the underlying OS details from a snap | 08:54 |
| mvo | ackk: no need to meet just for that :) | 08:55 |
| mvo | ackk: we can cover it next time | 08:55 |
| ackk | right | 08:55 |
| pedronis | zyga: they are test-only assertions, they look a bit like snap-declarations and snap-revisions | 09:01 |
| pedronis | zyga: but unless you reviewed the other ones in this chain, I wouldn't recommend looking a tit | 09:02 |
| pedronis | heh | 09:02 |
| pedronis | at it | 09:02 |
| pedronis | mborzecki: so that test failed again with the timeout | 09:04 |
| pedronis | mvo: some the unit tests changes that came with jamesh PR that was merged yesterday are failing regularly but is not super clear what is going on | 09:05 |
| pedronis | mvo: do you need to do a pre4 ? | 09:06 |
| ackk | mvo, I have a very basic snap with just meta/snap.yaml (to test the system-files interface). I snap pack it but when I install I get - Run configure hook of "testsnap" snap if present (run hook "configure": cannot snap-exec: no such file or directory) | 09:08 |
| ackk | mvo, why is it trying hooks when there are none? | 09:08 |
| zyga | ackk: do you have meta/hooks/configure? | 09:09 |
| ackk | zyga, nope | 09:09 |
| zyga | though the snap-exec error is also curious | 09:09 |
| ackk | just meta/snap.yaml and "script" , which is a bash script | 09:09 |
| ackk | I'm using base: bare if that mattes | 09:09 |
| ackk | *matters | 09:09 |
| zyga | ah | 09:09 |
| zyga | bare base has no shell? | 09:10 |
| zyga | no /bin/sh | 09:10 |
| ackk | oh, | 09:10 |
| zyga | though the configure hook is weird still | 09:10 |
| zyga | it's bare for a reason :D | 09:10 |
| zyga | in fact, it has nothing | 09:10 |
| zyga | you must provide a static binary | 09:10 |
| ackk | right, I forgot | 09:10 |
| mborzecki | pedronis: can you reproduce it? | 09:11 |
| mborzecki | pedronis: or was that in some PR? | 09:11 |
| pedronis | mborzecki: a PR | 09:11 |
| mvo | pedronis: not sure if we need a pre4, I hope we can test what we have in beta and if it's all working fine I can do a 2.45 for real but it depends a bit. do you know already something that is broken? | 09:12 |
| pedronis | mvo: no, just wondered because sudoers bits are not in pre3 | 09:12 |
| pedronis | so you cannot really test recover | 09:13 |
| pedronis | afaiu | 09:13 |
| mvo | pedronis: yeah, that was silly of me, I cherry picked it and pushed to release/2.45 and rebuild the beta. so it will be ~pre3+git but that should be ok for testing :) | 09:13 |
| mborzecki | pedronis: commit d16c44f0aa mentiones some issues with shutting down the session agent | 09:14 |
| pedronis | I know | 09:15 |
| pedronis | maybe we should try undoing and then it happens for often and try to really understand | 09:15 |
| pedronis | s/for/more/ | 09:15 |
| pedronis | anyway this is very annoying | 09:16 |
| mborzecki | hm wonde rif it's possible that the agent hits idle timeout and stops listening | 09:25 |
| pedronis | mborzecki: ah | 09:26 |
| pedronis | maybe should be possible to write a test to see what happens in that case | 09:26 |
| mborzecki | trying to provoke that in tests | 09:26 |
| pedronis | but there's shoould be an Accept around | 09:26 |
| pedronis | in that case, no? | 09:26 |
| pedronis | sorry, there shouldn't be | 09:26 |
| mborzecki | ah right | 09:27 |
| pedronis | zyga: I did another pass on #8566 | 09:30 |
| mup | PR #8566: c/snaplock/runinhibit: add run inhibition operations <Created by zyga> <https://github.com/snapcore/snapd/pull/8566> | 09:30 |
| pstolowski | hmm we could use systemd version check to avoid LazyUnmount on core16 (which generates a lot of noise in system log) | 10:48 |
| zyga | pedronis: thank you for the review, I replied to one comment and I'll adjust the rest later today | 10:49 |
| zyga | pstolowski: +1 | 10:49 |
| zyga | pstolowski: but then we never rewrite .mount units | 10:49 |
| zyga | pstolowski: so -1 unless you know how to do that on systemd upgrades | 10:49 |
| pstolowski | ah, hmm | 10:50 |
| pstolowski | anyway, something to think about | 10:50 |
| mup | PR snapd#8625 opened: wrappers: tweak the order of restoring, use client timeout in services test teardown <Skip spread> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8625> | 11:01 |
| mborzecki | pedronis: ^^ maybe this will give us more insight | 11:01 |
| mborzecki | hm removed that thing poking the client in teardown and can reproduce the timeout now | 11:07 |
| pstolowski | zyga: maybe we could make systemd version part of systemkey | 11:11 |
| zyga | pstolowski: we don't rewrite mount units on system key | 11:12 |
| pstolowski | zyga: yes, but maybe then we would | 11:12 |
| zyga | pstolowski: what happens when you rewrite mount unit | 11:12 |
| zyga | I tried to propose that but it got stuck on this discussion | 11:13 |
| zyga | do we remount things? we cannot really | 11:13 |
| pstolowski | allright... just thinking aloud | 11:13 |
| zyga | no no, that's good | 11:13 |
| zyga | it's just we need to think how to do that in practice | 11:13 |
| pedronis | zyga: interesting, firefox does this HOME: $SNAP_USER_COMMOM in its snap.yaml | 11:15 |
| zyga | I hope it's COMMON :) | 11:15 |
| zyga | but interesting, yeah | 11:15 |
| zyga | we give people the means and they get creative | 11:15 |
| mborzecki | meh, go test -timeout is kinda silly | 11:26 |
| mup | PR snapd#8626 opened: tests: fix passing stdin via session-tool <Created by zyga> <https://github.com/snapcore/snapd/pull/8626> | 11:26 |
| zyga | mborzecki: ^ a trivial patch and an annoying discovery | 11:27 |
| === pedronis_ is now known as pedornis | ||
| === pedornis is now known as pedronis | ||
| pedronis | mborzecki: your PR is green, so maybe we were trying to talk to the real agent? | 11:27 |
| pedronis | do we need that poking code? | 11:27 |
| mborzecki | yeah, but it'd hard to explain why there would be another agent | 11:28 |
| zyga | maybe we activate it via socket? | 11:28 |
| zyga | do we mask the real agent? | 11:28 |
| mborzecki | however, go test ./... runs all package tests in parallel, if there's another package starting the agent, we could try to talk to taht other agent ? | 11:28 |
| zyga | ah, unit tests | 11:28 |
| mborzecki | (assuming mocking elewhere is incorrect too) | 11:28 |
| pedronis | mborzecki: that sounds fragile, well there are agents own tests of course | 11:29 |
| pedronis | mborzecki: anyway I'm still unsure why we need that poking code | 11:30 |
| mborzecki | right | 11:30 |
| mborzecki | perhaps we could land #8625 while i'll try to poke further | 11:31 |
| mup | PR #8625: wrappers: tweak the order of restoring, use client timeout in services test teardown <Skip spread> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8625> | 11:31 |
| pedronis | mborzecki: yes, that's fine | 11:31 |
| zyga | it's finally warm enough to use the standing desk in the colder corner of the office :) | 11:38 |
| cachio | zyga, hi | 11:44 |
| zyga | hi | 11:44 |
| cachio | yesteday I left some errors related to session tool | 11:44 |
| cachio | I see you created a new PR with fixes | 11:44 |
| zyga | oh? | 11:44 |
| zyga | I didn't see those | 11:44 |
| zyga | which fixes? :) | 11:45 |
| cachio | zyga, is it related to this? https://paste.ubuntu.com/p/9Svw3Qd7yB/ | 11:45 |
| cachio | https://paste.ubuntu.com/p/k7fyPJch78/ | 11:45 |
| zyga | no | 11:45 |
| zyga | restore EOFs? | 11:45 |
| cachio | zyga, ahh, these fixes #8626 | 11:45 |
| mup | PR #8626: tests: fix passing stdin via session-tool <Test Robustness> <Created by zyga> <https://github.com/snapcore/snapd/pull/8626> | 11:45 |
| zyga | that is related to another PR but not to the pastebin | 11:46 |
| zyga | is this EOF a one-off or something that always happens/ | 11:46 |
| cachio | zyga, restore EOF and also the other test | 11:46 |
| cachio | zyga, the test failing on this https://paste.ubuntu.com/p/k7fyPJch78/ | 11:46 |
| cachio | zyga, cannot connect to server -> curl --unix-socket /run/user/12345/snapd-session-agent.socket -D- -X POST -H 'Content-Type: application/json' -d '{"action": "daemon-reload"}' http://localhost/v1/service-control | 11:48 |
| cachio | zyga, this is also failing on uc20 https://paste.ubuntu.com/p/NgHhWVVPFW/ | 11:50 |
| cachio | zyga, the problem is that as it is failing on restore, it breaks the whole test suite | 11:50 |
| cachio | zyga, my concern is why it is not failing in google execution | 11:50 |
| cachio | but fails on edge and beta validation | 11:51 |
| zyga | re | 12:06 |
| zyga | cachio: probably random | 12:06 |
| zyga | cachio: unless it happens each time when executing a specific test | 12:07 |
| mborzecki | cachio: can you reproduce the problem in tests/main/snap-session-agent-service-control? | 12:07 |
| cachio | zyga, mborzecki it happens 100% of the time | 12:07 |
| cachio | mborzecki, yes I can | 12:07 |
| zyga | cachio: did you collect information about the other session? | 12:07 |
| zyga | cachio: from the session-tool failure? | 12:08 |
| zyga | ah | 12:08 |
| zyga | wait | 12:08 |
| zyga | sorry, I misread | 12:08 |
| zyga | it's the EOF | 12:08 |
| zyga | so no new data | 12:08 |
| mborzecki | perhaps the session agent isn't running yet/already/at all | 12:08 |
| cachio | zyga, no, but I can do it | 12:08 |
| zyga | it's only interesting iff we can reproduce it in isolation | 12:08 |
| zyga | cachio: if you can run session-tool test _alone_ | 12:08 |
| cachio | zyga, sure | 12:09 |
| zyga | ok | 12:09 |
| cachio | zyga, running | 12:10 |
| cachio | gime me 5 minutes until it fails | 12:10 |
| cachio | zyga, I reproduced the error | 12:21 |
| cachio | is any info which I could provide? | 12:21 |
| cachio | I have an ssh opened | 12:22 |
| zyga | what's the last thing that was logged in the spread run? | 12:22 |
| cachio | 2020-05-08 09:15:58 Error restoring external:ubuntu-core-18-64:tests/main/session-tool:test (external:ubuntu-core-18-64) : + session-tool --restore -u test | 12:24 |
| cachio | 2020-05-08 09:15:58 Error debugging external:ubuntu-core-18-64:tests/main/session-tool:test (external:ubuntu-core-18-64) : EOF | 12:24 |
| cachio | 2020-05-08 09:15:58 Restoring external:ubuntu-core-18-64:tests/main/ (external:ubuntu-core-18-64)... | 12:24 |
| cachio | 2020-05-08 09:15:58 Error restoring external:ubuntu-core-18-64:tests/main/ (external:ubuntu-core-18-64) : EOF | 12:24 |
| cachio | I could run with the flag to show the output | 12:25 |
| zyga | cachio: how do you log into the system? | 12:25 |
| cachio | it is a vm here | 12:25 |
| zyga | cachio: how does spread log into the system? | 12:25 |
| cachio | I it does not log | 12:25 |
| zyga | well, does it execute test by magic? | 12:26 |
| cachio | I have a spreac which show ssh on real time | 12:26 |
| zyga | it must connect to the system somehow | 12:26 |
| cachio | zyga, I have an image to reproduce the error if you want | 12:26 |
| zyga | no | 12:26 |
| zyga | I'd like to understand if this is a normal spread run that uses ssh to connect | 12:26 |
| cachio | zyga, it is a normal spread run | 12:27 |
| zyga | ok | 12:27 |
| cachio | using external backend | 12:27 |
| zyga | is it using ssh to log in as the root user on the device under test? | 12:27 |
| zyga | or is there any other user used | 12:28 |
| cachio | zyga, it uses root | 12:29 |
| cachio | same as in google backend | 12:29 |
| mup | PR snapd#8627 opened: c/snap-bootstrap: port mount state mocking to the new style on master (2.45) <Created by pedronis> <https://github.com/snapcore/snapd/pull/8627> | 12:29 |
| zyga | ok | 12:29 |
| zyga | cachio: can you log into the system | 12:29 |
| zyga | and run, as root | 12:29 |
| zyga | session-tool --prepare -u test | 12:30 |
| zyga | session-tool --restore -u test | 12:30 |
| cachio | zyga, done | 12:32 |
| zyga | did it EOF? | 12:32 |
| cachio | no | 12:32 |
| zyga | ok | 12:32 |
| zyga | is there anything in journal from the time of the failure? | 12:32 |
| cachio | zyga, https://paste.ubuntu.com/p/bfJYGthXwB/ | 12:34 |
| cachio | this is the only I see which seems to be relevant | 12:35 |
| cachio | I see it many times | 12:35 |
| zyga | cachio: what specifically? | 12:35 |
| cachio | May 08 12:15:58 localhost systemd[1]: session-tool-c24e7dda-59d7-4fbd-a871-fc2431b4f5d1.service: Main process exited, code=exited, status=1/FAILURE | 12:36 |
| zyga | the test is running "false" | 12:36 |
| zyga | Starting session-tool running false as test... | 12:37 |
| zyga | to check that exit status is forwarded | 12:37 |
| cachio | ah | 12:37 |
| zyga | cachio: K w | 12:39 |
| zyga | I wonder if this is just a network timeout over ssh | 12:39 |
| zyga | this test takes a while to run | 12:39 |
| zyga | maybe tweak it so that there's fewer iterations | 12:39 |
| zyga | and see if that changes anything | 12:39 |
| cachio | I am running again but now showing the output | 12:40 |
| cachio | so I'll see on which line fails | 12:40 |
| cachio | zyga, perhaps it helps | 12:41 |
| zyga | yeah, let's try | 12:41 |
| cachio | it is looping | 12:43 |
| zyga | cachio: any failures? | 12:56 |
| cachio | still preparing | 12:56 |
| zyga | ok | 12:56 |
| ackk | zyga, am I doing something wrong here https://bugs.launchpad.net/maas/+bug/1876217/comments/5 ? | 12:56 |
| cachio | I had to re-execute | 12:56 |
| mup | Bug #1876217: Controllers report Ubuntu Core version in the Snap <MAAS:In Progress by ltrager> <snapd:Incomplete> <https://launchpad.net/bugs/1876217> | 12:56 |
| cachio | because once it fails, then the session is not correctly cleaned and fails the next run | 12:56 |
| ackk | zyga, (wrt system-files plug) | 12:57 |
| cachio | zyga, I see same behaviour | 12:57 |
| cachio | as it calls itself it goes into an infinite loop | 12:57 |
| zyga | ackk: yeah | 12:58 |
| zyga | ackk: probably ..l | 12:58 |
| zyga | ls -ld /etc/os-release | 12:58 |
| zyga | is it a symlink? | 12:58 |
| ackk | zyga, outside or inside the snap? | 12:58 |
| zyga | well | 12:59 |
| zyga | outside | 12:59 |
| zyga | the hostfs one is failing | 12:59 |
| ackk | /etc/os-release -> ../usr/lib/os-release | 12:59 |
| zyga | so | 12:59 |
| ackk | ah I see | 12:59 |
| ackk | so I have to use the actual file? | 12:59 |
| zyga | apparmor grants you rights to read /var/lib/snapd/hostfs/etc/os-release | 12:59 |
| zyga | but you that is a symlink | 12:59 |
| zyga | so you need /var/lib/snapd/hostfs/usr/lib/os-release | 12:59 |
| zyga | and you need to understand this in your app | 13:00 |
| zyga | it kind of sucks because symlinks seen via hostfs are "hard" | 13:00 |
| zyga | it's good we don't have absolute paths in them | 13:00 |
| zyga | extend the read section | 13:00 |
| zyga | to say /var/lib/snapd/hostfs/usr/lib/os-release | 13:00 |
| zyga | (in addition to the etc line) | 13:00 |
| zyga | and you should be good | 13:00 |
| ackk | zyga, so I suspect that won't work for the purpose of being os-agnostic, as on other OSes that might not be a symlink | 13:00 |
| zyga | yeah | 13:00 |
| ackk | zyga, so it won't work even if the interface allows both? | 13:00 |
| zyga | the denial would also tell you | 13:00 |
| zyga | well | 13:00 |
| zyga | the symlink can in theory point anywhere | 13:01 |
| zyga | in practice it either is not a symlink | 13:01 |
| zyga | or it points to a finite set of files | 13:01 |
| zyga | which do differ by OS flavour (I recommend booting a fedora workstation) | 13:01 |
| ackk | zyga, I see, thanks | 13:02 |
| zyga | good luck! :) | 13:02 |
| zyga | oh | 13:02 |
| zyga | standup!! | 13:02 |
| zyga | ackk: I would prefer a snapctl os-release -like API | 13:03 |
| zyga | where you could just ask | 13:03 |
| zyga | and we'd tell you without this mess | 13:04 |
| ackk | zyga, yeah, that'd be good | 13:04 |
| ackk | zyga, alternatively the actual /etc/os-release content could be exposed in some other file, like /etc/os-release-host or something | 13:05 |
| ackk | although that would only work for core* bases | 13:05 |
| zyga | ackk: that's tricky too | 13:05 |
| zyga | it's easier to have an API | 13:05 |
| ackk | zyga, yeah I assumed it would be trickier :) | 13:05 |
| cachio | zyga, this is the last part of the output | 13:06 |
| cachio | https://paste.ubuntu.com/p/mysVkdXQ4M/ | 13:06 |
| zyga | "last" :D | 13:06 |
| zyga | I don't know where to look | 13:07 |
| zyga | the end looks like just cleanup | 13:07 |
| cachio | I am logging the full run now | 13:08 |
| zyga | no | 13:08 |
| zyga | maybe just look at the log | 13:08 |
| zyga | I mean | 13:08 |
| zyga | I just don't know what to look at | 13:08 |
| zyga | there are 35K lines there | 13:08 |
| zyga | so a longer log doesn't really help much | 13:09 |
| zyga | do you see where it failed? | 13:09 |
| cachio | zyga, https://paste.ubuntu.com/p/2NT6zGG5QB/ | 13:09 |
| cachio | this could help | 13:09 |
| zyga | what is the status of user-1001.slice? | 13:09 |
| zyga | cachio: that does not seem to be a fragment of the earlier log | 13:10 |
| zyga | as I cannot see the EOF messages there | 13:10 |
| cachio | zyga, no, it is a new run | 13:11 |
| cachio | zyga, this is from journal https://paste.ubuntu.com/p/hkcJwwpsV3/ | 13:11 |
| zyga | May 08 13:07:54 localhost sshd[19701]: pam_unix(sshd:session): session closed for user test | 13:12 |
| zyga | this is weird | 13:12 |
| zyga | how come are we logging in as the test user? | 13:12 |
| cachio | cheking | 13:12 |
| zyga | so | 13:13 |
| zyga | cachio: let me tell you what I think | 13:14 |
| zyga | session-tool --restore does stop the slice of the test user | 13:14 |
| zyga | that stops all the services running as the test user | 13:14 |
| zyga | and kills all the test users's processes | 13:14 |
| zyga | the only explanation for the EOF | 13:14 |
| zyga | is that we ssh as the test user | 13:14 |
| zyga | not as root | 13:14 |
| zyga | maybe both, dunno | 13:14 |
| zyga | but that message looks like we just ssh as test | 13:14 |
| cachio | zyga, I ran with -vv and I see it is running as root | 13:19 |
| cachio | export USER="root" | 13:19 |
| zyga | cachio: can you find the moment where we ssh | 13:20 |
| zyga | or log into the machine and see if we've ssh'd as root or as test/ | 13:20 |
| zyga | pstolowski: maybe move the window to the mac's main screen? | 13:21 |
| zyga | mvo: if you review the shell fix, please merge it, it failed on mount-protocol-error :/ | 13:22 |
| mvo | zyga: will do | 13:23 |
| zyga | thanks | 13:23 |
| zyga | cachio: I know what the problem is | 13:28 |
| zyga | cachio: look at spread.yaml | 13:28 |
| zyga | cachio: and look for user: test | 13:28 |
| zyga | that's the problem | 13:28 |
| zyga | we do connect as the test user | 13:29 |
| zyga | no idea why | 13:29 |
| zyga | it's probably not a good idea | 13:29 |
| zyga | (username: test) | 13:29 |
| cachio | zyga, yes | 13:29 |
| cachio | you are right | 13:29 |
| cachio | Let me update the test user and try again | 13:30 |
| zyga | ok | 13:30 |
| zyga | some tests failed on the store | 13:31 |
| zyga | updates: got unexpected HTTP status code 408 via POST to "https://api.snapcraft.io/v2/snaps/refresh" | 13:32 |
| zyga | pedronis: ^ is that expected? | 13:32 |
| cachio | zyga, https://paste.ubuntu.com/p/fww8t62NbC/ | 13:37 |
| cachio | thanks! | 13:37 |
| zyga | excellent | 13:37 |
| zyga | thanks! | 13:37 |
| cachio | I'll create a PR with the fix | 13:37 |
| mup | PR snapd#8628 opened: cmd/snap: coldplug auto-import assertions from all removable devices <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8628> | 13:40 |
| zyga | mvo: if you look at https://tracker.debian.org/pkg/snapd it says there were 8K new commits since release | 13:44 |
| zyga | https://qa.debian.org/cgi-bin/vcswatch?package=snapd | 13:44 |
| zyga | mvo: overrides for https://github.com/snapcore/snapd/pull/8626 and https://github.com/snapcore/snapd/pull/8614 would be great | 13:50 |
| mup | PR #8626: tests: fix passing stdin via session-tool <Test Robustness> <Created by zyga> <https://github.com/snapcore/snapd/pull/8626> | 13:50 |
| mup | PR #8614: tests: don't create root-owned things in ~test <Test Robustness> <Created by zyga> <https://github.com/snapcore/snapd/pull/8614> | 13:50 |
| zyga | I could re-trigger but that would just waste time | 13:50 |
| mvo | zyga: sure, OMW | 13:56 |
| zyga | thank you | 13:57 |
| mup | PR snapd#8614 closed: tests: don't create root-owned things in ~test <Test Robustness> <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8614> | 13:57 |
| mup | PR snapd#8626 closed: tests: fix passing stdin via session-tool <Test Robustness> <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8626> | 13:57 |
| zyga | thanks! | 13:58 |
| mup | PR snapcraft#3111 opened: elf: fix parsing of notes after patchelf mangling <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/3111> | 14:04 |
| ijohnson | mvo: you might want to cherry-pick https://github.com/snapcore/snapd/pull/8623 onto 2.45 as well, since spread tests on the release/2.45 branch will fail there without that PR | 14:13 |
| mup | PR #8623: tests/lib/prepare.sh: delete patching of the initrd <UC20> <⚠ Critical> <Created by anonymouse64> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8623> | 14:13 |
| ijohnson | mvo: I milestoned the PR as 2.45 for you | 14:13 |
| mvo | ijohnson: \o/ | 14:17 |
| mup | PR snapd#8629 opened: tests: new test user is used for external backend <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/8629> | 14:35 |
| zyga | cachio: thank you :) | 14:36 |
| cachio | zyga, yaw | 14:37 |
| cachio | zyga, thaanks to you for helping with the issue | 14:37 |
| zyga | :) | 14:37 |
| zyga | yeah, it was a fun thing to discover :) | 14:37 |
| pedronis | #8577 needs a 2nd review, maybe ijohnson ? | 14:39 |
| mup | PR #8577: secboot,cmd/snap-bootstrap: move initramfs-mounts tpm access to secboot <UC20> <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/8577> | 14:39 |
| * zyga is sorry for missing the TGIF | 14:39 | |
| zyga | lunch | 14:39 |
| ijohnson | pedronis: yes it's been in my queue, will take a look | 14:43 |
| pedronis | I forgot about your 8559 | 14:44 |
| pedronis | mborzecki: so I double checked, the workaround would have poked the real agents, so it cannot have a useful effect | 14:46 |
| mborzecki | pedronis: i think we should just drop that bit that pokes the agent and see how it will perform in the unit tests accross more runs | 14:49 |
| mborzecki | let me push that change actually | 14:49 |
| pedronis | mborzecki: one other thing I notice is that the agent uses SdNotify but I don't see it mocked | 14:51 |
| mborzecki | pedronis: that code shoudl be noop mostly withouth the env vars | 14:52 |
| mup | PR snapd#8630 opened: tests: inject snapd from edge into seeds of the image in manual preseed test <Created by stolowski> <https://github.com/snapcore/snapd/pull/8630> | 15:09 |
| pstolowski | cachio: ^ this should fix the problem, passes for me on 19.10 & 20.04 | 15:10 |
| mborzecki | pedronis: so i dropped that bit that poked the session agent, and it failed now ;P | 15:11 |
| pedronis | mborzecki: it failed here too, but only with 1.9 | 15:12 |
| zyga | :D | 15:12 |
| mborzecki | maybe that shitdown is buggy | 15:12 |
| zyga | is it a bug in go? | 15:12 |
| mborzecki | shutdown | 15:12 |
| zyga | shitdown :D | 15:13 |
| mborzecki | heh, funny typo | 15:13 |
| zyga | shit happens | 15:13 |
| zyga | shutdown may hang | 15:13 |
| pedronis | likely yes, but is not obvious because the shutdown code seems the same | 15:13 |
| pedronis | so it would be lower level | 15:14 |
| zyga | maybe it relies on infra beneath | 15:14 |
| mborzecki | if we only could restart successful unit test jobs | 15:14 |
| zyga | oh about that | 15:14 |
| zyga | mvo: what about bit pusher? | 15:14 |
| mvo | zyga: bit-cacher? still an option, someone needs to review my pr | 15:14 |
| mborzecki | got to go, my father's name day, bringing him some cake and all | 15:15 |
| zyga | mborzecki: ^ you have a solution there | 15:15 |
| zyga | mborzecki: o/ | 15:15 |
| pedronis | mmh | 15:21 |
| pedronis | there is something odd about the code | 15:21 |
| pedronis | maybe it relates to the problem | 15:21 |
| pedronis | or not | 15:21 |
| seb128 | kenvandine, btw, I reported https://bugs.launchpad.net/snapd/+bug/1877610 now | 15:28 |
| mup | Bug #1877610: The gdb option takes over the user configurations <snapd:New> <https://launchpad.net/bugs/1877610> | 15:28 |
| seb128 | about the 'gdb screws the configuration and breaks your application' | 15:29 |
| kenvandine | seb128: thanks | 15:29 |
| cachio | pstolowski, great, thanks | 15:29 |
| * cachio lunch | 15:37 | |
| mup | PR snapd#8631 opened: image,cmd/snap,tests: add support for store-wide cohort keys <Created by xnox> <https://github.com/snapcore/snapd/pull/8631> | 15:45 |
| pedronis | maybe I understand what is going on | 15:49 |
| mup | PR snapd#8624 closed: cmd/snap: fix the order of positional parameters in help output <Simple 😃> <Skip spread> <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8624> | 16:18 |
| mup | PR snapd#8627 closed: c/snap-bootstrap: port mount state mocking to the new style on master (2.45) <Created by pedronis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8627> | 16:18 |
| mup | PR snapd#8629 closed: tests: new test user is used for external backend <Simple 😃> <Created by sergiocazzolato> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8629> | 16:19 |
| mup | PR snapd#8632 opened: configcore: only reload journald if systemd is new enough (2.45) <Created by mvo5> <https://github.com/snapcore/snapd/pull/8632> | 16:23 |
| mup | PR snapd#8633 opened: tests: detect and report root-owned files in /home <Created by zyga> <https://github.com/snapcore/snapd/pull/8633> | 16:27 |
| zyga | ijohnson: I guess with this ^ we could drop the FIXME comments in the pulseaudio tests | 16:29 |
| ijohnson | zyga: there's a lot of things there, which one? | 16:30 |
| pedronis | I think I have a fix for the wrappers mystery, actual the workaround would have also helped (if it talks to the right agent as now in Maciej PR) | 16:30 |
| zyga | ijohnson: ah sorry, this code says BZZ BROKEN if a test leaks root-owned files in /home | 16:30 |
| zyga | ijohnson: this is the code that showed the three tests fixed today | 16:31 |
| pedronis | the issue is that on older go it seems if you call Shutdown before Serve, you get a race and Shutdown will not quite work | 16:31 |
| zyga | pedronis: nice | 16:31 |
| zyga | pedronis: what was the problem? | 16:31 |
| zyga | ah :) | 16:31 |
| pedronis | in normal usage that shouldnd't happen, but because this is setting up the agent also for a lot of cases that don't use it | 16:31 |
| pedronis | it happens | 16:31 |
| zyga | ijohnson: if you scroll to the bottom, you can see how this works | 16:31 |
| zyga | pedronis: which go version is fixed? | 16:32 |
| zyga | I was looking at some hangs that I could not explain and perhaps those are related | 16:32 |
| ijohnson | zyga: no sorry I meant which PR were you referring to, there was like 8 messages from mup above your msg | 16:32 |
| ijohnson | the "^" was ambiguous | 16:32 |
| zyga | ijohnson: ah :D | 16:32 |
| zyga | ijohnson: see here please https://github.com/snapcore/snapd/pull/8633/files#diff-01bb876cbb65b7a336cd99a41c4b97a9R5 | 16:33 |
| mup | PR #8633: tests: detect and report root-owned files in /home <Created by zyga> <https://github.com/snapcore/snapd/pull/8633> | 16:33 |
| pedronis | zyga: I don't know | 16:33 |
| pedronis | actually | 16:33 |
| ijohnson | ack looking now | 16:33 |
| pedronis | zyga: it's not a change in Shutdown itself | 16:33 |
| pedronis | so it's not very easy to track | 16:33 |
| pedronis | and haven't looked deeply | 16:33 |
| zyga | ijohnson: some things we do in ad-hoc way could move there, like looking for selinux denials or apparmor denials or messages from broken udev rules | 16:34 |
| zyga | ijohnson: I have one more checker, that shows we don't have processes running as "test" user | 16:34 |
| ijohnson | this checker looks nice and I like having more checker things like this that make sure tests clean up after themselves | 16:35 |
| zyga | one thing this buys us | 16:35 |
| zyga | is perhaps a way to accelerate prepare/restore | 16:35 |
| zyga | as now it's somewhat heaveyweight because it must be very conservative | 16:35 |
| zyga | we could maybe make it faster over time | 16:36 |
| ijohnson | right | 16:36 |
| ijohnson | It would be good to see what kind of performance that is though, because I wouldn't be surprised if it's not actually that slow | 16:37 |
| zyga | that fast or that slow? | 16:37 |
| ijohnson | or rather than not doing automatic cleanup doesn't net us that much total performance | 16:38 |
| zyga | ah, I see | 16:38 |
| ijohnson | *performance gain | 16:38 |
| zyga | I would be happy if we get a clear message from broken tests :) | 16:38 |
| ijohnson | yes I think that's the MVP we should be looking for right now :-) | 16:38 |
| zyga | and not "one of the tests before, guess which please" but "this test right here" :) | 16:38 |
| zyga | agreed | 16:39 |
| ijohnson | also zyga what happened to "let's write all the -tool's in python :-P | 16:39 |
| ijohnson | " | 16:39 |
| zyga | well | 16:39 |
| zyga | I did | 16:39 |
| zyga | believe me | 16:39 |
| zyga | but it's 20K lines now | 16:39 |
| ijohnson | haha what | 16:39 |
| zyga | and I just realized this will never get merged | 16:39 |
| zyga | I wrote this | 16:39 |
| zyga | waaaay more complex | 16:39 |
| zyga | that's testbed-tool | 16:39 |
| zyga | it supports python and shell plugins | 16:40 |
| zyga | and has tests | 16:40 |
| zyga | and man | 16:40 |
| zyga | this shell script is good enough | 16:40 |
| ijohnson | ah I see what you mean | 16:40 |
| ijohnson | indeed | 16:40 |
| zyga | but | 16:40 |
| ijohnson | also much easier to review than 20000 lines of python | 16:40 |
| zyga | since this is not a source-me shell program | 16:40 |
| zyga | we can :) | 16:40 |
| zyga | that's the important distinction | 16:40 |
| pedronis | zyga: seems it's this issue: https://github.com/golang/go/issues/20239 and it's still thee 1.10 but fixed in 1.11 | 16:41 |
| zyga | and we are building with 1.10? | 16:42 |
| pedronis | we use 1.9 and 1.10 | 16:42 |
| zyga | I see | 16:42 |
| pedronis | and 1.13 | 16:42 |
| zyga | well, maybe after beta we could discuss having more recent builds | 16:42 |
| pedronis | in our tests | 16:42 |
| zyga | maybe for snapd.snap | 16:42 |
| zyga | dunno | 16:42 |
| zyga | I guess apart from core / snapd nobody is on go this old anymore | 16:43 |
| zyga | but after beta | 16:43 |
| zyga | pedronis: and great find! | 16:43 |
| zyga | ok I should EOW now | 16:44 |
| zyga | ijohnson: I'll push a few more checker patches that covert existing random checks into something more organized but probably later tonight only | 16:45 |
| ijohnson | sure sounds godo | 16:45 |
| ijohnson | good even | 16:45 |
| zyga | thanks, have a great weekend everyone :) | 16:45 |
| zyga | o/ | 16:45 |
| ijohnson | you too zyga o/ | 16:45 |
| mup | PR snapd#8625 closed: wrappers: tweak the order of restoring, use client timeout in services test teardown <Skip spread> <Created by bboozzoo> <Closed by pedronis> <https://github.com/snapcore/snapd/pull/8625> | 16:47 |
| mup | PR snapd#8634 opened: usersession/agent,wrappers: fix races between Shutdown and Serve <Created by pedronis> <https://github.com/snapcore/snapd/pull/8634> | 16:53 |
| pedronis | mvo: ijohnson: ^ | 16:53 |
| ijohnson | mmm pedronis I haven't been following this discussion super close, is it ok if I review a bit later, trying to wrap up the reboot from recover -> run PR to propose first | 16:54 |
| pedronis | yes | 16:54 |
| pedronis | mostly I pointed it out because it should help with some of the red we are seeing | 16:55 |
| ijohnson | ack, sounds good | 16:55 |
| mvo | pedronis: nice! | 17:36 |
| cmatsuoka | mvo: is the network fix for recover mode already in 2.45? | 17:59 |
| cmatsuoka | mvo: well it seems so, it worked now (but for some reason it failed once before) | 18:12 |
| ijohnson | cmatsuoka: yes it should be in the kernel snaps available on edge/beta | 18:30 |
| mup | PR snapd#8635 opened: o/devicestate: support doing system action reboots from recover mode <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8635> | 18:35 |
| === ijohnson is now known as ijohnson|lunch | ||
| cachio | zyga, any idea for this? https://paste.ubuntu.com/p/MRVj2xkQJf/ | 18:44 |
| cachio | it is happening with the new bionic image | 18:44 |
| cachio | this happens with session tool as I see | 18:45 |
| === ijohnson|lunch is now known as ijohnson | ||
| pedronis | ijohnson: I added some small comments/questions to that PR | 19:06 |
| ijohnson | pedronis: ack I'm addressing them now | 19:06 |
| zyga | cachio: re | 19:06 |
| zyga | cachio: I think it's the race I mentioned during standup, I'll send a patch that probably fixes it next week | 19:07 |
| cachio | zyga, nice, thanks | 19:07 |
| zyga | so, why do our test images have an "ubuntu" useR? | 19:22 |
| zyga | sorry | 19:22 |
| zyga | why does spread-created core-16 and core-18 have an ubuntu user? | 19:22 |
| zyga | it's really weird | 19:22 |
| zyga | there's a /home/ubuntu/.ssh/authorized keys that's empty | 19:22 |
| zyga | but /home/ubuntu is owned by root:root | 19:23 |
| zyga | but /home/ubuntu/.ssh is ubuntu:ubuntu | 19:23 |
| zyga | do we have a snap changes change for creating users? | 19:23 |
| zyga | ok, found the bug | 19:30 |
| cachio | zyga, did you find it? | 19:37 |
| zyga | no, it was a false trail | 19:37 |
| zyga | I added a few more checks | 19:37 |
| zyga | if you know I'm all ears :) | 19:37 |
| cachio | zyga, I also see this error + session-tool -u test --has-systemd-and-dbus | 19:38 |
| cachio | grep error: pattern not found, got: | 19:38 |
| cachio | no user dbus.socket | 19:38 |
| cachio | is it related? | 19:38 |
| zyga | no | 19:38 |
| zyga | where do you see it? | 19:38 |
| zyga | 16.04? | 19:38 |
| cachio | bionic | 19:38 |
| cachio | using the new image | 19:38 |
| cachio | which I didnt publish yet | 19:39 |
| cachio | image: test-bionic-1 | 19:39 |
| cachio | for this test google:ubuntu-18.04-64:tests/main/session-tool-support:test | 19:39 |
| zyga | no-install-recommends probably ate dbus-user-session | 19:39 |
| cachio | should I install dbus-user-session ? | 19:40 |
| cachio | as a dependency | 19:40 |
| zyga | if it was preinstalled in the other image, yeah | 19:41 |
| zyga | or adjust the test, the test really shows us what the system defaults are | 19:41 |
| pedronis | now we hit a different snapd-session-agent issue | 19:43 |
| cachio | zyga, it is not installed in the previous version | 19:44 |
| zyga | cachio: that's odd, the test really checks if you have dbus.socket as systemctl --user | 19:45 |
| zyga | cachio: and we seem to have it in a vanilla bionic image | 19:45 |
| zyga | cachio: otherwise it would not pass | 19:45 |
| zyga | cachio: perhaps something just removed it | 19:45 |
| zyga | cachio: if you have a debug shell, maybe it is in apt/dpkg history log | 19:45 |
| cachio | zyga, I think the problem is that now we have installed dbus-x11 installed and previously we didnt | 19:46 |
| zyga | pedronis: what did you run into? | 19:46 |
| zyga | cachio: dbus-x11 is not checked by session-tool-support | 19:46 |
| zyga | cachio: look at the test and at the filesystem please | 19:46 |
| pedronis | zyga: test are failing in on core 16 for user services saying there's a start limit issue with the snapd-session-agent | 19:46 |
| zyga | pedronis: start limit, as in it starts and crashes quicky? | 19:47 |
| pedronis | maybe | 19:47 |
| zyga | pedronis: core 16 and core 18 did not ship dbus for user sessions (20 did) | 19:47 |
| zyga | but did that test ever pass there? | 19:47 |
| pedronis | maybe is my code change, but unsure it happens only on core 16 | 19:47 |
| zyga | I see | 19:47 |
| zyga | oh fun | 19:47 |
| zyga | Friday :/ | 19:47 |
| pedronis | I'm trying to run the test alone, to see if it's flaky or always fail | 19:48 |
| pedronis | zyga: alone it works | 19:57 |
| pedronis | fwiw | 19:57 |
| pedronis | zyga: do we start sessions for root in tests? | 20:01 |
| zyga | pedronis: if the test explicitly wants to, as in runs session-tool --prepare (-u root is default) and session-tool .... | 20:02 |
| zyga | pedronis: IIRC some tests do but most don't | 20:02 |
| zyga | pedronis: probably only tests that I wrote do that | 20:02 |
| pedronis | maybe those tests are leaving something behind | 20:02 |
| zyga | pedronis: try with -repeat 30 or something like this | 20:02 |
| zyga | it is good at catching weird mistakes | 20:02 |
| zyga | like test breaking itself | 20:03 |
| zyga | as well as some chance of catching races | 20:03 |
| zyga | pedronis: perhaps, which test did you see fail? | 20:03 |
| zyga | if you dump all the information you have here I may have luck over weekend | 20:03 |
| pedronis | zyga: https://github.com/snapcore/snapd/pull/8634/checks?check_run_id=657046716 | 20:04 |
| mup | PR #8634: usersession/agent,wrappers: fix races between Shutdown and Serve <Test Robustness> <Created by pedronis> <https://github.com/snapcore/snapd/pull/8634> | 20:04 |
| zyga | - Make snap "test-snapd-user-service" (unset) available to the system (Post http://0/v1/service-control: read unix @->/run/user/0/snapd-session-agent.socket: read: connection reset by peer) | 20:05 |
| pedronis | yes, see /0 is root | 20:05 |
| zyga | yeah | 20:05 |
| zyga | let me check the test quickly | 20:05 |
| pedronis | but the tests really set up a session only for test | 20:05 |
| pedronis | sorry the test | 20:05 |
| pedronis | I think there's an issue with the tests but also probably a robustness issue in general | 20:06 |
| zyga | well | 20:06 |
| zyga | the root user has a session | 20:06 |
| zyga | because spread logs in over ssh | 20:06 |
| zyga | perhaps we should mask session services for root | 20:06 |
| zyga | at least for testing | 20:07 |
| zyga | until we understand what's going on | 20:07 |
| zyga | so, without using session-tool, there is a session | 20:07 |
| zyga | loginctl shows it | 20:07 |
| zyga | it's the ssh we use to connect | 20:07 |
| pedronis | maybe yes | 20:07 |
| zyga | so you get session services there | 20:07 |
| zyga | including auto-start and other things | 20:07 |
| zyga | if snapd does something to per-user sessions now | 20:07 |
| zyga | (I kind of forgot how this works) | 20:08 |
| zyga | it's plausible it talks to root's session as well | 20:08 |
| zyga | as a quick way to check | 20:08 |
| pedronis | it does, the logic is very naive in some ways | 20:08 |
| pedronis | that's why I said we might have to think a bit | 20:08 |
| pedronis | but indeed my concern right now is not even the feature | 20:08 |
| pedronis | but more test robustness | 20:09 |
| zyga | systemctl --user mask snapd.session-agent.{socket,service} | 20:09 |
| zyga | right | 20:09 |
| zyga | that line in core-16 prepare should disable this for the root user | 20:09 |
| zyga | so no socket, no autostart | 20:09 |
| zyga | if you want to get sanity quickly, maybe that's the solution | 20:09 |
| pedronis | not today at this point | 20:10 |
| zyga | sure, | 20:10 |
| zyga | it's super late | 20:10 |
| pedronis | I probably should look a bit more how the world looks when it works | 20:10 |
| zyga | the one problem with root session | 20:10 |
| zyga | is that we cannot "wipe the slate" like we do for test | 20:10 |
| zyga | since we're connected as root | 20:10 |
| zyga | so I guess we must be more careful with what happens across tests | 20:11 |
| pedronis | well it seems we get the user agent for root not working somehow | 20:13 |
| pedronis | ijohnson: I don't think I will get to re-review that PR today, also seems the check I made you add is redundant otoh I don't know how covered all that is | 20:14 |
| zyga | the connection reset by peer is simply the agent closing? | 20:14 |
| ijohnson | pedronis: no problem, I think it's reasonably well tested the only thing I don't have more tests for there that I could add would be that doing various combos of some mode to a differnet mode with a different system | 20:14 |
| ijohnson | pedronis: but as mentioned all such cases currently just fail | 20:14 |
| ijohnson | err are not supported, and thus fail | 20:15 |
| pedronis | zyga: it's socket activated | 20:15 |
| pedronis | or should be | 20:15 |
| zyga | maybe the agent crashes? | 20:15 |
| zyga | do we get a journal entry? | 20:15 |
| zyga | note | 20:15 |
| zyga | it's funny | 20:15 |
| pedronis | or mabye the stop on idle is naive | 20:15 |
| zyga | because journalctl --user is distinct | 20:15 |
| zyga | so maybe we should look at journalctl --user as ewll | 20:15 |
| zyga | *well | 20:15 |
| zyga | the test does this in debug | 20:16 |
| zyga | + session-tool -u test systemctl --user status snapd.session-agent.service | 20:16 |
| zyga | we should also do | 20:16 |
| zyga | + session-tool -u root systemctl --user status snapd.session-agent.service | 20:16 |
| zyga | or really just | 20:16 |
| zyga | systemctl --user status | 20:16 |
| zyga | (without the extra fanfare) | 20:16 |
| zyga | May 08 20:23:40 may082014-475733 passwd[1570]: password for 'ubuntu' changed by 'root' | 20:28 |
| zyga | cloud init? | 20:28 |
| zyga | is our core16 system affected by cloud init data from GCE? | 20:28 |
| zyga | ha | 20:29 |
| zyga | it seems so | 20:29 |
| zyga | :D | 20:29 |
| zyga | May 08 20:23:40 may082014-475733 passwd[1570]: password for 'ubuntu' changed by 'root' | 20:29 |
| zyga | that's from cloud-init.service | 20:30 |
| zyga | pedronis: ^^ lol | 20:30 |
| zyga | is this expecteD? | 20:30 |
| zyga | cachio: so, do you know why we get the "ubuntu" user on core16? | 20:38 |
| zyga | cachio: we do have a weird ubuntu user that belongs to grup "niemeyer" | 20:38 |
| zyga | this looks very much like an artefact of cloud-init | 20:38 |
| zyga | but I cannot put my finger on it | 20:38 |
| zyga | specifically the group is surprising | 20:39 |
| pedronis | how do we build images, do we take a image, ssh to it, add package and then clean up and snapshot it? | 20:40 |
| zyga | we build an image and copy some bits but I confirmed that /home/ubuntu does *not* exist at that stage | 20:42 |
| zyga | it really shows up during boot | 20:42 |
| zyga | even timestamps support that | 20:42 |
| zyga | I'm unfamiliar with cloud init detials, looking at some of what the logs say now | 20:42 |
| zyga | (so the directory and the user get added on first boot) | 20:44 |
| zyga | and it really seems that it's cloud init | 20:44 |
| zyga | and I would not really complain | 20:44 |
| zyga | except that it's buggy | 20:44 |
| zyga | the owner of /home/ubuntu is root :D | 20:44 |
| zyga | another fun find | 20:46 |
| zyga | core 16 ships this file | 20:46 |
| zyga | /usr/share/click/frameworks# cat ubuntu-core-15.04-dev1.framework | 20:46 |
| zyga | (messed up but you get the idea) | 20:47 |
| zyga | we also have /usr/share/upstart | 20:47 |
| zyga | https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1639955 | 21:10 |
| mup | Bug #1639955: bad test for snappy systems <cloud-init:Confirmed> <cloud-init (Ubuntu):Confirmed> <https://launchpad.net/bugs/1639955> | 21:10 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!