| === setuid is now known as zZZZzzetuid | ||
| mborzecki | morning | 05:14 |
|---|---|---|
| mvo | mborzecki: great job on 7166 - thanks that this is landed now | 06:56 |
| mborzecki | mvo: yw, it was fun :P | 06:56 |
| mborzecki | mvo: btw. got a report from fedora user about snapd dying due to ABRT, got the logs eventually | 06:56 |
| mborzecki | mvo: what happens is that the system goes to sleep, once it's woken up systemd kills snapd due to watchdog timeout :/ | 06:57 |
| mvo | mborzecki: oh no! | 06:57 |
| mvo | mborzecki: we can't be the only ones affected by this, this sounds like a very generic issue - or are we using the watchdog wrong in some way? | 06:58 |
| mvo | (I guess its a bit early to ask these questions :) | 06:58 |
| zyga | good morning | 07:00 |
| mborzecki | zyga: hey | 07:00 |
| zyga | mvo: thank you for the review on https://github.com/snapcore/snapd/pull/7549 | 07:02 |
| mup | PR #7549: cmd/libsnap: use cgroup.procs instead of tasks <Created by zyga> <https://github.com/snapcore/snapd/pull/7549> | 07:02 |
| zyga | I'm sorry for being absent on Friday | 07:02 |
| zyga | I was a full-time mom for the last three days | 07:02 |
| pokk | Good morning | 07:04 |
| mvo | hey zyga - good morning! | 07:04 |
| pokk | zyga: Speaking from experiance imho that is both wonderful and sucks. I hope you get decently paid while home with sick kids at least | 07:04 |
| zyga | pokk: paid with hugs and diapers :) | 07:06 |
| pokk | zyga: that's great :) | 07:06 |
| zyga | mborzecki: can you do a 2nd review on https://github.com/snapcore/snapd/pull/7549 please | 07:19 |
| mup | PR #7549: cmd/libsnap: use cgroup.procs instead of tasks <Created by zyga> <https://github.com/snapcore/snapd/pull/7549> | 07:19 |
| mvo | sil2100: good morning! could you please have a look at the pending snapd SRU in the unapproved queue? and who should I poke about universe SRUs? I did a drive-by rxtx sru to bionic on sunday to fix the arduino IDE in bionic, pretty trivial and was wondering if I need to poke someone else? | 07:27 |
| sil2100 | mvo: hey! Let me take a look at those then | 07:35 |
| mvo | sil2100: thank you | 07:35 |
| sil2100 | hm, weird, someone accepted snapd to disco but didn't for bionic and xenial | 07:36 |
| sil2100 | Anyway, looking into that | 07:36 |
| sil2100 | Ah | 07:36 |
| sil2100 | mvo: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1840740/comments/3 | 07:36 |
| mup | Bug #1840740: [SRU] 2.41 <verification-needed> <verification-needed-disco> <snapd (Ubuntu):Fix Released> <snapd (Ubuntu Xenial):New> <snapd (Ubuntu Bionic):Incomplete> <snapd (Ubuntu Disco):Fix Committed> <https://launchpad.net/bugs/1840740> | 07:36 |
| sil2100 | mvo: can you address that by any chance? | 07:37 |
| mvo | sil2100: uh, sorry, I must have missed that | 07:37 |
| mvo | sil2100: will look into a fix, please reject | 07:37 |
| sil2100 | mvo: ok! Give me a poke once you re-upload, I can review those then - in the meantime, I'll look at rxtx | 07:38 |
| mvo | sil2100: thanks, will probably take a bit until I will reupload, need to medidate over this a bit :/ | 07:39 |
| mvo | (mostly the test-case part) | 07:39 |
| sil2100 | mvo: I suppose xenial also has the same change, so I'll reject that as well | 07:40 |
| mup | PR snapd#7544 closed: tests: fix snapd-failover test for core18 tests on boards <Created by sergiocazzolato> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/7544> | 07:47 |
| mup | PR snapd#7549 closed: cmd/libsnap: use cgroup.procs instead of tasks <Created by zyga> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/7549> | 07:50 |
| mborzecki | mvo: based on https://github.com/golang/go/issues/19810 i think we could try to replace the time.Ticker in cmd/snapd/main.go with time.After(), but that's about the only idea i have atm | 07:51 |
| mborzecki | and we ping systemd at twice the required frequency, while systemd should be smart enough to use monotonic timers | 07:52 |
| === pstolowski|afk is now known as pstolowski | ||
| pstolowski | morning | 07:57 |
| mborzecki | pstolowski: hey | 08:00 |
| mvo | mborzecki: ta | 08:00 |
| mborzecki | mvo: updated https://github.com/snapcore/snapd/pull/7543 | 08:12 |
| mup | PR #7543: release: make forced dev mode look at cgroupv2 support <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7543> | 08:13 |
| mvo | mborzecki: thank you, much appreciated. sorry for the nitpicking there but it took me a couple of minutes(!) before my first cup of tea to understand the defer. which probably says more about me than the code :/ | 08:26 |
| mborzecki | mvo: no worries :) | 08:27 |
| mup | PR snapd#7430 closed: overlord/snapstate: add SetTaskSnapSetup helper + unit tests <Created by anonymouse64> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/7430> | 08:52 |
| mborzecki | zyga: some comments in #7547 | 08:52 |
| mup | PR #7547: many: use a dedicated named cgroup hierarchy for tracking <Created by zyga> <https://github.com/snapcore/snapd/pull/7547> | 08:52 |
| mborzecki | zyga: iirc we'd stil want to degrade gracefully, but maybe my memory is playing tricks | 08:54 |
| mup | PR snapd#7437 closed: wrappers/services.go: add disabled svc list arg to AddSnapServices <Created by anonymouse64> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/7437> | 08:54 |
| zyga | mborzecki: I'm not sure, we'll see | 08:55 |
| zyga | mborzecki: thank you for the review, I should have posted the more fresh version last week but nothing is lost :) You'll see it soon enough | 08:55 |
| Chipaca | mvo: pedronis: more and more i wonder why we did /v2/download instead of just having a download action on /v2/snaps | 08:55 |
| pedronis | Chipaca: because of different response ? | 08:56 |
| Chipaca | i guess it makes it easier to change our mind / fine-tune the rules for access, but eh | 08:56 |
| Chipaca | pedronis: pr'aps. Or maybe because then we can use it to download things that aren't snaps? | 08:57 |
| pedronis | that might happen | 08:57 |
| Chipaca | pedronis: mvo: have you talked about the assertions side of 'snap download' when going via snapd? | 08:57 |
| Chipaca | pedronis: mvo: also 'snap known --remote' | 08:58 |
| pedronis | Chipaca: mvo added a remote option to the assertion end point | 08:58 |
| mvo | Chipaca: yeah | 08:58 |
| pedronis | it has landed | 08:58 |
| pedronis | afaik | 08:58 |
| Chipaca | neat | 08:58 |
| mvo | Chipaca: we are ready pretty much, hold on a sec | 08:58 |
| mvo | Chipaca: I just chunked the PR up into small bits | 08:59 |
| mborzecki | zyga: so far it's looking good, all things considered the change should be rather simple as we discussed before :) | 08:59 |
| zyga | mborzecki: yeah, I was just thinking about how to stage it | 08:59 |
| zyga | mborzecki: perhaps the next chunk to land for real would be the cgroup alone, without the agent | 08:59 |
| zyga | mborzecki: and after that just the agent | 08:59 |
| zyga | mborzecki: does that sound okay? | 08:59 |
| zyga | mborzecki: I worry that together it will be less approachable | 09:00 |
| mborzecki | zyga: yeah, just the cgroup change would be functionally identical to what we have now | 09:00 |
| mvo | Chipaca: https://github.com/snapcore/snapd/compare/master...mvo5:snap-download-via-snapd?expand=1 this is the full pr, it needs a bit of polish still around the edges that are not proposed yet but once my other two prs are merged the rest is hopefully eay | 09:00 |
| mvo | Chipaca: fwiw, 7533 and 7510 | 09:00 |
| Chipaca | mvo: pedronis: completely unrelated and so I either write it down as a TODO, or forget about it: do we want to change the cache (as in store/cache) to use the same format for sha3 as 'snap info'? right now it's different which makes it harder to use the two together | 09:01 |
| mvo | Chipaca: I think that makes sense | 09:01 |
| pedronis | Chipaca: in which sense together? | 09:01 |
| pedronis | the cache is an implementation detail | 09:02 |
| Chipaca | pedronis: 'snap info --verbose some-snap' tells you the sha, but you can't then look in the cache to know which file it is | 09:02 |
| pedronis | is this for debugging? | 09:02 |
| Chipaca | yes | 09:03 |
| pedronis | Chipaca: it means we need to look at both formats for a while? | 09:04 |
| Chipaca | pedronis: you can always 'sudo find /var/lib/snapd/cache | sudo xargs snap info --verbose ' but that is really expensive, particularly on slow machines | 09:04 |
| Chipaca | pedronis: yes | 09:04 |
| Chipaca | i've added a card | 09:07 |
| pedronis | Chipaca: we could also have snap debug list-cache or something | 09:07 |
| pedronis | anyway it's kind of bottom of the queue given all the things | 09:07 |
| Chipaca | yeah :) | 09:07 |
| Chipaca | pedronis: ooh, snap debug cache might be useful | 09:08 |
| mvo | yeah, thats a nice idea | 09:08 |
| Chipaca | not just for the snap cache | 09:08 |
| pedronis | Chipaca: but given is low, it should really go to the forum as backlog, not as a card, we already have cards that need to move back to the forum | 09:11 |
| Chipaca | ah | 09:12 |
| Chipaca | sorry | 09:12 |
| pedronis | Chipaca: 19.10 TODO is really for stuff we must do | 09:12 |
| pedronis | it can go to upcoming if we were sure it would be done soon | 09:12 |
| pedronis | but we aren't sure of that :) | 09:12 |
| zyga | coffee | 09:50 |
| mborzecki | pedronis: #7541 can land | 10:05 |
| mup | PR #7541: seed/seedwriter: support for extra snaps <Created by pedronis> <https://github.com/snapcore/snapd/pull/7541> | 10:05 |
| pedronis | mborzecki: mvo: thanks | 10:08 |
| mup | PR snapd#7541 closed: seed/seedwriter: support for extra snaps <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/7541> | 10:08 |
| pedronis | mvo: mborzecki: pstolowski: #7556 is ready for review, is the actual switch to use seedwriter in image.go, there is quite a bit of red in it | 10:13 |
| mup | PR #7556: image,seed/seedwriter: switch image to use seedwriter.Writer <Created by pedronis> <https://github.com/snapcore/snapd/pull/7556> | 10:14 |
| pedronis | mvo: #7462 needs a re-review | 10:14 |
| mup | PR #7462: asserts: introduce explicit support for grade for Core 20 models <Created by pedronis> <https://github.com/snapcore/snapd/pull/7462> | 10:14 |
| mvo | pedronis: thanks | 10:15 |
| mvo | pedronis: I have a look | 10:16 |
| mvo | pedronis: just sent you a mail about something unrelated, would be great if you could have a look | 10:16 |
| mvo | pedronis: (but not high priority) | 10:16 |
| Chipaca | mvo: #7533 now suffering from the context conflict that mborzecki called out | 10:18 |
| mup | PR #7533: client: add support to use the new "download" API <Needs Samuele review> <Created by mvo5> <https://github.com/snapcore/snapd/pull/7533> | 10:18 |
| pstolowski | pedronis: ty | 10:19 |
| mvo | Chipaca: let me fix it | 10:20 |
| pedronis | I'll swithc to doing some reviewing after lunch | 10:20 |
| pedronis | *switch | 10:21 |
| mvo | pedronis: thanks | 10:30 |
| * Chipaca takes a break | 10:31 | |
| mborzecki | - Download snap "test-snapd-dbus-consumer" (6) from channel "beta" (stream error: stream ID 1; PROTOCOL_ERROR) | 10:47 |
| mborzecki | meh, again | 10:47 |
| mborzecki | how did we fix it the last time? | 10:48 |
| mvo | mborzecki: maybe its failing 3 times? | 10:48 |
| mvo | mborzecki: we only retry for protocol errror | 10:49 |
| mvo | mborzecki: so if it happens multiple times in the same stream eventually it fails iirc | 10:49 |
| mvo | mborzecki: what pr is that? | 10:49 |
| mborzecki | mvo: https://github.com/snapcore/snapd/pull/7543 | 10:49 |
| mup | PR #7543: release: make forced dev mode look at cgroupv2 support <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7543> | 10:49 |
| mborzecki | mvo: https://travis-ci.org/snapcore/snapd/jobs/594466695 | 10:49 |
| mborzecki | mvo: from the logs it does not look like snapd retried due to PROTOCOL_ERROR | 10:52 |
| mvo | mborzecki: I see some retries here | 10:52 |
| mvo | mborzecki: let me look further | 10:53 |
| mborzecki | mvo: i must be looking at other tests, I see a retry due to unexpected EOF and then there's protocol err but no retry | 10:53 |
| mvo | mborzecki: oh, interessting, I have one tests here with "Retry because of: stream error: stream ID 1; PROTOCOL ERROR | 10:55 |
| mvo | mborzecki: but I have no looked at all of them yet | 10:56 |
| mborzecki | mvo: one failure also kept getting &errors.errorString{s:"unexpected EOF"} | 10:56 |
| mvo | mborzecki: yeah | 10:57 |
| mvo | mborzecki: I think I see what you see, one test seems to not retry | 10:57 |
| mvo | mborzecki: I wonder if different bits of the code in http2 produce slightly different protocol errors or what is going on there, its very strange | 10:58 |
| * mvo should look closer before making strong assertions like "very strange" | 10:58 | |
| mborzecki | mvo: perhaps, but we look for 'PROTOCOL_ERROR' which appears in the non retry case too | 10:58 |
| mborzecki | mvo: and it's the same ubuntu 16.04-32 system, so cannot be go version change at play | 10:59 |
| mvo | mborzecki: right, in the log it seems like the total time of the attemp is ~3m30 so things looks pretty bad | 11:03 |
| mvo | mborzecki: to download yq | 11:03 |
| mvo | mborzecki: so maybe just a fastly issue? | 11:03 |
| zyga | mborzecki: I updated https://github.com/snapcore/snapd/pull/7547 | 11:07 |
| mup | PR #7547: many: use a dedicated named cgroup hierarchy for tracking <Created by zyga> <https://github.com/snapcore/snapd/pull/7547> | 11:07 |
| zyga | mborzecki: I think it has a chance to pass and get merged now | 11:07 |
| zyga | mborzecki: I will put the agent on hold for a sec and do a branch from this to fix setting up cgroup for classic snaps as well (the omission I mentioned earlier) | 11:08 |
| zyga | mborzecki: then come back to integrate the agent | 11:08 |
| zyga | rogpeppe: hello | 11:10 |
| rogpeppe | zyga: hiya | 11:10 |
| zyga | rogpeppe: do you have any updates for us? is there a chance to get the SD image? | 11:10 |
| rogpeppe | zyga: unfortunately my dad thought he'd dropboxed the image before going away, but only did a zip file with just the script instead, sadly | 11:11 |
| zyga | ah, I see | 11:11 |
| rogpeppe | zyga: so i'm waiting for him to get back from his travels (soon, I think) | 11:11 |
| zyga | that's okay, just keep us posted if you can get the image | 11:11 |
| zyga | thank you :) | 11:11 |
| zyga | we are moving some stuff behind the scenes to make the pi more reliable in conjunction with other developers from beyond the snapd team | 11:12 |
| zyga | it's not something that will be done overnight but I think that at least one annoying thing can be fixed in the next one or two releases (the reboot getting stuck there) | 11:12 |
| Chipaca | mborzecki: did you save the protocol error log? | 11:19 |
| mborzecki | Chipaca: i still ahve the log opened | 11:20 |
| Chipaca | mborzecki: can you get at the 'raw' log? | 11:20 |
| pedronis | pstolowski: I did pass over 7355, just nitpicks of various importance | 11:20 |
| pedronis | *a pass | 11:20 |
| mborzecki | Chipaca: sure, just a sec | 11:20 |
| pstolowski | pedronis: ty | 11:21 |
| mborzecki | Chipaca: https://paste.ubuntu.com/p/bFgCsXWvzX/ | 11:22 |
| Chipaca | hah! | 11:23 |
| pedronis | pstolowski: a question could we/should we consider using SetupMany in setupSecurityByBackend ? | 11:24 |
| Chipaca | mborzecki: mvo: AFAICT the problem is that we got protocol error not on doing the request but on downloading | 11:29 |
| Chipaca | mborzecki: mvo: i.e., we might need to add the same check to ShouldRetryHttpResponse? | 11:29 |
| mvo | Chipaca: aha, I missed that | 11:29 |
| Chipaca | not sure tho | 11:29 |
| mborzecki | ah, it's called directly in the store package | 11:30 |
| mborzecki | Chipaca: duh, but we call SHouldRetryError() there too | 11:31 |
| Chipaca | mborzecki: yeah i was wrong (surprise!) | 11:31 |
| Chipaca | the RetryHttpResponse will say yes because it's 206 | 11:32 |
| Chipaca | wait, no, it'll say no for that same reason | 11:32 |
| Chipaca | huh | 11:32 |
| pstolowski | pedronis: indeed! good idea, we could save a little there as well. i can do in a followup | 11:41 |
| mup | PR snapd#7557 opened: interfaces/mount: account for cgroup version when reporting supported features <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7557> | 11:50 |
| mborzecki | zyga: ^^ | 11:51 |
| zyga | ack | 11:51 |
| zyga | looking | 11:51 |
| zyga | +1 | 11:52 |
| zyga | mborzecki: we don't have 31 tests yet, right? | 11:52 |
| mborzecki | zyga: we don't, there was an image cachio prepared a while ago, but that was before 31 branched iirc | 11:52 |
| zyga | I see | 11:52 |
| zyga | ok, | 11:52 |
| zyga | it would be good to discuss this today | 11:52 |
| zyga | I could use 31 image | 11:52 |
| * Chipaca takes a deep breath and reminds himself he's not here to remove the stupid from random people | 11:53 | |
| * Chipaca goes to check on lunch | 11:53 | |
| * mvo hugs juliank for his command-nof-found upload | 11:54 | |
| ogra | will that finally fix core18 ? | 11:56 |
| juliank | :) | 11:56 |
| mborzecki | off to pick up the kids | 12:12 |
| pedronis | mborzecki: did a pass with some questions on the timeutil bits of #7443 | 12:14 |
| mup | PR #7443: timeutil: fix schedules with ambiguous nth weekday spans <Bug> <Needs Samuele review> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7443> | 12:15 |
| pedronis | Chipaca: do you remember why given it takes at most one snap we support taking multiple in the json to /download? | 12:40 |
| mup | PR snapd#7558 opened: boot,dirs,image: various refinements in the prepare-image code switched to seedwriter <Created by pedronis> <https://github.com/snapcore/snapd/pull/7558> | 12:50 |
| === ricab is now known as ricab|lunch | ||
| mup | PR snapd#7543 closed: release: make forced dev mode look at cgroupv2 support <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/7543> | 13:13 |
| bloodearnest | Chipaca: o/ - to follow up on refresh issues from last week - when do the systemd files get regenerated in the refresh action? It looks like at some point we're getting services started that are pointing at old snaps working directory | 13:21 |
| Chipaca | bloodearnest: if you have a recent 'refresh' change | 13:25 |
| Chipaca | bloodearnest: you can 'snap tasks --last=refresh' | 13:25 |
| bloodearnest | Chipaca: thanks | 13:26 |
| Chipaca | bloodearnest: but it is: run pre-refresh hook, stop services, remove aliases, remove current symlink, copy data, … , post-refresh hook, start serviecs | 13:27 |
| bloodearnest | Chipaca: does start services include generating the systemd files? | 13:32 |
| bloodearnest | Because the services seem to be starting up with the old working directory | 13:32 |
| zyga | mborzecki: hey | 13:38 |
| zyga | mborzecki: can you quickly decide if I should open the named cgroup PR (away from draft mode) | 13:39 |
| zyga | mborzecki: or finish the change to include classic snaps as well first | 13:39 |
| mborzecki | zyga: i think it's ok to open it now while to code is small, then classic can be done later as part of #7302 (or an intro to that PR) | 13:40 |
| mup | PR #7302: cmd/snap-confine: add support for parallel instances of classic snaps, global mount ns initialization <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7302> | 13:40 |
| zyga | okay, I'll do so now | 13:40 |
| zyga | mborzecki: https://github.com/snapcore/snapd/pull/7547 is open now | 13:42 |
| mup | PR #7547: many: use a dedicated named cgroup hierarchy for tracking <Created by zyga> <https://github.com/snapcore/snapd/pull/7547> | 13:42 |
| zyga | mborzecki: do you want to add that snap mount unit to 7302? | 13:42 |
| ackk | zyga, hi, it seems I managed to reproduce the issue with "snap try" and rsyncing files, leading to snapd not finding the executable in the snap | 13:43 |
| zyga | hey, cool, do you have it scripted by any chance? | 13:43 |
| mborzecki | zyga: i'll try a separate PR, just to see the whole spread suite run | 13:43 |
| zyga | something we can play with and debug? | 13:43 |
| zyga | mborzecki: Ok | 13:43 |
| ackk | zyga, no, I don't have a way to reproduce it, it just happens | 13:44 |
| ackk | zyga, but I could debug, if you have commands I can run | 13:44 |
| zyga | ackk: what is the error message you see that is the immediate problem? | 13:44 |
| ackk | $ maas | 13:45 |
| ackk | execv failed: No such file or directory | 13:45 |
| ackk | zyga, ^ | 13:45 |
| zyga | are you able to execute more commands now to explore this? | 13:45 |
| ackk | zyga, and of course, $SNAP/command-maas.wrapper and $SNAP/bin/maas (which is called by that one) are there | 13:45 |
| ackk | zyga, yeah | 13:45 |
| zyga | ackk: can you ls -ld /snap/maas/current | 13:45 |
| ackk | zyga, lrwxrwxrwx 1 root root 2 Oct 7 10:23 /snap/maas/current -> x1 | 13:46 |
| zyga | ackk: snap run --strace=raw maas | 13:46 |
| zyga | ackk: is x1 the expected correct revision? | 13:46 |
| zyga | ackk: cat /proc/self/mountinfo | grep -F '(deleted') | 13:46 |
| zyga | er | 13:46 |
| ackk | zyga, yes | 13:46 |
| zyga | fix the quote please | 13:46 |
| ackk | zyga, nothing there | 13:46 |
| ackk | also: | 13:47 |
| ackk | $ snap run --strace=raw maas | 13:47 |
| ackk | error: exit status 1 | 13:47 |
| zyga | ackk: cat /proc/self/mountinfo | grep maas | 13:47 |
| ackk | zyga, FTR this is in a bionic container | 13:47 |
| zyga | right, I recall those are used | 13:47 |
| zyga | and are most likely a factor | 13:47 |
| zyga | I'd love to understand what's the key bit that is broken | 13:47 |
| ackk | zyga, https://paste.ubuntu.com/p/8VcMZhmtrS/ | 13:47 |
| zyga | ackk: dmesg | grep DENIED | 13:48 |
| zyga | ackk: also, sudo nsenter -m/run/snapd/ns/maas.mnt -- in the resulting shell cat /proc/self/mountinfo | grep -F deleted | 13:49 |
| ackk | zyga, on the host or in the container? | 13:49 |
| ackk | (the dmesg) | 13:49 |
| zyga | ackk: hmmm, should be the same AFAIK | 13:49 |
| zyga | so either | 13:50 |
| ackk | zyga, I used to run snappy-debug on the host, or I would miss some denials | 13:50 |
| ackk | not sure if it's relevant | 13:50 |
| zyga | hmm, perhaps there's something I don't understand about lxd | 13:50 |
| zyga | try on the host please | 13:50 |
| ackk | zyga, https://paste.ubuntu.com/p/JSXhbrR4PV/ | 13:50 |
| ackk | zyga, empty result for deleted mountinfo | 13:51 |
| zyga | hum | 13:51 |
| zyga | [92186.616879] audit: type=1400 audit(1570431426.676:2750): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-maas_<var-snap-lxd-common-lxd>" profile="/snap/snapd/4605/usr/lib/snapd/snap-confine" pid=4605 comm="snap-confine" family="netlink" sock_type="raw" protocol=15 requested_mask="send receive" denied_mask="send receive" | 13:51 |
| zyga | <- this is interesting, separately from the issue that affects you | 13:51 |
| zyga | ackk: can you navigate around in that nsenter shell, to see if the executables are where you expect | 13:52 |
| ackk | zyga, this would be basically $SNAP right? | 13:52 |
| zyga | yeah, but SNAP is not set, so manually | 13:52 |
| ackk | sure | 13:52 |
| zyga | ackk: one more idea, SNAP_CONFINE_DEBUG=yes snap run ... (as you run maas otherwise) | 13:52 |
| ackk | $ SNAP_CONFINE_DBUG=yes snap run -- maas --help | 13:53 |
| ackk | execv failed: No such file or directory | 13:53 |
| ackk | zyga, ^ | 13:53 |
| ackk | zyga, afaict executables are there in the right place | 13:54 |
| zyga | SNAP_CONFINE_DEBUG=yes | 13:54 |
| zyga | DBUG vs DEBUG | 13:54 |
| zyga | interesting, perhaps it is failing to exec something other than the app | 13:54 |
| ackk | ugh | 13:54 |
| ackk | zyga, http://paste.ubuntu.com/p/jdKzRHbFsH/ | 13:55 |
| zyga | holly molly | 13:55 |
| zyga | I suspect I know what this is | 13:55 |
| zyga | from the nsenter shell | 13:55 |
| zyga | can you ls -ld /usr/lib/snapd/snap-exec | 13:55 |
| Chipaca | bloodearnest: sorry, missed your question. Actually writing the service files is part of the "link-snap" step, i.e. "make snap available to the system" | 13:56 |
| zyga | ackk: this is a core18 system? | 13:56 |
| zyga | ackk: as in | 13:56 |
| zyga | ackk: a classic system with core18 and snapd snap? | 13:56 |
| ackk | zyga, correct | 13:56 |
| ackk | zyga, snap-exect not found | 13:56 |
| ackk | zyga, bingo :) | 13:57 |
| Chipaca | bloodearnest: i'm very curious to know what the cause of the weirdness you're seeing is, fwiw | 13:57 |
| zyga | and you happen to follow which snapd channel? | 13:57 |
| zyga | ackk: ls -ld /usr/lib/snapd | 13:57 |
| Chipaca | bloodearnest: if you're looking at 'snap tasks --last=refresh' note you might also find interesting info in 'snap debug timings --last=refresh' | 13:57 |
| ackk | zyga, http://paste.ubuntu.com/p/wTsT8yPBHF/ tl;dr all stable | 13:57 |
| ackk | zyga, /usr/lib/snapd is there | 13:58 |
| zyga | is it a directory? | 13:58 |
| zyga | what's in it? | 13:58 |
| ackk | zyga, yes: drwxr-xr-x 2 root root 0 Oct 1 05:44 /usr/lib/snapd | 13:58 |
| ackk | zyga, but, empty | 13:58 |
| zyga | uhuhhh | 13:58 |
| zyga | okay | 13:58 |
| zyga | I strongly suspect I know what this is | 13:58 |
| zyga | mvo: ^ | 13:59 |
| zyga | ackk: the good news, it's high up our stack | 13:59 |
| zyga | ackk: the bad news, it's not your system, it's a real general bug | 13:59 |
| zyga | ackk: can you do snap changes please? | 13:59 |
| ackk | zyga, well, makes me feel less bad :) | 13:59 |
| ackk | zyga, in the nsenter or outside? | 13:59 |
| mvo | zyga: oh no | 13:59 |
| zyga | ackk: outside please | 13:59 |
| zyga | ackk: is this reported anywhere? | 13:59 |
| zyga | ackk: I think you might have already | 14:00 |
| zyga | mvo: I think the tools problem is more severe | 14:00 |
| ackk | zyga, http://paste.ubuntu.com/p/2MdPHqdyfd/ | 14:00 |
| zyga | mvo: with enough rotations I suspect we unmount /usr/lib/snapd from actively used namespaces | 14:00 |
| ackk | zyga, I don't remember if I did report it, unfortunately I never found a reliable way to make it fail. since last time we talked about it, it hasn't happened anymore and I tried a loop of rsync's | 14:00 |
| zyga | ackk: can you attempt to pinpoint when the issue occurred for you this time? | 14:00 |
| zyga | ackk: in that log, at which time was it | 14:00 |
| zyga | ackk: was it _before_ all of this? | 14:01 |
| ackk | zyga, no, I mean I've "snap try"'d and removed a bunch of times | 14:01 |
| ackk | zyga, but basically snap try , bunch of rsync -> broken | 14:01 |
| ackk | zyga, I didn't snap try from an already installed snap, if that helps | 14:01 |
| zyga | hmm | 14:01 |
| zyga | hmm | 14:01 |
| zyga | right | 14:02 |
| zyga | I wonder if snapd refreshed in the middle of all the "snap try" commands anywhere | 14:02 |
| Chipaca | mvo: pedronis: wrt downloads, if we want top drop action and change snaps to snap, we should do that before or together with 7510 | 14:02 |
| ackk | zyga, the snapd snap says refreshed 33d ago | 14:03 |
| zyga | hmm | 14:03 |
| zyga | in that case I know less, it's not my initial assumption | 14:03 |
| pedronis | Chipaca: for clarity (hopefully) I wrote this, related to channel stuff, https://trello.com/c/GTSFCPG5/292-do-not-switch-tracks-with-risk-only-channel-specification-snap-refresh-switch#comment-5d9b456d31aa8d1b410bcdc2 | 14:03 |
| zyga | ackk: the maas snap has a configure hook? | 14:03 |
| ackk | zyga, yes | 14:04 |
| zyga | right? | 14:04 |
| zyga | ok | 14:04 |
| ackk | which does a lot | 14:04 |
| zyga | so each snap try runs stuff | 14:04 |
| ackk | zyga, correct | 14:04 |
| zyga | ackk: can you please share the exact rsync command you are using | 14:04 |
| zyga | I'll try to reproduce this with a demo snap with a configure hook | 14:04 |
| mup | PR snapd#7559 opened: tests: change regex to validate access to cdn during snap download <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7559> | 14:05 |
| ackk | zyga, https://paste.ubuntu.com/p/B69smBJ73N/ | 14:05 |
| ackk | zyga, where build/dev-snap/prime is the prime dir that gets snap try'd | 14:06 |
| mvo | Chipaca: in a meeting right now | 14:06 |
| zyga | and on the container you "snap try build/dev-snap/prime? | 14:06 |
| ackk | zyga, correct | 14:06 |
| zyga | ok | 14:07 |
| zyga | hmm, for now I have no more ideas | 14:07 |
| zyga | thank you for reporting this again | 14:07 |
| zyga | I'll write a test tomorrow and see if I can hit something | 14:07 |
| zyga | your host is 16.04 or something newer? | 14:07 |
| zyga | the container was 18.04 IIRC | 14:07 |
| bloodearnest | Chipaca: thanks, testing a new release now, will report back | 14:08 |
| ackk | zyga, my host is bionic | 14:10 |
| ackk | zyga, sorry, it's dingo | 14:10 |
| zyga | ah, perfect. | 14:10 |
| zyga | so I have the same setup readty | 14:10 |
| zyga | ok, I'll keep you posted, thank you again | 14:10 |
| ackk | zyga, thank you for the help | 14:10 |
| === ricab|lunch is now known as ricab | ||
| pedronis | Chipaca: mvo: I would do that I think (I find the current state confusing with all the extra checks) | 14:15 |
| pedronis | mvo: what's the status of this https://github.com/snapcore/core/pull/83 ? | 14:40 |
| mup | PR core#83: move most of the ubuntu-core config deb into the snap snap build <Created by mvo5> <https://github.com/snapcore/core/pull/83> | 14:40 |
| mvo | pedronis: I think it never got reviews :/ | 14:41 |
| mvo | pedronis: I can fix the conflict and double check that its up-to-date | 14:41 |
| pedronis | mvo: reviews from whom? foundations? | 14:42 |
| pedronis | there's a review by ogra fwiw | 14:42 |
| mvo | sil2100: silly question, should the split of snapcore/pi3-gadget be on snapcore/pi-gadget (the "unified" one we use in core18) | 14:42 |
| mvo | sil2100: actually I guess I should ask in the PR | 14:43 |
| ogra | pedronis, well, more a yoga lesson :) but yeah ... +1 for that one | 14:44 |
| * cachio lunch | 14:46 | |
| pedronis | zyga: is this being or was fixed: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1785410 ? | 14:46 |
| mup | Bug #1785410: Can not open Gnome snaps as they are not connected to «gnome platform snap» <amd64> <apport-bug> <bionic> <snapd:Triaged by zyga> <snapd (Ubuntu):Confirmed> <https://launchpad.net/bugs/1785410> | 14:46 |
| mvo | Chipaca: I missed the earlier bits, what should happen with 7510? we remove "action=download"? | 14:47 |
| Chipaca | mvo: either in 7510 or before it, we drop action and change snaps to snap | 14:48 |
| mvo | Chipaca: ok, will do | 14:49 |
| pedronis | mvo: probably best to call it snap-name, we might have download by id | 14:57 |
| mvo | pedronis: great advise | 14:58 |
| zyga | pedronis: not sure if this instance, many similar issues were | 14:59 |
| zyga | many issues having similar observable result | 14:59 |
| zyga | I'll inspect that bug and respond | 15:00 |
| pedronis | zyga: you made a card specifically about this one, that's why I'm asking: https://trello.com/c/CFDRWAg5/206-content-connection-issues-https-launchpadnet-bugs-1785410 | 15:00 |
| * zyga breaks to rest | 15:08 | |
| mup | PR snapd#7560 opened: daemon: change /v2/download API to take "snap-name" as input <Created by mvo5> <https://github.com/snapcore/snapd/pull/7560> | 15:18 |
| mvo | Chipaca: -^ | 15:20 |
| mvo | pedronis: a quick review on 7560 would be great, should be quick and simple | 15:32 |
| pedronis | mvo: I was already looking at it | 15:33 |
| * mvo hugs pe | 15:35 | |
| * mvo hugs pedronis | 15:35 | |
| pedronis | pstolowski: this is done now, right? https://trello.com/c/2lsJLlXB/316-revisit-fix-per-revision-snap-configuration-handling | 16:19 |
| pstolowski | pedronis: yes | 16:22 |
| pedronis | pstolowski: is it in 2.42 or will it be in 2.43 ? | 16:27 |
| pedronis | Chipaca: this is done now: https://forum.snapcraft.io/t/expose-a-more-consistent-subset-of-systemds-service-directives/2268 ? | 16:45 |
| mup | PR snapd#7560 closed: daemon: change /v2/download API to take "snap-name" as input <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7560> | 16:48 |
| mup | PR snapd#7561 opened: tests: update system-usernames test now that opensuse-15.1 works <Simple 😃> <Created by mvo5> <https://github.com/snapcore/snapd/pull/7561> | 16:50 |
| Chipaca | pedronis: not yet, not in full | 16:50 |
| cachio | zyga, hey | 16:51 |
| cachio | zyga, I am trygin to fix an issue in opensuse | 16:52 |
| cachio | https://travis-ci.org/snapcore/spread-cron/builds/594484689#L1747 | 16:52 |
| pedronis | Chipaca: when you have a moment could you update it to reflect what is missing? or maybe create a new post if you think it's too old to reopen, in which case remove the tagging on the first, thank you | 16:52 |
| cachio | it is just happening with the last update | 16:52 |
| pstolowski | pedronis: 2.42. however it was only a small cleanup (not a fix) | 17:06 |
| pstolowski | pedronis: i mean, nothing for release notes imho | 17:06 |
| zyga | cachio: snapd is built without apparmor on opensuse leap | 17:11 |
| zyga | that test should not run there | 17:11 |
| zyga | cachio: check exclusion rules please | 17:11 |
| cachio | zyga, systems: [-ubuntu-core-*, -debian-*] | 17:14 |
| cachio | it is currently running on leap 15.1 | 17:14 |
| pedronis | pstolowski: thx, just wanted to know which done lane to use | 17:32 |
| mup | PR snapcraft#2709 closed: incorporate content provider snaps in dependency resolution <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2709> | 18:25 |
| === pstolowski is now known as pstolowski|afk | ||
| joedborg | hey zyga, have you got 5 minutes? | 19:09 |
| mup | PR snapd#7562 opened: tests: chech the apparmor_parser when the file exists on snap-confine test <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7562> | 19:28 |
| === zZZZzzetuid is now known as setuid | ||
| mup | PR snapcraft#2743 opened: meta: warn about command mangling <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2743> | 20:52 |
| mup | PR snapcraft#2744 opened: project_loader: load build-environment after snapcraft environment <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2744> | 22:19 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!