| === katnip` is now known as katnip | ||
| === katnip is now known as Guest66061 | ||
| === Guest66061 is now known as katnip` | ||
| jdstrand | roadmr: thanks! I'm hoping to have another one soon :) | 06:53 |
|---|---|---|
| roadmr | jdstrand: sure thing, just let me know | 07:03 |
| zyga | rogpeppe: good morning | 07:06 |
| rogpeppe | zyga: hiya | 07:07 |
| zyga | rogpeppe: we're still sprinting so attention is spotty, I will not be able to do much interactive debugging today | 07:07 |
| zyga | I wanted to check if the board misbehaved today | 07:08 |
| rogpeppe | zyga: yup, it's down currently | 07:08 |
| rogpeppe | zyga: i need to work out what's going on with the memory. perhaps using too much memory is killing the whole thing. | 07:08 |
| zyga | rogpeppe: perhaps if the service is leaking memory you can restart it on a timer | 07:09 |
| zyga | say hourly | 07:09 |
| rogpeppe | zyga: i'm also wondering if it's possible that swap was configured by default before but isn't any more | 07:09 |
| zyga | rogpeppe: I touched on the swap situation yesterday in the team but we're not sure, we need to look | 07:09 |
| zyga | people here suggest that my memory is rusty and it's off by default but you can turn it on | 07:10 |
| rogpeppe | zyga: i just added a pprof handler to the service, so hopefully i can look into it and find out what's taking all the memory | 07:10 |
| rogpeppe | zyga: it's also possible that the newer Go version isn't garbage collecting as much | 07:10 |
| zyga | rogpeppe: which version are you on? | 07:11 |
| rogpeppe | zyga: 1.13 | 07:11 |
| zyga | rogpeppe: we're building snapd with 1.10 | 07:11 |
| rogpeppe | zyga: ancient :) | 07:12 |
| zyga | rogpeppe: there were some changes in go memory handling but we have not seen actual leaks outside of a super specific case with a custom kernel | 07:12 |
| zyga | rogpeppe: compromise for practicality :) | 07:12 |
| rogpeppe | zyga: neither me. there's a tweakable parameter too - I could try tweaking GOGC | 07:14 |
| zyga | rogpeppe: perhaps do one more thing | 07:14 |
| zyga | rogpeppe: add a timer or something like that | 07:15 |
| zyga | or even a screen session | 07:15 |
| zyga | that logs dstat / top | 07:15 |
| zyga | so you see what's going on after it crashes | 07:15 |
| zyga | if it's really taking all the memory | 07:15 |
| rogpeppe | yeah, that's a good idea | 07:15 |
| zyga | top has a continuous mode, perhaps on a minute interval it would be useufl | 07:15 |
| rogpeppe | BTW I think i've also been bitten by that systemd bug where it loses the last few messages written to the command's output | 07:16 |
| rogpeppe | which means that it loses the whole panic message when it panics (not that that matters too much - i know where the panic was coming from: the clock issue) | 07:16 |
| zyga | rogpeppe: is it the rate limiting behavior? | 07:17 |
| zyga | I'm not familiar with the bug itself | 07:17 |
| zyga | you can also perhaps log to a file explicitly but I think journald should not behave this way | 07:17 |
| zyga | (it could be just a bug though) | 07:17 |
| rogpeppe | zyga: no, the problem is that systemd uses unix socket tricks to get the process id of the writer, but when the writer has just exited, it can't do that, so the messages don't get associated with the right unit | 07:18 |
| rogpeppe | zyga: (i can't remember if they're just discarded or if they just end up somewhere else) | 07:18 |
| rogpeppe | zyga: i think it might be this bug https://github.com/systemd/systemd/issues/2913 | 07:19 |
| zyga | checking | 07:21 |
| zyga | wow, that's useful | 07:21 |
| mup | PR snapcraft#2722 closed: meta: fixes for desktop file handling <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2722> | 07:29 |
| mup | PR snapd#7478 opened: snapstate: increase settleTimeout in TestRemodelSwitchToDifferentKernel <Created by mvo5> <https://github.com/snapcore/snapd/pull/7478> | 07:30 |
| mup | PR snapd#7479 opened: tests: skip centos when running nigthly suite <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7479> | 07:40 |
| mup | PR snapcraft#2724 opened: docs: add a Code of Conduct <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2724> | 07:44 |
| sergiusens | Chipaca: ^ | 07:44 |
| sergiusens | I used the contributor template on Github, it seemed a bit more project related and less Ubuntu specific (after reading the Ubuntu one) | 07:45 |
| sergiusens | please tell me what you think :-) | 07:45 |
| Chipaca | sergiusens: SGTM but I'd want to check it with somebody ubuntu-facing in canonical | 07:47 |
| sergiusens | Chipaca: I checked with Wimpress and popey , they are +1 on this one | 07:48 |
| mup | PR snapd#7480 opened: store: download propagates options to delta download <Created by chipaca> <https://github.com/snapcore/snapd/pull/7480> | 07:48 |
| Chipaca | verterok: https://github.com/snapcore/snapd/pull/7480 | 07:48 |
| mup | PR #7480: store: download propagates options to delta download <Created by chipaca> <https://github.com/snapcore/snapd/pull/7480> | 07:48 |
| Chipaca | sergiusens: schweet | 07:48 |
| verterok | Chipaca: that was fast! | 07:48 |
| Chipaca | things are either fast, or impossible | 07:49 |
| verterok | heh | 07:49 |
| verterok | thanks | 07:49 |
| Chipaca | now it needs to pass pesky things like peer review | 07:49 |
| Chipaca | mvo: https://github.com/snapcore/snapd/pull/7480.patch | 07:55 |
| mup | PR #7480: store: download propagates options to delta download <Created by chipaca> <https://github.com/snapcore/snapd/pull/7480> | 07:55 |
| mup | PR snapd#7481 opened: data/selinux: allow snapd to issue sigkill to journalctl <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7481> | 07:55 |
| mup | PR snapcraft#2724 closed: docs: add a Code of Conduct <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2724> | 07:56 |
| mup | PR snapd#7482 opened: tests/main/listing: account for dots in ~pre suffix <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7482> | 08:07 |
| mup | PR snapd#7482 closed: tests/main/listing: account for dots in ~pre suffix <Simple 😃> <Created by bboozzoo> <Closed by pedronis> <https://github.com/snapcore/snapd/pull/7482> | 08:16 |
| mup | PR snapd#7483 opened: We need a CoC, because people can't be nice <Created by chipaca> <https://github.com/snapcore/snapd/pull/7483> | 08:31 |
| Chipaca | zyga: #1841137 | 08:34 |
| mup | Bug #1841137: /dev/loopX devices left around for removed snap revisions <snapd:Confirmed for zyga> <https://launchpad.net/bugs/1841137> | 08:35 |
| mup | PR snapcraft#2725 opened: Update HACKING.md <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2725> | 08:56 |
| Chipaca | sergiusens: CLOSE ME? | 08:56 |
| sergiusens | Chipaca: just playing with autopkgtest | 08:57 |
| Chipaca | ah ok | 08:57 |
| mup | PR snapcraft#2725 closed: Update HACKING.md <Created by sergiusens> <Closed by sergiusens> <https://github.com/snapcore/snapcraft/pull/2725> | 08:59 |
| mup | PR snapd#7480 closed: store: download propagates options to delta download <Created by chipaca> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7480> | 09:26 |
| mup | PR snapd#7483 closed: docs: Add Code of Conduct <Created by chipaca> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7483> | 09:29 |
| Chipaca | verterok: on master, fwiw | 09:38 |
| verterok | cool | 09:38 |
| Chipaca | verterok: and on the 2.42 branch | 09:39 |
| verterok | \o/ | 09:40 |
| mup | PR snapd#7484 opened: osutil: generalize SyncDir with FileStater interface <Created by zyga> <https://github.com/snapcore/snapd/pull/7484> | 10:13 |
| zyga | dot-tobias: I'm investigating that issue now | 10:26 |
| dot-tobias | zyga: 🙏 | 10:30 |
| mup | PR snapd#7481 closed: data/selinux: allow snapd to issue sigkill to journalctl <Simple 😃> <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7481> | 11:39 |
| mup | PR snapd#7474 closed: release: 2.42~pre1 <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/7474> | 11:41 |
| jamesh | zyga: https://discourse.gnome.org/t/sandboxing-portal/1651/2 | 11:53 |
| zyga | thanks | 11:53 |
| mup | PR snapd#7476 closed: usersession/userd: make sure to export DBus interfaces before requesting a name <Created by bboozzoo> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/7476> | 12:14 |
| mup | PR snapd#7485 opened: data/selinux: allow snapd/snap to do statfs() on the cgroup mountpoint <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7485> | 12:26 |
| mup | PR snapcraft#2726 opened: Update README.md - Update information, shady staff <Created by JohnRLentonIsALyingBitch> <https://github.com/snapcore/snapcraft/pull/2726> | 12:26 |
| roadmr | ^^ has this been reported to github abuse? | 12:35 |
| diddledan | roadmr: I think we're probably heading that way if not already done | 12:40 |
| roadmr | I did :) | 12:40 |
| diddledan | we all know who it was | 12:40 |
| mup | PR snapcraft#2726 closed: Update README.md - Update information, shady staff <Created by JohnRLentonIsALyingBitch> <Closed by cjp256> <https://github.com/snapcore/snapcraft/pull/2726> | 12:47 |
| cwayne | roadmr: yep | 12:49 |
| zyga | dot-tobias: hello | 14:04 |
| zyga | dot-tobias: can you report snap version with the context of https://forum.snapcraft.io/t/snap-does-not-start-after-reboot/5750/10 aka https://bugs.launchpad.net/snapd/+bug/1844496 | 14:05 |
| mup | Bug #1844496: “Device or resource busy” error during snap refresh when using layout with variable <snapd:In Progress by zyga> <https://launchpad.net/bugs/1844496> | 14:05 |
| zyga | dot-tobias: I attempted to reproduce it on 2.42~pre1.1 without success (aka snapd from edge) | 14:05 |
| zyga | dot-tobias: I believe my test may be imperfect but I want to check with you first | 14:05 |
| zyga | dot-tobias: I wrote a small test, I'll run it against older release in a moment | 14:07 |
| joedborg | zyga: could you please give a once over to https://forum.snapcraft.io/t/apparmor-error-when-trying-to-connect-plugs/13303? | 14:10 |
| zyga | joedborg: sure | 14:10 |
| jdstrand | joedborg, I know what that is | 14:11 |
| jdstrand | zyga: ^ | 14:11 |
| zyga | ack | 14:11 |
| joedborg | jdstrand: ah awesome | 14:11 |
| jdstrand | joedborg: can you show me your snapcraft.yml? | 14:11 |
| zyga | thanks, I'm no longer looking unless jdstrand says otherwise | 14:12 |
| zyga | seems like something that instantiates the interface twice | 14:12 |
| zyga | so we get a duplicate | 14:12 |
| joedborg | https://www.irccloud.com/pastebin/BkZH0sq0/ | 14:12 |
| jdstrand | yes. and I think the two flavors might be used in the same app | 14:12 |
| joedborg | jdstrand: ^ | 14:12 |
| jdstrand | hmm | 14:13 |
| zyga | ks8-{kubelet,kubeproxy} are probably applied to a hook | 14:13 |
| jdstrand | joedborg: can you give me the snap.yaml from the snap? | 14:13 |
| jdstrand | zyga: oh, yes. | 14:13 |
| joedborg | from the actual built snap? | 14:14 |
| jdstrand | joedborg: did you add another hook beyond configure? | 14:14 |
| jdstrand | joedborg: nm, answer that ^ | 14:14 |
| joedborg | jdstrand: umm im not sure what that mean | 14:14 |
| joedborg | *means | 14:14 |
| zyga | joedborg: find meta/ | 14:15 |
| jdstrand | joedborg: what is in meta/hooks | 14:15 |
| zyga | jdstrand: in the snap | 14:15 |
| zyga | er, joedborg ^ | 14:15 |
| joedborg | ahhhhh, sorry, i see | 14:15 |
| joedborg | no, just configure | 14:15 |
| jdstrand | joedborg: can you paste unsquasfs -lls ./your.snap|grep meta | 14:16 |
| jdstrand | joedborg: err, unsquashfs | 14:16 |
| joedborg | https://www.irccloud.com/pastebin/hIMNXh1L/ | 14:17 |
| zyga | jdstrand: I'm surprised now | 14:17 |
| jdstrand | joedborg: perhaps put the snap somewhere so I can look? (and perhaps zyga) | 14:18 |
| jdstrand | joedborg: I need to step into a meeting in a moment | 14:18 |
| zyga | indeed | 14:19 |
| joedborg | no worries, 2 seconds | 14:19 |
| joedborg | i'll upload it to our google drive and drop the link | 14:24 |
| zyga | thank you | 14:26 |
| joedborg | https://drive.google.com/open?id=1fZXdTCW0cGHxGxSHNdkX0mQOa4OZCtAa | 14:29 |
| joedborg | zyga: ^ | 14:30 |
| zyga | downloading now | 14:30 |
| zyga | joedborg: I have the snap now | 14:37 |
| zyga | joedborg: reproduced | 14:38 |
| zyga | joedborg: can you please file a placeholder bug, I'll use it for tracking | 14:38 |
| jamesh | travis seems really slow today | 14:42 |
| joedborg | zyga: i have a guess what it might be - i'm rebuilding now | 14:42 |
| zyga | joedborg: whatever it really is it's a bug on snapd for sure | 14:43 |
| zyga | joedborg: can you share what your idea is? | 14:43 |
| zyga | unless the snap is manipulating profiles directly, that is | 14:43 |
| joedborg | zyga: i removed the kube-proxy app from the list of apps, but retained mentions to it elsewhere | 14:43 |
| zyga | I don't see that app mentioned in the yaml | 14:44 |
| zyga | (or that string at all) | 14:44 |
| zyga | oh | 14:45 |
| zyga | I see | 14:45 |
| zyga | the bug is as that this interface is then added to all the apps in the snap | 14:45 |
| zyga | jdstrand: ^ | 14:45 |
| mup | PR snapd#7486 opened: tests: add regression test for lp: #1844496 <Created by zyga> <https://github.com/snapcore/snapd/pull/7486> | 14:47 |
| zyga | joedborg: it's an invalid snap.yaml but the validation layer doesn't pick this up | 14:48 |
| zyga | well | 14:48 |
| zyga | not syntactically invalid | 14:49 |
| zyga | joedborg: I can file a bug now if you don't have one already | 14:49 |
| joedborg | zyga: sorry I was in a meeting, I can raise it unless you have? Yeah I agree, snapcraft should complain | 14:54 |
| zyga | joedborg: I'll handle that, thank you for reporting | 14:54 |
| zyga | joedborg: to unblock you, remove the "unused" interface declaration | 14:54 |
| zyga | joedborg: the one for k8s-proxy | 14:54 |
| zyga | and you should be good | 14:54 |
| joedborg | zyga: yeah it’s still building from that, I’ll let you know :) thanks! | 14:56 |
| zyga | joedborg, jdstrand: filed as https://bugs.launchpad.net/snapd/+bug/1844546 | 15:00 |
| mup | Bug #1844546: usage of multiple flavours of kubernetes-support interface for a single app is invalid but not validated <snapd:Confirmed> <https://launchpad.net/bugs/1844546> | 15:00 |
| jdstrand | zyga: note that microk8s strict already does this | 15:03 |
| zyga | jdstrand: hmm, does what exactly? | 15:04 |
| jdstrand | zyga: https://github.com/ubuntu/microk8s/blob/feature/strict-v2/snapcraft.yaml | 15:04 |
| jdstrand | zyga: uses both flavor of the kubernetes interface in the same snap (but not same command) | 15:05 |
| zyga | Right | 15:05 |
| zyga | That’s the issue here | 15:05 |
| jdstrand | zyga: note the 'plugs: []' which is why I asked about other hooks | 15:05 |
| jdstrand | zyga: I'm saying with microk8s, there wasn't the install issue | 15:05 |
| jdstrand | zyga: I didn | 15:06 |
| jdstrand | 't see what was different about the new k8s-worker | 15:06 |
| zyga | Ah | 15:06 |
| jdstrand | (yet, I'm in a meetin) | 15:06 |
| jdstrand | so I was confused why there was a problem | 15:06 |
| jdstrand | zyga: it is known (at least to me and samuele) that there is a problem if you don't specify plugs with hooks | 15:07 |
| jdstrand | not sure I filed it | 15:07 |
| jdstrand | zyga: but I couldn't see the difference between the yaml I pasted and the yaml joedborg pasted | 15:08 |
| jdstrand | zyga: gotta run | 15:08 |
| zyga | jdstrand: sure | 15:08 |
| zyga | jdstrand: read the bug once you have the time to see if you agree with my analysis | 15:08 |
| jdstrand | ok | 15:08 |
| zyga | thank you so much :) | 15:08 |
| jdstrand | zyga: briefly, what you decribe is definitely a bug (and related to the one I mentioned above), but that doesn't explain joedborg's afaics since he *did* use both k8s interfaces in two places and *did* use plugs in the configure hook | 15:23 |
| jdstrand | which should've avoided | 15:23 |
| jdstrand | it | 15:23 |
| jdstrand | meh, gotta go again | 15:24 |
| mup | PR snapd#7485 closed: data/selinux: allow snapd/snap to do statfs() on the cgroup mountpoint <Simple 😃> <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/7485> | 15:24 |
| mup | PR snapd#7477 closed: packaging: use snapfuse_ll to speed up snapfuse performance <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7477> | 17:43 |
| zyga | jdstrand: I think I have a different opinion on that but it's something I should revisit tomorrow with fresh eyes | 21:38 |
| jdstrand | joedborg (oh, the snap you put in drive doesn't have kube-proxy. you should k8s-kubeproxy from toplevel slots | 23:29 |
| jdstrand | meh | 23:29 |
| jdstrand | joedborg: that was for you, but I meant to (cc zyga) ^ :) | 23:30 |
| jdstrand | zyga: now I understand where you were coming from. the pastes had snap.yamls with a kube-proxy command. I'll comment in the but | 23:30 |
| jdstrand | bug* | 23:31 |
| joedborg | Yeah I figured that after telling zyga, I think the issue is that it passes validation | 23:31 |
| jdstrand | joedborg: yes. I'm mentioning that to unblock you | 23:31 |
| jdstrand | joedborg: sounds lik eyou didn't need me though. sorry for the delay. meeting then team dinner | 23:32 |
| joedborg | jdstrand: no worries! Thanks, hope Paris is going well. I also sent an email with a future blocker | 23:33 |
| joedborg | No rush, just FYI | 23:33 |
| jdstrand | ok | 23:34 |
| jdstrand | I'll comment in zyga's bug and see how much gas I have left in the tank :) | 23:34 |
| jdstrand | and yes, Paris is good :) | 23:34 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!