| Son_Goku | kyrofa, I think you've made the steps in snapcraft.yaml too magic | 03:00 |
|---|---|---|
| Son_Goku | to me, prep, build, etc should be stages where you explicitly describe what it should do | 03:00 |
| Son_Goku | rather than having to do the backwards thing of overriding default behaviors | 03:01 |
| === chihchun_afk is now known as chihchun | ||
| zyga | Caelum: do you have more info? | 05:05 |
| mborzecki | morning | 05:07 |
| zyga | Hey | 05:11 |
| zyga | Maciej, can you please boot tumbleweed | 05:11 |
| zyga | If you have one | 05:11 |
| mborzecki | don't have one installed atm | 05:12 |
| zyga | Ah, ok | 05:12 |
| zyga | Ill check | 05:12 |
| mborzecki | what's with tw? | 05:12 |
| zyga | Something broke apparently | 05:13 |
| zyga | Just woke up, will check soon | 05:13 |
| mup | PR snapd#5003 closed: cmd/snap-seccomp: graceful handling of non-multilib host <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/5003> | 05:14 |
| mborzecki | zyga: ok, let me know if you need help, i have a spare t60 i could try it on, though i don't remember if there's an hdd inside | 05:14 |
| zyga | I thought you have VMs on your beefy box :-) | 05:15 |
| mborzecki | zyga: cloud vms yes, graphical ones, not so much | 05:15 |
| zyga | re | 05:36 |
| mup | PR snapd#5028 opened: cmd/snap-seccomp: graceful handling of non-multilib host (2.32) <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5028> | 05:41 |
| mborzecki | trivial review ^^ | 05:44 |
| * zyga updates his opensuse tunmbleweed snapd to what is in system:snappy repo | 05:51 | |
| zyga | Caelum: I cannot see any issue | 05:55 |
| zyga | please tell me what you experienced | 05:55 |
| zyga | mborzecki: can you look at https://github.com/snapcore/snapd/pull/4957 | 05:59 |
| mup | PR #4957: cmd/snap-update-ns: remove the need for stash directory in secure bind mount implementation <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/4957> | 05:59 |
| zyga | 3rd review just to be sure | 06:00 |
| mborzecki | ok | 06:00 |
| * zyga -> breakfast | 06:05 | |
| zyga | chihchun: tee-hee-hee | 06:26 |
| chihchun | zyga: tee-hee-hee | 06:27 |
| zyga | I hope this boosts code reviews | 06:27 |
| chihchun | zyga: I guess you pinged the wrong person | 06:28 |
| zyga | oh | 06:28 |
| zyga | sorry, yes | 06:29 |
| zyga | chipaca sometimes has a nickname similar to the one you use | 06:29 |
| zyga | I missed that :) | 06:29 |
| chihchun | :) | 06:29 |
| zyga | morning mvo | 06:31 |
| mvo | zyga: good morning! how are you? | 06:31 |
| zyga | good morning, I'm not taking a longish walk today :) | 06:32 |
| zyga | so hopefully I'll make some code | 06:32 |
| mvo | zyga: :) | 06:40 |
| mvo | zyga: no reports yet about any issues with 2.32.3 afaict? | 06:49 |
| mvo | zyga: did you see anything? | 06:50 |
| zyga | no, nothing about 2.32.3 yet | 06:50 |
| mborzecki | mvo: hey, can we land #5028? | 06:54 |
| mup | PR #5028: cmd/snap-seccomp: graceful handling of non-multilib host (2.32) <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5028> | 06:54 |
| mup | PR snapd#5028 closed: cmd/snap-seccomp: graceful handling of non-multilib host (2.32) <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5028> | 07:07 |
| mborzecki | mvo: thanks! | 07:07 |
| mvo | np! 5026 needs a second review, very trivial | 07:08 |
| mup | PR snapd#5026 closed: tests: add check for OOM error after each test <Created by mvo5> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/5026> | 07:11 |
| mup | PR snapd#4987 closed: tests: add test to ensure `snap refresh --amend` works with different channels <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4987> | 07:12 |
| mvo | mborzecki: 5024 looks very good, just some testing tweaks and it is golden | 07:12 |
| === pstolowski|afk is now known as pstolowski | ||
| pstolowski | mornings | 07:12 |
| zyga | morning | 07:13 |
| mvo | hey | 07:13 |
| mborzecki | mvo: yes, thanks for you suggestions, i'll be pushing a patch soon | 07:13 |
| mborzecki | pstolowski: hello | 07:13 |
| mvo | and a second look at 5020 would be great, sorry for being pushy, want to get the SRU out this morning :) | 07:13 |
| mvo | hey pstolowski | 07:13 |
| kalikiana | good morning | 07:17 |
| zyga | hey ho | 07:17 |
| kalikiana | o/ zyga | 07:17 |
| mup | PR snapd#4845 closed: snap, store: store version numbers in the commands database <Squash-merge> <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/4845> | 07:19 |
| mvo | mborzecki: curious why is 4942 marked as blocked? it looks like its ready to go in, no? modulo the small ugliness in the journal but thats ok IMO for now | 07:20 |
| mborzecki | mvo: heh, forgot about the label :) let me fix that | 07:21 |
| mborzecki | mvo: squash merge right? | 07:21 |
| mvo | mborzecki: yes please | 07:24 |
| mup | PR snapd#4942 closed: cmd/snap: user session application autostart v3 <Squash-merge> <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/4942> | 07:27 |
| mup | PR snapd#5029 opened: cmd/snap: user session application autostart v3 (2.32) <Created by mvo5> <https://github.com/snapcore/snapd/pull/5029> | 07:34 |
| mup | PR snapd#5030 opened: packaging/amzn2: initial packaging of 2.32.3 for Amazon Linux 2 <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5030> | 07:34 |
| mborzecki | need more coffee | 07:35 |
| mup | PR snapd#5020 closed: errtracker: check for whoopsie.service instead of reading /etc/whoopsie <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5020> | 07:35 |
| zyga | mborzecki: question about 5030, why are we using /var/lib/snapd/snap there? | 07:37 |
| mborzecki | zyga: it's very rhel like and also lists `ID_LIKE="centos rhel fedora"` in /etc/os-release | 07:38 |
| zyga | so just because? | 07:39 |
| mborzecki | zyga: yeah, we already have the right switches to cover rhel/fedora | 07:39 |
| zyga | sure but we can add a new distro to the list | 07:40 |
| zyga | and just use /snap | 07:40 |
| mup | PR snapd#5031 opened: errtracker: check for whoopsie.service instead of reading /etc/whoopsie (2.32) <Created by mvo5> <https://github.com/snapcore/snapd/pull/5031> | 07:41 |
| mborzecki | zyga: no strong opinions here, this just saves vetting DistroLike switches and there's one rpmlint warning less | 07:43 |
| zyga | I'd go for /snap | 07:43 |
| zyga | mborzecki: added three comments | 07:49 |
| mborzecki | zyga: thanks | 07:49 |
| zyga | willcooke: hey, there's an issue that skype crashes wayland on startup | 08:01 |
| zyga | is that something on anyone's radar on your team? | 08:01 |
| willcooke | zyga, oh, no, not something I was aware of | 08:02 |
| Chipaca | zyga: why do you torture yourself with wayland :-) | 08:03 |
| zyga | willcooke: just login into wayland ssh in and look at the journal | 08:03 |
| zyga | run Skype and boom | 08:03 |
| zyga | Chipaca: x11 corrupts screen on my thinkpad | 08:03 |
| zyga | Chipaca: not sure why, wayland works better for that | 08:03 |
| zyga | (apart from taking down gnome-session when things go) | 08:03 |
| Chipaca | zyga: intel board? | 08:03 |
| om26er | popey: ping | 08:03 |
| zyga | Chipaca: my thinkpad | 08:03 |
| popey | om26er: good morning | 08:04 |
| om26er | good day popey :) | 08:04 |
| popey | om26er: I spoke to the store team, turns out we cannot rename snaps, so we should publish s-t and remove s-t-3 | 08:04 |
| Chipaca | zyga: btw is the fix for #1760841 in a core already? | 08:04 |
| mup | Bug #1760841: snapd does not parse /etc/fstab properly when using mhddfs <Snappy:Fix Committed by zyga> <https://launchpad.net/bugs/1760841> | 08:04 |
| zyga | Chipaca: yes, it's in .3 | 08:04 |
| om26er | aha | 08:05 |
| om26er | popey: so lets merge that one then ? | 08:05 |
| om26er | well actually my current "ping" was for https://github.com/snapcrafters/android-studio/pull/19 :) | 08:05 |
| mup | PR snapcrafters/android-studio#19: Fix app startup based on latest snapcraft changes <Created by om26er> <https://github.com/snapcrafters/android-studio/pull/19> | 08:05 |
| om26er | popey: android studio release that we pushed yesterday won't start so this fixes it | 08:08 |
| om26er | speaking of which, I think if we(I) enable the CI for that snap, we won't have that kind of issue again. | 08:09 |
| om26er | "build and try to run" | 08:10 |
| zyga | Chipaca: some comments on 5027 | 08:17 |
| zyga | and sorry for the branch summary change ;) | 08:17 |
| zyga | lol :D | 08:18 |
| zyga | 10 best patches you read this week | 08:18 |
| willcooke | zyga, confirmed. LP: #1762954 | 08:22 |
| mup | Bug #1762954: Running the skype snap under the Wayland session crashes the whole session <amd64> <apport-bug> <bionic> <wayland-session> <gnome-shell (Ubuntu):New> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1762954> | 08:22 |
| zyga | thank you! | 08:22 |
| willcooke | np, thanks for the ping | 08:22 |
| zyga | probably a good chunk of our users will just use wayland for some reason and will bump into this | 08:23 |
| zyga | thanks for the review on 4957 | 08:24 |
| zyga | I'll get right to it | 08:24 |
| Chipaca | willcooke: zyga: FWIW I think there's already a bug for that issue | 08:31 |
| Chipaca | or is it an issue for that bug | 08:31 |
| Chipaca | ah, maybe i'm thinking of https://forum.snapcraft.io/t/skype-crashes-gnome-on-ubuntu-18-04/4927 | 08:32 |
| mup | PR snapd#5031 closed: errtracker: check for whoopsie.service instead of reading /etc/whoopsie (2.32) <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5031> | 08:33 |
| Chipaca | or https://forum.snapcraft.io/t/vs-code-makes-shell-crash/4362 | 08:35 |
| Chipaca | willcooke: zyga: from that last one, https://bugs.launchpad.net/snappy/+bug/1760252 | 08:36 |
| mup | Bug #1760252: starting slack crashes xwayland on 18.04 <bionic> <Snappy:New> <https://launchpad.net/bugs/1760252> | 08:36 |
| willcooke | so generally classic snaps then | 08:36 |
| Chipaca | on wayland | 08:36 |
| Chipaca | i mean, it's wayland (or xwayland?) that crashes | 08:36 |
| zyga | maybe bundled lib input? | 08:36 |
| willcooke | yeah, was just looking at that, seems like it could be xwayland | 08:37 |
| * willcooke uploading stack traces | 08:37 | |
| zyga | it was xwayland from what I've seen | 08:37 |
| Chipaca | call me ornery but imo if a client can crash wayland, there might be a bug in the client but there is a bug in wayland | 08:38 |
| willcooke | one or two | 08:38 |
| Chipaca | :-) | 08:38 |
| Caelum | zyga: sorry, looks like the problem disappeared | 08:38 |
| zyga | no worries, what was the problem? | 08:39 |
| Caelum | snaps weren't starting from gnome app menu | 08:39 |
| willcooke | k, hopefully LP will do some retracing and I will tidy up all the various bugs once that's happened. | 08:42 |
| willcooke | Off to the office now, will be back later on | 08:42 |
| zyga | sure, thank you willcooke | 08:42 |
| seb128 | willcooke, you are on a wayland session? | 08:42 |
| willcooke | seb128, I was to test that | 08:43 |
| seb128 | oh, k | 08:43 |
| seb128 | thx | 08:43 |
| * Chipaca is off to the dentist's. ttfn! | 08:52 | |
| popey | om26er: ok. | 09:09 |
| === chihchun is now known as chihchun_afk | ||
| === chihchun_afk is now known as chihchun | ||
| mup | PR snapd#4957 closed: cmd/snap-update-ns: remove the need for stash directory in secure bind mount implementation <Created by jhenstridge> <Merged by zyga> <https://github.com/snapcore/snapd/pull/4957> | 09:46 |
| mup | PR snapd#5032 opened: repo: pass and return ConnRef via pointers <Created by stolowski> <https://github.com/snapcore/snapd/pull/5032> | 09:46 |
| zyga | pstolowski: why by pointer? | 09:47 |
| pstolowski | zyga: to save on passing, that was pointed out by pedronis in interface hooks review | 09:48 |
| zyga | mvo: is .4 ready? | 09:54 |
| mvo | zyga: .3.1 | 09:55 |
| mvo | zyga: yes it just got pushed to xenial-proposed and once my test builds finished also to the other releases | 09:56 |
| zyga | 3.1? is that a deb only or a real upstream release? | 09:56 |
| mvo | zyga: its pretty small and *finally* use git-buildpackage which makes me incredible happy | 09:56 |
| zyga | nice, thank you! | 09:56 |
| mvo | zyga: its deb only but we could still put it into the beta channel for consistency | 09:56 |
| zyga | is that only the OOM thing? | 09:57 |
| mvo | zyga: either way .4 needs to wait a little bit until we are sure we don't need to do anything for 2.32.3, i.e. no regressions | 09:57 |
| mvo | zyga: a little bit more, one sec | 09:57 |
| zyga | should it be on releases page on GitHub? | 09:57 |
| mvo | zyga: https://github.com/snapcore/snapd/blob/2.32.3.1/packaging/ubuntu-16.04/changelog | 09:58 |
| zyga | mvo: let me know when you push the tag | 09:58 |
| mvo | zyga: probably, I'm a bit hesitant because .4 will happen soon and I don't want to put too much burden on the downstream packagers but otoh I think its more correct to have a release on GH | 09:58 |
| mvo | zyga: it is pushed | 09:58 |
| zyga | I can drop it on the release page | 09:58 |
| zyga | thanks! | 09:59 |
| mvo | zyga: also the release script can now be simplified because the orig.tar.xz is sane now (snapd-2.32.3.1) | 09:59 |
| zyga | whee, good | 09:59 |
| zyga | so orig.tar.gz doesn't need renaming internally | 10:00 |
| mvo | zyga: yeah, I upload the file to the ppa in a sec | 10:02 |
| pedronis | mvo: we should discuss at the standup when to cut .4 | 10:06 |
| mvo | pedronis: +1 | 10:07 |
| mvo | pedronis: my strawman would be tomorrow morning | 10:07 |
| mvo | pedronis: but happy to discuss | 10:07 |
| mborzecki | 14.04 journalctl, no --show-cursor, no --after-cursor, no --identifier, damn | 10:16 |
| zyga | yeah | 10:19 |
| zyga | lack of cursor sucks | 10:19 |
| mborzecki | does anyone mind not running user session app autostart spread test on 14.04? | 10:19 |
| zyga | it's fine, 14.04 was supposed to be server-only | 10:22 |
| mborzecki | mvo: want me to look into #5029? | 10:28 |
| mup | PR #5029: cmd/snap: user session application autostart v3 (2.32) <Created by mvo5> <https://github.com/snapcore/snapd/pull/5029> | 10:28 |
| zyga | mborzecki: want to look at https://github.com/snapcore/snapd/pull/5033 | 10:39 |
| mup | PR #5033: cmd: generalize locking to global, snap and per-user locks <Created by zyga> <https://github.com/snapcore/snapd/pull/5033> | 10:39 |
| zyga | it's very straightforward | 10:39 |
| mup | PR snapd#5033 opened: cmd: generalize locking to global, snap and per-user locks <Created by zyga> <https://github.com/snapcore/snapd/pull/5033> | 10:39 |
| mup | PR snapd#5034 opened: userd: set up journal logging streams for autostarted apps <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5034> | 11:07 |
| pstolowski | mborzecki: hey, how do we go vendor stuff in fedora? it's seems to be the only one unhappy about my new dependencies (go-udev..)? | 11:14 |
| mborzecki | pstolowski: which pr? | 11:15 |
| zyga | pstolowski: we package | 11:15 |
| pstolowski | mborzecki: https://github.com/snapcore/snapd/pull/4940 | 11:15 |
| mup | PR #4940: RFC: added UDevMonitor for future hotplug support <Created by stolowski> <https://github.com/snapcore/snapd/pull/4940> | 11:15 |
| pstolowski | zyga: package as separate rpm for every go package we need? | 11:16 |
| zyga | pstolowski: package according to the way we need to in fedora, yes | 11:16 |
| mborzecki | pstolowski: btw. do have a solution for go-dev already? other deps from from rps | 11:16 |
| mborzecki | and iirc go-udev was not packaged for fedora yet | 11:17 |
| pstolowski | mborzecki yes, that's very probable, it's a small and not very popular project i think | 11:17 |
| zyga | gofed is pretty nice | 11:17 |
| pstolowski | zyga. mborzecki ok, if that's the case then I think I need a general +1 on the PR first before investing time into packaging (it's ~3 go packages as go-udev pulls others) | 11:18 |
| mborzecki | anyways i think the tests were using vendored deps on fedora | 11:18 |
| mborzecki | pstolowski: do you know what the transitive deps are? govendor list -v should show it | 11:19 |
| Son_Goku | mborzecki, they'd better not be | 11:28 |
| Son_Goku | I delete the vendor directory in prep | 11:28 |
| Son_Goku | I've already had that happen once and found out that we were testing Fedora wrong | 11:29 |
| mborzecki | Son_Goku: yeah, saw that, i'm thinking of something to just unblock spread in that pr | 11:29 |
| mborzecki | or maybe we should just package go-udev for fedora and submit it for review | 11:29 |
| Son_Goku | if you package the godeps, I'll review and merge them into the archive | 11:29 |
| Son_Goku | as zyga knows, I can be very fast with go package reviews | 11:30 |
| zyga | yes | 11:30 |
| Son_Goku | and go deps are easy to package because the gofed tool does nearly all the work | 11:30 |
| mborzecki | i can probably take a look, unless pstolowski you want to give it a 'go' :) | 11:31 |
| pstolowski | mborzecki: happy to do it, although as I said perhaps it makes sense to get general +1 on using go-udev first, because it hasn't been decided yet | 11:32 |
| Son_Goku | dependencies aren't free :) | 11:33 |
| mborzecki | pstolowski: hm thought we were already good with using it | 11:33 |
| mup | PR snapcraft#2058 closed: python plugin: install python-distutils when run on bionic <bug> <Created by bjornt> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2058> | 11:35 |
| pstolowski | mborzecki: I'm not sure, Gustavo hasn't yet commented on the branch (and on the HLD) | 11:35 |
| pstolowski | Son_Goku: thanks for the hints! | 11:36 |
| mborzecki | gofed is in .... python? | 11:37 |
| Son_Goku | yes | 11:37 |
| Son_Goku | most of Fedora tooling is in Python | 11:37 |
| Son_Goku | most of openSUSE tooling is in ruby :P | 11:37 |
| Son_Goku | and most of Debian tooling is in Perl | 11:38 |
| mup | PR snapcraft#2062 closed: packaging: simplify snapcraft.yaml <enhancement> <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2062> | 11:47 |
| mborzecki | Son_Goku: gofed generates a busted spec, https://paste.debian.net/1019710/ there's no newline after %changelog | 11:57 |
| Son_Goku | I know :/ | 11:58 |
| Son_Goku | the changelog entry is also wrong | 11:58 |
| mborzecki | other than that the rpm gets built :) | 11:59 |
| mborzecki | off to pick up the kids, coming back for standup | 12:01 |
| pedronis | cachio: mvo: I'm looking again at tests again staging, seems we need to copy canonical-livepatch to staging, and test-snapd-only-in-edge | 12:02 |
| pedronis | we probably want to copy core from edge to edge as well | 12:03 |
| cachio | pedronis, sure,I'll do it | 12:07 |
| pedronis | cachio: once we have solved the cred problems with spread-cron, we should revisit the various store tests branches there and add back staging I think | 12:08 |
| cachio | pedronis, ok, we could have a nightly exec | 12:08 |
| pedronis | I think we could trigger on store deploys | 12:08 |
| pedronis | but we'll see | 12:09 |
| pedronis | probably next week at this point | 12:09 |
| cachio | ok | 12:09 |
| Chipaca | whoo, dentist anesthetic waring off | 12:25 |
| * Chipaca is not having fun | 12:25 | |
| Chipaca | wearing off* | 12:25 |
| cachio | pedronis, my internet is so slow today | 12:41 |
| cachio | it is taking long time to upload the core | 12:41 |
| === sergiusens_ is now known as sergiusens | ||
| cachio | pedronis, snaps ready | 12:59 |
| Chipaca | pedronis: standup? | 13:03 |
| mborzecki | something new: 2018-04-11 11:08:43 Cannot allocate linode:fedora-27-64: cannot allocate new Linode server for fedora-27-64: no open slots for this plan!t | 13:13 |
| mborzecki | pstolowski: sorry, i've just restarted the travis build in 4940 | 13:25 |
| pstolowski | mborzecki: np, it's going to fail unless you pushed something to the branch? | 13:32 |
| mborzecki | pstolowski: no, i did not, it's going to fail ;) | 13:33 |
| pstolowski | ok ;) | 13:33 |
| pedronis | mvo: should IÂ squash-merge #5002 given that it has many commits? otoh it means that the next two might need to be changed because of conflicts with the squash | 14:11 |
| mup | PR #5002: many: use the new install/refresh /v2/snaps/refresh store API (2.32) <Critical> <Created by pedronis> <https://github.com/snapcore/snapd/pull/5002> | 14:11 |
| mvo | pedronis: no squash please | 14:11 |
| pedronis | ok | 14:11 |
| mvo | pedronis: this will be a super messy merge back otherwise | 14:11 |
| mvo | pedronis: thanks | 14:12 |
| mup | PR snapd#5002 closed: many: use the new install/refresh /v2/snaps/refresh store API (2.32) <Critical> <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/5002> | 14:13 |
| pedronis | mvo: merging | 14:13 |
| mup | PR snapd#5022 closed: overlord/snapstate: on multi-snap refresh make sure bases and core are finished before dependent snaps (2.32) <Critical> <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/5022> | 14:14 |
| pedronis | mvo: my 2.32 stuff is merged | 14:16 |
| mup | PR snapd#5021 closed: overlord/snapstate: introduce envvars to control the channels for bases and prereqs (2.32) <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/5021> | 14:16 |
| mvo | pedronis: thank you | 14:18 |
| mvo | mborzecki: 5029 needs a review - just double checking that its the right commit etc | 14:19 |
| mup | PR snapd#4840 closed: many: add `core.problem-reports.disabled` option <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4840> | 14:20 |
| * cachio afk | 14:26 | |
| mup | PR snapd#5029 closed: cmd/snap: user session application autostart v3 (2.32) <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5029> | 14:29 |
| sergiusens | Wimpress and kenvandine does this (https://forum.snapcraft.io/t/snap-application-and-snap-themes/4946) not somewhat align with what was discussed around a month ago? That theme does a bit more, I'll let you comment on that though :-) | 14:39 |
| kenvandine | sergiusens, yes, i replied with a link to jamesh's post | 14:46 |
| mup | PR snapd#5035 opened: release: snapd 2.32.4 <Created by mvo5> <https://github.com/snapcore/snapd/pull/5035> | 14:49 |
| zyga | Chipaca: hey, could you look at https://github.com/snapcore/snapd/pull/5033 | 15:01 |
| mup | PR #5033: cmd: generalize locking to global, snap and per-user locks <Created by zyga> <https://github.com/snapcore/snapd/pull/5033> | 15:01 |
| zyga | nothing major | 15:01 |
| sergiusens | kenvandine: your the man! | 15:03 |
| sergiusens | 're | 15:03 |
| diddledan | who fancies digging into snapcraft? https://forum.snapcraft.io/t/error-while-building-argument-list-too-long/4948 | 15:12 |
| kyrofa | diddledan, yuck | 15:14 |
| diddledan | I aim to please :-p | 15:14 |
| kyrofa | diddledan, can you try on edge? | 15:14 |
| diddledan | I did | 15:14 |
| diddledan | the second paste | 15:14 |
| kyrofa | diddledan, scriptlets give better errors there | 15:14 |
| kyrofa | Oh | 15:14 |
| * kyrofa scrolls down further | 15:14 | |
| kyrofa | Dang, still not helpful | 15:15 |
| kyrofa | diddledan, but yeah, I'll own this one | 15:16 |
| diddledan | yey | 15:16 |
| diddledan | did you look at the yaml yet? ;-) | 15:16 |
| diddledan | a mere snip at 1350 lines | 15:16 |
| kyrofa | diddledan, no, you're scaring me | 15:16 |
| * kyrofa cries | 15:16 | |
| diddledan | haha | 15:17 |
| * diddledan cuddles kyrofa | 15:17 | |
| Chipaca | diddledan: hey, at least three of those lines are comments now | 15:17 |
| diddledan | :-p | 15:18 |
| kyrofa | diddledan, ignoring the actual cause, the fact that this is so opaque is problematic. In an ideal world, how would this be presented to you? | 15:25 |
| diddledan | well to begin with I think actually telling me what command was being executed would help me to understand what it's doing when it fails | 15:25 |
| === chihchun is now known as chihchun_afk | ||
| diddledan | at least then I'd be able to determine whether there is a workaround while the problem gets fixed | 15:26 |
| diddledan | I guess it would also help me discern whether it's my fault with wonky build configuration or an endemic problem with snapcraft itself | 15:28 |
| Chipaca | zyga: is #5033 part of not landing new stuff? | 15:29 |
| mup | PR #5033: cmd: generalize locking to global, snap and per-user locks <Created by zyga> <https://github.com/snapcore/snapd/pull/5033> | 15:29 |
| pedronis | Chipaca: it was proposed in the morning | 15:29 |
| niemeyer | pstolowski: Do you want to have a call in a few mins to discuss it? | 15:33 |
| zyga | haha, but that was posted in the morning | 15:33 |
| zyga | :) | 15:33 |
| zyga | no more newies | 15:33 |
| pstolowski | niemeyer: what is that? | 15:34 |
| niemeyer | We tried to have a call on Monday but it didn't work.. wondering if you'd like to discuss now.. there's not hurry since per the meeting we should be stabilizing until next week | 15:35 |
| pedronis | mvo: I completely missed that 2.32.4 is now in beta | 15:37 |
| noise][ | pedronis: .4 has new refresh API? | 15:38 |
| pedronis | noise][: yes | 15:39 |
| pstolowski | niemeyer: ah, about udev? if so then yes, i'd be happy to do it in a moment | 15:40 |
| mvo | pedronis: it happend ~15sec ago | 15:45 |
| mvo | pedronis: ok, maybe a bit longer but armhf literally finished in the last 5min | 15:45 |
| pedronis | I just saw it in info for amd64 | 15:45 |
| mvo | pedronis: yeah, that finished ~ 15min ago or so | 15:46 |
| mvo | jdstrand: bad news, the issue https://forum.snapcraft.io/t/oom-for-interfaces-many-on-bionic-i386/4101 is back it seems https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/s/snapd/20180411_152411_beda4@/log.gz is the most recent autopkgtest, we added a check for lowmem and it seems like its running out of low kernel mem during the tests | 15:49 |
| zyga | :-( | 15:49 |
| mvo | jdstrand: for refrence this is from http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#snapd | 15:50 |
| zyga | low memory is that special area that is below 4G (not-PAE) or is that the special memory limit so that PCI devices can map it with their limited address space? | 15:50 |
| zyga | jdstrand, jjohansen: do we have any news on the kernel memory leak when apparmor profiles are unloaded/reloaded | 15:51 |
| zyga | mvo: I have alternative solution that would make that test pass | 15:51 |
| zyga | mvo: but it's new code and definitely risky at this stage | 15:51 |
| zyga | mvo: I talked about this to jdstrand without association to the leak issue that I was unaware of at the time | 15:51 |
| zyga | mvo: I need to double check by looking at the test first | 15:52 |
| zyga | but the general idea is that we would defer apparmor parser invocation until end of transaction | 15:52 |
| mup | Bug #1763071 opened: Error message installing a paid snap as unauthenticated user <Snappy:New> <https://launchpad.net/bugs/1763071> | 15:52 |
| zyga | mvo: now we run it way more often than needed, in some cases | 15:53 |
| mvo | cachio: all well with the sru validation so far? | 15:54 |
| zyga | mvo: actually, I think it would not help with this issue but let me read this more carefully :/ | 15:55 |
| zyga | mvo: no, it would not help here | 15:57 |
| zyga | Son_Goku: 2.32.4 is out | 15:58 |
| zyga | :-) | 15:58 |
| zyga | Son_Goku: how is that server WG proposal coming along | 15:58 |
| zyga | can I help in writing down the plan somehwere? | 15:59 |
| Son_Goku | I will have something drafted in a few minutes | 16:00 |
| Son_Goku | gotta grab lunch :) | 16:00 |
| mvo | zyga: thanks, well, we need to figure out what is going on. it also relatively new | 16:00 |
| zyga | aww, my pohne just died | 16:01 |
| mvo | zyga: lets see if we also see it in artful | 16:01 |
| zyga | Son_Goku: awesome, I'll say in touch | 16:01 |
| mvo | zyga: that can't be - its an iphone ;) | 16:01 |
| zyga | I forgot to charge it yesterday | 16:01 |
| zyga | mvo: but I take the joke | 16:01 |
| zyga | mvo: still, way happier with it than with my android phones | 16:02 |
| mvo | zyga: I know, just teasing | 16:02 |
| zyga | mvo: the current software shows battery health now and it claims my battery is at 83% | 16:03 |
| zyga | not so low that I'd be tempted to replace it yet | 16:03 |
| * zyga -> grocieres | 16:04 | |
| zyga | mvo: and btw, thinkpad modems are worth every penny :) | 16:04 |
| zyga | I love working outside | 16:04 |
| * mvo hugs zyga | 16:04 | |
| popey | mvo: i have had a user ping me that they have a serious issue when installing a snap and had a full lock up and reboot | 16:36 |
| popey | wondered if i get them in here you or someone else on the team might be able to help if you're around | 16:36 |
| popey | the machine may have state info on it that could help debugging perhaps? | 16:37 |
| mup | PR snapd#4832 closed: tests: move fedora 27 to google backend <Created by sergiocazzolato> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/4832> | 16:37 |
| pedronis | popey: do we know with what version and what snap? | 16:38 |
| pedronis | a full lock and reboot is quite a lot | 16:38 |
| pedronis | IÂ mean snapd version | 16:38 |
| popey | pedronis: a snap he made, he installed devmode | 16:38 |
| popey | not yet, he's booting a live cd now | 16:38 |
| popey | will hopefully drop by here soon | 16:38 |
| ogra_ | out of diskspace ? | 16:38 |
| === jkridner|pd is now known as jkridner_ | ||
| === jkridner_ is now known as jkridner__ | ||
| popey | we'll see. | 16:41 |
| === jkridner__ is now known as jkridner | ||
| * zyga is back from shopping | 16:45 | |
| zyga | popey: I could help | 16:45 |
| zyga | popey: ping me if you have anything I can jump on | 16:46 |
| popey | thanks. he's having trouble getting online | 16:46 |
| * zyga hopes vmware workstation will work on the next wave of LTS distros | 16:49 | |
| zyga | I cannot use the zoo of libvirt-based things as they all are more or less broken in more or less obvious ways :/ | 16:50 |
| pstolowski | Son_Goku: what's the process of proposing go-udev package in fc once i've it ready (currently spinning fc27 vm up)? do you have a wiki describing it? | 16:52 |
| Son_Goku | yes | 16:52 |
| suebt | Howdy, popey led me here: I ran into the following issue with snap: I installed a package I created (daemon snap, devmode) but unfortunately it led to completely locking up/freezing the whole system. After some time I did a hardware shutdown and tried to reboot. The system starts until displaying the cursor but then completely locks up/freezes again (tried several times now). Is there anything I can do to try debug this issue before tr | 16:53 |
| popey | zyga: ^ | 16:54 |
| Son_Goku | pstolowski: https://fedoraproject.org/wiki/Package_Review_Process | 17:05 |
| pstolowski | Son_Goku: thanks | 17:05 |
| Son_Goku | pstolowski, zyga and kyrofa can walk you through it too, as they've both done it | 17:06 |
| kyrofa | pstolowski, yeah, docs are pretty good, let me know if you want any help | 17:06 |
| jdstrand | mvo (cc zyga and jjohansen): we never did anything about that leak. the conclusion was that while there was a small apparmor leak, it was just that. I recorded that in the forum. jjohansen was going to look at the small leak | 17:10 |
| jdstrand | mvo (cc jjohansen): zyga's idea of batching the loads makes a lot of sense to me-- I think it would might help with timeouts in the spread tests (separate issue) | 17:11 |
| jdstrand | s/would might/might/ | 17:11 |
| pstolowski | thanks kyrofa! I think i'll attack this tomorrow morning; do you have a link to any of your specs? i wonder how much cleanup is expected from the gofed-generated spec | 17:11 |
| kyrofa | pstolowski, totally, although I'm not sure I'd suggest mine as model specs, haha. Let me get them for you | 17:11 |
| pstolowski | kyrofa: if it passed the review, it's model ;) | 17:12 |
| kyrofa | :D | 17:12 |
| Son_Goku | https://src.fedoraproject.org/user/zyga | 17:12 |
| Son_Goku | you can see zyga's packages | 17:12 |
| Son_Goku | and kyrofa's: https://src.fedoraproject.org/user/kyrofa | 17:12 |
| kyrofa | Yep, there you go. With git repos to everything | 17:13 |
| pstolowski | awesome, thanks! | 17:13 |
| Son_Goku | and then mine... https://src.fedoraproject.org/user/ngompa | 17:14 |
| kyrofa | Don't look at Son_Goku's, they'll make you feel inferior | 17:14 |
| Son_Goku | :D | 17:15 |
| pstolowski | lol | 17:15 |
| pedronis | so we found another instance where there reading of snaps ignoring errors is hiding other problems | 17:16 |
| popey | i believe zyga is away, pedronis are you able to help suebt ? | 17:21 |
| pedronis | popey: probably needs to boot into emergency mode and look around at logs , it's a bit hard to understand what from snapd would take over the system so much | 17:26 |
| suebt | Hey pedronis. I'm currently logged into a live session on the machine. Can I access logs from there? | 17:27 |
| ogra_ | suebt, that sounds a bit like your daemon simply goes mad | 17:29 |
| ogra_ | do you have the surce for that snap public ? | 17:29 |
| suebt | Yeah, I assume so. Could be constantly rebooting or something. Wondering whether snapd doesn't stop it at some point from doing that, though? | 17:30 |
| ogra_ | *source | 17:30 |
| Caelum | zyga: you said there was some other gnome software that needs packaging, can I help with this? | 17:30 |
| suebt | Nope, it's neither public nor finished yet, unfortunately, was just the first try of snapping it up ._. | 17:31 |
| ogra_ | there are some self checks for snaps, but if the daemon first starts fine and then ... say ... fills up all RAM, snapd wont be able to do anything about that | 17:31 |
| suebt | s/rebooting/restarting/ | 17:31 |
| suebt | ogra_: yeah, right ... | 17:31 |
| pedronis | suebt: once is installed snapd mostly let the management be done by systemd | 17:31 |
| pedronis | for services | 17:31 |
| ogra_ | yeah | 17:31 |
| suebt | Okay, is there any log I could check to find out? | 17:31 |
| ogra_ | you should be able to just remove the systemd units from disk | 17:31 |
| ogra_ | from your live session | 17:32 |
| ogra_ | then you can reboot into your normal system | 17:32 |
| pedronis | /var/log/syslog and then you would need to point journalctl from the live session to the logs on your main disk | 17:32 |
| pedronis | which should be possible, but IÂ never tried myself it seems (I see there are relevant options though) | 17:33 |
| ogra_ | well, syslog from disk should be enough | 17:33 |
| ogra_ | at least if the daemon logs anything | 17:33 |
| ogra_ | if it just goes crazy in I/O or fills your ram (just enough to not hit OOM) you might not see anything at all | 17:34 |
| pedronis | otherwise it seems journalctl --root=your/main/disk/root .. could help | 17:34 |
| suebt | Okay, thanks, I'm not seeing anything helpful, just a bunch of 00\00\00\00\00\00 at the end | 17:34 |
| suebt | Okay, well, when there are no additional snap-specific logs that could be helpful, I'll try to restore my system as you proposed ogra_. Thanks! | 17:36 |
| pedronis | suebt: btw it seems you can also disable your daemon using systemctl --root=... disable snapd.snap-service... | 17:37 |
| ogra_ | snaps usually just log to logger ... which writes to syslog and journald ... | 17:37 |
| ogra_ | oh, wow, i didnt know that one | 17:37 |
| suebt | oh ok, thanks ogra_ pedronis | 17:37 |
| pedronis | ogra_: yea, haven't quite tried but apparently both journactl and systemctl can take an alternative root dir | 17:38 |
| ogra_ | very cool | 17:38 |
| suebt | pedronis: systemctl --root worked just fine, so that was easier than I thought it would be. Thanks! Okay, will report in case I find anything interesting which is not related to my application going wild for some reason .-. | 17:42 |
| ogra_ | make a wrapper script around your daemon that reports ram and disk usage or some such ... and prehaps add a timeout so you dont kill the system again | 17:44 |
| zyga | re | 17:46 |
| zyga | I'm home | 17:46 |
| Caelum | zyga: hey, you said earlier we need to package some other things for OS, can I help with any of that? | 17:46 |
| zyga | yes, there is a glib wrapper for talking to snapd that is a dependency of gnome-software | 17:47 |
| zyga | but I think we should see what it would take to submit the package to factory | 17:47 |
| zyga | as the extra package I'm talking about won't work without snapd | 17:48 |
| Caelum | ah ok | 17:48 |
| Caelum | well let me know if you need help with anything | 17:48 |
| pedronis | zyga: seems there's a real bug where during install we think a snap is mounted but is not, somebody hit it with beta, but it's not a regression (it fails strangely because of the ignore error code we have in readInfo) | 17:51 |
| pedronis | it's rare though we have a report now and one from long ago , Chipaca might know more | 17:53 |
| zyga | pedronis: interesting, so we think it is mounted and then go an and try to use it | 17:56 |
| zyga | pedronis: I added some code that prevents snaps with snapInfo.Broken from entering the repository | 17:56 |
| zyga | pedronis: but it is a new thing that landed in 2.32.3 first | 17:56 |
| zyga | Caelum: I think we should send a mail out to the packaging mailing list | 17:56 |
| zyga | Caelum: and propose the current incarnation of snapd | 17:56 |
| zyga | Caelum: and see what the response is | 17:56 |
| pedronis | zyga: but it fails much earlier in this case, we really need to revist readInfo | 17:56 |
| pedronis | and really decide when returning broken vs error makes sense | 17:57 |
| zyga | Caelum: I bet we will get some quick things like "change that, fix this" etc | 17:57 |
| zyga | pedronis: I agree | 17:57 |
| pedronis | zyga: we did to support listing and removals | 17:57 |
| zyga | pedronis: but I also think this is a secondary effect, we should understand what is wrong with mounting | 17:57 |
| zyga | pedronis: yes, I remember | 17:57 |
| pedronis | but right now it makes other places fail in very strange ways | 17:57 |
| zyga | pedronis: for interfaces we should probably fail | 17:57 |
| pedronis | because they were never changed to look at broken | 17:57 |
| pedronis | nor tbh it makes a lot of sense for them | 17:58 |
| zyga | pedronis: though now we kind of do (but again, since just a moment ago) | 17:58 |
| pedronis | they would much better get an error | 17:58 |
| pedronis | zyga: yea, but it is earlier, it seems doMountSnap itself can explode (strangely because of the ignored error) on a not really mounted snap | 17:58 |
| mvo | jdstrand: re oom> thanks for your update. so you say that apparmor leak is too small to trigger this oom situation and something else is mostly likely eating the lowmem? | 17:59 |
| zyga | Caelum: so I think we should definitely just do that | 17:59 |
| pedronis | zyga: it means do that our mount code has problems , we trust something that isn't correct (or yet something else is going on but unclear what) | 17:59 |
| zyga | Caelum: let's propose what we have | 17:59 |
| zyga | Caelum: and see what the feedback is | 18:00 |
| zyga | Caelum: can you do that? | 18:00 |
| pedronis | s/do/though/ | 18:00 |
| zyga | pedronis: I would like to know if it happens inside a container or inside a non-container | 18:00 |
| zyga | pedronis: perhaps FUSE is the thing that makes it more "broken" | 18:00 |
| pedronis | I can ask about the last case | 18:00 |
| zyga | pedronis: who was the last case? what do we know about it? | 18:01 |
| pedronis | somebody on the store team | 18:01 |
| pedronis | I'm asking him (but he was having lunch) | 18:02 |
| pedronis | zyga: not a container apparently | 18:03 |
| zyga | that's good information, so it probably is a generic issue | 18:03 |
| pedronis | xenial | 18:03 |
| zyga | I'll get some tea, keep talking about what we know | 18:03 |
| jdstrand | mvo: that was the conclusion before: https://forum.snapcraft.io/t/oom-for-interfaces-many-on-bionic-i386/4101/7 | 18:04 |
| zyga | back | 18:04 |
| * zyga checks what systemd does for mount units in xenial | 18:05 | |
| jdstrand | mvo: nothing was changed in apparmor wrt this. the fact that it went away and came back coupled with ^ suggests it is something else. I can't comment on the progress of the work to make the 30M smaller or the 2M leak go away | 18:05 |
| pedronis | zyga: he also said he remove the snap and reinstalled, might also be some interaction between lazy detaching and remounting | 18:06 |
| pedronis | mvo: somebody hit a old bug with the beta, seems we really have some corner cases in which SetupSnap returns without errors but the snap is not really mounted (yet) | 18:08 |
| zyga | pedronis, mvo: let's add some simple code that after the mount call, checks | 18:09 |
| zyga | we can wait, raise red flags, etc | 18:09 |
| zyga | and then run it in a very long loop | 18:09 |
| zyga | maybe something will come up | 18:09 |
| zyga | opinions? | 18:09 |
| Caelum | zyga: sure | 18:11 |
| mvo | jdstrand: well, we have more tests I think it never went away we just didn't trigger it because we ran less tests. but fair enough, I start looking tomorrow again, its pretty important as it blocks us from entering bionic right now | 18:12 |
| mvo | pedronis: oh, interessting | 18:13 |
| zyga | mvo: is it easy to reproduce? | 18:13 |
| cachio | mvo, do you know who is the owner of the dragonboard-kernel snap | 18:13 |
| cachio | dragonboard-kernel_45.snap is getting stuck starting | 18:13 |
| mvo | zyga: I think so, I think its a matter of installing removing a snap | 18:13 |
| zyga | cachio: I think the kernel team owns all kernel snaps | 18:13 |
| cachio | zyga, do you know who? | 18:14 |
| mvo | zyga: http://paste.ubuntu.com/p/26xzSJ8NHJ/ | 18:14 |
| zyga | mvo: on which kernel? any? | 18:15 |
| zyga | specifically i386 bionic? | 18:15 |
| mvo | zyga: bionic/i386 | 18:15 |
| zyga | ok, I'll try to reproduce it now | 18:15 |
| sergiusens | jdstrand: hi there, we are having an isolated issue and might find it interesting if you could provide insight (snapcraft snap inside lxd). Here's a paste I have https://paste.ubuntu.com/p/mkhz73q7K7/ where on first run everything works but on a second one (where we do not install core nor snapcraft again) it fails with "cannot change profile for the next exec call: No such file or directory" | 18:16 |
| mvo | zyga: also artful :/ | 18:16 |
| mvo | zyga: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-artful/artful/i386/s/snapd/20180411_144134_835a7@/log.gz | 18:16 |
| zyga | sergiusens: what prints that line | 18:16 |
| zyga | cannot change profile for the next exec call: No such file or directory | 18:16 |
| zyga | is that snap-confine? | 18:16 |
| sergiusens | zyga: running snapcraft | 18:16 |
| sergiusens | zyga: yeah, most likely | 18:17 |
| zyga | sergiusens: right, but as a part of that you are runnig some snaps, right? | 18:17 |
| zyga | can you add export SNAP_CONFINE_DEBUG=yes please | 18:17 |
| sergiusens | snapcraft is a snap | 18:17 |
| zyga | and reproduce | 18:17 |
| zyga | may be some hint | 18:17 |
| zyga | what this tells us is that we're trying to switch to some profile | 18:17 |
| zyga | but that profile doesn't exist | 18:17 |
| sergiusens | zyga: yeah, not me, popey and it seems he can consistently reproduce. This also only happens for him on on lxd 3.0.0 | 18:17 |
| zyga | you can also get one more thing | 18:17 |
| zyga | when it fials | 18:17 |
| mvo | niemeyer: hm, did spread change and is no longer compatible with go1.6 or something? I see in the xenial autopkgtest output: + go get -u github.com/snapcore/spread/cmd/spread | 18:17 |
| mvo | package context: unrecognized import path "context" (import path does not begin with hostname) | 18:17 |
| sergiusens | zyga: yeah, interesting that it works on the first pass and fails on the second, so it seems something is lost on container stop/start | 18:18 |
| zyga | when it fails run "sudo cat /sys/kernel/security/apparmor/profiles" | 18:18 |
| zyga | sergiusens: yeah, likely so | 18:18 |
| zyga | mvo: yes, spread changed and it is not compatible with go 1.6 anymore | 18:18 |
| zyga | mvo: we discussed this last week | 18:18 |
| sergiusens | zyga: the `cat` inside the container or outside or is it irrelevant? | 18:18 |
| mvo | zyga: right, the next question is - what can we do about it :) | 18:19 |
| sergiusens | zyga: also, snapcraft is classic, so there should be no profile to switch to, right? | 18:19 |
| jdstrand | mvo: with the way that interfaces are connected (run apparmor_parser after each snap connect/disconnect, even if a particular command has multiple interfaces rather than only running it once) and sooo many tests, I wonder if the leak, even though it is small, might compound and therefore aggravate the situation. I don't really know how the leak happens though, so that might be a question for jjohansen when he is around again. he seems timme shifted, | 18:19 |
| zyga | mvo: we discussed this at length, one idea is to get the pre switch spread in a branch and build it for go 1.6 | 18:19 |
| mvo | niemeyer: anything we can do to make spread 1.6 compatible again? right now this blocks the autopkgtests in xenial. or should we use a fork for spread on xenial? | 18:19 |
| mvo | zyga: right, did we discuss with gustavo? | 18:20 |
| mvo | zyga: yet? | 18:20 |
| zyga | mvo: no, not yet | 18:21 |
| zyga | mvo: both you and gustavo were away at that time | 18:21 |
| sergiusens | mvo: you could probably enable backports and pin in your adt test to get the latest go | 18:22 |
| mvo | jdstrand: ok, thank you. its late in my TZ but i can try to gather data tomorrow | 18:22 |
| mvo | zyga: aha, indeed | 18:22 |
| sergiusens | or install the go snap, prior to enabling squashfuse to get the arches running in containers covered | 18:22 |
| mvo | sergiusens: that is a good idea | 18:23 |
| mvo | sergiusens: we don't support testing in containers currently so that is ok | 18:23 |
| sergiusens | unless spread is part of the packaging, then I have no ideas | 18:23 |
| mvo | sergiusens: also 2.32 fixes the squashfuse issue :) | 18:23 |
| zyga | sergiusens: actually, I don't think that is true | 18:23 |
| zyga | sergiusens: classic has profiles, just very open | 18:23 |
| * zyga checks | 18:23 | |
| zyga | sergiusens: yes, classic has profiles and is confined | 18:23 |
| sergiusens | mvo: but armhf runs in a container, or do you get special hw for adt? | 18:23 |
| jdstrand | mvo: did you see the bit about jjohansen time-shifted? looking back, it seems my comment may have been too long | 18:24 |
| sergiusens | zyga: ok, just a lean and mean one :-) | 18:24 |
| mvo | sergiusens: we skip if we detect containers, we don't run tests there right now unfortunately | 18:24 |
| zyga | jdstrand: do you know which timezone jj currently inhabits/ | 18:24 |
| mvo | jdstrand: heh, I did see that, thank you :) | 18:24 |
| sergiusens | mvo: ok, I was looking at installing squashfuse in debian/test/control to figure out if we could get that going | 18:25 |
| jdstrand | mvo: did you see the bit about jjohansen time-shifted? looking back, it seems my comment may have been too longwhen he is around again. he seems timme shifted, so you might ask him when you come online tomorrow" | 18:25 |
| sergiusens | I'll report back, as it might interest you (we currently skip build-snaps tests on armhf and such) | 18:25 |
| jdstrand | man | 18:25 |
| jdstrand | mvo: ok, you saw it, I'll stop trying to paste it :) | 18:25 |
| mvo | niemeyer: about spread and go1.6 - a fix in the spread upstream repo would be great as it would allow us to avoid another upload. if that is hard/impossible I can try to workaround it via installing a different go when building spread or using the snap or trying to be creative in other ways. | 18:26 |
| suebt | ogra_, pedronis: Hey, regarding my lockup issue we just talked about: I found out that the application basically just crashes and exits. I just reproduced it with a one-line app that just panics and quits. Looks like snap by default makes daemons auto-restart in case of failure. Is it expected that this will lead to locking up the whole system when the app is quitting straight away? | 18:26 |
| zyga | suebt: hey, sorry for being absent earlier | 18:26 |
| zyga | I think system will back off eventually | 18:26 |
| suebt | hey zyga, no problem :) | 18:26 |
| zyga | but even if you essentially create a "while true; crash; done" app | 18:27 |
| zyga | it should not bring the system down | 18:27 |
| suebt | I can post the code, give me a sec | 18:27 |
| jdstrand | sergiusens: fyi, what zyga asked for is what is needed to understand what is happening. the profile (likely for the lxc command if I were to guess) seems to have been unloaded | 18:27 |
| zyga | can you provide as much information as possible please | 18:27 |
| zyga | suebt: oh, perfect | 18:27 |
| zyga | jdstrand: maybe when the container is stopped and started something is off and apparmor profiles are not loaded | 18:27 |
| zyga | mvo: I think the "stable quiet period" is a good thing for now | 18:28 |
| zyga | we have plenty of things to attack | 18:28 |
| mvo | zyga: oh yes! | 18:29 |
| jdstrand | zyga (cc sergiusens): that's interesting and plausible. a snapd restart on container start would workaround that | 18:29 |
| zyga | jdstrand: note that what is super odd | 18:29 |
| zyga | jdstrand: is that snap-confine would say "aha, I'm not confined" | 18:29 |
| zyga | jdstrand: and would bail out way before reaching that code | 18:29 |
| zyga | jdstrand: so its own profile must have been loaded | 18:29 |
| zyga | jdstrand: but it is only loaded by apparmor init scripts | 18:30 |
| zyga | jdstrand: and whenever we install core | 18:30 |
| zyga | jdstrand: so ... ? | 18:30 |
| zyga | something is off | 18:30 |
| zyga | or | 18:30 |
| zyga | well, that's silly | 18:30 |
| zyga | or the profiles from /var/lib/snapd/apparmor are not loaded | 18:30 |
| jdstrand | snapd will load it if it detects overlay or nfs too | 18:30 |
| zyga | we changed one thing recently | 18:30 |
| suebt | zyga: here you go: https://github.com/tim-sueberkrueb/snap-daemon-lockup-example I'm on Ubuntu 17.10 | 18:30 |
| zyga | we have the system key | 18:30 |
| jdstrand | not that this is the case, just mentionign that | 18:30 |
| zyga | we don't reload profiles like we used to do, all the time | 18:30 |
| jdstrand | does reexec make sure it is there? | 18:31 |
| jdstrand | anyway, need more data | 18:31 |
| zyga | suebt: perfect, me too | 18:31 |
| suebt | get a live stick ready xD | 18:31 |
| zyga | jdstrand: reexec yes but only when core is installed (that is during that operation) | 18:31 |
| zyga | jdstrand: what I am saying is that now with the system key we are not loading profiles on startu | 18:31 |
| zyga | jdstrand: so if there was a bug since forever in lxd | 18:31 |
| zyga | jdstrand: it was masked | 18:31 |
| zyga | jdstrand: but not anymore | 18:31 |
| zyga | suebt: my desktop runs 17.10 | 18:31 |
| suebt | ok, I just mean in case of locking your system up ^^ | 18:32 |
| zyga | suebt: inspecting now | 18:32 |
| suebt | thanks :3 | 18:32 |
| zyga | thank you for using snaps :) | 18:33 |
| zyga | and sorry for the bad experience | 18:33 |
| * zyga loves rust | 18:33 | |
| suebt | Yeah, rust is awesome | 18:33 |
| suebt | just started learning it though :) | 18:34 |
| * suebt is still a noob in rust | 18:34 | |
| jdstrand | I wonder if this is related to https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1746463 | 18:34 |
| mup | Bug #1746463: apparmor profile load in stacked policy container fails <aa-kernel> <verification-needed-artful> <snapd:Triaged> <apparmor (Ubuntu):Confirmed> <linux (Ubuntu):Confirmed> <linux-gcp (Ubuntu):Fix Released> <apparmor (Ubuntu Xenial):Won't Fix> <linux (Ubuntu Xenial):Invalid> <linux-gcp | 18:34 |
| mup | (Ubuntu Xenial):Fix Released> <apparmor (Ubuntu Artful):Fix Committed> <linux (Ubuntu Artful):Fix Released> <linux-gcp (Ubuntu Artful):Invalid> <apparmor (Ubuntu Bionic):Confirmed> <linux (Ubuntu Bionic):Confirmed> <linux-gcp (Ubuntu Bionic):Fix Released> <https://launchpad.net/bugs/1746463> | 18:34 |
| jdstrand | perhaps the kernel is out of date | 18:34 |
| jdstrand | sergiusens (cc zyga): ^ | 18:35 |
| jdstrand | anyway, I need to move to another task now. if there is more data, we can look | 18:35 |
| zyga | jdstrand: perhaps, good call | 18:35 |
| zyga | suebt: what does snap version say? | 18:37 |
| zyga | are you on 2.32.3/ | 18:37 |
| suebt | yep | 18:37 |
| zyga | perfect | 18:37 |
| zyga | ok, installing now | 18:37 |
| zyga | (I hope it doesn't explode that hard :)" | 18:37 |
| suebt | ok, hope you can reproduce it :D | 18:37 |
| zyga | suebt: nope | 18:38 |
| zyga | it starts, it is restarted a few times | 18:38 |
| zyga | nothing happens | 18:39 |
| zyga | I'm typing this from that system | 18:39 |
| suebt | Well, this is both good and bad ^^ | 18:39 |
| suebt | Hmm ... | 18:39 |
| zyga | can you ssh into your machine | 18:39 |
| zyga | from something else | 18:39 |
| suebt | I didn't build it with cleanbuild | 18:39 |
| zyga | run journalctl -f | 18:39 |
| suebt | But that doesn't make any difference | 18:39 |
| zyga | neither did I | 18:39 |
| zyga | and then install the snap | 18:40 |
| zyga | and let's see what you get | 18:40 |
| suebt | Okay | 18:40 |
| sergiusens | jdstrand: well popey is on kde neon | 18:41 |
| jdstrand | sergiusens: perhaps popey should be talking to us :) popey, make sure your kernel is up to date (there was a fix last week for the above bug) and try the lxd stuff again | 18:41 |
| Pharaoh_Atem | zyga: review? https://paste.fedoraproject.org/paste/IBruVUkBu79voKY439~bOg | 18:46 |
| suebt_ | zyga: uhm, ya, writing this from the other machine: https://paste.ubuntu.com/p/nxqmQydfrp/ that's all | 18:46 |
| sergiusens | jdstrand: heh, I wanted to rule out snapcraft any lxd interaction, but it seems the error is not snapcraft originated (while it is snapcraft driven) | 18:46 |
| zyga | Pharaoh_Atem: ack | 18:47 |
| zyga | suebt_: nothing there, what happens with your system | 18:47 |
| zyga | is the ssh session dead? | 18:47 |
| suebt_ | Yep it is xD | 18:47 |
| suebt_ | it completely locked up again | 18:47 |
| suebt_ | The other suebt just died | 18:47 |
| zyga | hmmmm | 18:47 |
| jdstrand | sergiusens: yeah, I think it is how snapcraft is driving lxd that is uncovering the issue | 18:47 |
| zyga | I wonder if your kernel crashed | 18:47 |
| zyga | is it a laptop/desktop? | 18:47 |
| zyga | do you have a screen | 18:48 |
| suebt_ | It's a laptop. | 18:48 |
| zyga | can you reboot it | 18:48 |
| zyga | go to vt4 | 18:48 |
| zyga | or something without X | 18:48 |
| zyga | log in into the consoel | 18:48 |
| zyga | ssh in remotely | 18:48 |
| zyga | trigger the error remotely | 18:48 |
| zyga | if the kernel crashed there's bound to be something on the tty | 18:48 |
| sergiusens | jdstrand: so the line that fails almost looks like `lxc exec <container-name> -- snapcraft` | 18:48 |
| suebt_ | Okay, will try | 18:48 |
| zyga | suebt_: ok, perfect, thank you | 18:49 |
| zyga | Pharaoh_Atem: reading now | 18:49 |
| jdstrand | sergiusens: yeah, that is what I figured | 18:49 |
| sergiusens | jdstrand: the difference from the first run and second is that on the first run we snap install core and snapcraft while on the second run it is already there (so we do not install) | 18:49 |
| * jdstrand nods | 18:49 | |
| sergiusens | jdstrand: and it works for me too; but I did not go through an upgrade path of 2.0 to 3.0.0 as he has (I installed 3.0.0 from scratch) | 18:50 |
| sergiusens | oh, and I am on bionic | 18:50 |
| zyga | sergiusens: that hints at the fact that core installation triggers profile setupo | 18:50 |
| jdstrand | sergiusens: I did go from 2 to 3. what do I need to do to reuse the container? (I use cleanbuild all the time) | 18:51 |
| jdstrand | that said, I suspect it has nothing to do with lxd | 18:51 |
| sergiusens | jdstrand: look at the top of the paste for the feature flag | 18:51 |
| sergiusens | export SNAPCRAFT_CONTAINER_BUILDS=1 | 18:51 |
| sergiusens | it behaves like a local build but inside a container, so you can run `snapcraft pull` and it will do the pull in the container and you get to see the bits locally | 18:52 |
| jdstrand | sergiusens: that's nice | 18:53 |
| jdstrand | sergiusens: yes, it works fine here (bionic) | 18:54 |
| === grumblr is now known as grumble | ||
| jdstrand | sergiusens: I'd like to see what kernel popey has | 18:54 |
| sergiusens | ok, we can probaly use a smaller test case for this when he's back by installing a small classic confined snap and go with that | 18:55 |
| jdstrand | sergiusens: a good reproducer would be nice. that said, I can snapcraft twice on a small snap just fine and fast | 18:57 |
| zyga | Pharaoh_Atem: it looks good | 19:01 |
| zyga | let me think if there's anything to tweak | 19:01 |
| suebt_ | Hey zyga: On the laptop screen I get: "watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [snap-exec:10B49] ..." | 19:02 |
| zyga | aha | 19:02 |
| zyga | let it run | 19:02 |
| zyga | it's not dead | 19:02 |
| suebt_ | how long should I let it run? | 19:02 |
| zyga | I wonder what's the kernel shortcut to get a backtrace from stuff | 19:02 |
| zyga | a few minutes | 19:03 |
| suebt_ | okay | 19:03 |
| zyga | looks like a deadlock | 19:03 |
| zyga | suebt_: can you send me the snap you are running | 19:03 |
| zyga | the built one | 19:03 |
| zyga | (next time after reboot) | 19:03 |
| zyga | but don't reboot yet | 19:03 |
| zyga | and uname -a | 19:03 |
| zyga | and tell me if you have any kernel modules you've built from source or dkms | 19:04 |
| suebt_ | Sure, as soon as I'm able too reboot again. Okay, then I'm going to wait 5 more minutes? | 19:04 |
| zyga | Pharaoh_Atem: +1 from me | 19:04 |
| zyga | I think it's a start of a new era :) | 19:05 |
| zyga | suebt_: yes, I think there's a keyboard combo that can be useful | 19:05 |
| zyga | but I don't recall it | 19:05 |
| mborzecki | zyga: sysrq? | 19:05 |
| zyga | yes | 19:05 |
| zyga | mborzecki: do you know which one panics the kernel or shows something useful (backtrace) | 19:06 |
| mborzecki | hmm i have muscle memory for sending one over uart ;) | 19:06 |
| mborzecki | zyga: echo l > /proc/sysrq-trigger | 19:06 |
| mborzecki | zyga: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/sysrq.rst#n51 | 19:07 |
| mup | Issue snapcraft#2026 closed: Implement a passthrough feature <bug> <enhancement> <Created by sergiusens> <Closed by kyrofa> <https://github.com/snapcore/snapcraft/issue/2026> | 19:08 |
| mup | PR snapcraft#2053 closed: meta: implement pass-through of properties to snap.yaml <Created by kalikiana> <Merged by kyrofa> <https://github.com/snapcore/snapcraft/pull/2053> | 19:08 |
| zyga | mborzecki: do you know what to press on suebt_'s keyboard to get useful backtrace? | 19:08 |
| mborzecki | zyga: On x86 - You press the key combo :kbd:`ALT-SysRq-<command key>`. and then l or t, probably l if it's a lockup | 19:09 |
| zyga | Pharaoh_Atem: can you please cross-post this to the forum | 19:10 |
| zyga | suebt_: ^^ what mborzecki said | 19:10 |
| zyga | can you try? | 19:10 |
| suebt_ | okay, sure, so alt print l? | 19:11 |
| mborzecki | suebt_: did it work? | 19:14 |
| suebt_ | hmm nothing happens, maybe I try the wrong combo for my laptop | 19:14 |
| suebt_ | tried alt+print+l | 19:15 |
| mborzecki | if it's a laptop you may need to press Fn+prtscr | 19:15 |
| suebt_ | ok | 19:15 |
| suebt_ | Nothing, also tried with the external keyboard | 19:15 |
| mborzecki | hm it may be disabled then :/ | 19:16 |
| mborzecki | on arch & thinpad x220, it's press alt, press Fn, press prscr, release prscr, release fn, press l, and then ther's a nice message in dmesg "This sysrq operation is disabled" | 19:17 |
| mup | Issue snapcraft#2064 opened: Support for set-grade <Created by kyrofa> <https://github.com/snapcore/snapcraft/issue/2064> | 19:20 |
| suebt_ | hmm | 19:20 |
| suebt_ | in the lenovo forum FN + S seems to be also a thing, but that doesn't work either | 19:21 |
| suebt_ | okay, meh, sorry, then I'll reboot now, I gues? | 19:22 |
| suebt_ | *ss | 19:22 |
| zyga | suebt_: yeah, go for it | 19:24 |
| zyga | suebt_: send me uname -a | 19:25 |
| zyga | and your snap | 19:25 |
| zyga | I will look into it | 19:25 |
| zyga | but not tonight, I have some high priority work now | 19:25 |
| suebt_ | okay, will provide the info you requested in a few minutes, thanks! | 19:29 |
| Pharaoh_Atem | zyga: why would I post this to the forum? | 19:31 |
| Pharaoh_Atem | this is the email I'm going to send to server WG | 19:31 |
| zyga | Pharaoh_Atem: cross post, this is a big and important topic | 19:31 |
| zyga | this way people will know it happens | 19:31 |
| * Pharaoh_Atem sighs | 19:32 | |
| Pharaoh_Atem | I guess | 19:32 |
| suebt | snap package: https://drive.google.com/open?id=1eXHPcZLn-lVFf38EWvke-k8SWflOW6l- | 19:34 |
| suebt | system info: https://paste.ubuntu.com/p/ZySVSw8GQn/ | 19:34 |
| suebt | No custom kernel | 19:34 |
| suebt | anything I forgot? | 19:34 |
| suebt | zyga ^^^ | 19:35 |
| zyga | looking | 19:36 |
| zyga | ah, | 19:36 |
| zyga | I'm on a custom kernel! | 19:36 |
| zyga | I'll boot to -38 and try | 19:36 |
| zyga | thank you, that's all for now | 19:36 |
| zyga | can you hop in tomorrow | 19:36 |
| zyga | and check with us again please? | 19:36 |
| suebt | Sure, I will try to be around 18 UTC :) | 19:37 |
| suebt | *18 pm | 19:37 |
| suebt | 18 pm is not a thing nevermind xD | 19:38 |
| suebt | ok, thanks! | 19:38 |
| mvo | meh, too late - I have the answer for sysreq on the new lenovo keyboards: https://mvogt.wordpress.com/2017/05/24/sysreq-on-lenovo-x250/ - oh well | 19:53 |
| petan | is there a way to display very old builds? | 19:53 |
| petan | I see only like last 10 | 19:54 |
| petan | I need to check some like 100 builds ago | 19:54 |
| mvo | petan: snapcraft history <snapname> iirc | 19:55 |
| petan | ok | 19:55 |
| mvo | mwhudson: did I mention today how great "snap install go --channel=1.6/stable" is? | 19:56 |
| mvo | mwhudson: thank you so much for this | 19:56 |
| popey | jdstrand: 4.13.0-37-generic. KDE neon | 19:59 |
| * diddledan peeks | 20:03 | |
| mvo | niemeyer: I pushed a possible fix for the spread go1.6 compat in https://github.com/snapcore/spread/pull/56 - as a stop-gap. if that is acceptable I can trigger the autopkgtests again on xenial (after that is merged of course). but please do let me know if you prefer a different approach | 20:10 |
| mup | PR spread#56: add go1.6 compatibility <Created by mvo5> <https://github.com/snapcore/spread/pull/56> | 20:10 |
| niemeyer | mvo: What's that about? | 20:12 |
| * niemeyer ooks | 20:12 | |
| niemeyer | looks | 20:12 |
| mvo | niemeyer: in a nutshell our autopkgtests on xenial are broken because they try to build spread with go1.6 | 20:12 |
| niemeyer | mvo: That's an easy one | 20:13 |
| niemeyer | mvo: It's in | 20:13 |
| mvo | niemeyer: \o/ | 20:13 |
| mvo | niemeyer: man, thank you so much | 20:13 |
| niemeyer | mvo: Thanks, and sorry for missing this earlier | 20:13 |
| mvo | niemeyer: I will trigger the tests to re-run now, thank you! | 20:14 |
| === sergiusens_ is now known as sergiusens | ||
| === TinoGuest_ is now known as TinoGuest | ||
| === MrGeneral_ is now known as MrGeneral | ||
| === davdunc_ is now known as davdunc | ||
| === iatrou_ is now known as iatrou | ||
| === icey_ is now known as icey | ||
| jdstrand | popey: linux (4.13.0-38.43) artful; urgency=medium | 20:31 |
| jdstrand | popey: that ^ has the fix for the bug I mentioned. please upgrade to that and try again | 20:32 |
| popey | jdstrand: I am on xenial | 20:53 |
| jdstrand | popey: linux-hwe (4.13.0-38.43~16.04.1) xenial; urgency=medium | 21:08 |
| === sergiusens_ is now known as sergiusens | ||
| === sergiusens_ is now known as sergiusens | ||
| === sergiusens_ is now known as sergiusens | ||
| === sergiusens_ is now known as sergiusens | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!