| mup | PR snapcraft#3095 opened: plugins: break out rosdep resolve parsing for external use <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/3095> | 00:57 |
|---|---|---|
| mup | PR snapcraft#3096 opened: pluginhandler: export SNAPCRAFT_BUILD_BASE to build environment <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/3096> | 01:39 |
| mup | PR snapcraft#3015 closed: [WIP] ros2 (colcon) extension preview <Created by cjp256> <Closed by cjp256> <https://github.com/snapcore/snapcraft/pull/3015> | 01:48 |
| mup | PR snapcraft#3097 opened: [WIP] colcon v2 plugin + ros2 extension <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/3097> | 01:58 |
| === xnox|sessions is now known as xnox | ||
| mup | Bug #1875543 opened: Ubuntu 20.04 "A stop job is running for Snappy Daemon" during shutdown <20.04> <ubuntu> <Snappy:New> <https://launchpad.net/bugs/1875543> | 05:46 |
| zyga | Good morning | 06:25 |
| mborzecki | morning | 06:44 |
| mvo | good morning mborzecki | 06:46 |
| mborzecki | mvo: hey, meeting is at 11 right? | 06:47 |
| mvo | mborzecki: yes | 06:48 |
| zyga | Hey | 06:49 |
| zyga | How was day one? | 06:49 |
| mborzecki | mvo: cool, i'll go for a walk with kids and do groceries, will be back before 11 | 06:50 |
| mvo | mborzecki: sounds good, note that you are optional for the meeting, no worries about this | 06:52 |
| pstolowski | morning | 07:08 |
| zyga | good morning pawel :) | 07:13 |
| zyga | let's see if the store works better today | 07:38 |
| pedronis | hello, how are tests doing today? | 07:56 |
| zyga | pedronis: running now, all yesterday was red red red | 08:00 |
| zyga | pedronis: I'll let you know if a single run can pass | 08:01 |
| pstolowski | hi pedronis, i just kicked one of my PRs, will know soon | 08:01 |
| mborzecki | re | 08:22 |
| pedronis | zyga: pstolowski: I have a PR that I would like to land to let people test in edge, but I will not trigger it if it's hopeless | 08:23 |
| pedronis | zyga: pstolowski: thanks | 08:23 |
| zyga | ack | 08:23 |
| * zyga tests a fix for opensuse | 08:24 | |
| zyga | google:ubuntu-core-16-64:tests/main/core-persistent-journal failed | 08:38 |
| zyga | + snap set core journal.persistent=false | 08:38 |
| zyga | 886 | 08:38 |
| zyga | grep error: pattern not found, got: | 08:38 |
| zyga | 887 | 08:38 |
| zyga | error: cannot communicate with server: Put http://localhost/v2/snaps/core/conf: EOF | 08:38 |
| zyga | preseed tests also failed on 19.10 | 08:38 |
| zyga | ++ find /mnt/cloudimg/var/lib/snapd/seed/snaps/ -name 'core_*.snap' | 08:39 |
| zyga | 1316 | 08:39 |
| zyga | + CORE_IMAGE= | 08:39 |
| zyga | 1317 | 08:39 |
| zyga | + unsquashfs '' | 08:39 |
| zyga | 1318 | 08:39 |
| zyga | Could not open , because No such file or directory | 08:39 |
| zyga | 1319 | 08:39 |
| zyga | pstolowski: let's finish that branch that fixes this | 08:39 |
| zyga | pstolowski: you said that you had more improvements to my PR? | 08:39 |
| pstolowski | zyga: i pushed them all to #8528 | 08:42 |
| mup | PR #8528: tests: fix for pre-seeding failures <Created by zyga> <https://github.com/snapcore/snapd/pull/8528> | 08:42 |
| zyga | super, let's see | 08:42 |
| zyga | when did you restart this last? | 08:42 |
| pstolowski | zyga: let me merge master & push | 08:43 |
| pstolowski | was last week | 08:43 |
| zyga | pstolowski: thanks | 08:43 |
| zyga | pstolowski: perhaps rebase, the merges are just confusing | 08:43 |
| pstolowski | zyga: hmm too late | 08:43 |
| zyga | no worries | 08:43 |
| pstolowski | zyga: do you have a link to core-persistent-journal failure? | 08:43 |
| zyga | yes, one sec | 08:44 |
| zyga | https://github.com/snapcore/snapd/pull/8565/checks?check_run_id=625210861 | 08:44 |
| mup | PR #8565: osutil: expand FileLock to support shared locks and more <Created by zyga> <https://github.com/snapcore/snapd/pull/8565> | 08:44 |
| pstolowski | i cannot reproduce google:ubuntu-core-16-64:tests/main/core-persistent-journal failure | 09:13 |
| zyga | may have been a random one :/\ | 09:15 |
| pstolowski | zyga: we have some selinux denials on centos, and tests failing because of them, looking | 09:26 |
| zyga | I saw that but I'm unsure how to fix it | 09:27 |
| pstolowski | zyga: did we change snap-update-ns recently? | 09:27 |
| zyga | noo | 09:27 |
| zyga | no* | 09:28 |
| pstolowski | hmm | 09:28 |
| zyga | I suspect it's the nss modules | 09:28 |
| zyga | but I didn't go deep on that | 09:28 |
| pstolowski | indeed, only a few irrelevant changes there | 09:33 |
| pstolowski | zyga: pressed PR failed, i think 19.10 now has core18+snapd too, fun | 09:45 |
| pstolowski | zyga: i'm on it | 09:45 |
| zyga | heh | 09:45 |
| zyga | thank you | 09:45 |
| zyga | I'm looking at base policy issue | 09:45 |
| zyga | pedronis: is it possible to look at earlier revisions of assertions somehow? I'd like to see snap declarations | 09:49 |
| zyga | pedronis: no longer needed! | 10:00 |
| pedronis | zyga: the answer is no basically, not easily | 10:02 |
| pstolowski | zyga: pushed fix to #8528 | 10:20 |
| mup | PR #8528: tests: fix for pre-seeding failures <Created by zyga> <https://github.com/snapcore/snapd/pull/8528> | 10:20 |
| zyga | \o/ | 10:21 |
| zyga | thank you! | 10:21 |
| pstolowski | zyga: have you found anything about opensuse failures? | 10:24 |
| zyga | I looked but my trivial fix failed, I need to boot suse and just look around | 10:25 |
| zyga | I suspect I know what it is | 10:25 |
| zyga | I will probably couple that with package update | 10:25 |
| zyga | but after the meeting today | 10:25 |
| zyga | brb | 10:39 |
| === pedronis_ is now known as pedronis | ||
| jdstrand | diddledan: hey, I responded in the forum | 10:46 |
| pstolowski | zyga: #8525 green everywhere except for opensuse and centos 8 | 11:25 |
| mup | PR #8525: tests: ignore user-12345 slice and service <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8525> | 11:25 |
| pstolowski | zyga: ups, i mean #8528 | 11:26 |
| mup | PR #8528: tests: fix for pre-seeding failures <Created by zyga> <https://github.com/snapcore/snapd/pull/8528> | 11:26 |
| zyga | pstolowski: on unstable systems, can we merge regardless? | 11:33 |
| pstolowski | zyga: yes, we should. needs 2nd review | 11:34 |
| zyga | ok | 11:34 |
| zyga | maybe ijohnson is around now? | 11:34 |
| ijohnson | whats up zyga | 11:34 |
| zyga | ah, on a call | 11:34 |
| ijohnson | also good morning | 11:34 |
| zyga | ijohnson: just queue it for later ^^^ | 11:34 |
| ijohnson | 8258 ? | 11:34 |
| pstolowski | hey ijohnson | 11:34 |
| ijohnson | hey pstolowski | 11:35 |
| pstolowski | ah, actually it needs 2 reviews ;) | 11:36 |
| zyga | I just cannot review as I wrote some part of it | 11:36 |
| pstolowski | i suppose i can't review it | 11:36 |
| pstolowski | yeah | 11:36 |
| pstolowski | zyga: maybe your and my review would count 0.5 each ;) | 11:36 |
| zyga | haha, yeah | 11:41 |
| zyga | I think that's ok | 11:41 |
| * juergh_ juergh | 11:50 | |
| juergh_ | I need to install an old version of snapd that doesn't seem to be in any of the channels anymore. How would I go about that? | 11:50 |
| === juergh_ is now known as juergh | ||
| zyga | juergh_: hey, unless it's available in one of the channels you cannot do that | 11:51 |
| zyga | juergh: is there a particular reason you need to use a older snapd? | 11:51 |
| juergh | zyga, 2.43.x introduced a workaround for what some people believe is a kernel issue. I need to test with the old snapd that dosn't have the workaround to debug this. | 11:52 |
| zyga | juergh: perhaps you can download a package from the archive (.deb) and see if that's sufficient | 11:55 |
| zyga | juergh: I think there are some places that cache them | 11:55 |
| juergh | zyga, are you saying we don't archive old snaps? | 11:56 |
| zyga | juergh: we do but we don't allow everyone to download them | 11:57 |
| juergh | zyga, and you're saying as an Ubuntu kernel maintainer and Canonical employee I'm not allowed :-) | 11:57 |
| zyga | juergh: only the publisher of a given snap can get every revision | 12:02 |
| roadmr | (and collaborators) | 12:03 |
| juergh | zyga, whom do I need to talk to get the revision that I need? | 12:16 |
| zyga | juergh: sorry, in a call | 12:17 |
| zyga | juergh: I think someone from the store or snapd might be able to help you | 12:17 |
| zyga | juergh: you need to find a revision of core or snapd that has what you want | 12:17 |
| zyga | and download that | 12:17 |
| zyga | juergh: I'll break for lunch now, I can try to help you after the break | 12:38 |
| zyga | mvo: not sure if you have time but we need https://github.com/snapcore/snapd/pull/8528 to unbreak master | 12:41 |
| mup | PR #8528: tests: fix for preseeding failures <Created by zyga> <https://github.com/snapcore/snapd/pull/8528> | 12:41 |
| zyga | mvo: its green apart from unstable systems and has one review from Pawel who contributed a good chunk of the work | 12:41 |
| mup | PR snapd#8574 opened: tests, selinux: update SELinux rule affecting snap-update-ns on centos 8 <Created by stolowski> <https://github.com/snapcore/snapd/pull/8574> | 12:55 |
| pstolowski | mborzecki: ^ if you have a sec.. not sure if there is a magic macro anywhere in the policy, but this should do .. | 12:55 |
| cmatsuoka | zyga, cachio: going to SU? | 13:01 |
| zyga | no, I'm in the public review session | 13:01 |
| cachio | yes | 13:01 |
| cachio | but trying t o login | 13:01 |
| cachio | mvo, hi | 13:04 |
| zyga | cachio: mvo is speaking now | 13:04 |
| cachio | zyga, ouch | 13:05 |
| cachio | zyga, thanks for the heads up | 13:05 |
| ijohnson | zyga: pstolowski: I have reviewed 8528 | 13:16 |
| pstolowski | ijohnson: thanks! i'll remove the TODO comment on next occasion | 13:17 |
| ijohnson | sounds good | 13:17 |
| mvo | cachio: hi | 13:23 |
| cachio | mvo, sorry for the interruption | 13:24 |
| cachio | cwayne, asked me to create a test-snapd-tools-core20 snap | 13:24 |
| cachio | mvo, you created the other 2 ones | 13:24 |
| cwayne | i did do that | 13:24 |
| cachio | and I cant find where you have the snapcraft.yaml | 13:25 |
| mvo | cachio: thanks, let me check | 13:25 |
| cachio | and the snap recipe | 13:25 |
| mvo | cachio: looking now | 13:25 |
| cwayne | that's the main stuff failing on uc20 runs for us | 13:25 |
| cachio | mvo, thanks | 13:32 |
| cmatsuoka | cachio: where can I find the core image inside the classic test machine? | 13:32 |
| cachio | cmatsuoka, /tmp/work-dir/image | 13:34 |
| cachio | there | 13:34 |
| mvo | cachio: it's a hand crafted one, I pushed a new one and shared with you, we should make it owned by test-snaps@c.c | 13:36 |
| mvo | cachio: anyway, should unblock you | 13:36 |
| cachio | mvo, thanks | 13:36 |
| mup | PR snapd#8528 closed: tests: fix for preseeding failures <⚠ Critical> <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/8528> | 13:39 |
| cmatsuoka | cachio: one of the flags made the nested vm boot | 13:40 |
| cachio | cmatsuoka, awesome | 13:40 |
| cachio | cmatsuoka, panic? | 13:40 |
| cmatsuoka | cachio: now I must find which one, there are four possibilities | 13:40 |
| cachio | cmatsuoka, I can help | 13:41 |
| cmatsuoka | cachio: I tried it with -cpu host,-vmx-apicv-register,-vmx-apicv-vid,-vmx-ple,-vmx-rdrand-exit | 13:42 |
| cmatsuoka | cachio: but I also ran with -smp 4 and it rebooted later, so perhaps -smp 1 could help in this case? | 13:43 |
| cachio | cmatsuoka, yes | 13:43 |
| cmatsuoka | cachio: what I actually did was to run my usual qemu script inside the google-nested machine with -nographics | 13:44 |
| cmatsuoka | adding the extra flags to cpu | 13:44 |
| cachio | ahh, I am trying now with your parameters | 13:44 |
| cmatsuoka | cachio: there are four flags there but probably one one fixes the msr issue | 13:45 |
| cachio | cmatsuoka, I am trying now the regular nested with those parameters | 13:46 |
| cmatsuoka | cachio: -vmx-rdrand-exit prevents the crash, let's see if it doesn't cause entropy starvation problems | 14:15 |
| cachio | cmatsuoka, awesome | 14:16 |
| cachio | cmatsuoka, so far I couldn't connect through ssh | 14:16 |
| cmatsuoka | cachio: I think it's better to monitor the console to actually see what's going on there, it could be many things now including our dreaded entropy starvation | 14:17 |
| cmatsuoka | I'm not sure what exactly this flag does and documentation isn't great | 14:18 |
| cachio | cmatsuoka, yes | 14:26 |
| cachio | cmatsuoka, I see that goes into an infinite loop | 14:28 |
| cachio | using -smp 1 | 14:28 |
| cmatsuoka | cachio: it freezes? | 14:28 |
| cachio | cmatsuoka, no | 14:28 |
| cmatsuoka | cachio: or is it a reboot loop? | 14:28 |
| cmatsuoka | does this problem also happen without -cpu? | 14:30 |
| cachio | it rebooted in install mode many times | 14:37 |
| cachio | without -cpu I didnt see those reboots usign -smp 1 | 14:37 |
| cmatsuoka | and without -cpu, where did the problem happen again? | 14:39 |
| cachio | cmatsuoka, so | 14:44 |
| cachio | I don't see the menu anymore | 14:44 |
| cachio | so I cant update the kernel command line any more in run mode | 14:45 |
| cachio | cmatsuoka, also when I run console-conf I dont see any network: https://paste.ubuntu.com/p/2J4vjjxvHw/ | 14:57 |
| cachio | cmatsuoka, I am going to have lunch now | 15:01 |
| * cachio lunch | 15:02 | |
| mup | PR snapd#8575 opened: packaging/fedora: disable FIPS compliant crypto for static binaries <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8575> | 15:23 |
| mborzecki | pstolowski: ^^ | 15:24 |
| mup | PR snapd#8574 closed: tests, selinux: update SELinux rule affecting snap-update-ns on centos 8 <Created by stolowski> <Closed by stolowski> <https://github.com/snapcore/snapd/pull/8574> | 15:26 |
| === stgraber_ is now known as stgraber | ||
| pstolowski | drat, another failure of google:ubuntu-core-16-64:tests/main/core-persistent-journal, something is flaky after all | 15:43 |
| cmatsuoka | cachio: so if you're reaching console conf the system is booting correctly to run mode, except network? | 15:58 |
| cachio | cmatsuoka, yes | 16:29 |
| cmatsuoka | cachio: and this is with -cpu or without? | 16:31 |
| cachio | without | 16:31 |
| cmatsuoka | ok, so entropy starvation was not an issue | 16:31 |
| cmatsuoka | good | 16:31 |
| cachio | with -cpu it goes to a infinite loop | 16:31 |
| cachio | cmatsuoka, do you know how to edit the kernel commandline now? | 16:33 |
| cachio | the menu does not appear anymore | 16:33 |
| cmatsuoka | do you want to edit it for install mode or run mode? | 16:34 |
| cmatsuoka | for run mode it's part of the tpm measurements so you can't change it | 16:34 |
| cachio | cmatsuoka, ahh | 16:35 |
| cachio | that makes sense | 16:35 |
| cmatsuoka | now this network problem seems strange, did you see something similar before? | 16:36 |
| cachio | no | 16:38 |
| cachio | perhaps this is the reason why ssh cannot be stablished | 16:38 |
| mup | PR snapd#8575 closed: packaging/fedora: disable FIPS compliant crypto for static binaries <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8575> | 17:03 |
| mborzecki | yay, thanks mvo! | 17:04 |
| mborzecki | hm maybe we could move centos8 back to stable systems now | 17:05 |
| cachio | cmatsuoka, now without -cpu I see a reboot loop again | 17:25 |
| cachio | cmatsuoka, this is dmesg output on the host machine https://paste.ubuntu.com/p/JD5WgQkBQG/ | 17:25 |
| cachio | cmatsuoka, for each reboot I see a vcpu0, guest rIP: 0xffffffffb5e788b4 | 17:26 |
| cachio | xnox, hey, any idea about what be causing that error? | 17:27 |
| cmatsuoka | this seems to be just a notification message, not necessarily an error | 17:31 |
| cachio | cmatsuoka, but this vcpu0, guest rIP: | 17:35 |
| cachio | means that something happens and hte vm was killed right? | 17:35 |
| cmatsuoka | I think it's just a message telling that the guest is trying to access the msr | 17:44 |
| cmatsuoka | if something is crashing you should see a more detailed message | 17:45 |
| Saviq | zyga: LOL, `sejwy`? ;D | 17:49 |
| zyga | Saviq: *comprehensible* ;) | 17:49 |
| zyga | Saviq: are you following all new posts? :D | 17:50 |
| Saviq | zyga: only the interesting ones :P | 17:50 |
| Saviq | zyga: "remainder", btw :) | 17:50 |
| zyga | ah, thanks | 17:50 |
| zyga | fixed | 17:51 |
| zyga | Saviq: feedback welcome, it's just an idea at this stage | 17:51 |
| * zyga EODs | 18:00 | |
| === ijohnson is now known as ijohnson|lunch | ||
| cachio | cmatsuoka, did you remember the kvm parameter used to show extra info like crashes? | 18:24 |
| cachio | do you ? | 18:24 |
| cmatsuoka | cachio: hmm, no, I don't know this parameter | 18:27 |
| cmatsuoka | cachio: maybe Saviq knows? | 18:28 |
| Saviq | no, sorry | 18:28 |
| === ijohnson|lunch is now known as ijohnson | ||
| mup | PR snapd#8576 opened: tests/main/lxd: add test for snaps inside nested lxd containers <Test Robustness> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8576> | 18:32 |
| jdstrand | mvo: hey, fyi the discussion in https://github.com/snapcore/core20/issues/48#issuecomment-620509641 | 18:37 |
| mvo | jdstrand: thanks, checking | 18:38 |
| mvo | jdstrand: right, I agree we should use the updated libseccomp and rebuild snapd/core with that | 18:40 |
| mvo | jdstrand: timing is not great, I can probably work on this tomorrow (my) morning but it requires hte updated libseccomp to be available in ppa:snppay-dev/image | 18:40 |
| * mvo actually should write this in the bug | 18:40 | |
| ijohnson | also mvo I proposed the nested lxd test and it fails | 18:45 |
| ijohnson | need someone to look into why it fails | 18:45 |
| * ijohnson doesn't know | 18:45 | |
| mvo | ijohnson: might be worthwhile to check with stgraber | 18:46 |
| jdstrand | mvo: yeah. I'm not pushing for a particular time, just wanted to make sure you saw it | 18:46 |
| jdstrand | mvo: thanks | 18:46 |
| mvo | jdstrand: yeah, if snapcraft enables it we should fix the issue ASAP :) | 18:46 |
| mvo | ups, ijohnson -^ | 18:46 |
| ijohnson | yeah, I mean we could also ask snapcraft team to delay if really needed | 18:46 |
| ijohnson | stgraber: is running snapd inside a lxd container which is a nested lxd container supported? | 18:47 |
| ijohnson | stgraber: we test that snapd inside a lxd container works as expected and that we can create nested containers, but we heard from the juju team that sometime around eoan release it stopped working to install snaps inside the inner nested container | 18:47 |
| mvo | ijohnson: it's probably ok, I check in my morning, shouldn't be much work to do a 2.44.4, the only annoying part is that we only have a single beta channel and 2.45~pre1 is there right now, so it needs to go to candidate quickly or we test from a branch or something as we also need to build the beta cu20 image from the beta/ channel | 18:48 |
| ijohnson | mvo: I thought that ~pre* releases don't ever got to candidate ? | 18:49 |
| mvo | ijohnson: yeah, we usually put 2.44.x into beta so it would override 2.45~pre1 etc, I need to think a little bit about this | 18:49 |
| ijohnson | ack, let me know if there's anything I can help with | 18:49 |
| mvo | ijohnson: it's probably fine if we keep 2.44.4 in beta just shortly, 2.44.4 has almost no changes so the regression risk is minimal | 18:50 |
| mvo | ijohnson: thanks, will do! | 18:50 |
| mup | PR snapd#8577 opened: [RFC] secboot,cmd/snap-bootstrap: move initramfs-mounts tpm access to secboot <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/8577> | 18:52 |
| stgraber | ijohnson: apparmor only supports one level of nesting, so you can install snaps in a LXD container but you cannot install snaps inside a nested LXD container | 18:55 |
| stgraber | ijohnson: LXD itself if not installed through snapd can nest all the way to the max 32 levels deep and will just re-use the parent level's apaprmor profile when detecting that it can't create a new apparmor namespace | 18:56 |
| ijohnson | stgraber: ack, I will let Tim Penhey know | 18:56 |
| stgraber | I had someone reach out about this setup a week or so ago too | 18:57 |
| ijohnson | thanks, I can't seem to find him on IRC so I'll drop him an email and CC you on it | 18:58 |
| mup | PR snapd#8475 closed: tests: port snap-session-agent-* to session-tool <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/8475> | 20:23 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!