mupPR snapd#9326 closed: tests: fix for basic20 test running on external backend and rpi <Simple 😃> <Created by sergiocazzolato> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9326>00:03
=== jamesh_ is now known as jamesh
mupPR snapd#9362 closed: tests: skip nested images pre-configuration by default <Run nested> <Simple 😃> <Created by sergiocazzolato> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9362>01:59
mupPR snapd#9360 closed: tests: make gadget-reseal more robust <Run nested> <Simple 😃> <UC20> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9360>06:20
mvogood morning mborzecki06:37
mborzeckimvo: hey06:37
mborzeckimvo: so are we out of the woods yet?06:38
mvomborzecki: yeah, I think it's time for a (small) bottle of champagne06:39
mvomborzecki: resealing+pcr4 just landed06:39
mvomborzecki: so we should be good06:40
mupPR snapd#9277 closed: secboot: add boot manager profile to pcr protection profile <Run nested> <UC20> <Created by cmatsuoka> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9277>06:40
mupPR snapd#9356 closed: sysconfig,o/devicestate: mv DisableNoCloud to DisableAfterLocalDatasourcesRun <UC20> <Created by anonymouse64> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9356>06:40
mvomborzecki: I guess the next step is to test gadget udpate with the pc branch 20/beta/edition3 for real06:40
mvomborzecki: and if that works it's really time to celebrate06:40
mvomborzecki: I will branch 2.47 now06:41
mvomborzecki: afaict all is merged and the only missing pr needs a samuele review first06:41
mborzeckimvo: i'll work on the fix for updating structures without partitions on an encrypted device06:41
mvomborzecki: sounds great, thanks for this06:42
mvogood morning pstolowski07:02
mvopstolowski: do we happen to have a forum topic for disk-space awareness that I could link to?07:02
pstolowskimvo: i don't think so07:02
pstolowskimvo: i could publish the notes i sent to degville07:03
mvopstolowski: not urgent if degville publishes them I can link to that07:04
mvopstolowski: mostly wondering because I'm working on 2.47 release notes07:04
degvillemvo / pstolowski: the disk awareness doc is here: https://forum.snapcraft.io/t/disk-space-awareness/2000707:05
pstolowskidegville: nice, thank you!07:07
pstolowskidegville: there is a mistake there re snapd version (it says 2.37+ )07:07
degvillepstolowski: ah, thanks. I'll fix it now.07:08
mvodegville: \o/ thank you! I updated the snapd roadmap page now07:08
zygagood morning :)07:10
zygaI patched all the profiles last night and ran a test before EODing07:10
zygaI see that lxd and gdbserver failed, let me look at those07:10
zygalet's debug those07:12
zygamvo: is 2.47 ready now?07:12
zygaI saw ian and samuele fight some issues last night07:13
mvogood morning zyga07:13
mvozyga: yeah, I'm on 2.47~pre1 right now07:13
zygaI slept for 5 hours but I'm okay surprisingly07:13
zygacool, fingers crossed no more surprises07:13
mupPR snapd#9361 closed: tests/lib/cl_check.py: use python3 compatible code <Simple 😃> <Skip spread> <Created by anonymouse64> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/9361>07:15
zygaah missing snap-gdbserver-shim07:27
zygalet's look at lxd now07:27
mborzeckizyga: can you take a loot at https://github.com/snapcore/snapd/pull/9364 ? you like them python bits ;)07:35
mupPR #9364: tests/lib/cla_check: default to Python 3, tweaks, formatting <Simple 😃> <Skip spread> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9364>07:35
mborzeckizyga: fwiw, do you remember what distro is used for the workers/07:35
mupPR snapd#9364 opened: tests/lib/cla_check: default to Python 3, tweaks, formatting <Simple 😃> <Skip spread> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9364>07:35
zygare-formatted with black I presume07:36
mborzeckizyga: yeah, the trademark () style it uses07:36
mborzeckizyga: are you in power to install python3-launchpadlib in the containers used to run the cla workflow?07:37
zygaon mine, yes07:37
zygaone moment07:37
mborzeckizyga: who manages the other workers? that workflow runs on self-hosted (whatver that means)07:39
zygamborzecki: claudio07:39
zygainstalling now07:39
mborzeckizyga: cool, i'll ping him07:39
zygamborzecki: self hosted are workers which are managed buy us, not github07:40
mborzeckizyga: right, i though that maybe is runs/manages some of those for us07:40
zygathat's two nodes from me and one from claudio07:40
zygait's self-managed07:40
zygait will be ready in a few minutes07:40
mborzeckizyga: cool, thank you07:41
zygamborzecki: done, all workers under my control are ready07:55
mborzeckizyga: thanks07:56
zygamborzecki: as for self-hosted workers07:56
zygamborzecki: we could tweak actions to run cla checks on a pi easily07:56
zygamborzecki: no need for spread or root access07:56
zygamborzecki: just a small container with python and git07:56
zygamborzecki: so that's entirely viable07:57
zyganothing is running now07:58
zygarebooting for kernel update07:58
zygaworkers will be back in a minute or two07:59
zygaok, fixed the profile bit affecting lxd08:00
zygait's quite curious08:00
zygaMS_BIND really matters for MS_REMOUNT08:00
zygatesting again08:00
zygaall of main should pass now08:00
zygaI'll do core next and then add the feature flag I need for safety08:00
zygaand that should be it :)08:01
zygamvo: I'll start working on enabling raa today as we talked08:01
mborzeckihm built a uc20 image and it gets stuck in the efi shell08:09
mborzeckipretty sure i'm using the same ovmf that worked the last time08:09
zygaRemoving snapd (2.45.1ubuntu0.2) ...08:14
zygaFailed to execute operation: Connection timed out08:14
zygainside containers08:14
mupPR snapd#9365 opened: release: 2.47~pre1 <Simple 😃> <Skip spread> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9365>08:15
zygalxd passed08:24
zygaregression and core are in progress08:24
zygaafk, lucy needs me08:47
zygaboth core and regression passed08:47
zygajust the feature to restore old behavior in case we need it08:47
zygabut that's after wife is back in a moment08:48
pedronismvo: I made a comment in https://github.com/snapcore/snapd/pull/929308:48
mupPR #9293: snap: auto-import will not try to auto-create users on managed devices <Needs Samuele review> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9293>08:48
mvopedronis: \o/08:48
mvopedronis: sounds sensible08:48
pedronismvo: sorry, it's a bit more work, but maybe put us in a better spot08:49
mvopedronis: yeah, totally agreed, I like this08:49
mborzeckiduh, console-conf drops a backtrace when i try to configure the device09:07
zyga-mbpmborzecki physical or vm?09:08
mborzeckior snapd isn't quite up yet https://paste.ubuntu.com/p/3RZY7vmC4m/09:08
zyga-mbpwell, console conf should handle that error09:09
zyga-mbpand probably just retry09:09
zyga-mbpmaybe we just got restarted09:09
mvomborzecki: please file a bug and we need to put it onto the foundations radar09:34
mborzeckimvo: investigating still a bit, maybe it's snapd that's tarting slow09:35
mborzeckimvo: and seriously takes a bit of work to get a debug shell on the system :/09:36
mborzeckimvo: but i guess you already know the pain :)09:36
mvomborzecki: yes, but it's okay to just file a bug09:36
mborzeckiuhh, mark boot successful fails, `cannot identify kernel snap with bootloader grub: cannot read dangling symlink kernel.efi`09:37
mvomborzecki: and not dive too deep, if it's really a matter of retrying. but sorry, in a meeting so only 20% of my brain is available09:37
mvomborzecki: this is not the issue that ian identified the other day that snap-bootstrap older versions eat modeenv keys?09:38
mborzeckiyeah, omg, need to repack the kernel09:38
mborzeckii suppose i have what i wanted now anyway09:39
mborzeckimvo: a random thought, it'd be great to have at least a debug version of the snapd snap which has `systemd.debug-shell=1 dangerous` baked in09:40
mvomborzecki: yeah, I was wondering something similar - and snapd.debug=1 too :)09:43
mvomborzecki: maybe we should have this as part of the +testkeys or a similar new build-tag?09:44
zygare :)09:48
zygaI've updated mount-ns test to cope with the new tmpfs10:36
zyganow added the feature flag11:22
zyganow for some tests11:22
ijohnsonmborzecki: imho the proper way to set systemd.debug-shell=1 dangerous for tests is to make gadget specified kernel cmdlines work, then when we already repack the gadget we can set the extra cmdlines from there12:08
mborzeckiijohnson: well, i'm using https://github.com/snapcore/snapd/commit/a1ec67e26bc5fde9ef645bcc8d6e127345cee344 for now ;)12:09
mborzeckiijohnson: but yeah, i'd prefer the gadget way12:09
mborzeckilogin.ubuntu.com is down?12:19
mborzeckicannot create users in a UC20 vm12:19
mborzeckiehh, dropped too many modules when repacking the kernel12:22
mupPR snapd#9366 opened:  gadget: resolve device mapper devices for fallback device lookup <Run nested> <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9366>12:26
pedroniszyga: any news on google:ubuntu-16.04-64:tests/main/lxd:snapd_cgroup_neither ?12:27
zygapedronis: oh? is that broken?12:27
zygaI ran it today several times12:27
pedronisit failed here: https://github.com/snapcore/snapd/pull/9270/checks?check_run_id=112394007212:28
mupPR #9270: wrappers, systemd: allow empty root dir and conditionally do not pass --root to systemctl <Run nested> <Services ⚙️> <⛔ Blocked> <Created by stolowski> <https://github.com/snapcore/snapd/pull/9270>12:28
zygalet me look12:28
zygaah, wait, is that the issue we discussed recently12:29
zygaI remember now,12:29
cmatsuokazyga: I had multiple errors in that test yesterday12:29
zygait's weird, I did see the failure messages but in my case the test ultimately passed (it was slow though)12:29
zygaI'll check with stgraber about it and re-check with -shell-after, to see if it really passed for me12:30
pstolowskipedronis: there is no api to append to zip in Go... but gess what, there is an old juju package from rogpeppe to do this.. https://godoc.org/github.com/juju/zip#Reader.Append12:39
pedronispstolowski: mmh12:39
pstolowskipedronis: it's a fork of stdlib though ...12:39
pstolowskiand old12:39
pedronispstolowski: we need to chat again12:40
pstolowskifor context: https://groups.google.com/g/golang-dev/c/ZSubqlF2G4k?pli=112:40
pstolowskiand https://github.com/golang/go/issues/1562612:40
pstolowskiit got nowwhere in stdlib of go12:40
pedronispstolowski: I also I was confused about something12:46
pedronispstolowski: anyway let's chat again12:46
pstolowskipedronis: ok, now/later?12:46
pedronispstolowski: as you prefer, but not a lot of time before standup12:47
pstolowskipedronis: ok, after then12:47
mupPR snapd#9364 closed: tests/lib/cla_check: default to Python 3, tweaks, formatting <Simple 😃> <Skip spread> <Created by bboozzoo> <Merged by anonymouse64> <https://github.com/snapcore/snapd/pull/9364>13:01
mborzeckihmm, maybe we still need to improve error handling in nested prepare: https://paste.ubuntu.com/p/FBBRKSKBGH/14:01
ijohnsonif the snap command is available why would it be connection refused talking to the socket14:01
ijohnsonI guess sudo snap wait system seed.loaded in the nested_retry_until_success with a much shorter timeout on that14:02
mborzeckiijohnson: could it be that this ran after the socket was created by system, but before snapd.service started listening (or the service started)?14:06
ijohnsonmborzecki: but I thought that was the whole point of socket activation14:07
ijohnsonis that the stuff sent on the socket gets queued and "saved" so that when snapd is ready and starts listening using systemd's fd we just get what was sent before snapd finished starting14:08
mupBug #1887153 changed: Installing network-manager drops the network <Snappy:Invalid> <snappy-hwe-snaps:New> <https://launchpad.net/bugs/1887153>14:10
mupBug #1887153 opened: Installing network-manager drops the network <Snappy:Invalid> <snappy-hwe-snaps:New> <https://launchpad.net/bugs/1887153>14:13
mupBug #1887153 changed: Installing network-manager drops the network <Snappy:Invalid> <snappy-hwe-snaps:New> <https://launchpad.net/bugs/1887153>14:19
* cachio lunch14:51
mupPR snapd#9365 closed: release: 2.47~pre1 <Simple 😃> <Skip spread> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9365>15:27
mupPR snapd#9367 opened: tests: misc nested changes <Run nested> <Simple 😃> <Test Robustness> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9367>15:47
ijohnsonhmm does anybody remember the version of snapd we supported parallel installs of classic snaps?15:47
ijohnsondegville: when you have time could you review my update to https://forum.snapcraft.io/t/parallel-installs/7679 mentioning classic parallel snap support in 2.43+ ?15:51
ijohnsonhopefully I didn't botch it too badly :-)15:52
degvilleijohnson: yes, of course! Looks great - thanks for updating it.15:53
zygawoot, new test passes16:00
zygaa few more variants to cover everything16:21
zygaI'll go make tea16:21
zygaI like days like this16:21
zygaquiet focus16:21
cachiomvo, 2.47 is comming today, right?16:43
zyga-debug time16:54
=== King_InuYasha is now known as Conan_Kudo
=== Conan_Kudo is now known as King_InuYasha
ijohnsoncachio: could you look at my 9364 again, it seems my change to the debug: section for each suite didn't take when the nested tests failed, so I force pushed a different change which should work now I think17:22
=== ijohnson is now known as ijohnson|lunch
cachioijohnson|lunch, sure17:23
cachioijohnson|lunch, it is already merged17:24
ijohnson|lunchcachio: sorry I meant 936717:24
cachioijohnson|lunch, hehe, ok17:24
=== King_InuYasha is now known as Conan_Kudo
=== Conan_Kudo is now known as King_InuYasha
mvocachio: it's already in beta - at least the snapd snap17:49
mvocachio: systemd is yet again hold in the review queue17:50
=== ijohnson|lunch is now known as ijohnson
cachiomvo, ok, I'll start with snapd, thnanks17:50
mvocachio: great, thank you17:51
* zyga starts to prepare commits17:51
=== King_InuYasha is now known as Conan_Kudo
=== Conan_Kudo is now known as King_InuYasha
zygaok, one more case fixed18:57
zygathis test acts as extra documentation18:57
zygastill some failures on some systems19:11
zygaand core19:18
zygalet's dig at the classic set first19:18
mupPR snapcraft#3289 opened: build providers: fix issues running on Windows <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/3289>19:18
zygabut hey 29 / 40 passed :)19:19
cmatsuokaijohnson: did you try to build and install an edge system recently?19:27
ijohnsoncmatsuoka: yes, many19:27
ijohnsoncmatsuoka: let me read your mind19:27
ijohnsoncmatsuoka: it's broken isn't it?19:27
cmatsuokaijohnson: hum, I just tried it and it failed to install19:27
cmatsuokaah ok19:27
ijohnsonthe issue is that we need a kernel snap respin19:28
ijohnsonubuntu-core-initramfs was updated with new snap-bootstrap19:28
ijohnsonbut the kernel snap has not been rebuilt yet19:28
cmatsuokaah I see19:28
ijohnsonso edge is currently broken19:28
cmatsuokaah I thought it was published already19:28
ijohnsonnot yet19:29
* zyga takes the dog out19:30
cmatsuokamy idea was to build a edge image with maciek's gadget update snapd branch and try it with the edition3 gadget19:31
cmatsuokaand see if it reseals correctly19:31
ijohnsonyeah you'll need to repack the initramfs with the new version of ubuntu-core-initramfs19:32
cmatsuokado we have any estimate on when the kernel snap will be ready?19:33
ijohnsonno idea19:33
zygaback and looking at the debug prompt19:44
zygalooks like real byg19:48
zygaprintf debug time19:50
cachioijohnson, cmatsuoka I found something really insteresting about the issue on kvm20:22
cachioif I use the host system = debian 920:22
cachiono reboots20:23
cmatsuokathat's interesting20:23
cachiothe only diff is that on debian 9 I coundt make tpm work20:23
zygaok, interesting20:23
cachioI need to take a look to that20:23
zygabut looks like forced host tools20:23
zygabut why, maybe some silly bug20:24
cachiobut tmp does not affect on the reboots20:24
zygaanyway, thank for integration tests20:24
zygaI'll know on the next cycle20:24
cachioI tested ubuntu without tmp and ti reboots20:24
cachioso perhaps a solution for us is to replace ubuntu focal by debian 920:25
zygacachio: debian 9 has really old qemu20:25
zygaperhaps it doesn't do something fancy enough20:25
cachiozyga, I am gonna try usign the same qemu on focal20:25
zygaor doesn't interact with recent security mitigations20:25
cachiozyga, both things could be the problem20:26
ijohnsoncachio: but if you can't make tpm work with debian then it seems not useful since we need tpm20:27
cachiozyga, I'll try with debian sid20:27
cachioijohnson, that qemu does not support tpm emulation20:27
ijohnsoncachio: right but my point is that we need nested vm's with tpm to test uc2020:27
cachioijohnson, yes yes agre20:28
ijohnsonand afaik the reboots only happen with tpm20:28
cachioijohnson, no20:28
ijohnsonbut yes anyways very interesting point about using debian 920:28
cachioijohnson, reboots happen when we use ovmf20:28
cachioindependently if we use tpm20:28
cachioI tried removing tpm and the reboots still happen20:29
ijohnsoncachio: ah I see20:29
cachioijohnson, I'll try now with debian sid20:29
cachioit should support tpm by software20:30
cachioif sid fails the problem is in qemu/kvm version used20:30
ijohnsonYeah good find nonetheless20:32
cmatsuokasid has qemu 5.1 I believe20:32
mupPR snapcraft#3290 opened: build providers: unified provider refactoring for provider setup <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/3290>20:34
cachioI am creating a new image to support virtualization on sid20:34
cachiolet see20:34
zygawhat is /etc/sysconfig/snapd:SNAP_REEXEC=020:39
* zyga looks20:39
zygaat least that explains it20:42
zygathe test needs adjusting20:43
zygaif this passes the next error is on core20:58
zygaI'll keep pushing, open the PR and go to sleep20:59
zygabut I may be a bit tired tomorrow20:59
mupPR snapcraft#3289 closed: build providers: fix issues running on Windows <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3289>21:04
zygafedora passed21:04
zygaok, core21:04
zyganeed to stretch back21:08
zygaback with water,21:16
zygacore is rebooting21:16
zygaI'll know soon21:16
zygaok, weird21:20
zygalet's check21:20
zygaI think the test has a mistake21:20
cachioijohnson, cmatsuoka, zyga fails on debian sid21:25
cachioit reboots21:25
cachioso the qemu version21:25
cachioI need to find a version with supports tpm and no reboots21:26
cmatsuokais it using kvm or just software qemu?21:26
cmatsuokaif using kvm it should depend on the kernel too21:27
cachiocmatsuoka, in tha tpast I tried with a very old kernel21:27
cachioand it rebooted21:27
cachioan old kernel which supported kvm21:28
cachioI think I should find a version of qemu which works21:28
zygaI think we should ask some qemu people21:30
cmatsuokaour case is very specific even for qemu people (nested with ovmf)21:33
cmatsuokathe last time I asked some kvm engineers they said they never worked with that specific scenario :|21:34
cachiocmatsuoka,just ran in xenial and no reboots21:53
cachioqemu 2.521:53
cmatsuokaand does it work with tpm?21:54
cachionow I'' try to find which version works with tpm21:55
cachioI nee dto go now21:55
cachioI'll continue after I am back21:55
cachioneed to buy some stuff21:55
zygafigured it out21:57
zygaand documented21:57
zygaman so many sharp edges21:57
zygarestarting another run21:58
zygaall systems now21:58
zygaI bail22:11
zygasee you all22:11

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!