=== benfrancis8 is now known as benfrancis
[05:12] <mup> PR snapd#9279 closed: github: run tests also on push to release branches <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9279>
[05:16] <mborzecki> morning
[05:18] <mvo> hey mborzecki - good morning
[05:18] <mborzecki> mvo: hey
[05:19] <mborzecki> heh, my son got fever today, and it's only been like 4 days of school, didn't take long for him to catch something
[05:25] <mvo> mborzecki: oh no!
[05:25] <mvo> mborzecki: good luck
[05:26] <mvo> mborzecki: let's hope it goes away again quickly
[05:59] <mborzecki> and i can reproduce the problem with govendor sync on TW: https://paste.ubuntu.com/p/mDHNQV7kTD/
[05:59] <mborzecki> still, makes very little sense
[06:22] <mborzecki> quick errand, need to drive my daughter to school
[06:52] <mborzecki> re
[07:02] <pstolowski> morning
[07:03] <mvo> pstolowski: good morning
[07:03] <mvo> mborzecki: hm, memory exaution or something?
[07:03] <mborzecki> pstolowski:  hey
[07:03] <mborzecki> mvo:  idk, running with strace now
[07:03] <mvo> mborzecki: \o/
[07:04] <mborzecki> mvo: even `if ! govendor sync -v; then echo failed; fi` fails and shuts down the connection
[07:05] <mborzecki> mvo: oh, and it does not fail when i add exit 1 before the line in prepare-restore, and then run govendor sync from the debug shell
[07:08] <mvo> mborzecki: you run it with or without quiet?
[07:08] <mvo> mborzecki: I wonder if that has anything to do with it :/
[07:08] <mborzecki> mvo: with or without has same effect
[07:09] <mvo> :(
[07:09] <mvo> woah
[07:23] <zyga> good morning
[07:24] <mvo> good morning zyga
[07:24] <zyga> how are you doing? it's very warm for autumn this year
[07:25] <zyga> I'm still looking after Lucy, just getting ready for our call
[07:28] <mvo> zyga we can postpone if you want
[07:33] <mborzecki> zyga: ehy
[07:33] <mborzecki> hey
[07:33] <mborzecki> ;P
[07:37] <zyga> mvo: that's fine, lets's have the call
[07:38]  * zyga needs to restart avahi after setting up some bridges last night
[07:38] <zyga> anyway
[07:42] <mvo> zyga sure thing
[07:43]  * mvo is happy that zyga build bridges instead of burning them
[07:43] <zyga> hahaha
[07:43] <zyga> I have a routable riscv64 VM now
[07:43] <zyga> something that was in the back of my head for a while
[07:50] <mborzecki> Sep 07 07:49:52 sep070746-567919 kernel: sshd[1445]: segfault at 278 ip 000055e17401a553 sp 00007ffc9aba67c0 error 4 in sshd[55e173fd8000+74000]
[07:50] <mborzecki> Sep 07 07:49:52 sep070746-567919 kernel: Code: 1f 84 00 00 00 00 00 48 85 ff 0f 84 d7 00 00 00 41 54 55 48 89 fd 48 83 ec 08 48 8b 87 20 01 00 00 48 85 c0 0f 84 95 00 00 00 <8b> 40 18 a8 02 7
[07:50] <mborzecki> 5 46 a8 04 0f 85 8e 00 00 00 48 8b 7d 08 e8 05 f0
[07:50] <mborzecki> on tumbleweed when disconnecting spread
[07:50] <mborzecki> maybe that's why debug shell cannot be connected?
[07:50] <zyga> seg.vfault() :(
[07:51] <zyga> yeah, this could be a problem
[07:51] <zyga> maybe we need to reboot after upgrades?
[07:52] <mborzecki> zyga: looking at zypper log there's  no updates pulled in spread stats
[07:52] <zyga> hmm
[07:53] <zyga> maybe that's something exploiting an unknown bug in ssh?
[07:53] <mborzecki> oh wow, kernel bug? https://paste.ubuntu.com/p/bDGGb7BhWX/
[07:53] <zyga> those machines are under constant barrage of ssh connections
[07:53] <mborzecki> the network is down at this point
[07:53] <zyga> interesting
[07:53] <zyga> xfs
[07:53] <mborzecki> idk, maybe this snapshot is just bad :/
[07:54] <zyga> that's 5.8?
[07:55] <mborzecki> yes
[07:55] <mborzecki> there's a bunch of updates (kernel among those)
[07:56] <mborzecki> maybe we should ask cachio to update the images and we'll try again
[07:56] <zyga> yeah
[07:56] <zyga> good idea
[07:56] <zyga> I updated my TW install and didn't see any issues
[07:56] <zyga> and I released 2.46 there, waiting for .1 tarballs
[07:57] <zyga> mvo: do you remember if 2.46.1 tarball contains the libexec patch from Maciej?
[07:58] <mborzecki> zyga: it does not, the PR is still up https://github.com/snapcore/snapd/pull/9280 and we cannot merge it because the spread job on TW does not run
[07:58] <mup> PR #9280: packaging/opensuse: fix for /usr/libexec on TW, do not hardcode AppArmor profile path <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9280>
[07:58] <zyga> mborzecki: aha
[07:58] <zyga> ok, I'll cherry pick that as distro patch
[07:58] <zyga> it's interesting that openSUSE is the redhat-like distro but with apparmor
[07:59] <zyga> can we do libexec in ubuntu 14.04+ and remove all patches now ;-) ?
[07:59] <zyga> (no, we cannot because debian stable)
[07:59] <mup> PR core#118 opened: hooks: update motd handling and remove 50-motd-news <Created by mvo5> <https://github.com/snapcore/core/pull/118>
[08:24]  * pstolowski errand
[08:43] <zyga-kaveri> re
[08:43] <zyga-kaveri> in the office
[08:44] <mup> PR core#38 closed: Add another pi-config option <Created by sergey-borovkov> <Closed by pedronis> <https://github.com/snapcore/core/pull/38>
[08:44] <mup> PR core#110 closed: Disable dynamic motd <Created by vorlonofportland> <Closed by mvo5> <https://github.com/snapcore/core/pull/110>
[08:44] <mup> PR core#111 closed: live-build/hooks/motd: disable dynamic motd, disable motd services <Created by bboozzoo> <Closed by mvo5> <https://github.com/snapcore/core/pull/111>
[09:11]  * zyga-kaveri goes upstairs for a pullover
[09:11] <zyga-kaveri> mvo: when convenient please provide 2.46.1 tarballs
[09:11] <mvo> zyga-kaveri: will do, sorry!
[09:27] <zyga-kaveri> mvo: no problem
[09:38] <mvo> zyga-kaveri: it's up now
[09:38] <zyga-kaveri> mvo: thanks
[09:43] <mup> PR snapd#9289 opened: github: run tests also on push to release branches <Simple 😃> <Skip spread> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9289>
[09:48] <mup> PR snapd#9290 opened: release: 2.46.1 <Skip spread> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9290>
[10:00] <pstolowski> re
[10:18] <mup> PR snapd#9289 closed: github: run tests also on push to release branches <Simple 😃> <Skip spread> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9289>
[10:38] <mup> PR snapd#9210 closed: daemon: add /v2/systems "reboot" action API <UC20> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9210>
=== pedronis_ is now known as pedronis
[11:03]  * zyga-kaveri breaks for coffee and then goes to write manager hooks
[11:12] <mvo> I updated 9021, hopefully a relatively simple review. for snap reboot
[11:24] <mborzecki> hmm `snapd   2.46.1-1-dirty` mkversion getting confused by something there?
[11:35] <pedronis> mvo: I answered your question there, but didn't do a review yet
[11:40] <mborzecki> quick errand, back in 30
[12:03] <mborzecki> re
[12:07] <mborzecki> cachio: hi, can you take a look whether we can update opensuse tumbleweed images maybe?
[12:07] <cachio> mborzecki, hi, yes
[12:08] <mborzecki> cachio: looks like the current image is broken, and that's what is causing issues with govendor sync
[12:08]  * zyga-kaveri writes more tests
[12:08] <mborzecki> cachio: i was able to observe a kernel panic when govendor sync runs, breaking the host
[12:08] <pedronis> pstolowski: is #9270 ready for reviews ?
[12:08] <mup> PR #9270: [RFC] wrappers, systemd: allow empty root dir and conditionally do not pass --root to systemctl <Run nested> <Services ⚙️> <Created by stolowski> <https://github.com/snapcore/snapd/pull/9270>
[12:10] <pstolowski> pedronis: yes but i'd wait for spread tests, so far i saw many failures which seemed unrelated
[12:10] <pstolowski> but maybe not, hard too tell, too much noise there
[12:11] <pstolowski> i'm keeping an eye on that one
[12:12] <pstolowski> pedronis: if you could advise on error code for #9259 that would be great
[12:12] <mup> PR #9259: client, api: handle insufficient space error <Disk space awareness> <Needs Samuele review> <Created by stolowski> <https://github.com/snapcore/snapd/pull/9259>
[12:12] <pedronis> I plan to review that soon
[12:33] <mup> PR snapd#9285 closed: many: seed.Model panics now if called before LoadAssertions <UC20> <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/9285>
[12:38] <mborzecki> pedronis: hm building on sid with nosecboot must have been failing for a while now
[12:39] <pedronis> mborzecki: it passes in other PRs
[12:40] <mborzecki> pedronis: maybe go is smart to figure out what it needs during the build, anyways, had to fix debian packaging to not remove secboot/encrypt.go (added earlier i believe) and gadget/install/params.go (added in claudio's PR)
[12:40] <mborzecki> and yeah, the fun code copying over stuff to _build
[12:41] <pedronis> mborzecki: something is wrong
[12:43] <pedronis> mborzecki: why params and encrypt?
[12:44] <mborzecki> pedronis: install_dummy references Options (defined in params.go), and SystemInstallObserver.ChosenEncryptionKey() takes secboot.EncryptionKey (defined in secboot/encrypt.go)
[12:44] <pedronis> yes, but secboot/encrypt.go doesn't refer to anything interesting
[12:44] <pedronis> what am I missing
[12:45] <pedronis> I don't think you can build devicestate without params
[12:45] <mborzecki> pedronis: hm maybe i wasn't clear, afaict secboot/encrypt.go is needed to get secboot.EncryptionKey, used by gadget/install/params.go
[12:46] <pedronis> ah, not remove
[12:46] <mborzecki> pedronis: and the code that sets up _build would remove everything from gadget/install and secboot but a few select files
[12:46] <pedronis> sorry, I misread your comment
[12:46] <mborzecki> pedronis: cool, all is clear now ;)
[12:49] <mborzecki> cachio: let me know when you have a new opensuse tumbleweed image to try, wonder if that kernel bug is no longer happening
[12:49] <cachio> mborzecki,  it is in the queue
[12:50] <cachio> to be updated, i think in 20 minutes should be ready
[12:53] <mvo> pedronis: thankyou!
[12:56] <mup> PR snapcraft#3279 opened: Set VDPAU_DRIVER_PATH appropriately <Created by flexiondotorg> <https://github.com/snapcore/snapcraft/pull/3279>
[13:32] <mborzecki> cachio: can you take a look at the console of spread node `sep071325-531734`?
[13:33] <cachio> mborzecki, sure
[13:34] <cachio> mborzecki, can't see that instances
[13:34] <mborzecki> cachio: hmm ok, i died when i tried to run the tests
[13:35] <mborzecki> cachio: let me start another one
[13:35] <mborzecki> cachio: can you observe the console of this one `sep071335-735209` ?
[13:38] <mborzecki> **** fought a bit with Sid packaging in Claudio's PR https://github.com/snapcore/snapd/pull/9265
[13:38] <mup> PR #9265: many: move seal code from gadget/install to boot <Run nested> <UC20> <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/9265>
[13:38] <mborzecki>      - we have some special code that sets up a separate source tree under _build
[13:38] <cachio> mborzecki, kernel BUG at mm/filemap.c:1315!
[13:38] <mborzecki>      - it would remove all unnecessary files, but a few selected ones
[13:38] <mborzecki>      - with the changes from the PR, files that were removed are actually needed
[13:38] <mborzecki>        to build with nosecboot tag
[13:38] <mborzecki> another bug? https://paste.ubuntu.com/p/273tJhBxVd/
[13:38] <pedronis> pstolowski: I reviewed #9259
[13:38] <mup> PR #9259: client, api: handle insufficient space error <Disk space awareness> <Needs Samuele review> <Created by stolowski> <https://github.com/snapcore/snapd/pull/9259>
[13:38] <mborzecki> cachio: yeah, and now it's dead isn't it?
[13:39] <mborzecki> duh, secondary clipboard
[13:40] <cachio> mborzecki, https://paste.ubuntu.com/p/GXsBpHsmD7/
[13:40] <cachio> still running
[13:43] <mborzecki> cachio: hmm BUG: workqueue lockup - pool cpus=0 node=0 flags=0x0 nice=0 stuck for 60s!
[13:43] <mborzecki> zyga-kaveri: ^^
[13:44] <mborzecki> cachio: tumbleweed is not an official image, we build it ourselves right?
[13:45] <cachio> mborzecki, yes
[13:45] <cachio> no official cloud image
[13:45] <cachio> mborzecki, this is hte script we use for that https://github.com/snapcore/spread-images/blob/master/tasks/google/add-opensuse-tumbleweed-64/task.yaml
[13:54] <mup> PR snapd#9290 closed: release: 2.46.1 <Skip spread> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9290>
[14:02] <pstolowski> pedronis: ty
[14:33] <mborzecki> mvo: i can try a reproduce the problem ian had on rpi4, got 1.2rev here
[14:35] <mvo> mborzecki: cool, if it's not too much hassle, that would be great
[14:44] <mup> PR snapd#9291 opened: client: implement RebootToSystem <Created by mvo5> <https://github.com/snapcore/snapd/pull/9291>
[15:04]  * cachio lunch
[16:16] <mvo> did anyone look into the 503 we get from the store for the main/searching test ? has anyone more info on that? or contacted the store team?
[16:21] <cachio> mvo, I'll do it
[16:22] <mvo> thanks cachio
[16:27] <pstolowski> i'm seeing a lot of that too
[16:29] <cachio> pstolowski, can you reproduce the isuse doing -> snap find --section=photo-and-video vlc ?
[16:30] <pstolowski> cachio: no
[16:30] <cachio> pstolowski, tx
[16:31] <mvo> me neither
[16:46] <pstolowski> however, mistyping section name gives 403, was it always like that?
[18:33] <zyga> back from PT
[18:34] <zyga> issues with pi?
[18:35] <zyga> ijohnson, cachio: anything new?
[19:07] <cachio> zyga, sorry, issues?
[19:07] <cachio> beta validation was ok for rpi
[19:08] <cachio> didn't see anything weird
[19:08] <zyga> ok
[19:10]  * zyga finishes supper and will look at writing one more function