[04:46] <mup> PR snapd#8326 opened: [wip] boot: move initramfs-mounts logic to boot pkg <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8326>
[06:48] <mborzecki> morning
[06:53] <mborzecki> ayy new kernel, brb
[06:55] <mborzecki> re
[07:06] <zyga> Good morning
[07:06] <zyga> I will make coffee first
[07:10] <mborzecki> zyga: hey
[07:29] <jamesh> mborzecki: thanks for your feedback on the xdg-open-to-portal PR.  The test is now successfully running on Arch.
[07:30] <jamesh> mvo: I think https://github.com/snapcore/snapd/pull/8318 is ready, although it hit two unrelated spread test failures
[07:30] <mup> PR #8318: tests: ensure sockets target is ready in session agent spread tests <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/8318>
[07:42] <mborzecki> jamesh: thanks for updating the PRs!
[07:43] <mborzecki> zyga: do you remember whether spread sets nullglob in bash?
[07:46] <mvo> jamesh: yeah, pushed one fix for this test failure
[07:46] <mup> PR snapd#8327 opened: tests: update proxy-no-core to match latest CDN changes <Simple 😃> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8327>
[07:46] <mvo> jamesh: looking at the other now
[07:47] <jamesh> mvo: the other is a "google-unstable" task, so perhaps doesn't matter
[07:48] <mvo> jamesh: yeah, the unstable ones are not blocking merges
[07:49] <mborzecki> mvo: hm gperf seems to be gone from our 18.04 images, causing snap-seccomp-syscalls tests to fail
[07:51] <jamesh> mborzecki: the xdg-open-portal spread test fails when I was running it on debian-9 and opensuse-15.1.  I'm not sure if it is down to the old xdg-desktop-portal versions, or if fakeportalui is using some Python API not available in those release
[07:51] <zyga> mborzecki: I don't know what nullglob is
[07:51] <zyga> espanding non-matching glob to nothing?
[07:51] <zyga> hey mvo
[07:51] <mborzecki> zyga: yes
[07:51]  * zyga checks if travis responds 
[07:52] <zyga> yesterday I could not get travis-ci.io to load at all
[07:52] <zyga> I see it's fixed now
[07:52] <mborzecki> zyga: so foo* gives nothing instead of 'foo*' ;)
[07:52] <zyga> yeah
[07:53] <zyga> huh
[07:53] <zyga> master failed
[07:53] <zyga> please install gperf
[07:53] <zyga> what the f***
[07:53] <zyga> configure: error: please install gperf
[07:53] <zyga> *why* do we need g-frelling-perf now
[07:53] <mborzecki> zyga: on it now
[07:53] <zyga> thank you
[07:53] <zyga> sigh
[07:53] <zyga> I want to get us off autotools
[07:54] <mborzecki> zyga: seems it was preinstalled in 18.04 images before, but is no longer there
[07:54] <zyga> we're not using it, why is it mandatory?
[07:54] <zyga> thank you maciek
[07:54]  * zyga tries something new
[07:54] <mborzecki> zyga: we build libseccomp upstream and they use gperf, since like 3 months now ;)
[07:55] <zyga> ah
[07:55] <zyga> oh boy
[07:55] <zyga> man
[07:55]  * zyga is unusually grumpy today, sorry
[07:55] <mup> PR snapd#8328 opened: tests/main/snap-seccomp-syscalls: install gperf <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8328>
[07:55] <zyga> mborzecki: but it's weird that we depend on it
[07:56] <zyga> it's usually a build-dep
[07:56] <zyga> not a runtime dep
[07:56] <mborzecki> zyga: the test depends on it, not snapd
[07:56] <zyga> ah
[07:56] <zyga> sorry, I missed that we build libseccomp upstream
[07:56] <mborzecki> zyga: look at what the test does
[07:56] <zyga> btw, why do we do that? to check for upcoming regressions?
[07:56] <mborzecki> zyga: yes, and we know when the list of syscalls we know of needs to be updated
[07:57] <mborzecki> zyga: the part of not rebuilding seccomp profiles when not needed
[08:01] <mborzecki> zyga: haha so found something interesting in #8316 build log
[08:01] <mup> PR #8316: github: add prototype workflow running unit tests <Skip spread> <Created by zyga> <https://github.com/snapcore/snapd/pull/8316>
[08:02] <mborzecki> wtf is riscv_flush_icache syscalls?
[08:02] <zyga> mborzecki: riscv flush instruction cache?
[08:02] <zyga> mborzecki: what did you find?
[08:02] <mborzecki> zyga: left a comment there
[08:02] <zyga> ok
[08:03] <pstolowski> morning
[08:03] <zyga> hahaha
[08:03] <zyga> mborzecki: that's funny
[08:03] <zyga> I wonder how they do it
[08:03] <zyga> but they apparently do a better job than we do :D
[08:03] <mborzecki> zyga: hopefully #8295 will fix that
[08:03] <mup> PR #8295: osutil: do not leave processes behind after the test run <Created by mvo5> <https://github.com/snapcore/snapd/pull/8295>
[08:03] <zyga> mhm
[08:03] <mvo> mborzecki: yeah, looking at the snap-seccomp-syscalls test right now - unless someone else was faster, then I will stop
[08:03] <mborzecki> mvo: i'm on it already
[08:03] <zyga> thank you guys :)
[08:03] <mvo> mborzecki: cool
[08:04] <mvo> mborzecki: aha, nice
[08:11] <mborzecki> ouch, our PR title checker just got `urllib.error.HTTPError: HTTP Error 429: too many requests`
[08:12] <mup> PR snapd#8321 closed:  cmd/snap-failure,tests: try to make snap-failure more robust (2.44) <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8321>
[08:17] <mborzecki> hmm, actually why is oldwait4 removed?
[08:17] <mvo> mborzecki: yeah, this is strange
[08:17] <mvo> mborzecki: also, why did this change, what package changed?
[08:18] <pedronis> mvo: hi, did you see my comment in 8325 ?
[08:19] <mborzecki> mvo: hmm https://github.com/seccomp/libseccomp/commit/4bd8de391d5920bdbdb639b1d81ddd7bfe4d7549
[08:21] <mvo> mborzecki: I didn't, let me look
[08:21] <mvo> mborzecki: oh, interessting
[08:22] <mborzecki> mvo: i'll check the list of syscalls from HEAD and one before that commit
[08:22] <mvo> pedronis: yeah, will fix, makes sense
[08:23] <mvo> pedronis: I want to look at improving unit tests too but otherwise that would be the recover mode
[08:23] <mvo> mborzecki: aha, we git checkout to test for the syscalls, right? so we are always git-fresh?
[08:23] <mborzecki> mvo: yes
[08:26] <zyga> mvo: so, I guess we can kind of do nvidia tests
[08:26] <zyga> mvo: and pi tests
[08:27] <zyga> mvo: or aarch64 tests on specific board even if we wanted to
[08:27] <mvo> zyga: we can or we can't ?
[08:27] <zyga> we *can* :)
[08:32] <mborzecki> mvo: so this commit introduces the list of syscalls that is used right now, oldwait4 is no longer there https://github.com/seccomp/libseccomp/commit/f5b3166d6126f4a45b59d6af6780b5e00a0e9867
[08:34] <mvo> mborzecki: ok
[08:35] <mborzecki> there is no __NR_oldwait4 in my system either
[08:35] <zyga> mborzecki: does it spell trouble?
[08:35] <pedronis> mvo: you might or not conflict with Ian draft PR, otoh that one needs work, it feels too complicated atm
[08:35] <mvo> pedronis: ok
[08:36] <mborzecki> zyga: mvo: intersting, we have that syscall in our template
[08:37] <zyga> har har
[08:37] <zyga> I wonder why they removed it
[08:37] <zyga> is it really gone
[08:37] <zyga> was it ever available
[08:37] <mborzecki> zyga: mvo: and snap-seccomp ignores the syscalls that are unknown to libseccomp library, maybe the test should be smarter about checking?
[08:37] <zyga> on 14.04?
[08:37] <zyga> it only ignores because error checking in seccomp is not great
[08:38] <zyga> we have not way to differentiate "this is a valid syscall but your arch doesn't have it" from "lol, no, that is bogus"
[08:38] <mvo> it's a bit mysterious, linux famously never breaks abi with userland so just removing it if it was there sounds strange
[08:38] <mborzecki> zyga: fwiw, i don't see that syscall in my kernel source tree either
[08:38] <zyga> mborzecki: try 14.04
[08:41] <pedronis> mvo: I noticed the problem in #8237 last night, I pinged cachio about it
[08:42] <mup> PR #8237: interfaces/{docker,kubernetes}-support: updates for lastest k8s - 2.44 <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8237>
[08:42] <pedronis> grumble, 8327
[08:43] <zyga> ?
[08:44] <mvo> pedronis: what problem in 8237 is that? the k8s update looks ok to me, what am I missing?
[08:44] <mvo> oh, sorry
[08:44] <pedronis> mvo: I mistyped
[08:44] <mvo> pedronis: let me look at 8327 instead :)
[08:45] <mvo> pedronis: but sergio did not do a pr yet, did he?
[08:45] <pedronis> seems not
[08:45] <pedronis> it was late for him as well I suppose
[08:45] <mvo> ok, should be fine, the fix is trivial
[08:45] <mborzecki> zyga: mvo: we should be fine afaict https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a0673fdbcd42105261646cd4f3447455b5854a32
[08:46] <pedronis> pstolowski: hi, what's the state of the search v2 PR? there are still some comments to address, right?
[08:46] <mborzecki> mvo: zyga: this was done in 4.17
[08:47] <mborzecki> i'll update the commit message
[08:48] <mvo> mborzecki: hm,hm, we support 4.4 back in 16.04, still harmless?
[08:49] <pedronis> UC 18 is 4.15
[08:49] <pedronis> that's also older than 4.17
[08:49] <pedronis> only 20 is newer than 4.17 at 5.4
[08:49] <mborzecki> mvo: aiui it's only used on 'score' architecture, which got removed in 4.17 as well
[08:50] <zyga> brb, some stuff at home
[08:52] <pstolowski> pedronis: i think i addressed all the comments except for potential specialial-casing of some errors from error-list, let me reply to all comments
[08:52] <mvo> mborzecki: ok, spread should tell us is there is any issue
[08:52] <pstolowski> pedronis: ah, sorry, missed one about copyNonZero..
[08:57] <pedronis> pstolowski: I added 2 more comments to #8309
[08:57] <mup> PR #8309: o/configcore: implement Apply method for early configuration of core <Created by stolowski> <https://github.com/snapcore/snapd/pull/8309>
[08:57] <pstolowski> ty
[08:57] <pedronis> pstolowski: but let's try to wrap search v2 first
[08:58] <pstolowski> pedronis: okay; we will need to wait for store rename still
[08:58] <pedronis> pstolowski: yes, but let's try to get the rest in order and also get a 2nd review
[08:58] <pstolowski> sure
[08:59] <pedronis> mvo: mborzecki: when you have a moment can you review #8253
[08:59] <mup> PR #8253: snap-bootstrap: expand data partition on install <UC20> <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/8253>
[08:59] <mborzecki> mvo: just to be sure, i pinged jdstrand and updated the commit message with relevant links
[09:00] <mborzecki> mvo: as i understand it, unless we supported that architecture (which nobody did?), the syscall should not come up
[09:00] <mborzecki> pedronis: sure
[09:02] <zyga> re
[09:05] <pedronis> mvo: so my comment in your recover auth PR was really about the original one? #8010
[09:05] <mup> PR #8010: snap-bootstrap: add support for "recover" mode <UC20> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8010>
[09:06] <mvo> pedronis: yes, but it's fine, we can use only a single PR, the copy auth data is a bit complex so I was splitting but it's not long so either way is fine with me
[09:06] <mup> PR snapd#8327 closed: tests: update proxy-no-core to match latest CDN changes <Simple 😃> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8327>
[09:08] <pedronis> mvo: no, let's try to merge 8010 first?
[09:08] <mvo> pedronis: +1
[09:08] <pedronis> but the Buffer stuff needs fixing
[09:09] <mup> PR snapd#8313 closed: release: 2.44.1 <Simple 😃> <Skip spread> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8313>
[09:09] <pedronis> mvo: what happens right now in recover mode? do we run console-conf or did we turn that off?
[09:10] <mvo> pedronis: right now recover mode just errors, with 8010 as is we would run console-conf and allow you to create a new login (it's unmanaged) and then you could ssh into it and inspect recovery-ubuntu-data
[09:11] <mvo> pedronis: so we want the auth commit too
[09:11] <mvo> pedronis: (that was the question, right?)
[09:11] <pedronis> mvo: did we turn off console-conf in install?
[09:11] <mvo> pedronis: let me quickly check, iirc we have a ConditionKernelCommandline for it, but not sure how targeted that is
[09:14] <mvo> pedronis: it looks like it's active, I checked the released source and I see no kernelcommandline condition, let me double check with git
[09:16] <mup> PR snapd#8295 closed: osutil: do not leave processes behind after the test run <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8295>
[09:16] <mvo> no kernel commandline condition in git either
[09:16] <pedronis> pstolowski: you already have those apiV1 checks at the end of the test, the commented bits just needed removing afaict
[09:17] <pstolowski> pedronis: ahh, of course, you're right, thanks, missed them
[09:24] <pedronis> mvo: we need to do something about that
[09:25] <pedronis> mvo: either with the unit or setting something from bootstrap
[09:25] <pedronis> mvo: remember that we use it for the chooser as well now? mborzecki
[09:26] <pedronis> so we need to be careful what we do?
[09:29] <zyga> mvo: do you want to merge https://github.com/snapcore/snapd/pull/8328 before jamie review (it fixes master)
[09:29] <mup> PR #8328: tests/main/snap-seccomp-syscalls: install gperf <Simple 😃> <⚠ Critical> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8328>
[09:32] <mvo> pedronis: right, so I think we need to not run it in install mode, for recover mode we probably don't want to run it either, the only corner case would be if the device has no user
[09:32] <mvo> zyga: yes
[09:33] <pedronis> mvo: yes, but I think it has code that checks for managed vs not, the problem is that your copying of auth alone will not have snapd give the right answer
[09:34] <pedronis> because that requires users in state
[09:34] <mvo> pedronis: exactly, fixing this would be hard, we would have to have some smarts to just extract the users from state and copy that over to the ephemeral part
[09:35] <mvo> pedronis: but then a state.json would prevent the first time seeding so :(
[09:35] <pedronis> mvo: that's not true, seeding is based on seed flag
[09:35] <pedronis> mvo: anyway first we need to decide what is the right thing to do, 2nd what we can do now
[09:35] <mvo> pedronis: right
[09:37] <mvo> pedronis: indeed, I was thinking of https://github.com/snapcore/core20/blob/master/static/usr/lib/core/run-snapd-from-snap#L57 but ian fixed it a while ago so we can actually transfer the user
[09:37] <mvo> pedronis: in this case it seems like the right thing would be to transfer the "user" part of the state to the ephemeral state in "run" mode, yes? and disable console-conf in install mode as it's meaningless here
[09:38] <pedronis> mvo: yes, something like that
[09:38] <pedronis> while keeping the chooser working
[09:39] <mup> PR snapd#8318 closed: tests: ensure sockets target is ready in session agent spread tests <Created by jhenstridge> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8318>
[09:39] <mvo> pedronis: indeed
[09:40] <mvo> mborzecki: if we want to disable console-conf in "install" mode (but keep the recovery-chooser) what is the right thing to do? can we just add a "kernel command line not snapd_recover_mode=install" or will that break the chooser now that they are more interwined?
[09:41] <mup> PR snapd#8322 closed: client: use Assert when checking for error <Simple 😃> <Skip spread> <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8322>
[09:41] <mup> PR snapd#8323 closed: many: improve comments, naming, a possible TODO <Created by pedronis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8323>
[09:42] <mborzecki> mvo: the chooser doesn't really run in install mode
[09:42] <mvo> mborzecki: aha, nice
[09:42] <mvo> mborzecki: what prevents it from running?
[09:43] <mborzecki> mvo: hm maybe i wasn't clear, it does not need to run in install mode, probably the same for trigger
[09:44] <mborzecki> mvo: so we could add ConditionKernelCommandLine=!snapd_recovery_mode=run right?
[09:44] <mborzecki> uh, way
[09:44] <mup> PR snapd#8319 closed: github: run unit tests and spread via github actions <Created by zyga> <Closed by zyga> <https://github.com/snapcore/snapd/pull/8319>
[09:44] <mup> PR snapd#8324 closed: github: cache go build cache and pkg dirs <Created by zyga> <Closed by zyga> <https://github.com/snapcore/snapd/pull/8324>
[09:45] <mborzecki> mvo: snapd_recovery_mode=run, and we need to make sure that run is always passed in the mode
[09:45] <mvo> mborzecki: yeah, I think we don't prevent it from running right now and should. so adding your suggestion sounds right
[09:46] <mvo> mborzecki: pretty sure we always set snapd_recover_mode
[10:08] <pedronis> pstolowski: +1 to #8309 but with some final suggestions which I hope make sense
[10:08] <mup> PR #8309: o/configcore: implement Apply method for early configuration of core <Created by stolowski> <https://github.com/snapcore/snapd/pull/8309>
[10:10] <pstolowski> pedronis: thanks!
[10:23] <pedronis> mvo: made a couple of suggestions: https://github.com/snapcore/snapd/pull/8325#discussion_r397044896
[10:24] <mup> PR #8325: snap-bootstrap: copy auth data from real ubuntu-data in recovery mode <UC20> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8325>
[10:25] <mvo> pedronis: thank you!
[11:02] <mup> PR snapd#8316 closed: github: add prototype workflow running unit tests <Skip spread> <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8316>
[11:05] <zyga> please let me know if the new github actions misbehaves
[11:23] <mborzecki> opened the chooser PR https://github.com/CanonicalLtd/subiquity/pull/655 ;)
[11:23] <mup> PR CanonicalLtd/subiquity#655: console_conf: implement UC20 recovery chooser <Created by bboozzoo> <https://github.com/CanonicalLtd/subiquity/pull/655>
[11:23] <mborzecki> i should probably add some tests, because there's none for console-conf atm
[11:23] <mborzecki> at the same time, there's some snapd work to do still
[11:53] <mup> PR snapd#8329 opened: allow raw access to USB printers <Created by ogra1> <https://github.com/snapcore/snapd/pull/8329>
[11:54] <zyga> mvo: ping
[11:57] <ogra> zyga, hmm, your actions port writes werid emails ... i just got sent a link to someone elses merge it seems
[11:57] <zyga> what?
[11:58] <zyga> can you be more specific please
[11:59] <ogra> zyga, i added a PR ... https://github.com/snapcore/snapd/pull/8329 ... and got a mail https://paste.ubuntu.com/p/G5bFF7TyJk/ ... clicking the link in the mail gets me to: https://github.com/ogra1/snapd/runs/530255923
[11:59] <mup> PR #8329: allow raw access to USB printers <Created by ogra1> <https://github.com/snapcore/snapd/pull/8329>
[11:59] <zyga> interesting
[12:00] <zyga> what was in the mail?
[12:00] <zyga> ah, one sec
[12:00] <ogra> the tests for my PR seems to be fine though
[12:00] <zyga> yeah, it's weird
[12:00] <zyga> I can see the test passed just fine
[12:00] <ogra> the pastebin has all the content of the mail
[12:00] <zyga> wait
[12:00] <zyga> that's different
[12:00] <ogra> subject was: 	[ogra1/snapd] Run failed: Quick Go unit tests - master (ff4e1ba)
[12:00] <zyga> it failed on _your_ master
[12:00] <ogra> why would it run anything on my master ?
[12:01] <zyga> because you pulled actions
[12:01] <zyga> https://github.com/ogra1/snapd/runs/530255923
[12:01] <ogra> bah
[12:01] <zyga> ha
[12:01] <zyga> funny
[12:01] <zyga> ok, I know how to fix it
[12:01] <zyga> thank you for reporting
[12:01] <zyga> I will handle it :)
[12:01] <zyga> go's very picky about repository name
[12:01] <zyga> thanks ogra :)
[12:01] <ogra> np :)
[12:06] <zyga> ogra: done
[12:06] <zyga> https://github.com/snapcore/snapd/pull/8330
[12:06] <mup> PR #8330: github: always checkout to snapcore/snapd <Created by zyga> <https://github.com/snapcore/snapd/pull/8330>
[12:06] <mup> PR snapd#8330 opened: github: always checkout to snapcore/snapd <Created by zyga> <https://github.com/snapcore/snapd/pull/8330>
[12:06] <zyga> can you cherry-pick this into your master
[12:06] <zyga> to see if it fixes
[12:06] <zyga> just fetch from my remote and cherry pick and push to your fork's master branch please
[12:07] <cachio> mvo, pedronis sorry, yesterday I missed the comment about the proxy
[12:11] <cachio> mvo, I just promoted core to candidate
[12:26] <mvo> cachio: nice, thanks
[12:26] <mvo> cachio: proxy> no worries, it's fixed now
[12:27] <cachio> mvo, about cloud config https://paste.ubuntu.com/p/TbTrf4Dcfp/
[12:27] <mvo> cachio: is that working?
[12:27] <cachio> I am doing that to update the image
[12:28] <cachio> but It didnt work yesterday
[12:28] <cachio> now I am going to boot locally the image
[12:28] <cachio> mvo, then I could not continue because of 2.44.1
[12:29] <cachio> mvo, do you see anything that could be improved in that code?
[12:30] <mup> PR snapd#8331 opened: github: run spread via github actions <Created by zyga> <https://github.com/snapcore/snapd/pull/8331>
[12:32] <mvo> cachio: it llooks ok
[12:32] <mup> PR snapd#8332 opened: gadget: CoreDefaults helper for FilesystemOnlyApply (2/N) <Created by stolowski> <https://github.com/snapcore/snapd/pull/8332>
[12:33] <cachio> mvo, ok, I'll boot the image here in that case because yesterday didn't work
[12:35] <mvo> cachio: uh, ok - would be nice to inspect the resulting image, i.e. if the file is in the right place on ubuntu-seed and ubuntu-data
[12:35] <mvo> cachio: maybe it's something silly during the data copy
[12:37] <cachio> mvo, sure, I'll check that
[12:46] <mup> PR snapd#8330 closed: github: always checkout to snapcore/snapd <Skip spread> <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8330>
[12:46] <zyga> ta
[13:11] <mup> PR snapd#8328 closed: tests/main/snap-seccomp-syscalls: install gperf <Simple 😃> <⚠ Critical> <Created by bboozzoo> <Merged by zyga> <https://github.com/snapcore/snapd/pull/8328>
[13:16] <zyga> mvo: ^ fyi, I think master should be not broken anymore
[13:35] <ijohnson> hello folks
[13:35] <roadmr> ijohnson!!!!!!!!!!!!!!!!!!!!!!
[13:35] <roadmr> hello!
[13:35] <roadmr> hehe :)
[13:35] <ijohnson> hey roadmr
[13:36]  * ijohnson thinks about running away with that many exclamation marks
[13:36] <roadmr> yes it's typically followed by a very complicated ask or something
[13:36] <roadmr> but no - just saying hi :)
[13:36] <ijohnson> :-)
[13:37] <ijohnson> pedronis: do you want to chat after the SU today? I realize my PR is a bit complicated, but I rather like the fact that all the base snap modeenv changes is done in bootstate20 alongside all the other boot state changes we do during normal running
[13:38] <ijohnson> pedronis: I will try to look at your comments before the SU
[13:39] <pedronis> ijohnson: I don't the issue is bootstate20 vs not, the issue is that is following patterns that are bit overkill for it inside it, I think there should be a complexity middle ground, but yes, happy to chat
[13:39] <ijohnson> pedronis: okay
[13:39] <pedronis> ijohnson: the other issue is that boot doesn't care about snapd so far, and that I think we should keep it that way
[13:40] <ijohnson> pedronis: but for uc20, now the snapd snap is part of the boot process on first boot run mode and in install mode
[13:40] <pedronis> snapd itself doesn't require to touch bootloader states or modeenv anyway atm
[13:40] <jdstrand> mborzecki: I looked at the PR and said it LGTM
[13:40] <mborzecki> jdstrand: thanks!
[13:41] <ijohnson> pedronis: the snapd snap here is a bit of an odd egg, but it feels cleaner to me IMHO
[13:41] <pedronis> ijohnson: it's not cleaner, in my opinion, without moving even more stuff
[13:41] <pedronis> and the win is unclear at this point
[13:41] <pedronis> cleanliness is only one factor here
[13:42] <ijohnson> right one of the points I made in the PR is that I wanted to move more stuff from cmd_initramfs_mounts.go to the boot package
[13:42] <ijohnson> i.e. install mode mount choosing
[13:42] <pedronis> ijohnson: formulated in those terms is not a win or not
[13:42] <pedronis> why is it good or bad?
[13:43] <pedronis> ijohnson: anyway my recommendation is to leave snapd alone at this point
[13:44] <ijohnson> alright I will bring my reasoning with to our meeting
[13:44]  * ijohnson still needs to have breakfast
[13:45] <pedronis> ijohnson: refactoring is to fix other TODOs properly, is not a goal in itself (especially at this point in time)
[13:46] <pedronis> ijohnson: anyway if you worry that you'll have to start this from scratch, I don't think so
[13:47] <ijohnson> well fwiw I can easily ressuect the simpler version of this that I started with which just moves code to the boot package but it feels much more mechanical and at that point it seems what's the point if we just move the same code from one file to another
[13:47] <pedronis> ijohnson: that we can call it from tests
[13:48] <pedronis> ijohnson: anyway, I'm happy to discuss, I'm not sure what the mechnical version does
[13:48] <pedronis> so I can't say it's better or worse
[13:48] <pedronis> ijohnson: I can see where the current code as proposed could go
[13:54] <mup> PR snapcraft#2991 opened: yaml_utils: don't sort keys when dumping <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/2991>
[13:57] <mvo> zyga: yay
[13:57] <zyga> re :)
[13:57] <cachio> cmatsuoka, hey
[13:58] <cmatsuoka> hi cachio
[14:01] <sergiusens> zyga: if we implement https://bugs.launchpad.net/snapcraft/+bug/1868460 then this stops being yaml https://yaml.org/spec/1.2/spec.html#id2765608
[14:01] <mup> Bug #1868460: environment sections are reordered <Snapcraft:Confirmed for cjp256> <https://launchpad.net/bugs/1868460>
[14:04] <cjp256> sergiusens/zyga:  i was just writing something like that up for LP :D  at first i thought it may have been a change in behavior either due to something I did, or the recent pyyaml uprev, but I think not
[14:09] <zyga> sergiusens: I'm okay if that stops being yaml in a way
[14:09] <zyga> sergiusens: I think we cannot reorder it to be a sequence now
[14:09] <zyga> cc pedronis ^
[14:09] <ijohnson> let's just all move everything to toml and call it a day
[14:10] <sergiusens> zyga: you can accept both inputs or get degville to fix the docs to say this is NOT yaml, but yaml-like
[14:10] <zyga> sergiusens: I think we can be reasonable, it's yaml but we respect the order
[14:11] <zyga> sergiusens: if we need to fix this snapd side we can but I would rather have snapcraft maintain the order
[14:13] <sergiusens> zyga: well, I would have preferred that snapd had implemented a sequence, as I bet kyrofa would have too
[14:13] <zyga> sergiusens: sure but that's too late, the spec is set
[14:15] <sergiusens> zyga: you know that YAML spec is also set
[14:15] <zyga> sergiusens: I know, I'm not asking you to change that spec
[14:15] <sergiusens> zyga: we will make the change, but fix the docs
[14:41] <zyga> sergiusens: ack, I think environment is not documented (I could not find it) so I'll see what we can do about that first
[14:42] <mup> PR snapcraft#2988 closed: tests: add two more workers to the 18.04 systems <Created by sergiusens> <Closed by sergiusens> <https://github.com/snapcore/snapcraft/pull/2988>
[14:47] <mborzecki> mvo: can you upload 2.44.1 release tarballs?
[14:50] <rbasak> ijohnson: thank you for your answer (on refresh hook exit status https://forum.snapcraft.io/t/supported-snap-hooks/3795/12) - should the behaviour be documented then? It's only alluded to in the official docs.
[14:56] <mvo> mborzecki: uh, sorry, sure
[14:57] <mborzecki> pedronis: cmatsuoka: there is a bug with generating timer schedules, but i can't really reproduce the OOM thing
[14:58] <mborzecki> pedronis: cmatsuoka: the bug is that we don't effectively generate a schedule if if it's a single `mon` thing (but we have a plenty of weird, complicated schedules)
[15:03] <pstolowski> mborzecki: hey, i was just talking with mvo about https://bugs.launchpad.net/snapd/+bug/1868706, can you help with this?
[15:03] <mup> Bug #1868706: Snapd postinst script hangs <snapd:Triaged> <https://launchpad.net/bugs/1868706>
[15:16]  * zyga -> lunch
[15:16] <pstolowski> any idea if .postinst stdout goes to a log file somewhere?
[15:17] <ijohnson> rbasak: yes it should be documented somewhere, degville perhaps can help with that
[15:18] <mborzecki> pstolowski: let me see
[15:18] <mborzecki> pstolowski: wow
[15:18] <mborzecki> pstolowski: hmm that's the interactive ask for password prompt right?
[15:19] <pstolowski> mborzecki: whoot? where?
[15:19] <pstolowski> 2388?
[15:20] <mborzecki> pstolowski: systemd-tty-ask
[15:20] <mborzecki> pstolowski: afaik it's actually calling systemctl start which is added by dh-systemd
[15:21] <mvo> the systemd-tty-ask might be a red herring
[15:21] <cachio> mvo, I just see mnt/system-data/etc/cloud/cloud-init.disabled
[15:21] <mborzecki> mvo: it might, but it will block further operations
[15:22] <mvo> cachio: aha, interessting. is this a grade dangerous model?
[15:22] <cachio> mvo, yes
[15:26] <kyrofa> sergiusens, "I would have preferred that snapd had implemented a sequence, as I bet kyrofa would have too" -> you know it
[15:27] <cachio> mvo, this is the partions https://paste.ubuntu.com/p/x9n6wmNtQQ/
[15:27] <mvo> mborzecki: interessting, keep me updated please
[15:27] <mvo> cachio: odd, can you point me to the full branch that you use? I will try to run it here
[15:28] <cachio> mvo, https://github.com/sergiocazzolato/snapd/tree/tests-extend-uc20-nested-suite
[15:28] <ijohnson> pedronis: wdyt of adding a "Rewrite()" to modeenv, turns out we have places we need to artificially create a modeenv then write it out somewhere, so I think we should have two methods, Rewrite() for writing to where we read it, and Write() for writing to anywhere
[15:28] <cachio> mvo, this is the test https://github.com/sergiocazzolato/snapd/blob/tests-extend-uc20-nested-suite/tests/nested/manual/uc20-smoke/task.yaml
[15:28] <mvo> cachio: thanks
[15:29] <ijohnson> pedronis: i.e. https://pastebin.ubuntu.com/p/gdQdfxCt3V/
[15:31] <ijohnson> actually slightly more sensible version https://pastebin.ubuntu.com/p/FJwFQzgXcd/
[15:32] <tkamppeter> ijohnson, jdstrand, pedronis: We have our meeting now.
[15:32] <ijohnson> tkamppeter: I thought the meeting was in 30 minutes?
[15:32] <ijohnson> tkamppeter: that's what the calendar invite says?
[15:33] <tkamppeter> I have a calendar invite for 4:30pm.
[15:33] <jdstrand> tkamppeter: mine also says 30 minutes from now
[15:33] <ijohnson> I mean I'm free now if we want to meet now I guess
[15:33] <tkamppeter> So then let us have it in 30 minutes, no problem.
[15:33] <ijohnson> okay, in 30 minutes, thanks tkamppeter
[15:42] <mup> PR snapd#8333 opened:  wrappers: fix timer schedules that are days only <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8333>
[15:43] <pedronis> ijohnson: sounds ok, either that or Write and WriteTo
[15:44] <ijohnson> ah I like WriteTo and Write better
[15:44] <ijohnson> thanks
[15:45] <mborzecki> pedronis: cmatsuoka: please take a look at #8333
[15:45] <mup> PR #8333:  wrappers: fix timer schedules that are days only <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8333>
[15:46] <pedronis> mborzecki: looks alright
[15:49] <tkamppeter> ijohnson, jdstrand, pedronis: If you are all free we could start right away now.
[15:50] <ijohnson> give me a few minutes to wrap up what I'm doing right now
[15:50] <tkamppeter> OK.
[15:52] <pstolowski> zyga: does this https://bugs.launchpad.net/snappy/+bug/1868620 fall into the category of any known nvidia/GL issues?
[15:52] <mup> Bug #1868620: Snaps based on Wine fail with nvidia driver 440  <Snappy:New> <https://launchpad.net/bugs/1868620>
[15:53] <zyga> I noticed that bug a moment ago, I would have to check
[15:54] <tkamppeter> ijohnson, jdstrand, pedronis: I am connected now.
[16:01] <ijohnson> sorry pulseaudio problems
[16:01]  * jdstrand shakes fist at pulseaudio
[16:03] <mvo> cachio: fwiw, I did a manual test, created a new image, copied things onto it and the cloud-init data.cfg got copied to the right place. the user is not created and I'm not sure why but the file is in the right place. I will collect the steps and pastebin them
[16:04] <cachio> mvo, ahh, thanks for testing that
[16:05] <mvo> cachio: http://paste.ubuntu.com/p/gMTKqZxPSt/
[16:06] <mvo> cachio: I look now why the user is not there, maybe the cloud-init logs have hints
[16:07] <cachio> snapd on esge already has the cloud init code right?
[16:09] <mvo> cachio: yes, I created the image with --channel=edge
[16:09] <mvo> cachio: I see in /run/cloud-init/cloud-init-generator.log: "cloud-init is enabled but no datasource foudn, disabling"
[16:10] <mvo> cachio: looks like either you need to write a more complex data source or I need to implement the uc16 compatbile cloud-init overrides, I think this is what we want anyway so probably a good idea to wait for this
[16:10] <mvo> cachio: unless tweaking the data.cfg is easy
[16:10]  * mvo does not know
[16:10] <cachio> mvo, ahh, ok
[16:11] <cachio> I'll update the cfg and then once we have that in place I'll revert the config
[16:11] <cachio> thanks for the info
[16:15] <mvo> cachio: my pleasure
[17:36] <zyga> cachio: is ubuntu-20.04-64 using a desktop image?
[17:36] <zyga> cachio: it's significantly slower than other systems
[17:37] <cachio> I found some interesting stuff
[17:37] <cachio> zyga, the most important was related to systemd
[17:37] <cachio> all the systemd operations took more time on ubuntu 20.04
[17:37] <zyga> cachio: so it's not a desktop image?
[17:38] <zyga> cachio: no gdm, no extra packages?
[17:38] <cachio> it uses the cloud image published by ubuntu
[17:38] <zyga> vanilla?
[17:38] <cachio> no deskto
[17:38] <cachio> p
[17:38] <zyga> IIRC we had our own images derived form cloud
[17:39] <cachio> zyga, right
[17:39] <cachio> we are using one which is a copy from the published by ubuntu-cloud
[17:40] <zyga> and we have not, by any chance, installed gdm?
[17:41] <cachio> zyga, gcloud compute images list --project ubuntu-os-cloud-devel | grep ubuntu-2004-lts
[17:41] <cachio> zyga, I can check manually that
[17:41] <zyga> cachio: thanks, I remember seeing gdm in some logs
[17:41] <zyga> on 20.04 classic
[17:41] <zyga> which made me wonder if there's something odd that would also explain the extra time
[17:44] <cachio> zyga,
[17:44] <cachio> google:ubuntu-20.04-64 .../tasks/google/start-instance# apt-cache policy gdm3
[17:44] <cachio> gdm3:
[17:44] <cachio>   Installed: 3.34.1-1ubuntu1
[17:44] <cachio>   Candidate: 3.34.1-1ubuntu1
[17:44] <cachio>   Version table:
[17:44] <cachio>  *** 3.34.1-1ubuntu1 500
[17:44] <cachio>         500 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/main amd64 Packages
[17:44] <cachio>         100 /var/lib/dpkg/status
[17:44] <zyga> woot
[17:44] <zyga> so it's there?
[17:44] <zyga> can you try to guess why
[17:44] <zyga> I wonder if the 20.04 images are wrong
[17:45] <zyga> or we pulled it via dependencies somehow
[17:45] <cachio> let me check the base image
[17:45] <zyga> super, thank you
[17:51] <mvo> mborzecki: any clues re bug 1868706 ?
[17:51] <mup> Bug #1868706: Snapd postinst script hangs <snapd:Triaged> <https://launchpad.net/bugs/1868706>
[17:52] <zyga> mvo: maybe seeding bug on old focal images?
[17:52] <mborzecki> mvo: left some ideas for pstolowski
[17:52] <cachio> zyga, look at this https://paste.ubuntu.com/p/N7Fc5KsWY2/
[17:52] <zyga> looking
[17:52] <mvo> mborzecki: nice
[17:52] <mborzecki> mvo: but imo it's the systemd-ttyu-ask-password is runnig and waiting for prompt
[17:52] <zyga> O M G
[17:52] <zyga> installing evolution-data-server pulls in GDM and the desktop
[17:52] <zyga> gosh
[17:52] <mborzecki> mvo: which is weird because systemctl believes it's runnign on tty and not under uid 0
[17:52] <zyga> can you file a bug on that
[17:52] <zyga> and we can fight it later
[17:53] <zyga> now we can at least remember
[17:53] <zyga> thank you for checking!
[17:53] <cachio> zyga, is it a bug?
[17:53] <zyga> I think we don't want to run the desktop session in our images
[17:53] <zyga> we should adjust our tests
[17:53] <zyga> and the script that regenerates images
[17:53] <zyga> so that we don't have gdm3 installed
[17:53] <zyga> look at what this pulled
[17:54] <zyga> mvo: ^ fyi
[17:54] <zyga> why stuff is slower
[17:54] <cachio> I compare systemd  on 18.04 nad 20.04
[17:55] <cachio> and takes much more time on focal than in bionic
[17:55] <cachio> zyga, let me cehck if I can find the source data
[17:56] <cachio> I addded that to the standup notes but I think i was purged
[17:56] <zyga> ok
[18:04] <cachio> mvo, in 1~2 hour snapd will be ready to go to candidate
[18:05] <cachio> mvo, internet is very slow today
[18:10] <cachio> zyga, lp:1868857
[18:10] <zyga> thank you :)
[18:10] <cachio> zyga, yaw
[18:12] <mvo> cachio: ok
[18:14] <mup> PR snapd#8334 opened: many: add core.resiliance.vitality-hint config setting <Created by mvo5> <https://github.com/snapcore/snapd/pull/8334>
[18:21]  * zyga EODs
[18:21] <zyga> o/
[20:01] <cachio> pedronis, I am debugging uc20 with encription which is still rebooting constantly
[20:01] <cachio> pedronis, any idea why it could restart almost always after showing in the console output the line
[20:01] <cachio> [   21.191888] systemd[1]: Starting Create Static Device Nodes in /dev...
[20:02] <pedronis> no, not sure
[20:02] <pedronis> cmatsuoka maybe knows more or can help
[20:03] <cachio> pedronis, ok, thanks
[20:07] <cmatsuoka> cachio: is it rebooting in a loop, or just once after it reaches run mode?
[20:07] <cmatsuoka> cachio: could you make an image available for me to test locally?
[20:08] <mup> PR snapd#8335 opened: many: introduce naming.WellKnownSnapID <Created by pedronis> <https://github.com/snapcore/snapd/pull/8335>
[20:09] <cachio> cmatsuoka, it is not reaching run mode
[20:09] <cachio> keeps rebooting in install mode
[20:11] <cmatsuoka> cachio: oh, that's something we didn't see before, right?
[20:11] <cachio> cmatsuoka, I reproduce it with this https://paste.ubuntu.com/p/HJQt9Rxh7g/
[20:12] <cachio> in gce it fails 100% of the times
[20:12] <cachio> cmatsuoka, I'll try the same now but using ubuntu image snap instead
[20:19] <cmatsuoka> cachio: just for me to understand where we are, what was changed since the last image that worked?
[20:20] <cachio> cmatsuoka, I think the problem is ubuntu-image
[20:20] <cachio> works different using the snap and the deb
[20:21] <cachio> cmatsuoka, confirmed
[20:21] <ijohnson> cachio: yes the deb is definitely behind the snap
[20:21] <cachio> works bad when the image is created using ubuntu-image from deb
[20:24] <cachio> ijohnson, agree, but   I though both should work
[20:24] <ijohnson> yeah I think eventually they both will work, but it takes time for the deb to get all the features that the snap gets
[20:28] <cmatsuoka> mm, something strange in #8333, travis test passed but github thinks it's still in progress
[20:28] <mup> PR #8333:  wrappers: fix timer schedules that are days only <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8333>
[20:29] <ijohnson> cmatsuoka: yeah that has been happening recently
[20:29] <ijohnson> cmatsuoka: what fixed it last time is to close and re-open
[20:30] <cmatsuoka> ouch
[20:32] <mup> PR snapd#8336 opened: boot,many: add modeenv.WriteTo, make Write take no args <Simple 😃> <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8336>
[20:37] <cmatsuoka> ijohnson: well it seems that closing/re-opening didn't work this time :\
[20:38] <ijohnson> oh sorry what I meant is that closing /re-opening would make the tests re-run and then after running they would update travis properly
[20:38] <cmatsuoka> ijohnson: and it spawned another set of github quick tests
[20:38] <ijohnson> err travis would update github properly after re-running
[20:38] <cachio> cmatsuoka, using the ubuntu image snap I can reproduce the error https://paste.ubuntu.com/p/CyqDHtrvPF/
[20:38] <cmatsuoka> ijohnson: ah ok, I'll wait to see if it works
[20:39] <ijohnson> cachio: did you try edge channel of ubuntu-image snap ?
[20:39] <cachio> ijohnson, no
[20:40] <cachio> ijohnson, I'll try
[20:40] <cachio> reset
[20:43] <cachio> ijohnson, same error
[20:43] <ijohnson> hmm
[20:44] <cachio> ijohnson,
[20:44] <cachio> trying with an image with is not updated
[20:44] <cachio> a pure one
[20:45] <cachio> cmatsuoka, that could affets?
[20:45] <cachio> affect
[20:45] <cachio> if I add a file to the image and the vm is using secure boot?
[20:46] <cmatsuoka> hmm, it shouldn't unless your file is something used in the boot process
[20:46] <cachio> cloud init config
[20:47] <cmatsuoka> cachio: and we're not measuring anything yet, so sb should only check shim and friends
[20:47] <cachio> cmatsuoka, I see
[20:47] <cmatsuoka> cachio: at which point it reboots? does it boot the target system and seeds?
[20:47] <cachio> BdsDxe: loading Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
[20:47] <cachio> BdsDxe: starting Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
[20:47] <cachio> error: file `/EFI/ubuntu/grubenv' not found.
[20:47] <cachio> error: no such device: ubuntu-boot.
[20:48] <cachio> error: no such device: ubuntu-boot is new
[20:48] <cmatsuoka> cachio: I think the missing grubenv is only a warning and it's not fatal
[20:49] <cmatsuoka> no ubuntu-boot is interesting, because it's created during install
[20:49] <cmatsuoka> cachio: and you said it fails in install mode?
[20:49] <cachio> yes
[20:49] <cachio> once I went to run mode
[20:49] <cachio> but it is really randome
[20:50] <cachio> sometimes goes to run mode sometimes still rebooting in install mode
[20:50] <cmatsuoka> ah, so you actually reboot in run mode and then it fails?
[20:50] <cmatsuoka> ah
[20:50] <cachio> now it is not going to run mode
[20:51] <cmatsuoka> and are you able to reproduce this behavior locally?
[20:51] <cachio> cmatsuoka, didn't try
[20:51] <cmatsuoka> because maybe it's not completing the install?
[20:51] <cachio> cmatsuoka, because my internet sucks today
[20:51] <cachio> cmatsuoka, let my try
[21:10] <mup> PR snapd#8337 opened: cmd/snap-bootstrap/initramfs-mounts/tests: use dirs.RunMnt over s.runMnt <Simple 😃> <Test Robustness> <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8337>
[21:55] <pedronis> ijohnson: I tried to answer your question,  I think we should try to get rid of the modeEnv hanging on DeviceManager at some point soon
[21:55] <pedronis> we don't keep a bootloader around like that
[23:14] <mup> PR snapcraft#2992 opened: [WIP] repo: ubuntu implementation refactoring with initial package-management <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/2992>