[03:41] PR snapd#9098 opened: tests: new organization for nested tests <⛔ Blocked> [06:02] PR snapd#9089 closed: many: introduce content write observer, install mode glue, initial seal stubs [06:06] morning [06:37] PR snapd#9099 opened: github: do not skip gofmt with Go 1.9/1.10 [06:50] good morning [06:50] I will start at 9:00 [07:00] * zyga starts the day [07:03] morning [07:06] good morning pstolowski [07:08] mvo: pstolowski: zyga: hey [07:08] o/ [07:09] pstolowski: #9099 should catch the formatting errors you had in the interfaces PR early in the ci run [07:09] PR #9099: github: do not skip gofmt with Go 1.9/1.10 [07:09] (and it's trivial btw) [07:10] thx [07:10] nice [07:28] mvo: hey, as i said yesterday evening, backporting all the debug preseeding api was easy process, except for preseed spread tests. they diverged over time in master (including some associated tests/lib helpers). a lot of cherry picking and crossing lines with various tweaks that Sergio implemented in the meantime. not sure what to do [07:29] mvo: on the other hand would be good to have snap debug seeding .. in the spread test to confirm everything is in place [07:35] fwtw that was my older PR that attempted to bridge the gap (but we didn't land it) https://github.com/snapcore/snapd/pull/8980 [07:35] PR #8980: tests: backport preseed test fixes to 2.45 [07:36] they are shooting a movie outside our house again [07:40] pstolowski: I think we can live without the spread tests if it's too muhc of a hassle [07:40] pstolowski: we need to do a manual test then but it's not the end of the world as long as it's in master [07:41] zyga: good morning! how do you feel? [07:41] good morning! [07:41] mborzecki: good morning to you as well :) I have not forgoten 9078 [07:41] so far so good [07:41] mvo: hahah, thanks [07:44] I'm almost done with 9057 [07:46] quick errand, may be late for the desktop meeting [07:47] mvo: ok [07:52] PR snapd#9100 opened: many: cherry-pick debug seeding api for 2.45 [07:57] re, heh, not even a real errand, pickup guy waited for me at the gate [07:58] thought i'd have to drive a little [07:58] ^ there is one commit there touching spread test that could be omitted (since i haven't backported other changes to this tests and it will most likely won't work) [08:17] I just realized my note-taking notebook is upstairs [08:18] time to clime the mountain [08:18] brb [08:34] re [08:43] PR snapd#9099 closed: github: do not skip gofmt with Go 1.9/1.10 [08:45] PR core#115 closed: live-build/hooks: add chroot hook to rm cloud-init config file we don't want [08:51] ok, small break to lie down [09:13] PR snapd#9101 opened: interfaces/systemd: use emulation mode when preseeding [09:18] pstolowski: hi, #9100 is missing a couple of things, see my comment there [09:18] PR #9100: many: cherry-pick debug seeding api for 2.45 [09:18] pedronis: thanks for spotting! [09:23] PR snapd#9102 opened: corecfg: add "system.timezone" setting to the system settings [09:32] pedronis: updated [09:38] mvo: https://github.com/snapcore/snapd/pull/9102#pullrequestreview-462363510 [09:38] PR #9102: corecfg: add "system.timezone" setting to the system settings [09:40] zyga: nice catch, I will fix this [09:40] zyga: I looked over all timezones I could find but obviously I missed some [09:41] zyga: maybe I should relax this a lot, not sure if I'm too cautious [09:44] yeah, I was thinking if we should validate [09:45] I guess it's a good thing if it gets checked early [09:45] at snap build time (gadget defaults) [09:45] but probably not good otherwise (we need to match another implementation) [09:45] * mvo hugs mborzecki for 9103 [09:47] mvo: we had a TODO there, pedronis correctly pointed out it's probably time to fix it to avoid incomplete writes when updating grub efi binaries [09:48] PR snapd#9103 opened: gadget, osutil: use atomic file copy, adjust tests [09:51] mborzecki: yeah, hence the *hug* :) [09:51] mborzecki: thanks for fixing this [09:51] pstolowski: thx [09:53] pedronis: btw. ListsEqual or ListEqual in 9096? it's a trivial patch i can push and maybe we can land it before ijohnson comes online [09:54] mborzecki: ListEqual [09:54] ack [09:58] mvo: https://github.com/snapcore/snapd/pull/9102#pullrequestreview-462374184 [09:58] PR #9102: corecfg: add "system.timezone" setting to the system settings [10:01] mvo, i assume using #9102 is not mandatory ? i.e. using timezone-control from a snap will still work ? [10:01] PR #9102: corecfg: add "system.timezone" setting to the system settings [10:02] mvo: some comments/questions in #9094 [10:02] PR #9094: corecfg: add "system.hostname" setting to the system settings [10:02] ogra_: yes, the main use-case for image building. but mixing snap set and other means to set the timezone will be problabmatic [10:03] ogra_: the main use case *is* image-building for uc20 [10:03] mvo: we'll need to make this virtual like netplan at some point [10:03] pedronis: exactly [10:03] yeah ... as long as using i.e. a snap that looks up the TZ via geoip and setting the TZ still works all is fine [10:03] maybe we need TODOs for that [10:04] pedronis: happy to add one [10:04] same for hostname [10:04] pedronis: will do [10:04] mvo: anyway some questions there, probably relevant also for timezone [10:05] ogra_: yeah, that's fine as long as it's not mixed (i.e. sometimes snap set, sometimes timedatectl directly) [10:05] right [10:05] pedronis: yeah, thank you [10:05] mvo: https://github.com/snapcore/snapd/pull/9094 is not bi-directional, right? [10:05] PR #9094: corecfg: add "system.hostname" setting to the system settings [10:05] if I set a system hostname, get will tell me it is "" [10:06] unless I used set to set it [10:06] right? [10:06] zyga-x240: yes, until we have virtual config nodes [10:06] ok [10:06] mvo, so we shoudl better not set it by default (wrt UC20 images) to not force the world to use snapd-control 😉 ... perhaps timezone-control could grant access to the setting though ? [10:07] oh [10:07] interesting [10:08] ogra_: we will support virtual configuration soon(ish) then this will become a non-issue [10:08] ogra_: but yeah, until then it's a problem [10:08] i dont think baking a TZ into an image at build time is a good idea at all though ... in case thats the plan [10:08] I guess we could cheat for hostname/timezone and simply import if unset but feels a bit hacky [10:09] ogra_: heh :) I'm just a code-monkey, this was a request from the people building images [10:09] well, but do they understand the implications ? [10:09] ogra_: apparently one of the things that happend in uc18 is that writable is modified to change timezone/hostname [10:10] ogra_: if you are a telco doing consumer hw for a small EU country, baking the timezone is OK [10:10] I think it's a valid use case [10:10] zyga-x240, not talking about a telco that uses a brand store and has access to snapd-control ... [10:10] sure but it's the out-of-the box thing that works before you get online fully [10:11] I think it's not wrong [10:11] but about our community images where people only have timezone-control to set it [10:11] having snaps re-set my TZ to whats baked into the image on boot wouldnt be something i expect [10:11] ogra_: we will not have anything in our community images that will set this [10:11] s/snaps/snapd/ [10:11] ah, k [10:12] we could have a ubuntu-core-20-brexit image [10:12] with hard-coded TZ [10:12] all fine then 🙂 [10:12] * zyga-x240 runs [10:12] ogra_: but you make an interessting point. I guess "snap get system.timezone" should really get whatever /etc/localtime tells it regardless of the setting [10:12] and blue splash screen [10:12] mvo: I think those are both properties [10:13] with a big long hex number and the word "SUCCESS" [10:13] not actual variables [10:13] mvo: yes, that means virtual config. we really need to start on that sooner or later [10:13] you can read / write but the system is authoritative [10:15] pedronis: +1 [10:17] pstolowski: btw, did you see my comment in #9088 ? [10:17] PR #9088: cmd/snap-preseed: use snapd from the deb if newer than from seeds [10:19] pedronis: yes, thank you, just got sidetracked by other stuff, will get back to this PR in a bit [10:23] ok, thx [10:45] zyga-x240: I applied your suggestions (and more) in 9094 (the companion PR for hostname). if it looks good there I will also put them into the timezone one [10:45] mvo: thanks, I'll review the hostname one as well [10:45] looking at your changes now [10:53] mvo: https://github.com/snapcore/snapd/pull/9094#pullrequestreview-462410202 [10:53] PR #9094: corecfg: add "system.hostname" setting to the system settings [10:59] zyga-x240: thank you! [11:39] * zyga-x240 moved the locking earlier [11:39] now to look at tests [11:48] PR snapd#9100 closed: many: cherry-pick debug seeding api for 2.45 [12:15] pstolowski: I re-reviewed #9088 [12:15] PR #9088: cmd/snap-preseed: use snapd from the deb if newer than from seeds [12:38] pedronis: thanks [12:40] * zyga-x240 is making progress on tests [12:47] morning folks [12:47] morning ijohnson ! [12:47] hey pstolowski [12:49] hey ijohnson ! [12:49] morning zyga [12:49] brb, see you at the standup [14:01] 2/3 sid passed without an error [14:04] zyga-x240: https://pipelines.actions.githubusercontent.com/xS8oSnypZkPEQZqiZgDaRp2kdvQJKbOY08TesHp7E8vn7g4hYR/_apis/pipelines/1/runs/10375/signedlogcontent/60?urlExpires=2020-08-06T14%3A04%3A51.3637247Z&urlSigningMethod=HMACV1&urlSignature=CF%2B2ibNYy9HIFj7kxbQb7%2Fh9DQ71vy%2F6z2rksCZgawA%3D that's sid [14:04] ah, thanks [14:04] I'll look into those [14:06] zyga-x240: this link is probbaly better: https://github.com/snapcore/snapd/runs/953201904 [14:06] I will focus on a specific case and dig into it, but first some tests need finishing [14:07] ? [14:20] pedronis: mmm actually maybe we do need a short sync, I can get rid of the Origin() method, but I'm not sure if I should go the route of implementing both the deepEqual method and a copy method, or if there's some way I can get away with only one additional method on *Modeenv compared to master [14:21] Bug #1890610 opened: 'snap install' | "INFO Waiting for automatic snapd restart..." [14:21] pedronis: I don't see how to do it with just a single additional method, but I can get rid of deepEqual if I have a Duplicate method and something like m.SyncUpdates(m2) [14:21] perhaps it's fine to have both deepEqual and a Duplicate method even if their impls will largely be the same [14:24] Bug #1890610 changed: 'snap install' | "INFO Waiting for automatic snapd restart..." [14:27] Bug #1890610 opened: 'snap install' | "INFO Waiting for automatic snapd restart..." [14:33] ijohnson: we can sync quickly if you want [14:33] pedronis: yes if you have time, should be simple, just not sure which methods you would want [14:34] jdstrand: When you've got time, could you have a look over https://github.com/snapcore/snapd/pull/8943 (dbus activation support)? I know you reviewed a version of this in another PR, but the implementation has changed a bit so probably needs another pass [14:34] PR #8943: wrappers: generate D-Bus service activation files [14:34] ijohnson: let me join the standup again [14:34] k, joining [14:34] jdstrand: there is less code to review this time around though, which should help [14:35] cachio: i've updated #9019 [14:35] PR #9019: tests/nested/manual/minimal-smoke: run core smoke tests in a VM meeting minimal requirements [14:35] jamesh: yes, I'm hoping to get to that after something I'm doing for microk8s atm. thanks for the ping. I also owe you a response to the theme forum topic [14:36] mborzecki, nice, I'll take a look [14:45] jdstrand: thanks [14:51] mvo: I gave some feedback on your comment here: https://github.com/snapcore/snapd/pull/9103#discussion_r466349911 [14:51] PR #9103: gadget, osutil: use atomic file copy, adjust tests [14:51] pedronis: thank you [14:59] PR snapd#9104 opened: tests: fix for timing issues on journal-state test [14:59] ijohnson: do you know if your PR conflicts with #9078 ? [14:59] PR #9078: boot: fancy marshaller for modeenv values [15:00] pedronis: it might not actually, do you want me to double check [15:00] ? [15:00] no, just wondering [15:00] I'll try to review it soon [15:00] I don't think it should as mine is just adding things [15:10] mvo: I reviewed the recovery/reboot PRs [15:10] pedronis: \o/ [15:11] the reboot one needs a bit more work I think [15:14] * zyga-x240 finished with refresh checks and will focus on debian in 15 minutes [15:14] small break [15:16] should I merge #9096 or just close it now [15:16] PR #9096: strutil: add ListEqual helper [15:16] it's not used anymore [15:19] mvo: one suggestion to #9102 [15:19] PR #9102: corecfg: add "system.timezone" setting to the system settings [15:20] pstolowski: thank you [15:30] ijohnson: I would say close it [15:30] pedronis: ack [15:31] ok, let's fix debian [15:32] ijohnson: why did you call the new method Duplicate when we agreed to call it Copy ? [15:32] :) [15:33] I literally just re-force pushed it like 60 seconds ago [15:33] ah [15:34] ijohnson: it's somehow still called Duplicate, did you force push less than expected? [15:34] also if we now have DeepEqual, why do we have check.DeepEquals ?? [15:35] I also forgot to rename DeepEquals -> DeepEqual so I am force pushing that [15:35] give me like 3 minutes and all renames will be sorted out sorry [15:35] ijohnson: I don't think check is following go conventions there [15:35] fair [15:36] it's Time.Equal and reflect.DeepEqual [15:36] etc [15:43] I can't wait for the day that we can use a new gofmt :-/ [15:44] PR snapd#9096 closed: strutil: add ListEqual helper [15:46] ugh that took too long [15:46] everything should be in sync now, renamed and passing gofmt 1.9 [15:47] I also canceled all the queued / in progress github actions from my various force pushes [15:47] ijohnson, hey [15:47] hi cachio [15:47] I see thie error refreshing core20 [15:47] https://paste.ubuntu.com/p/XdF4Qq7nmS/ [15:48] cachio: you mean the "there was a rollback" message ? [15:48] this is after hte reboot [15:48] Aug 06 15:47:40 aug061535-531864 systemd-user-runtime-dir[35169]: Failed to acquire number of inodes for runtime directory: Unknown interface org.freedesktop.> [15:48] ijohnson, yes [15:48] Aug 06 15:47:40 aug061535-531864 systemd-user-runtime-dir[35169]: Failed to acquire number of inodes for runtime directory: Unknown interface org.freedesktop.login1.Manager or property RuntimeDirectoryInodesMax. [15:48] I reffreshed from beta to edge [15:48] after the reboot I see that [15:48] cachio: is this after your dropping of kvm ? or is this not a vm ? [15:49] s/vm/nested vm/ [15:49] ijohnson, it si a vm with kvm enabled [15:49] cachio: is it a nested vm ? [15:49] yes [15:49] cachio: so this could just be that random reboot bug [15:49] cachio: do you have the rest of the log ? [15:51] ijohnson, this is what I have https://paste.ubuntu.com/p/Wr2Dgjb3rs/ [15:51] ijohnson, I'll try again without kvm [15:52] cachio: seems something got broken in this branch you are using [15:52] qemu has `-serial file:/tmp/work-dir/serial.log` [15:52] but spread debug is looking at [15:52] `+ '[' -f /tmp/work-dir/serial-log.txt ']'` [15:53] that wouldn't break anything but it means we got less information than we need to in order to diagnose if this was the random reboot problem [15:53] that property is real but is missing from introspection data [15:54] ijohnson, you are right [15:54] fixing that [15:55] cachio: yes please try w/o kvm, or you do have #9083 applied locally where you are running this ? [15:55] PR #9083: tests: new parameters for nested execution [15:55] zyga-x240: broken systemd ? [15:55] perhaps, not sure yet [15:55] introspection data certainly disagrees with source [15:55] that seems bad [15:56] I'm running sid on one test machine (not vm) [15:56] so I guess I'm brave ;-) [15:58] very brave :-) [15:58] it's a ppc box [15:59] you go there to compile C code and see if it works [16:01] mvo: ijohnson: I reviewed but then ended up submitting some tweaks #9078 [16:01] PR #9078: boot: fancy marshaller for modeenv values [16:02] pedronis: did anyone ever figure out the deal with secboot's checksumSHA1 flip flopping around ? [16:02] no [16:03] ijohnson: what do you get there? [16:03] ijohnson, yes merged with 9083 [16:03] now runing without kvm enabled [16:03] ok, I think I have a partial fix [16:03] at least for one test [16:03] cachio: mmm then please fix the serial file so we can see the log, it may be a real problem then if we aren't having random reboots anymore [16:04] ijohnson, I already fixed that, let's see the results in 30/40 minuets [16:04] cachio: I don't see those different filenames on master, was that locally you changed that ? [16:04] ijohnson, thanks for the catch [16:04] cachio: ack sounds good [16:04] pedronis: one moment [16:05] pedronis: I have `fqejS2llZXw3gLnOYhg7pcSlY+Q=` like on master, but sometimes it changes for me as well, unclear when it changes, see for example my systemd-mount PR [16:05] brb [16:06] pedronis: ah actually my systemd-mount PR has the same change as your PR does https://github.com/snapcore/snapd/pull/9010/files#diff-bd290170e2912d3e8694db1a151066e5 [16:06] PR #9010: cmd/snap-bootstrap/initramfs-mounts: call systemd-mount instead of the-tool [16:08] ijohnson: anyway 9078 needs a review of my commit [16:08] pedronis: yes one moment lgtm I will submit a formal review [16:08] sirens outside all day [16:08] today new president got sworn [16:09] all kind of important folks are driven around [16:09] police, vip cars, more police [16:10] mvo: should I push the rename and arg swap to #9103 ? [16:10] PR #9103: gadget, osutil: use atomic file copy, adjust tests [16:10] pedronis: yeah, if feel free then we can merge [16:18] pedronis: I pushed an update to 9020 [16:22] sloooow prepare [16:22] testing fixes is slow [16:23] mvo: done here: https://github.com/snapcore/snapd/pull/9103/commits/46a7e60434a7115f5dc8a1910e5e160e09c4ac73 [16:23] PR #9103: gadget, osutil: use atomic file copy, adjust tests [16:24] pedronis: thank you [16:24] more sirens [16:24] feels like a fire truck [16:24] they have different horns [16:24] PR snapd#9057 closed: overlord: use new tracking cgroup for refresh app awareness [16:33] zyga-x240: could you look at #9101 ? [16:33] PR #9101: interfaces/systemd: use emulation mode when preseeding [16:34] sure [16:34] thx [16:34] given the arg surgery I did on #9103 it probably needs a 2nd review from somebody else than me [16:34] PR #9103: gadget, osutil: use atomic file copy, adjust tests [16:34] pedronis: do you need me to review more of pawel's preseeding PR's as they pertain to the backends ? [16:35] * ijohnson can't remember whether those are going into 2.45.4 or not [16:35] ijohnson: they are [16:36] ijohnson: for 2.45.4 #9088, #9101, and #9103 need 2nd reviews [16:36] PR #9088: cmd/snap-preseed: use snapd from the deb if newer than from seeds [16:36] PR #9101: interfaces/systemd: use emulation mode when preseeding [16:36] PR #9103: gadget, osutil: use atomic file copy, adjust tests [16:36] pedronis: ok I will try to look at all of those today, might not get to all of them [16:37] ijohnson: I think zyga will look as well at 9101, so start with the other two [16:37] sure [16:39] more on the sid error side: Aug 06 16:39:09 aug061610-263415 dbus-daemon[302]: [system] The maximum number of pending replies for ":1.6" (uid=0 pid=673 comm="/lib/systemd/systemd-logind ") has been reached (max_replies_per_connection=128) [16:39] * pedronis mostly eods [16:44] interesting that systemd-logind uses 450MB of ram when it's in a broken state [16:44] and about 2MB of ram when functional [16:44] wow [16:46] right? [16:46] I'll check earlier, I think this is broken at an earlier stage [16:47] just restarting systemd-logind.service seems to "fix" stuff [16:47] maybe some package upgrades we do leave it this way [16:47] * zyga-x240 adds "false" in a strategic place ;) [16:48] ijohnson: could you look at https://github.com/snapcore/snapd/pull/7825 [16:48] PR #7825: many: use transient scope for tracking apps and hooks [16:48] at the last comment only [16:48] and at the diffstat [16:48] haha I'm literally reviewing it now [16:48] :-) [16:49] haha cool :) [16:51] +1d [16:51] I saw, super! [16:51] * cachio lunch [16:53] ok, moved false to after prepare project [16:53] let's see if it's already broken then [16:53] oh, [16:53] I should have just upgraded in place [16:53] that would probably be enough [16:53] oh well, I'll know in a moment [17:02] yeah, just preparing project is enough [17:16] I understand the bug now [17:16] (for debian sid) [17:18] zyga-x240: is there are an order to #7700, #7825, #8573 . should they be reviewed in their PR number order? or something else? [17:18] PR #7700: cmd/snap: wait while inhibition file is present [17:18] PR #7825: many: use transient scope for tracking apps and hooks [17:18] PR #8573: overlord/snapstate: inhibit startup while unlinked [17:19] pedronis: I would recommend to look at 7700 first - it's rather basic and shows what happens on snap run [17:20] pedronis: then at 8573 as it shows how we acquire the new inhibition lock [17:20] and ignore 7825 for now [17:20] as it is a hollow branch that just removes code now [17:20] https://github.com/systemd/systemd/issues/16685 for context [17:20] xnox: do you know any systemd debian maintainers [17:20] there's a bug that cripples snapd tests that just rolled out to sid [17:20] the fix is a one liner [17:21] but requires some thought and consideration [17:30] I'm reporting a debian bug now, almost done [17:30] I'll attach a patch [17:35] ok, reported now [17:36] I'll send a patch to unbreak sid while we wait for the package to be fixed [17:50] testing a fix now [17:50] I should have EODd a while a go [17:51] *ago [17:51] zyga: normally patches go as pull requests on salsa.debian.org against systemd repo [17:54] xnox: the situation is more complex now, it's not a clear fix :/ [17:54] it's just kind of bad and there's no good solution [17:54] I've updated the upstream bug with references if you are interested [18:01] ok, fixed, I'll run a quick smoke test and open a PR [18:11] ijohnson: https://github.com/snapcore/snapd/pull/9105 [18:11] PR #9105: tests: work around bug in systemd/debian <⚠ Critical> [18:11] nice [18:15] PR snapd#9105 opened: tests: work around bug in systemd/debian <⚠ Critical> [18:22] OK [18:23] * zyga-x240 resolved some conflicts and EODs [18:23] it's been a long day [18:23] need to exercise a little [18:23] o/ [18:23] ijohnson: I resolved conflicts in https://github.com/snapcore/snapd/pull/8573 now [18:23] PR #8573: overlord/snapstate: inhibit startup while unlinked [20:05] PR snapd#9106 opened: gadget: remove partition table data from ondisk volume [20:25] ijohnson, I was refreshing reverting core20 and snapd snapd the whole afternoon without kvm and it worked 100% [20:25] nice [20:25] so the problem were the random reboots [20:26] still odd that you were running without kvm and hit that issue this morning though [20:26] I am running those tests in a look, at some point I'll reproduce them [20:31] thanks cachio [20:33] yaw [21:15] ijohnson, could you take a second review to #8942 plase? [21:15] PR #8942: tests: support different images on nested execution [21:15] cachio: sure [21:15] so if it is green I can squash merge it [21:15] thanks [21:15] many things seem to be stuck in queued state right now [21:15] yes [21:16] what I've been doing is to cancel individual spread runs via the github ui, then wait for all the other spread runs to be canceled, then go and hit re-run [21:16] not sure how helpful that has been [21:16] we may need Zygmunt to update the self hosted runners or something [21:24] np, I'll monitor that one [21:24] thanks [22:46] PR snapd#9103 closed: gadget, osutil: use atomic file copy, adjust tests