[00:39] <mup> PR snapd#9257 opened: bootloader: retrieve boot chains from bootloader <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/9257>
[05:03] <zyga> good morning
[05:25] <mborzecki> morning
[05:25] <mborzecki> starting a bit later, need to drive kids to school
[05:28] <zyga> mborzecki: ok
[05:41] <zyga> brb
[06:08] <zyga> brb again, time to fetch charger
[06:14] <zyga> I have the 12v power ;)
[06:15]  * zyga is struggling a little with the export provider algorithm, it probably indicates I need a different data model
[06:22] <mvo> good morning zyga
[06:22] <zyga> hello :)
[06:22] <zyga> mvo: first day of morning shift, I'm trying to re-think the data model for exports
[06:22] <zyga> mvo: good luck with your meetings
[06:23] <zyga> mvo: ian's feedback last night was very useful, may simplify zfs support
[06:24] <mvo> zyga - oh, nice! in what PR was that feedback provided?
[06:24] <mvo> zyga or was it here on irc?
[06:24] <zyga> mvo: the one about exchange
[06:25] <zyga> mvo: ian's comments made me think that all I may need, after all, is the plain old rename
[06:26] <mvo> zyga oh, ok
[06:27] <mvo> zyga: getting away with this would be great, less-code++
[06:34] <mborzecki> re
[06:34] <mborzecki> mvo: zyga: morning guys
[06:35] <mvo> good morning mborzecki
[06:35] <mborzecki> wow, the weather turned to shit pretty quickly this year
[06:35] <zyga> mborzecki: how was first day of school?
[06:35] <zyga> mborzecki: it's 2020
[06:35] <mborzecki> zyga: 13C, raining all of yesterday and during the night
[06:35] <mborzecki> zyga: ofc, also a short power outage yday, and my primary internet link is down ;P
[06:36] <zyga> mborzecki: nothing like 1st of September to remind everyone that holidays are over
[06:36]  * zyga cannot get used to the fact that some places start school mid-August 
[06:36] <mborzecki> yeah, bringing everyone up to speed with the harsh reality ;)
[06:37] <zyga> mborzecki: I have a question, a comment from ian relating to RENAME_EXCHANGE indicated you did some research about atomic properties of that operation
[06:37] <mborzecki> hm?
[06:37] <zyga> mborzecki: do you recall any problems with that?
[06:37] <zyga> it was in relation to AtomicSylink helpers
[06:38] <mborzecki> zyga: let me take a look at the notes
[06:39] <zyga> mborzecki: it's not a biggie if you don't find anything
[06:39] <zyga> just reading that comment made me wonder if you found something about rename+exchange that is still racy
[06:42] <mborzecki> zyga: hm not much in my notes, aside from #8045 and 8044/8043
[06:42] <mup> PR #8045: osutil: add helpers for creating symlinks and renaming in an atomic manner <Created by bboozzoo> <Merged by anonymouse64> <https://github.com/snapcore/snapd/pull/8045>
[06:44]  * zyga was tired, re-reading Ian's comment now makes a lot more sense
[06:44] <zyga> mborzecki: thanks!
[07:02] <pstolowski> morning
[07:03] <zyga> good morning pawel
[07:03] <mborzecki> pstolowski: hey
[07:05] <mvo> good morning ps
[07:05] <mvo> good morning pstolowski
[07:05] <pstolowski> o/
[07:18] <mborzecki> pedronis: hi, thanks for the cleanup in #9249
[07:18] <mup> PR #9249: boot,bootloader,gadget: apply new bootloader.Options.Role <Run nested> <UC20> <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/9249>
[07:22] <pedronis> mborzecki: thx
[07:22] <zyga> Lucy just woke up
[07:22] <zyga> afk
[07:36] <mup> PR snapd#9255 closed: o/snapstate, features: add feature flag for disk space check on remove (2.46) <Created by stolowski> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9255>
[09:03]  * zyga-kaveri starts the day
[09:14] <zyga-kaveri> mvo: there's some progress on building understanding of the apparmor notification protocol
[09:14] <zyga-kaveri> mvo: I will try to spend Fridays on that, there's a good chunk of code I can now write
[09:14] <zyga-kaveri> mvo: so the meetings work :)
[09:14] <mvo> zyga-kaveri: cool
[09:15] <zyga-kaveri> mvo: in a way the kernel is really full of murky #defines and bitfield and modes and masks and stuff
[09:15] <zyga-kaveri> writing a good, high-level abstraction for this is only possible with hands-on access to insight not present in the .h files
[09:18]  * mvo nods
[09:24] <mborzecki> pedronis: i've updated #9246
[09:24] <mup> PR #9246: boot: handle canceled update <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9246>
[09:24] <pedronis> mborzecki: thx, I put it back on my queue
[09:24] <mborzecki> pedronis: thanks!
[09:35] <mborzecki> quick errand, back in 30
[09:41] <mup> PR snapd#9258 opened: devicestate: add tests around logging in RequestSystemAction <Created by mvo5> <https://github.com/snapcore/snapd/pull/9258>
[10:11] <mborzecki> re
[10:31] <mup> PR snapd#9259 opened: client, api: handle insufficient space error <Disk space awareness> <Created by stolowski> <https://github.com/snapcore/snapd/pull/9259>
[10:32] <pstolowski> hah, nice touch from github: typing # in the text fields not only auto-completes PR numbers, it actually understands strings and finds PRs by titles. i wish I knew that earlier ;)
[10:36] <mborzecki> pstolowski: is that a new thing?
[10:37] <pstolowski> mborzecki: i have not idea, just discovered by accident
[10:37] <mborzecki> anyways nice, i would often launch a new tab, look for the PR and then use the number
[10:37] <pstolowski> i always assumed numbers
[10:37] <pstolowski> yeah, that's what i was always doing
[10:40] <mvo> pedronis: given that 9250 is green and there is a followup planned anyway, do you mind if I merge it (even without the doc updates?)
[10:41]  * pstolowski lunch
[10:43] <zyga-kaveri> pstolowski: neat :)
[10:44] <zyga-kaveri> pstolowski: maybe it does more, #fixthisbugplease
[10:49] <pedronis> mvo: it's fine, let's just not forget
[11:01] <mup> PR snapd#9250 closed: many: use BootFile type in load sequences <UC20> <Created by cmatsuoka> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/9250>
[11:05] <pedronis> mvo: we should try to land https://github.com/snapcore/snapd/pull/8920 no? it might need a master merge, and a re-run
[11:05] <mup> PR #8920: interfaces: update cups-control and add cups for providing snaps <Needs Samuele review> <Squash-merge> <Created by jdstrand> <https://github.com/snapcore/snapd/pull/8920>
[11:06] <mup> PR snapd#9260 opened: seed/seedwriter: test local asserted snaps with UC20 grade signed <UC20> <Created by pedronis> <https://github.com/snapcore/snapd/pull/9260>
[11:06] <mup> PR snapd#9261 opened: bootloader: tweak doc comments (thanks Samuele) <Created by mvo5> <https://github.com/snapcore/snapd/pull/9261>
[11:10]  * zyga-kaveri afk for 15
[11:24] <pedronis> pstolowski: what's the status of #8960?  should I stil wait to review it? do we need to discuss about it?
[11:24] <mup> PR #8960: o/snapstate,servicestate: use service-control task for service actions (9/9) <Needs Samuele review> <Services ⚙️> <Created by stolowski> <https://github.com/snapcore/snapd/pull/8960>
[11:29] <pstolowski> pedronis: i've a tentative fix and need to update unit tests for it, not pushed yet
[11:30] <pstolowski> pedronis: but it's not the workaround we discussed earlier this week
[11:31] <zyga-kaveri> re
[11:54] <pedronis> mvo: I re-reviewed, tbh not a fan of the new logging code refactor
[11:54] <pedronis> #9210
[11:54] <mup> PR #9210: daemon: add /v2/systems "reboot" action API <Created by mvo5> <https://github.com/snapcore/snapd/pull/9210>
[12:00] <mvo> pedronis: that's fine, happy to redo it, the important part is that we have tests now so it's easier
[12:08] <pedronis> mvo: maybe it's quicker if I push a re-refactor?
[12:11] <mvo> pedronis: please do, in a meeting
[12:25]  * zyga-kaveri takes a lunch break and looks at apparmor sources
[12:53] <pedronis> mvo: pushed
[12:53] <mvo> pedronis: thank you!
[12:54] <pedronis> hopefully is not worse
[12:55] <pedronis> https://github.com/snapcore/snapd/pull/9210/files#diff-0b649265c21137e8fd367d4a16607a82R1058
[12:55] <mup> PR #9210: daemon: add /v2/systems "reboot" action API <Created by mvo5> <https://github.com/snapcore/snapd/pull/9210>
[13:47] <mup> PR snapd#9262 opened: configcore: rework how console-conf is disabled <Created by mvo5> <https://github.com/snapcore/snapd/pull/9262>
[14:52] <mup> PR snapd#9263 opened: interfaces/fpga: add fpga interface <Created by alfonsosanchezbeato> <https://github.com/snapcore/snapd/pull/9263>
[15:35]  * cachio lunch
[15:45] <ijohnson> cachio: let me know when you're back
[15:47] <mup> PR snapd#9254 closed: sysconfig/cloudinit.go: add DisableNoCloud to CloudInitRestrictOptions <Simple 😃> <UC20> <Created by anonymouse64> <Merged by anonymouse64> <https://github.com/snapcore/snapd/pull/9254>
[15:53] <mup> PR snapd#9264 opened: [RFC] many: introduce ContentChange for tracking gadget content in observers <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9264>
[16:00] <pedronis> cmatsuoka: I did a pass on #9257
[16:00] <mup> PR #9257: bootloader: retrieve boot chains from bootloader <UC20> <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/9257>
[16:01] <cachio> ijohnson, hey
[16:01] <cachio> I am here
[16:02] <cmatsuoka> pedronis: thanks, I was in the process of making the mock trusted bootloader functions consistent with the mocked trusted assets
[16:03] <cachio> ijohnson, I see I can't boot anymore uc20 on nested
[16:04] <ijohnson> cachio: hey I need to write a manual nested with my own model assertion when building the image, do you think I should modify the function to get the model assertion to use a global var that I define in the test? Does that make sense?
[16:04] <ijohnson> cachio: re uc20 not booting in nested, what's wrong? can you provide logs? is this on your PR or on master ?
[16:04] <cachio> ijohnson, about the mode, yes
[16:05] <cachio> a var for the model sounds good
[16:05] <ijohnson> cachio: ack I will add that then
[16:06] <cachio> ijohnson, https://paste.ubuntu.com/p/rstDJgrkqq/
[16:06] <cachio> ijohnson, could be related to reseal?
[16:07] <ijohnson> cachio: mmm I don't think enough of that has landed to be related to resealing
[16:07] <ijohnson> cachio: was this from master or from your branch ?
[16:08] <cachio> it is my branch
[16:08] <cachio> I merged with master
[16:08] <cachio> not sure if it is the root cause
[16:13] <ijohnson> cachio: I'll have a look in a bit
[16:15] <cachio> I thnik it is something in edge channel because the iamges created from beta work well
[16:18] <ijohnson> cachio: I've got google-nested:ubuntu-20.04-64:tests/nested/core20 running from master now, let's see what happens
[16:19] <cachio> ijohnson, nice, manual also fails for some tests
[16:19] <ijohnson> cachio: which tests ? is it reproducible ?
[16:19] <cachio> minimal-smoke
[16:20] <ijohnson> cachio: yes I've seen that one fail intermittently on PR's
[16:20] <cachio> ijohnson, I am still waiting for more results
[16:20] <ijohnson> k
[16:20] <cachio> but fails to prepare, basically can't login
[16:21] <cachio> in the logs I don't see any reboot
[16:23] <cachio> ijohnson, the weird part is that I ran with kvm and there are not any reboot
[16:33] <ijohnson> cmatsuoka: have you ever seen this failure on gce before?
[16:33] <ijohnson> 2020-08-26T17:52:35.2828113Z Aug 26 17:46:06 ubuntu snapd[1116]: secboot_tpm.go:416: TPM provisioning error: cannot access resource at handle TPM_RH_LOCKOUT because an authorization check failed
[16:33] <ijohnson> 2020-08-26T17:52:35.2828332Z Aug 26 17:46:06 ubuntu snapd[1116]: taskrunner.go:271: [change 2 "Setup system for run mode" task] failed: cannot create partitions: cannot seal the encryption key: cannot provision TPM: cannot access resource at handle TPM_RH_LOCKOUT because an authorization check failed
[16:33] <ijohnson> seems like the TPM wasn't properly reset before it was used to boot this image?
[16:34] <ijohnson> cachio: yeah very odd, my run is almost done preparing the image, just waiting for it to either finish successfully or get stuck
[16:34] <cmatsuoka> ijohnson: yes, it happened to cachio a couple of times. according to chris this shouldn't happen, unless the TPM was not properly cleared
[16:34] <cmatsuoka> (or the simulator is faulty)
[16:34] <ijohnson> mmm yeah this is with the swtpm-mvo snap
[16:36] <cmatsuoka> ijohnson: do you see any possibility of the TPM not being cleared before this test?
[16:43] <pedronis> ijohnson: I tried to answer your considerations in #9253
[16:43] <mup> PR #9253: sysconfig/cloudinit.go: add AllowCloudInit and use GadgetDir for cloud.conf <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9253>
[16:43] <ijohnson> mmm seems my internet is out :-/
[16:44] <ijohnson> cmatsuoka: actually yes I see how it could happen, we don't ever reset the TPM from swtpm-mvo, if it's already installed, we should reset it if it's already installed when we boot a VM
[16:44] <ijohnson> I can send a pr when my internet is back
[16:45] <cmatsuoka> ijohnson: ah, mystery solved then. I thought the nv data file was deleted before each test
[16:45] <ijohnson> pedronis ack thanks for clarifying, I will leave the function as-is, copying gadget config if it is found
[16:46] <ijohnson> Err rather I'll refactor to first check if AllowCloudInit, then if it's allowed then copy the gadget config
[16:49] <ijohnson> yay internet is back
[16:51] <pedronis> ijohnson: I think the points are mostly order, rename and comments
[16:52] <cachio> ijohnson, cmatsuoka at some point we were reseting this
[16:52] <cachio> I added that when we originally had that problem and claudio told me to add that
[16:52] <ijohnson> Well apparently my internet is not back
[16:52] <cachio> don't know if it is gone with a refactor
[16:52]  * ijohnson is back to the phone 
[16:53] <ijohnson> pedronis ack yes I will push up an update later today
[16:53] <ijohnson> cachio I don't see it anywhere, where were we doing this?
[16:53] <ijohnson> cachio all I see on master is that `if ! snap list swtpm-mvo ...`
[16:54] <cachio> ijohnson, it is not there anymore
[16:54] <ijohnson> yeah
[16:54] <cachio> I think during a refactor we remove that
[16:54] <cachio> I'll take a look to the logs
[16:54] <cachio> I can add that to my PR as well
[16:55] <cachio> so perhaps that is affecting also
[16:55] <ijohnson> cachio: please don't add it to your pr
[16:55] <ijohnson> cachio: I really want to land that pr without further iteration
[16:55] <ijohnson> cachio: that pr is really big, I am almost done with another review for you
[16:55] <cachio> ijohnson, sure
[16:55] <cachio> ijohnson, ah, nice, thanks
[16:56] <cachio> it started small
[16:56] <cachio> :)
[16:56] <ijohnson> yes it started small, but is no longer small :-)
[16:59] <ijohnson> cachio: well re the failing uc20 nested tests, I lost my internet connection so I don't know if the test failed or not, I will try again now hopefully my internet doesn't cut out again
[17:03] <cachio> ijohnson, np
[17:04] <cachio> the test in this pr could be failing because the tpm or the kernel repack thing
[17:04] <cachio> which are already being fixed in other prs
[17:05] <cachio> ijohnson, here we have a lot of problems with the internet too, all the children having zoom meetings for school
[17:05] <ijohnson> pedronis: is it expected that `snap ack` on a system-user assertion doesn't "just work" and that the system-user assertion needs to be imported by a usb drive for example?
[17:05] <cachio> and so many people working from home in this area as well
[17:05] <ijohnson> cachio: yeah that's what I expect for our neighborhood too, it's all virtual learning here too pretty much
[17:06] <ijohnson> pedronis: `snap ack system-user.assert` returns 0, but no user is created
[17:07] <ijohnson> pedronis: ah I see from the docs on system-user assertions:
[17:07] <ijohnson> > The simple addition of such assertions to a device assertion database should not be enough to trigger the user creation
[17:07] <ijohnson> so nvm me, it is not enough to just `snap ack`
[17:14] <cmatsuoka> pedronis: addressed the issues but not sure if the grub NoSlashBoot fix is what you had in mind
[17:18] <pedronis> ijohnson: that's by design, auto import from usb does a step more, but ack alone is not alone
[17:18] <pedronis> *is not enough
[17:19] <ijohnson> Yes I see that now
[17:22] <pedronis> cmatsuoka: it's fine, but probably better not to use TrustedAssets in boot chain now we the vars
[17:22] <cmatsuoka> pedronis: thanks, will update accordingly
[17:31]  * cmatsuoka breaks for lunch
[17:40] <pedronis> ijohnson: could you look at #9246 ?
[17:40] <mup> PR #9246: boot: handle canceled update <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9246>
[17:40] <ijohnson> pedronis: sure I can look this afternoon
[17:53] <ijohnson> cachio: both of the tests/nested/core20 from master failed for me
[17:53] <ijohnson> cachio: I think what happened is that cloud-init took too long to run and got disabled by snapd
[17:53] <ijohnson> cachio: but I need to do another run with persistent logging to know for sure what happened
[17:54] <ijohnson> it's rather annoying that we don't have persistent logging enabled by default :-/
[18:03] <cachio> ijohnson, thanks for checking that
[18:13] <mup> PR snapcraft#3274 closed: schema: rename package-repository's "deb-types" to "format" <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3274>
[18:13] <pedronis> ijohnson: isn't our cloud-init timeout fairly generous?
[18:14] <pedronis> do we need to increase it?
[18:14] <ijohnson> pedronis: well without logs I don't know what happened, all I know is that cloud-init didn't create the user even though the right files were put onto /etc/cloud from ubuntu-seed, but the disabled file was installed by snapd from the previous boot
[18:15] <ijohnson> pedronis: we give cloud-init 5 minutes to finish running
[18:15] <ijohnson> pedronis: but also note that these VM's are excruciatingly slow
[18:15] <ijohnson> *nested VM's
[18:15] <ijohnson> so maybe it does actually take more than 5 minutes during a first boot for cloud-init to finish, I dunno
[18:16] <ijohnson> I have a run in progress with persistent logging enabled via the gadget so hopefully I can see what happened for sure
[18:17] <cachio> ijohnson, I'll be afk about 1 hour, need to go to the kinesiologist now, please let me know if you find anything else
[18:17] <ijohnson> cachio: sounds good
[18:17] <cachio> I'll go back to this once I am back
[18:17] <cachio> thanks
[18:18] <mup> PR snapcraft#3259 closed: cli: introduce set-default-track <enhancement> <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3259>
[18:18] <mup> PR snapcraft#3275 closed: cli: ignore sudo warning when using multipass <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3275>
[18:18] <mup> PR snapcraft#3276 closed: meta: add suite/component validation checks for package-repositories <Created by cjp256> <Closed by cjp256> <https://github.com/snapcore/snapcraft/pull/3276>
[18:20]  * cachio -> kinesiologist
[18:33] <mup> PR snapd#9260 closed: seed/seedwriter: test local asserted snaps with UC20 grade signed <Test Robustness> <UC20> <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/9260>
[18:38] <mup> PR snapd#9261 closed: bootloader: tweak doc comments (thanks Samuele) <Created by mvo5> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/9261>
[19:02] <ijohnson> cachio: I +1'd 9098, I think you should merge it when it goes green
[19:02] <ijohnson> let's just handle cleanups and followups, the PR is changing a lot and it's difficult to keep tracking
[19:53] <ijohnson> mmm this is odd
[19:53] <ijohnson> devicemgr.go:702: System initialized, cloud-init reported to be done, set datasource_list to [ None ]
[19:54] <ijohnson> seems cloud-init ran, it did in face take more than 5 minutes, but also it seems that cloud-init finished without actually doing anything
[19:54] <ijohnson> Sep 02 19:25:19 ubuntu systemd[1]: Reached target Cloud-init target.
[19:54] <ijohnson> Sep 02 19:25:19 ubuntu systemd[1]: Startup finished in 52.655s (kernel) + 5min 16.330s (userspace) = 6min 8.985s.
[20:21] <ijohnson> cachio: I think I understand why nested uc20 tests are broken
[20:21] <ijohnson> cachio: it is because of a PR landed to the core20 snap which breaks cloud-init
[20:40] <ijohnson> cachio: I have debugged it and will be opening a PR shortly to fix the problem
[20:45] <mup> PR core20#84 opened: static/writable-paths: use transition instead of none for /etc/cloud <Created by anonymouse64> <https://github.com/snapcore/core20/pull/84>
[20:48] <cachio> ijohnson, awesome
[20:48] <cachio> thanks
[20:50] <mup> PR core20#84 closed: static/writable-paths: use transition instead of none for /etc/cloud <Created by anonymouse64> <Merged by xnox> <https://github.com/snapcore/core20/pull/84>
[21:24] <cachio> ijohnson, please take a look to the answers I left in the PR #9098
[21:24] <mup> PR #9098: tests: new organization for nested tests <Run nested> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9098>
[21:24] <ijohnson> cachio: yes I saw, looks fine to me
[21:24] <cachio> ijohnson, thanks
[21:24] <ijohnson> cachio: as soon as the pr goes green, please merge I think it's ok without a second +1
[21:24] <cachio> ijohnson, nice, thank
[21:24] <cachio> s
[22:04] <mup> PR snapd#9265 opened: many: move seal code from gadget/install to boot <UC20> <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/9265>
[22:06] <cachio> zyga-kaveri, when you have 5 minutes, could you please take a final view to #9098
[22:06] <mup> PR #9098: tests: new organization for nested tests <Run nested> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9098>
[22:49] <mup> PR snapd#9266 opened: tests/lib/nested.sh: reset the TPM when we create the uc20 vm <Bug> <Run nested> <Test Robustness> <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9266>
[22:56] <cmatsuoka> ijohnson: is "partion" a real word or can I assume it's mispelled partition?
[23:04] <ijohnson> cmatsuoka: afaik it's not a word
[23:04] <cmatsuoka> ijohnson: from the context I'll assume it's intended to be "partition"
[23:07] <ijohnson> cmatsuoka: is this from one of my prs?
[23:08] <cmatsuoka> ijohnson: I found many instances, just grep -Ri partion. I'm preparing a PR to change them to partition
[23:16] <ijohnson> ack
[23:24] <mup> PR snapd#9267 opened: many: fix partion vs partition typo <Simple 😃> <Created by cmatsuoka> <https://github.com/snapcore/snapd/pull/9267>