[06:15] <zyga> Hey
[06:15] <zyga> Back to school day
[06:17] <mborzecki> morning
[06:17] <zyga> We need to land the critical PR
[06:17] <mborzecki> zyga: hey, which one?
[06:17] <zyga> Stuff is still broken, eh
[06:17] <zyga> Last one
[06:18] <zyga> 8178
[06:18] <mborzecki> uhh
[06:19] <mborzecki> zyga: that's why pushing snapd to store failed on friday?
[06:22] <zyga> re
[06:22] <zyga> checking what failed
[06:22] <zyga> yes
[06:22] <zyga> sucks, eh?
[06:22] <zyga> I saw some failover failures yesterday
[06:23] <zyga> and eh
[06:23] <zyga> fu**
[06:23] <zyga> https://www.irccloud.com/pastebin/RwhTm8UI/
[06:23] <mborzecki> fun? :)
[06:23] <zyga> where do we get foo frm?
[06:24] <zyga> crap from other test?
[06:24] <mborzecki> yeah, tha'd be my first guess
[06:25] <zyga> pushed a fix
[06:25] <mborzecki> there's security-private-tmp test
[06:26] <mborzecki> and it ran before
[06:27] <zyga> mmm
[06:27] <zyga> I'll get coffee and fix this properly
[06:27] <mborzecki> zyga: maybe as a workaround, we could also take down the setgid flag in postinst?
[06:28] <zyga> hmm?
[06:28] <zyga> postinst of the deb?
[06:28] <zyga> that would ruin the snap, no?
[06:28] <zyga> as in, that would cause the snap to be g-s
[06:28] <zyga> unless I don't understand how it's made
[06:29] <zyga> in either case, it's all temporary
[06:29] <zyga> we'll fix review tools today
[06:29] <zyga> deploy in a day or two
[06:29] <mborzecki> duh, yeah, that may produce a different result for deb and snap
[06:30] <mborzecki> hm looks like the snap is built by just extrating snapd, so we'd have to tweak the bit there too
[06:30] <mborzecki> but that would break review-tools again :/
[06:31] <zyga> mborzecki: I didn't test-build the deb because i didn't feel comfortable guessing how it's really built
[06:31] <zyga> mborzecki: but the deb, for sure, has no g+s anymore
[06:31] <zyga> er
[06:31] <zyga> *has*
[06:31] <zyga> so with some luck it will pass
[06:40] <zyga> brb
[07:07] <mup> PR snapd#8179 opened: interfaces: power control interface <Created by EthanHsieh> <https://github.com/snapcore/snapd/pull/8179>
[07:42] <zyga> re
[07:43] <zyga> mborzecki: can you review 8178
[07:43] <zyga> I'll land with one review
[07:51] <mup> PR snapd#8178 closed: packaging: work around review-tools and snap-confine <⚠ Critical> <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/8178>
[07:58] <mup> PR snapd#8180 opened: tests: rm -rf /tmp/snap.* in restore <Created by zyga> <https://github.com/snapcore/snapd/pull/8180>
[07:58] <zyga> mborzecki: ^ quick follow-up
[07:58] <zyga> hey mvo
[07:58] <zyga> good morning
[07:58] <zyga> mvo: FYI https://github.com/snapcore/snapd/pull/8178
[07:58] <mup> PR #8178: packaging: work around review-tools and snap-confine <⚠ Critical> <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/8178>
[07:59] <mvo> hey zyga
[07:59] <mvo> zyga: looking
[08:01] <pstolowski> morning
[08:01] <zyga> hey Pawel :)
[08:01] <zyga> mvo: this was pre-approved but I wanted to let you know it landed
[08:01] <zyga> mvo: it allows core/snapd snaps to be published again
[08:04] <mborzecki> mvo: pstolowski: hey
[08:04] <mvo> hey pstolowski and mborzecki
[08:04] <mvo> zyga: yeah, saw the mails over the weekend
[08:05] <zyga> failover still fails
[08:05] <mvo> zyga: do you know if jamie is aware that the review tools need fixing?
[08:05] <zyga> yes
[08:05] <mup> PR snapd#8136 closed: boot: write current_kernels in bootstate20, makebootable <UC20> <Created by anonymouse64> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/8136>
[08:05] <pedronis> mvo: yes, he said he was working on it
[08:05] <zyga> we talked (Jamie / Pedronis / me) about it together
[08:05] <mvo> cool
[08:05] <pstolowski> mvo: hey! it occured to me over the weekend that snap-pressed could be made a classic snap, which is what you probably meant when you suggested making it a snap :)
[08:06] <zyga> snapd-failover failure on top of current master https://www.irccloud.com/pastebin/pxEx4fth/
[08:06] <mvo> pstolowski: yeah, that will work
[08:09] <mup> PR snapd#8168 closed: snapcraft.yaml: add comments, rename snapd part to snapd-deb <Simple 😃> <Created by anonymouse64> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/8168>
[08:10] <pstolowski> mvo: good, i'll prep a PR for that (a snapcraft.yaml inside cmd/snap-presseed?) and will make a quick test
[08:11] <pedronis> pstolowski: do we know it's needed? I'm probably missing something
[08:11] <zyga> hey pedronis, good morning :)
[08:12] <pstolowski> pedronis: the only alternative afaict is to use a ppa right now, until a new deb lands in the distro
[08:14] <mvo> pstolowski: it's probably ok to wait for them for feedback first, their build setup is special and may not be suitable for snaps (I honestly don't know). so might be worth to double check before we invest time in this
[08:15] <mborzecki> btw. trying to find out why spotify core dumps on arch reminded me of the missing feature of having access to debug symbols, that we discussed but never got down to working on it
[08:16] <pedronis> mborzecki: there is been work on the snapcraft side on that
[08:16] <pedronis> we actually need to play with it
[08:17] <pedronis> but not much time atm
[08:17] <pstolowski> mvo: sure
[08:18] <mborzecki> pedronis: interesting, is there some info about the snapcraft bits?
[08:18] <pedronis> mborzecki: yes
[08:18] <pedronis> mborzecki: I can give some pointers in the standup
[08:19] <mborzecki> pedronis: please do, thanks!
[08:39] <pedronis> mvo: I made a comment on #8100, does #8142 need a master merge?
[08:39] <mup> PR #8100: httputil: add support for extra snapd certs <Created by mvo5> <https://github.com/snapcore/snapd/pull/8100>
[08:39] <mup> PR #8142: client: add support for "ResumeToken", "HeaderPeek" to download <Created by mvo5> <https://github.com/snapcore/snapd/pull/8142>
[08:42] <pedronis> mborzecki: did you see my remark on #6708 ?
[08:43] <mup> PR #6708: tests/main/buildmode: verify usability of PIE hardening for deb packages <⛔ Blocked> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/6708>
[08:43] <pedronis> pstolowski: are you waiting on reviews from me atm ?
[08:43] <pstolowski> pedronis: no. thanks for asking!
[08:45] <pedronis> mborzecki: when you have a second could you push master into #8147 ?
[08:45] <mup> PR #8147: cmd/snap-bootstrap: verify kernel snap is in modeenv before mounting it <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8147>
[08:45] <mborzecki> pedronis: sure
[08:45] <pedronis> thx
[08:58] <mvo> pedronis: thanks, will followup, 8142 probably needs a master merge
[09:28] <mborzecki> spread tests should be fixed now in #8156
[09:28] <mup> PR #8156: cmd/snap-bootstrap: subcommand to detect UC chooser trigger <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8156>
[09:33] <mborzecki> mvo: pedronis: the recovery chooser binary is /usr/lib/snapd/snap-recovery-chooser, does that sound ok? i want to open a PR to subiquity with changes to console-conf-wrapper
[09:34] <ackk> hi, how do I remove a broken (try) snap? it complains because it doesn't find the .service file
[09:37] <ackk> https://paste.ubuntu.com/p/r4H46ycwVq/ is what I get
[09:40] <ackk> oh, it seems umounting the snap before trying to remove it worked
[09:54] <mvo> mborzecki: sorry for the delay, was in a meeting. the name sounds good to me
[09:55] <mvo> ackk: thanks, this feels a bit like a bug on our side, we need to be robust against failures here
[09:56] <ackk> mvo, TBF this was a "snap try" whose prime dir ended up being removed/overwritten, so kinda of an edge case
[09:57] <mvo> ackk: agreed and yet we should handle it better :) but agreed on the edge-case so not that high of a priority right now
[09:57] <mvo> ackk: out of curiosity, did you guys made progress on the run-away supervisor processes?
[09:59] <ackk> mvo, sort of. BjornT experienced something similar installing maas on a rpi4. my gut feeling is that if for some reason there's a high cpu usage and everything slows down, supervisord times out on starting processes, kills and restarts them, making things worse
[10:00] <ackk> mvo, in my case because of squashfuse using all cpu, on the rpi because of limited cpu power
[10:00] <ackk> mvo, there doesn't seem to be any error in logs in these cases, just processes being started and killed
[10:00] <mup> PR snapd#8181 opened: packaging: revert "work around review-tools and snap-confine" <Skip spread> <⛔ Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8181>
[10:00] <mvo> ackk: woah, ok
[10:15] <zyga> mvo: https://github.com/snapcore/snapd/pull/8181 has two patches, only one should be reverted (the other will conflict once https://github.com/snapcore/snapd/pull/8180 lands
[10:15] <mup> PR #8181: packaging: revert "work around review-tools and snap-confine" <Skip spread> <⛔ Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8181>
[10:15] <mup> PR #8180: tests: rm -rf /tmp/snap.* in restore <Created by zyga> <https://github.com/snapcore/snapd/pull/8180>
[10:19] <mborzecki> yay, opened https://github.com/CanonicalLtd/subiquity/pull/636
[10:20] <mup> PR CanonicalLtd/subiquity#636: bin/console-conf-wrapper: detect whether snapd recovery chooser should run <Created by bboozzoo> <https://github.com/CanonicalLtd/subiquity/pull/636>
[10:20] <zyga> mborzecki: looking :)
[10:21] <mborzecki> i'm thinking that maybe w should have a list of ttys in snapd that the chooser can be invoked on, like /etc/securetty, this can easily be gadget dependent
[10:21] <zyga> mborzecki: looks innocent enough :)
[10:21] <mborzecki> yeah, it does ;)
[10:22] <mvo> zyga: aha, thanks, feel free to edit and force push, in a meeting righ tnow
[10:22] <zyga> mvo: ack, will do
[10:22] <mborzecki> /var/lib/snapd/choosertty
[10:25] <mborzecki> again soemthing with jenkins spread test?
[10:26] <zyga> mborzecki: I saw that today
[10:26] <zyga> their repo times out
[10:26] <zyga> eh
[10:26] <mup> PR core20#23 opened: hooks: ensure console-conf@ is started after snapd.recovery-chooser-trigger.service <Created by bboozzoo> <https://github.com/snapcore/core20/pull/23>
[10:27] <zyga> mborzecki: question
[10:27] <zyga> what happens if that service is disabled?
[10:27] <zyga> namely if you systemctl disable snapd.recovery-chooser-trigger.service
[10:27] <zyga> how does that affect dependency chain?
[10:27] <mborzecki> zyga: nothing, it's only Before/After ordering, not Requires
[10:28] <zyga> aaah
[10:28] <zyga> that's great
[10:28] <zyga> ok
[10:31] <zyga> mborzecki: wild west idea: a tool that reads from a google spreadsheet, allowing us to put a marker in a specific cell that means jenkins test is skipped
[10:31] <zyga> or that arch or tumbleweed are broken today
[10:31] <zyga> etc
[10:31] <zyga> mborzecki: the sheet could be ACLd and locked
[10:36] <mborzecki> zyga: we have s3 like storage for out gcp now, we could easily upload a list of disabled tests there, download it to the host, and skip at runtime, then track which run skipped what
[10:36] <zyga> mborzecki: spreadsheets are way easier to work with
[10:36] <zyga> mborzecki: and I bet the ACLs on entire file is not as nice as per-cell ACL
[10:38] <mborzecki> idk, i prefer plain text files :P
[10:38] <zyga> mborzecki: even changing files in s3-like thing is hard
[10:38] <zyga> and the other thing is self-documenting, you open a URL to a specific cell
[10:38] <zyga> and if you can, you can edit it
[10:38] <zyga> easy
[10:46] <zyga> core 20 setup is sloooooooow
[10:48] <mup> PR snapd#8133 closed: cmd/snap-confine: deny snap-confine to load nss libs <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/8133>
[10:59] <mup> PR snapd#8182 opened: build: enable type: snapd <⛔ Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/8182>
[11:03] <pedronis> zyga: we'll start taking care of some todos that should improve that a bit this week
[11:06] <zyga> that's good to know
[11:12] <mborzecki> #8176 is simple and green
[11:12] <mup> PR #8176: tests: adding amazon linux to google backend <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/8176>
[11:20] <zyga> brb, cold and somewhat hungry as a result
[11:37] <zyga> re
[11:37] <zyga> hot tea :)
[11:50] <zyga> did anyone debug why tests/go.{mod,sum} show up
[12:00] <zyga> kids back from school
[12:00] <zyga> also school is back
[12:00] <zyga> whee
[12:00] <zyga> :)
[12:13] <mborzecki> duh, wonder why nothing is printed out with termbox-go controlling ttyS0
[12:35] <zyga> mborzecki: maybe it establishes exclusive control over tty?
[12:35] <zyga> there's an ioctl for it
[12:35] <zyga> I used it once
[12:35] <zyga> https://github.com/snapcore/core-build/blob/master/initramfs/testing/init.c#L212
[12:37] <pedronis> mborzecki: some comments and questions on 8156
[13:00] <mborzecki> zyga: hm so input works, i'm loogginmg when keys are pressed and so on, but the output does not work
[13:00] <zyga> that's odd
[13:01] <cmatsuoka> zyga: are you aware of any mount flag that disables directories bind mounts but won't affect regular file bind mounts?
[13:01] <zyga> cmatsuoka: no, because that's not a property of a mount point
[13:02] <zyga> cmatsuoka: in other words, that's not possible AFAIK
[13:02] <cmatsuoka> zyga: I'm having this really weird bug where I can't bind mount directories but files work normally
[13:02] <cmatsuoka> I mean, bind mounting files work, but directories fail silently
[13:02] <zyga> cmatsuoka: can you tell me more please
[13:02] <zyga> cmatsuoka: also /proc/self/mountinfo would help
[13:02] <cmatsuoka> zyga: ok, so it's a long story. I'll start with the symptoms
[13:03] <zyga> cmatsuoka: sure
[13:03] <zyga> cmatsuoka: do you want to HO or is here find?
[13:03] <zyga> *fine
[13:03] <cmatsuoka> in a HO it will be easier to explain
[13:04] <zyga> sure
[13:04] <cmatsuoka> I'm in the SU room
[13:04] <zyga> one sec
[13:15] <pedronis> cmatsuoka: hasn't xnox made progress (see uc20)
[13:16] <xnox> cmatsuoka:  all i can see is that without a valid /etc/crypttab things that require writable mount and bind to it, all get torn down by systemd, because it says dev-mapper-ubuntu-data.device? i don't need that, stop it now.
[13:17] <xnox> cmatsuoka:  i'm now redoing to provide a valid /etc/crypttab in the mounted system, to ensure the post-switch-root systemd has left and right half of the brain connected together.
[13:21] <zyga> for reference, we discussed some ideas, including unbindable, and will look at mountinfo data to try to understand what is going on
[13:21] <cmatsuoka> xnox: thanks, I think the most interesting piece of information is that switching from a point as late as initrd-cleanup still works
[13:22] <zyga> snap session agent socket activation failed
[13:22] <zyga> https://www.irccloud.com/pastebin/8UEOWyMx/
[13:22] <zyga> jamesh: ^
[13:27] <xnox> cmatsuoka:  yes, it's the systemd from the rootfs that gets confused.
[13:27] <xnox> cmatsuoka:  you can boot with init=/bin/bash => such that switch-root pivots to a bash shell, where everything still looks sane
[13:28] <cmatsuoka> xnox: will try that
[13:28] <pedronis> mvo: what is happenign here:  https://api.travis-ci.org/v3/job/654356068/log.txt there's a SKIP_GOFMT=1 but an error about go fmt ??
[13:30] <zyga> pedronis: it runs gofmt
[13:30] <zyga> and ignores the result
[13:30] <zyga> that's how I remember it
[13:31] <zyga> (it's kind of weird)
[13:31] <zyga> pedronis: it really failed on FAIL: overlord_test.go:694: overlordSuite.TestEnsureLoopPruneAbortsOld
[13:31] <zyga> pedronis: roughly before the middle of the test log
[13:31] <pedronis> ah
[13:32] <pedronis> I was confused
[13:32] <pedronis> pstolowski: https://api.travis-ci.org/v3/job/654356068/log.txt
[13:32] <zyga> yeah, we should remove that gofmt thing on SKIP_GOFMT
[13:32] <zyga> it's silly
[13:32] <zyga> pedronis: I'll do it later today
[13:33]  * zyga looks outside at endless rain
[13:34] <zyga> good thing I have tea :)
[13:37] <cmatsuoka> xnox: confirmed, it works just after switch root and before init
[13:40] <mborzecki> zyga: what about SKIP_GOFMT?
[13:40] <zyga> mborzecki: it's really RUN_GOFMT_AND_NOT_FAIL_LOL=1
[13:41] <mborzecki> well, that's its purpose
[13:41] <zyga> mborzecki: it's a weird purpose
[13:41] <zyga> mborzecki: it's always noise
[13:41] <zyga> mborzecki: why would you want to look at gofmt from incompatible version of go?
[13:41] <mborzecki> zyga: not sure i follow
[13:41] <zyga> mborzecki: we set SKIP_GOFMT=1 when we know the result is garbage anyway
[13:42] <mborzecki> zyga: the problem is that travis builds and cheks with go 1.10, which i don't use locally
[13:42] <zyga> mborzecki: when it is equal to 1 then we _run_ gofmt, print a bunch of diff, and not fail on that
[13:42] <mborzecki> zyga: right, so that you know what's wrong wrt. 1.10 that travis uses
[13:42] <zyga> mborzecki: are you saying it's always wrong? :)
[13:42] <mborzecki> zyga: if go didn't break the formatting in 1.11 we wouldn't have the problem, but they did
[13:43] <zyga> mborzecki: anyway, if you look at what the code does now
[13:43] <zyga> mborzecki: I think you will agree that what we do is silly
[13:43] <zyga> mborzecki: we do run the gofmt check with the right go
[13:43] <zyga> mborzecki: (in other places)
[13:43] <zyga> mborzecki: but we always run and print useless diff regardless of anyone asking for it
[13:44] <pedronis> as data point I was confused by it and missed the real issue initially
[13:44] <mborzecki> zyga: but other palces isn't travis unit tests job , maybe it shouldn't check gofmt then
[13:45] <zyga> mborzecki: there's a task that checks gofmt with the right version
[13:45] <zyga> mborzecki: it's not the quick-and-early check
[13:45] <zyga> mborzecki: the quick and early check does run gofmt verification, wasting time, producing confusion and _ignores_ the result because go version mismatch
[13:45] <zyga> mborzecki: that's the problem
[13:46] <pedronis> mmh, this was not the early check
[13:46] <pedronis> this was unit/go
[13:46] <pedronis> afaict
[13:46] <zyga> pedronis: right
[13:46] <zyga> pedronis: because that's the same piece of code
[13:46] <zyga> it runs twice
[13:46] <mborzecki> zyga: because we run unit tests twice
[13:46] <zyga> I think we should just not run gofmt check at all if SKIP_GOFMT is set
[13:46] <zyga> mborzecki: correct
[13:47] <mborzecki> zyga: once in travis, then antoher one in spread, the 2nd may run on a new distro where go isn't 1.10
[13:47] <zyga> mborzecki: that's correct too
[13:48] <mborzecki> zyga: so shoudl the early check fail, you get a diff of the code wrt. 1.10, and you know what to fix or reformat
[13:48] <zyga> mborzecki: that's not what I'm saying
[13:48] <zyga> mborzecki: unless I'm confused: <zyga> I think we should just not run gofmt check at all if SKIP_GOFMT is set
[13:49] <zyga> mborzecki: this is sufficient to keep checking exactly what we check now and reduce the confusing diff intermixed with actually relevant unit test failures
[13:50] <zyga> jdstrand: hey
[13:50] <zyga> jdstrand: https://github.com/snapcore/snapd/pull/8123 is ready for a quick re-review
[13:50] <mborzecki> zyga: ah ok, so yeah, skip the whole thing then
[13:50] <mup> PR #8123: interfaces/network-control: bring /var/lib/dhcp from host (approach b) <Bug> <Created by zyga> <https://github.com/snapcore/snapd/pull/8123>
[13:50] <zyga> jdstrand: please check my comments but I think you will find that it implements what you outlined
[13:50] <jdstrand> zyga: I saw
[13:50] <jdstrand> well, I saw it was ready for re-review
[13:50] <zyga> jdstrand: I didn't implement the helper that makes the permissions side more understandable
[13:50] <jdstrand> I haven't gone through it yet
[13:50] <ijohnson> guten tag friends :-)
[13:51] <zyga> jdstrand: but you indicated that as optional for now
[13:51] <jdstrand> hey ijohnson :)
[13:51] <ijohnson> hey jdstrand
[13:51] <zyga> jdstrand: I'll get back to transient scope pr now
[13:51] <zyga> ijohnson: halo :)
[13:52] <ijohnson> how are you zyga? have a good weekend?
[13:52] <zyga> ijohnson: pretty good, yeah
[13:52] <zyga> ijohnson: winter holidays are over so kids are back to order ;)
[13:53] <ijohnson> nice, it got really warm here over the weekend (like 5-10 C I think) and we had a bbq at my mom's house, and I found some paczki to share for dessert!
[13:53] <zyga> ijohnson: oh, nice :)
[13:53] <zyga> ijohnson: what's the flavor?
[13:54] <ijohnson> zyga: it had custard cream in it with some powdered sugar on the outside
[13:54] <ijohnson> let me see if I can find a picture
[13:54] <zyga> NO :D
[13:54] <ijohnson> I guess it's a bit "americanized"
[13:54] <zyga> I didn't have lunch yet
[13:54] <ijohnson> hahaha
[13:54] <zyga> I'll be super hungry all standup :D
[13:55] <zyga> ijohnson: we usually have some jam inside, rose or plum
[13:55] <zyga> ijohnson: and carmelized orange peel on the outside
[13:55] <ijohnson> https://usercontent.irccloud-cdn.com/file/RhgAaQJe/paczki.png
[13:55] <zyga> (yummy)
[13:55] <ijohnson> ah that sounds really good
[13:55] <ijohnson> ^ that's what the ones I found look like
[13:55] <zyga> that's a sizable bag :D
[13:55] <zyga> oh well
[13:56] <zyga> I have some leftovers from dinner to warm up after standup
[13:56] <zyga> (not paczki though)
[13:56] <ijohnson> :-)
[13:57] <zyga> cmatsuoka: I see the mountinfo data
[13:58] <pstolowski> pedronis, zyga oh it failed despite the sleep bump last Friday? hmm
[13:58] <zyga> pstolowski: not sure but I think I saw one failure after rebasing
[13:59] <zyga> pstolowski: I don't know the state of the branch that Samuele was looking at
[13:59] <pstolowski> i see
[14:00] <zyga> cmatsuoka: and when it was in the broken state, what's the mount operation that you tried to perform exactly?
[14:00] <zyga> oh
[14:00] <zyga> standup
[14:39] <mup> PR snapd#8183 opened: tests: remove tmp dir for snap not-test-snapd-sh on security-private-tmp test <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/8183>
[14:41] <zyga> https://github.com/snapcore/snapd/pull/8180 needs a 2nd review and should be [simple]
[14:41] <mup> PR #8180: tests: rm -rf /tmp/snap.* in restore <Created by zyga> <https://github.com/snapcore/snapd/pull/8180>
[14:43] <jdstrand> zyga: ok, responded to PR 8123
[14:43] <mup> PR #8123: interfaces/network-control: bring /var/lib/dhcp from host (approach b) <Bug> <Created by zyga> <https://github.com/snapcore/snapd/pull/8123>
[14:43] <zyga> jdstrand: thank you
[14:43] <jdstrand> zyga: did you see my comment in PR https://github.com/snapcore/snapd/pull/7731#issuecomment-589829764
[14:43] <mup> PR #7731: usersession/userd: add "apt" to the white list of URL schemes handled by xdg-open <⛔ Blocked> <Created by oSoMoN> <https://github.com/snapcore/snapd/pull/7731>
[14:44] <zyga> jdstrand: not yet
[14:44] <jdstrand> zyga: you might want to see my other comments from friday
[14:44] <zyga> jdstrand: I will, github has some nice new notifications
[14:44] <zyga> I need to start using that again
[14:44] <jdstrand> zyga: I think we can merge it now, cherry-picking to 2.44 if needed
[14:44] <zyga> (the previous one was just noise and hard to find anything in)
[14:45] <zyga> jdstrand: ack, I'll review both shortly
[14:50] <mvo> meh, my inbox just exploded with snapd edge build failures :(
[14:53] <zyga> yeah
[14:53] <zyga> did you check
[14:53] <zyga> I'm considering checking or having quick lunch ahead of core 20 sync
[15:05] <roadmr> does a rabbit qualify as "quick lunch"?
[15:05] <roadmr> also - baby oil - which part of the baby does it come from? 🤔
[15:09] <cwayne> roadmr: i literally asked that yesterday
[15:12] <roadmr> you were around asking questions about baby oil on #snappy on sunday?
[15:14] <jdstrand> roadmr: you are on fire today :)
[15:14] <zyga> failure from launchpad builders https://www.irccloud.com/pastebin/i6AZBoZK/
[15:14] <roadmr> and it's only 10:15 am :)
[15:14] <jdstrand> \o/
[15:17] <zyga> pstolowski: ^ does that ring a bell
[15:18] <ijohnson>  zyga: have you seen this kind of spread failure ? https://pastebin.ubuntu.com/p/TQ3mkHcQW5/ I've seen this twice now in the past week
[15:18] <zyga> ijohnson: yes, it's fixed
[15:18] <zyga> ijohnson: https://github.com/snapcore/snapd/pull/8180 would help too
[15:18] <ijohnson> zyga: ack I'll merge master into my PR
[15:18] <ijohnson> thanks
[15:18] <mup> PR #8180: tests: rm -rf /tmp/snap.* in restore <Created by zyga> <https://github.com/snapcore/snapd/pull/8180>
[15:18] <zyga> ijohnson: if you review, it's +1 and green and simple
[15:19] <zyga> thank you
[15:19] <ijohnson> zyga: +1d and merged
[15:19] <mup> PR snapd#8180 closed: tests: rm -rf /tmp/snap.* in restore <Created by zyga> <Merged by anonymouse64> <https://github.com/snapcore/snapd/pull/8180>
[15:22] <zyga> little description is confusingly close to little encryption
[15:22] <pstolowski> zyga: no, not really
[15:22] <zyga> pstolowski: ok,
[15:41] <mvo> zyga: I see a lot of failures in the ppa build with "cannot set effective group to 0: Operation not permitted" - do you know anything about this?
[15:42] <mvo> zyga: https://launchpad.net/~snappy-dev/+archive/ubuntu/edge/+packages e.g. the s390x build for 19.10
[15:54] <mvo> zyga: fwiw, it happens when running the unit tests it seems
[15:57] <zyga> mvo: hmmm
[15:57] <zyga> mvo: probably the setgid change
[15:57] <zyga> mvo: I'll check
[15:57] <mvo> zyga: yeah, I was wondering if it might be
[15:57] <mvo> zyga: thanks for having a look!
[15:58] <mvo> zyga: the error does indicate it, maybe the priv dropping leaking into the unit tests?
[15:58] <mvo> zyga: anyway, strange that it suddently starts
[15:58] <zyga> mvo: perhaps that's the change I landed in the morning
[15:59] <mvo> zyga: that sounds plausible
[15:59] <mvo> zyga: that's around the time it started to fial
[15:59] <zyga> aaah
[15:59] <mvo> fail
[15:59] <zyga> I get it
[15:59]  * mvo hugs zyga
[15:59] <zyga> the test run as user
[15:59] <zyga> and this is new code path
[15:59] <zyga> I'll fix this today
[15:59] <zyga> sorry
[15:59] <zyga> it didn't come out in our tests because those ... run as root
[15:59] <zyga> OR
[16:00] <zyga> there's something else missing
[16:00] <zyga> I think we may have started to run tests as a user as well
[16:00] <mvo> zyga: yeah, it breaks the snap building so let's treat is as high priority
[16:01] <zyga> I'll finish dinner and fix it
[16:10] <mvo> zyga: if I can help please let me know, happy to investigate myself but I also feel it's probably silly because you know the details way better than me
[16:16] <mvo> ijohnson: a review of 8182 would be appreciated, I saw you self-requested a review, we would like to cordinate with the store to do this RSN
[16:18] <ackk> hi, does snapd need access to anything else than api.snapcraft.io:443 to for store access?
[16:19] <zyga> re
[16:19] <zyga> ackk: probably, for downloads
[16:19] <zyga> ackk: but dunno for sure
[16:21] <pedronis> ackk: yes, there is more, the store keeps an official list
[16:21] <mup> PR snapd#8182 closed: build: enable type: snapd <⛔ Blocked> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8182>
[16:22] <ackk> pedronis, thanks
[16:47] <ijohnson> pedronis: re: error wrapping in #8177, is it okay if I wrap errors in other places? or would you prefer that I just not wrap errors there for now and add a TODO:UC20: to wrap those errors later? because many of the seed errors are a bit unclear on an actual system to see why boot failed IMHO
[16:47] <mup> PR #8177: cmd/snap-bootstrap/initramfs-mounts: mount the snapd snap in run-mode too <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8177>
[16:47] <pedronis> ijohnson: I would prefer not to wrap for now
[16:47] <pedronis> because that code is in flux
[16:47] <ijohnson> pedronis: ack I will just remove the wrapping for now then
[16:48] <pedronis> ijohnson: there's some common code there, that probably should go in a single helper
[16:48] <ijohnson> pedronis: in this PR or later with  TODO:UC20:
[16:48] <ijohnson> ?
[16:48] <ijohnson> I know the TODO's are a bit piling up now
[16:48] <pedronis> ijohnson: later
[16:48] <ijohnson> ok
[16:49] <pedronis> ijohnson: I need to change a bit the signature of LoadMeta
[16:49] <mup> PR snapd#8142 closed: client: add support for "ResumeToken", "HeaderPeek" to download <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8142>
[16:49] <ijohnson> sure that would make it easier/quicker in run mode
[16:53] <ijohnson> pedronis: 8177 is ready again but still needs second review
[16:54]  * ijohnson updates 8147 now
[17:11] <pedronis> ijohnson: I reviewed both
[17:11] <ijohnson> thanks pedronis
[17:12] <ijohnson> I'll do a little bit more gardening of my open PR's then get back to opening the boot robustness tests PR
[17:12] <pedronis> thx
[17:40] <pedronis> ijohnson: I'm reading now grub TryKernel, I don't understand when it would return true and err at the same time
[17:41] <sarnold> maxiberta: hello :) what would you think about autoconnecting htop's necessary permissions? last week a user on #ubuntu had a seriously unhappy machine https://pastebin.com/raw/hJASew3R
[17:41] <ijohnson> pedronis: but we will eventually have multiple implementations of ExtractedRunKernelImageBootloader, i.e. uboot, lk, etc.
[17:41] <pedronis> ijohnson: they should all behave this way
[17:41] <pedronis> it's not a sane api
[17:41] <ijohnson> pedronis: but can't we be more safe ?
[17:42] <pedronis> ijohnson: yes, and no
[17:42] <pedronis> we can but that invariant broken is very confusing for readers
[17:42] <pedronis> so I'm not sure this is a long term sane state of things
[17:43] <ijohnson> would a comment help this to explain we're being extra cautious there ?
[17:43] <pedronis> ijohnson: maybe but honestly I wouldn't expect users of that api to need to be cautius
[17:43] <pedronis> it's really unusual for a go function to return an error and other values != zero
[17:43] <pedronis> it happens but it's rare
[17:43] <pedronis> and usually very documented
[17:44] <ijohnson> yes but when it happens it's a usually a bug so we're trying to be resilient here against a bug
[17:44] <ijohnson> if we aren't careful then we would panic and boot would fail
[17:44] <ijohnson> err rather
[17:44] <ijohnson> if we aren't careful and there is a bug, we would panic and boot would fail
[17:44] <ijohnson> I'm perfectly fine with your suggestion to change the order of the check
[17:45] <ijohnson> i.e. `if err == nil && tryKernelExists` seems fine to me
[18:17] <pedronis> ijohnson|lunch: related to this boot.ExtractedRunKernelImageBootloader probably merits more details docs about the methods behavior
[18:29] <mup> PR snapd#8184 opened: cmd/libsnap, tests: fix C unit tests failing as non-root <⚠ Critical> <Created by zyga> <https://github.com/snapcore/snapd/pull/8184>
[18:29]  * zyga pushes one small PR and EODs
[18:29] <zyga> please land that to unbreak PPA builds
[18:29]  * zyga is afk, tg in case of urgency
[18:30] <mup> PR snapd#8171 closed: cmd/snap-failure/snapd: rm snapd.socket, reset snapd.socket failed status <Simple 😃> <Created by anonymouse64> <Merged by anonymouse64> <https://github.com/snapcore/snapd/pull/8171>
[18:37] <mup> PR pc-amd64-gadget#37 opened: Drop duplicate grub.cfg-* <Created by xnox> <https://github.com/snapcore/pc-amd64-gadget/pull/37>
[18:47] <mup> PR snapd#8176 closed: tests: adding amazon linux to google backend <Created by sergiocazzolato> <Merged by cmatsuoka> <https://github.com/snapcore/snapd/pull/8176>
[19:33] <mup> PR snapd#8185 opened: tests: add uc20 kernel snap upgrade managers test, fix bootloadertest bugs <Test Robustness> <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8185>
[20:34] <ijohnson> hey mwhudson do you still maintain console-conf for ubuntu core?
[20:37] <mwhudson> ijohnson: "maintain"
[20:37] <ijohnson> :-)
[20:37] <ijohnson> quick question
[20:38] <ijohnson> this PR: https://github.com/CanonicalLtd/subiquity/pull/389 was merged to the core/bionic branch, but I don't see any other core/... branches, so if we need to do the same thing/similar thing for uc20, where would we propose that change to?
[20:38] <mup> PR CanonicalLtd/subiquity#389: services: run console-conf after core18.start-snapd.service <Created by mvo5> <Merged by sil2100> <https://github.com/CanonicalLtd/subiquity/pull/389>
[20:38] <ijohnson> or better yet, could we just land the equivalent change for uc20 to subiquity master and have it show up in focal eventually ?
[21:02] <ijohnson> mwhudson: ^
[21:03] <mwhudson> ijohnson: hmmm the version of console-conf in focal may well be a bit broken
[21:03] <mwhudson> ijohnson: in general xnox knows a lot more about uc20 things
[21:03] <ijohnson> mwhudson: ok, just curious about the branching situation with console-conf
[21:04] <mwhudson> ijohnson: magic 8 ball says "vague"
[21:04] <mwhudson> ijohnson: basically there isn't one i guess
[21:04] <ijohnson> mwhudson: haha, fair enough
[21:06] <ijohnson> mwhudson: also thoughts on having the console-conf service always specify `After=... <insert-ubuntu-core-specific-things ad infinitum>` in the systemd unit? I don't think that breaks anything because if the uc things don't exist if it's just After then it's just ignored IIRC
[21:06] <ijohnson> i.e. put that in master so it runs on classic and core
[21:06] <xnox> ijohnson:  i've dropped version number from the systemd job in core20, because it is silly.
[21:07] <mwhudson> ijohnson: sounds vaguely sane
[21:07] <mwhudson> ah hi xnox
[21:07] <xnox> ijohnson:  and yes, something similar needs to land to master + dput to focal
[21:07] <xnox> ijohnson:  ie. After=core.start-snapd.service After=core18.start-snapd.service to cover all basis
[21:07] <ijohnson> xnox: so should I file a PR to subiquity master with those changes then?
[21:07] <xnox> ijohnson:  we are overdue a consoleconf upload.
[21:07] <xnox> ijohnson:  yes please
[21:07] <ijohnson> ok, sounds good
[21:08] <xnox> ijohnson:  i can pre-test them against hand-built core20 snap too with a UC20 image
[21:08] <ijohnson> xnox: also not sure if you saw, but maciej opened this: https://github.com/snapcore/core20/pull/23, but I think the better way to do that is to just put it all in console-conf upstream
[21:08] <mup> PR core20#23: hooks: ensure console-conf@ is started after snapd.recovery-chooser-trigger.service <Created by bboozzoo> <https://github.com/snapcore/core20/pull/23>
[21:09] <xnox> ijohnson:  i think so
[21:10] <ijohnson> xnox: ok, I think I will file a PR with the core18 specific hacks first then maybe maciej re-opens his core20 PR against subiquity upstream then
[21:15] <ijohnson> thanks xnox and mwhudson
[21:26] <hellsworth> part
[21:36] <mup> PR snapd#8147 closed: cmd/snap-bootstrap: verify kernel snap is in modeenv before mounting it <UC20> <Created by anonymouse64> <Merged by anonymouse64> <https://github.com/snapcore/snapd/pull/8147>
[21:46] <pedronis> ijohnson: after further thinking, I still we should follow the pattern we use in other cases (like ErrNoState) and have a special error (ErrNoTryKernelRef ?) instead of the bool for TryKernel
[21:46] <ijohnson> pedronis: ok, that's easy enough to do and I think I can wrap that up succinctly in my next PR
[21:47] <pedronis> s/still/think/
[21:48] <pedronis> ijohnson: thx
[22:16] <zyga> ensure prune thing is still failing
[22:27] <zyga> https://github.com/snapcore/snapd/pull/8186 as promised
[22:27] <mup> PR #8186: run-checks: SKIP_GMFMT really skips formatting checks <Created by zyga> <https://github.com/snapcore/snapd/pull/8186>
[22:27] <zyga> and back to bed
[22:28] <mup> PR snapd#8186 opened: run-checks: SKIP_GMFMT really skips formatting checks <Created by zyga> <https://github.com/snapcore/snapd/pull/8186>
[22:55] <mup> PR snapd#8187 opened: boot: misc UC20 changes <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/8187>
[23:17] <mup> PR core18#148 opened: hooks: delete console-conf hack <Created by anonymouse64> <https://github.com/snapcore/core18/pull/148>
[23:20] <mup> PR core20#24 opened: hooks: delete console-conf hack <Created by anonymouse64> <https://github.com/snapcore/core20/pull/24>