[06:18] mvo: good morning [06:18] fun test failure [06:18] no void directory on core18 https://www.irccloud.com/pastebin/hjyPHGW5/ [06:18] zyga: uh, fun indeed [06:18] zyga: and easy to fix [06:18] zyga: thanks for finding this! [06:19] mvo: so [06:19] mvo: I have a problem [06:19] mvo: how do we fix this? [06:19] by hacking core18/ [06:19] mvo: can we create a package that is designed for core18-like systems? [06:19] mvo: that has the right startup core and mounting points [06:20] mvo: I would rather not have this conversation in core20 [06:20] zyga: I would add it to the build tree of UC18, this will become the build tree of UC20 [06:20] mvo: if we make the right package it will be simply inistalled in core18 systems and that will be enough [06:21] mvo: add it how? I'm arguing that it should be a standalone package built out of snapd source package, that is then installed in core18 system [06:22] zyga: yeah, it could be snap-skeleton-dirs or something but I'm not sure this is more robust, we might also forgot add things there instead of in the main snapd package just like we could forgot adding things to the UC18/uc20 snaps. or do you have something different in midn? [06:22] mind [06:22] mvo: we can very well forget but if we fix it the fix goes to core16, core18 and core20 [06:22] mvo: in one patch [06:22] mvo: and it would be in the snapd repo, not somewhere else [06:24] zyga: how would that look like in pratcise? what would the package be called and what would the dependencies be? [06:24] mvo: it would not depend on anything but it would conflict with snapd.deb [06:24] mvo: the name is irrelevant, maybe snapd-bootstrap-and-skeleton? [06:25] mvo: it would ship /var/lib/snapd structure, the /bin/snap symlink, the /usr/lib/snapd mount point and perhaps your bootstrap script [06:25] mvo: (anything appropriate) [06:26] zyga: it would be separate from the main package snapd which (afaics) has the same risk of getting our of getting out of sync. maybe we pull something from snapd git in to create the dirs/perms and use exactly this in the main snapd package build(s)? [06:28] mvo: hmm? [06:29] mvo: the snapd source package would have snapd.deb binary package and snapd-skeleton.deb binary (arch all) package [06:29] mvo: is that sufficient or do I misundertand your worry? [06:29] *misunderstand [06:29] * zyga preps breakfast for kids, responses may be slower [06:31] zyga: snapd/snapd-skeleton may get out of sync if we are not careful [06:32] zyga: because we add a new dir (or something) to snapd but forget to add it to snapd-skeleton [06:32] zyga: so we need something that ensures that this can't happen, one way would be to use a makefile/script/whatever as part of the snapd package build and use exactly the same script as part of the uc18/uc20 build [06:35] zyga: alternatively we could create some of these missing dirs ourselfs maybe? /var/lib/snapd is fully writable [06:35] zyga: (create them if missing) [06:35] zyga: I actually like this as it means its easier if people try to run snapd from a snapd git checkout etc (slightly easier I admit :) [06:38] morning [06:40] wow, some prs are actually green [06:41] hmmm [06:41] I'm not sure [06:41] zyga: hey [06:41] this shoud run before snap [06:42] before snapd [06:44] hey mborzecki [06:44] mvo: hey [07:00] zyga: idk if you noticed, but there's quite a few directories missing under /var/lib/snapd between core and core18 [07:06] mborzecki: yeah, zyga and I discussed ways to fix this, zyga had some ideas how to make sure we have the layout inside our tree somehow so that its used in uc18/uc20 and we are exploring which one is the best [07:14] funny that we noticed just now [07:14] in most cases those are created on demand [07:15] yeah, I think on demand-creation is a nice way out of this [07:15] and it has the advantage of making it easy to run from e.g. checkouts [07:16] but *shrug* there are downsides too [07:16] mvo: I don't disagree in principle but not everything can be created on demand [07:16] iirc void has some special permissions too, so this would have to be accounted for when we create it on the fly [07:19] mborzecki: can you do a 2nd review on https://github.com/snapcore/snapd/pull/6645 please? [07:19] haha looking at it just now :) [07:19] thank you :) [07:19] I want the rush of adrenalin from landing a PR [07:20] still, suprising usb.ids doesn't list oneplus [07:20] maybe it's a fake request? [07:20] zyga: no, i found some pages describing the process of getting oneplus working on linux and they list the same vendor id [07:21] aha, that's a good find [07:21] * zyga reviews https://github.com/snapcore/snapd/pull/6634 [07:23] mborzecki: silly question - what is needed for 6606 ? a second review or is there more ? [07:24] mvo: 2nd review [07:26] zyga: added some links under 6645 in case you're curious, there's even a patch from lineageos [07:26] mborzecki: looking [07:27] mborzecki: thanks, maybe pstolowski can do a second review on 6606, it looks pretty much ready to me [07:31] mborzecki: added my .0002¢ I think its fine but would feel better if pstolowski could do a final pass on 6606 [07:31] mvo: thanks! [07:47] brb, coffee [07:48] https://github.com/snapcore/snapd/pull/6597 <- 2nd review needed in case someone wants to look [07:48] just a quick service announcement - we have an updated systemd in the edge core with a race fix, if anyone notices anything strange in the tests related to systemd please shout [07:48] zyga: I can look [07:50] mvo: thank you [07:50] mvo: alternatively look at https://github.com/snapcore/snapd/pull/6621/files [07:50] it's much smaller [07:50] and more on the critical path [07:50] mvo: ack, about systemd, thank you for that! [07:50] zyga: ha! thanks, I take 6621 I think :) [07:51] zyga: the other one is *big* it seem [07:51] zyga: hopefully both, but for that I need more tea [07:51] yeah, this one is very small, just tests and a few lines of new code [07:59] pedronis: do you want to review 6630 again (snap warnings yamlish) or can I merge? john addressed your review points [08:03] morning [08:05] pstolowski: good morning! [08:05] pstolowski: heya [08:35] * dot-tobias says hi [08:36] mvo: hi, please merge [08:38] pedronis: ta [09:09] ppisati, morning! [09:09] ppisati, which is the apparmor status for 4.9 kernels? Is there any patch needed to run ubuntu core/snapd with it? [09:11] abeato: not that i'm aware [09:12] mvo I will merge 6605 after adding the extra comment [09:12] abeato: *none, that i'm aware of [09:12] mvo: just wrapping up a small new branch you asked for [09:12] mvo: thank you for the review! [09:14] ppisati, ok, thanks [09:16] pedronis: i've debugged the ssh.disabled issue [09:18] zyga: ta [09:23] pedronis: we don't support dotted notation at the nested level of the defaults definition, we expand dotted path at the root level only, e.g. that would work (works in a unit test i made): [09:23] defaults: [09:23] test-snap-ididididididididididid: [09:23] service.ssh.disabled: true [09:24] pstolowski: it sounds like we should error or do something though ? [09:24] we don't support dots in name in principle? [09:24] pstolowski: where is ther relevant code? [09:24] pedronis: and consequently, with the config that Sergio created you end up with unexpanded key in the config: map[string]interface {}{"ssh.disabled":true} [09:25] yea, that looks wrong [09:26] pedronis: also, this works fine in unit test: [09:26] defaults: [09:26] test-snap-ididididididididididid: [09:26] service: [09:26] ssh: [09:26] disabled: true [09:27] that I know [09:27] pedronis: we do support dots in name, but it needs to start at the root level. you cannot define map and then have dotted path deep inside [09:27] pstolowski: I understand, you said it already [09:28] and yes we should error out i suppose [09:28] yes, "ssh.disabled" should't be a valid key [09:28] we should not get there [09:28] let me see if i can find where it happens exactly in the code [09:34] mvo: https://github.com/snapcore/snapd/pull/6650 :-) [09:36] zyga: shouldn't we try to land 6583 first? [09:39] pedronis: i think the important bit is the "for key, value := range patch" loop in func (h *configureHandler) Before(). we call tr.Set there for top-level patch entries, but inside transaction's Set we only ParseKey once (for that top level) [09:40] pstolowski: so we are missing a check for dots in key (where unexpected) somewhere? [09:42] pedronis: I will be working on that today [09:42] zyga: you have 17 open PRs , that's not a feature :) [09:43] pedronis: I know but many of them are very close to landing [09:43] zyga: then please try to land before opening new ones [09:43] or I will start to close new ones [09:44] pedronis: with regards to landing: we need a decision on what to do here https://github.com/snapcore/snapd/pull/6642#discussion_r268780713 [09:46] zyga: I agree with jdstrand if there's a way to keep it working without security issues it's better, we have been bitten already by taking stuff away from people [09:46] pedronis: so moving to void if we cannot stay in the current directory due to permission checks? [09:47] yes [09:47] ok, I'll make the change [09:47] thanks, can you add a comment saying that in the PR, it will be easier for jamie to see [09:48] done [09:48] Thanks [09:53] pedronis: yes, you could say that. i think we could support that in PatchConfig, although the fix is not immediately obvious [09:54] pstolowski: anyway is not quite a priority right now, but we should plan to do something at some point [09:54] pedronis: ack [09:54] pstolowski: I'm also not sure if it's better to error or to make it work [09:54] (but splitting on dot9 [09:54] (by splitting on dots) [09:59] mvo: do we use the standup ? [09:59] pedronis: https://github.com/snapcore/snapd/pull/6649 is a low hanging fruit [10:00] ok [10:03] degville: do you know where the meeting is? [10:03] pedronis: nope. standup location? mvo is running a little late. === chihchun_afk is now known as chihchun [10:24] https://forum.snapcraft.io/t/ubuntu-18-04-fresh-install-default-snaps-are-updated-but-runtime-isnt/10579 this is interesting in a bad way [10:25] damn, Failed for "golang.org/x/crypto/cast5" (failed to clone repo): exit status 128 [10:27] shal we land https://github.com/snapcore/snapd/pull/6623 ? [10:29] mborzecki: seems so [10:29] it's sufficiently ripe :) [10:30] it has a jdstrand +1 and other reviews [10:30] at some point it sounded it needed to touch snap-confine, but is not the case [10:31] landed [10:34] eh [10:35] my tests also failed [10:35] 50 minutes of wasted time [10:35] fatal: unable to access 'https://go.googlesource.com/crypto/': Failed to connect to 2607:f8b0:400c:c06::52: Network is unreachable [10:35] this is why we need spread --fail-fast [10:35] this just sucks [11:11] * Chipaca hates all of the interfaces-many- spread tests [11:13] zyga: +1 [11:16] mup_: So.. what's up with you? [11:24] mup: How're you doing now? [11:24] niemeyer: I apologize, but I'm pretty strict about only responding to known commands. [11:51] 2019-03-26 10:33:46 Cannot allocate google:debian-sid-64: cannot perform Google request: Get https://www.googleapis.com/compute/v1/projects/computeengine/zones: oauth2: cannot fetch token: Post https://accounts.google.com/o/oauth2/token: dial tcp 74.125.126.84:443: i/o timeout [11:51] hmmm [11:52] gcp broke? [12:05] mborzecki: nevah! [12:05] also, spread takes too long, so: while true; do printf "\u$(printf "%x" $((0x2571 + RANDOM%2)))"; done [12:07] heh, and i got into a little mess with the 'gadget' package, some code is on review right now, and merging the branches breaks all of it :P [12:08] aand another job hit the time limit [12:09] it's quite intersting, so prepare fails on some nodes because golang.org is flaky and times out, then the nodes that actually made it execute all the tests and the job hits travis time limits [12:14] maybe it's because of https://twitter.com/MehreenKhn/status/1110509604176384000 [12:19] * zyga goes to the doctor [12:19] ttyl [12:22] PR snapcraft#2515 closed: build providers: support for provider setup [12:24] mvo: question about the snapd snap when you have a bit [12:25] pstolowski, https://api.travis-ci.org/v3/job/511064184/log.txt [12:25] mvo: the question is whether building with snapcraft from the snap (instead of from the deb), and adding -b -Zgzip to the dpkg-buildpackage in the plugin, make sense to you [12:25] this is the log for the ssh.disable: true [12:25] mvo: the first is because otherwise it leaves behind a bunch of packages that break the invariant [12:26] mvo: the second is because 300s → 90s [12:26] mvo: (I could just change the first, the second is merely opportunistic) [12:30] cachio: i know, i debugged it in the morning and discussed with pedronis; we don't currently support dotted paths nested deep inside defaults; it would work only from top-level, e.g. service.ssh.disabled: true. we should at the very least provide a meaningful error message or make it smarter [12:32] Chipaca: +1 on the second, I'm not fully getting the first one, sorry (probably a bit slow today) [12:34] cachio: btw i think it found the cause of interfaces-daemon-notify failure in my nested vm PR; i think it fails because of "plan: n1-standard-2" (?!); i run this single test a couple of times today with and without this change (also with just master) and it seems to be it. if that plan is enabled, then "journalctl -u snap.test-snapd-daemon-notify.notify.service" that we execute at the end of the test doesn't return any [12:34] output for that service. i debugged this manually and when it fails, system log has the expected "Permission denied" errors in it, it's just not reported by journalctl for whatever reasons. systemd/journalctl versions appear to be the same both with and without that plan. [12:35] i've juse reverted this plan from https://github.com/snapcore/snapd/pull/6491 and it passed [12:35] PR #6491: interfaces: hotplug nested vm test, updated serial-port interface [12:36] pedronis: ^ is green [12:36] cachio: I'll comment in the PR when I have a bit of time today [12:36] (about ssh.disable) [12:36] mvo: only builds the binary deb, uses gzip to compress it (instead of also building source and using xz) [12:36] mvo: oh wait [12:37] mvo: the first one, changing deb for snap, because 'apt install snapcraft" pulls in e.g. python3-crypto, which autoremove can't [12:37] can't autoremove i mean [12:37] because things recommend it or maybe suggest it i guess [12:37] pstolowski, which is the PR? [12:37] Chipaca: +1 [12:37] pedronis, thanks [12:38] Chipaca: oh, so "snap install snapcraft"? [12:38] mvo: yar [12:38] cachio: #6491 [12:38] Chipaca: this is the change? if so, yes, lets do it [12:38] ok [12:38] PR #6491: interfaces: hotplug nested vm test, updated serial-port interface [12:38] Chipaca: thank you! I think htis is just old code from the time when we had no snap (ironically) [12:38] mvo: it'll be part of the invariant pr unless you'd rather it went in alone [12:38] Chipaca: and nice idea on the dpkg-buildpackage one [12:38] cachio: can you think on any reason why this more performant plan makes that test fail? any difference in the images? [12:38] Chipaca: fine for me [12:38] k [12:38] Chipaca: its not a big change :) [12:38] Chipaca: thank you! [12:40] pstolowski, did you change the plan? [12:40] I didn't see that [12:58] cachio: yes i reverted that change innym PR to make it pass [12:58] cachio: see last commit [13:00] cachio: just apply it on master and it should fail the interfaces-daemon-notify test [13:00] PR snapd#6651 opened: tests: all the systems for google backend with 6 workers [13:01] * zyga off to the doctor [13:02] * Chipaca off to have lunch [13:02] zyga: lunch > doctor [13:02] * Chipaca wins [13:03] pstolowski, mmmm, ok [13:03] pstolowski, we can't change the plan [13:05] What is the status of spread / cloud? [13:05] Did we run out of money? === ricab is now known as ricab|lunch [13:08] cachio: hmm hey not? How Is It affecting 14.04? [13:12] PR snapd#6606 closed: selinux, systemd: support mount contexts for snap images [13:13] zyga: do you want to take a look at https://github.com/snapcore/snapd/pull/6485 ? [13:13] PR #6485: interfaces/seccomp: regenerate changed profiles only [13:29] jdstrand: Is there a way to force an NTP resync using the timezone-control DBus interface? https://www.freedesktop.org/wiki/Software/systemd/timedated/ indicates that SetNTP() method restarts systemd-timedated, but timedatectl still reports “NTP in sync: no”. Use case: Our device is offline until configured via an access point, so the device is booting without network. System does not seem to pick up the available network connection in the [13:29] minutes after its established. [13:32] back from doctor [13:32] bikes are amazing :) [13:32] mborzecki: sure, looking [13:56] * Chipaca ⇝ cuppa [14:00] PR core#38 closed: Add another pi-config option [14:00] PR core#83 closed: move most of the ubuntu-core config deb into the snap snap build [14:01] PR core#38 opened: Add another pi-config option [14:01] PR core#83 opened: move most of the ubuntu-core config deb into the snap snap build [14:05] dot-tobias: I'm not an expert on timedated. If I were to guess, I'd guess the clock is maybe too far off or that your device is unable to communicate with the configured server. you might also check your logs for policy violations, but it sounds like timedatectl is working ok and it is something else [14:07] jdstrand: Yes, the clock was off > 12h because that's the time when the image was created. Didn't know that in order to sync the clock has to be near the actual date. Will look into workarounds how to get in re-sync. [14:07] jdstrand: Thanks! [14:08] PR snapd#6329 closed: cmd/snap-confine, packaging: support SELinux [14:11] dot-tobias: cool. ogra may have some thoughts on that as well [14:14] hey jdstrand :) [14:14] jdstrand: I'm glad you noticed "goto the void" :-) [14:14] initially it was an else statement but then it was too good to pass :-) [14:17] zyga: hey, yeah :) [14:17] :-) [14:17] jdstrand: we also managed to uncover that core18 is missing some directories in that test, it's better late than never [14:18] zyga: I saw that, wild :) [14:21] jdstrand: I implemented the next branch but I will defer until my set of open PRs shrinks sufficiently [14:21] jdstrand: btw, not sure if you saw https://github.com/snapcore/snapd/pull/6649 [14:21] PR #6649: snap: reject layouts to /lib/{firmware,modules} === ricab|lunch is now known as ricab [14:49] PR snapd#6652 opened: sanity: use proper SELinux mount context when sanity checking squashfs mounts [14:50] * zyga -> tea / coffee [14:51] super simple selinux PR ^^ [15:06] zyga, mvo, do you gys know if anyone actually tested pi core18 images with wlan ? seems i dont get proper DNS resolution to finish console-conf here (network is up and i can ping the board but user creation times out with DNS issues finding the login.u.c api stuff) [15:07] this is with the stable released image... [15:08] ogra: I'm not sure we did, sorry. is this the dragonboard or pi3? [15:08] jdstrand, in case dot-tobias comes back while i'm not around ... https://github.com/ogra1/ntpcontrol [15:08] ogra: or both? [15:08] mvo, pi image on a pi 3 A+ [15:09] core 16 works fine [15:09] (i resorted to that in the end) [15:12] ogra: we need to look into this, in a meeting now unfortunately [15:13] mvo, all fine, just wanted to check if it was tested before [15:13] i also resoted to core16 for the project i needed the board for ... but i have another A+ for any tests if required [15:18] ogra: added to test/debug this to our trello, thanks for reporting [15:22] PR snapd#6649 closed: snap: reject layouts to /lib/{firmware,modules} === cwayne_ is now known as cwayne [15:34] * cachio lunch [15:35] ogra: sorry if this message is doubled, my ircs been weird but we do test wlan on Pi's, but using nm [15:35] well, that wont help much if NM isnt preinstalled :P [15:35] (also ... does console-conf actually support NM ??) === chihchun is now known as chihchun_afk [16:13] cachio: spread#75 reviewed again [16:13] PR spread#75: Make spread tests for spread project run on google backend [16:17] PR snapcraft#2516 opened: readme: add snap store badge [16:19] s pedronis renamed https://github.com/snapcore/snapd/pull/6621 [16:19] PR #6621: overlord/snapstate: track time of postponed refreshes [16:20] pedronis: to "inhibit" [16:20] it is green so I'm very tempted to merge it :) [16:20] everyone: I need 2nd review for https://github.com/snapcore/snapd/pull/6597 [16:20] PR #6597: cmd/snap-update-ns: refactor of profile application (1/N) [16:20] zyga: it doesn't have 2 +1 [16:20] pedronis: I know :) [16:20] that's why I'm saying [16:20] if that was your only reservation I'd love to get a 2nd one [16:21] zyga: I asked Chipaca to review it [16:22] great, I'll wait for the review then [16:22] Chipaca: could you look at #6621 [16:22] PR #6621: overlord/snapstate: track time of postponed refreshes [16:22] when you have time [16:22] yes [16:40] niemeyer, thanks [16:41] fyi to anybody using the go remote part, I just updated it in the wiki, so if anything breaks ping me [16:41] thank you ijohnson [16:42] huh [16:43] Chipaca: hmm? [16:43] zyga: what permissions should snap-confine have? [16:43] Chipaca: as in UNIX? [16:43] zyga: ye [16:44] for now u+rwXs,g+rXs,o:+rX [16:44] though I will drop the g+S part [16:44] why? [16:45] zyga: because it starts at 06755, as you say [16:45] zyga: but snap-confine-from-core sets it to 04755 [16:45] wait, what? [16:45] why? [16:45] * zyga hugs Chipaca for fixing the tests [16:46] restore: | [16:46] echo "Restoring host snap-confine" [16:46] chmod 4755 /usr/lib/snapd/snap-confine [16:46] zyga: ^ [16:46] la la la [16:47] man [16:47] ... [16:47] what just happened [16:47] I spent 10 minutes debugging [16:47] why snapd doesn't build [16:47] on ... macos [16:47] zyga: snap does :-) but not snapd :-( [16:47] wrong terminal tab on a common shared source [16:47] lol [16:47] * Chipaca is still happy that snap does :-) [16:47] I aborted a rebase in progress because I got fed up with this and wanted to think why everything is broken [16:48] zyga: what was it telling you wasn't implemented? [16:48] /Users/zyga/go/src/github.com/snapcore/snapd/interfaces/mount/backend.go:97:84: undefined: osutil.MountEntry [16:49] I totally ignored the path [16:50] * Chipaca nods [16:51] fwiw there was this crazy linux distro that didn't use the usual / layout, they could have had /User too [16:51] it's just noise [16:51] noise][1: no offense [16:51] :-) [16:51] mborzecki: yeah, I used it once [16:51] it had /Users and /Applications like on macos [16:53] mborzecki: I did a pass over the gadget validation PR [16:54] pedronis: thanks, sorry for it being so long [16:55] mborzecki: we'll need another layer of validation that looks at actual content for sizes as well? [16:55] pedronis: yes, that will need to access actual snap data [16:56] ok [17:02] zyga: are you waiting for review from me on anything but #6642 atm ? [17:02] PR #6642: cmd/snap-confine: prevent cwd restore permission bypass [17:02] checking [17:03] we need to look at https://github.com/snapcore/snapd/pull/6583 [17:03] PR #6583: cmd/snap-confine: move ubuntu-core fallback checks [17:03] but perhaps tomorrow morning [17:03] I'm not entirely happy with the code I saw the there the last time I looked [17:03] we can chat tomorrow [17:03] today I want to finish mending existing branches that I know what needs to happen wiht [17:03] ok [17:04] I'll try to still look at 6642 tonight [17:05] thank you! [17:05]