=== bdx_ is now known as bdx === mup_ is now known as mup === JanC_ is now known as JanC [02:07] PR snapcraft#1288 opened: store: collaboration UI for snaps [03:31] PR snapcraft#1289 opened: beta [05:43] PR snapcraft#1289 closed: beta [05:43] PR snapcraft#1290 opened: tests: refactor tests with build and stage packages [06:42] Pharaoh_Atem: hey, I'm working on a new snapd release currently and you mentioned you wanted some tarballs with vendor removed, do I remember this correctly? === chihchun is now known as chihchun_afk [07:07] mvo: hi, is the release draft missing mentioning release.schedule ? [07:07] sorry refresh.schedule === chihchun_afk is now known as chihchun [07:17] pedronis: yes indeed, I need to add that [07:23] o/ [07:23] mvo: hey, how are you feeling [07:23] mvo: ready for the sprint? [07:24] zyga: yes [07:24] fgimenez: new beta snaps available for an early smoke test of 2.25 [07:25] mvo: great thanks! on it, will let you know how it goes [07:25] fgimenez: and ubuntu-core as well [07:26] mvo: ack, we have spread tests for it too, but yet no spread-cron branches up, i'll trigger them manually [07:30] PR snapd#3246 closed: cmd/snap-confine: remove obsolete debug message [07:41] PR snapd#3249 opened: releasing package snapd version 2.25 [07:41] pstolowski: can you have a 2nd look on https://github.com/snapcore/snapd/pull/3225 - I think I addressed your questions [07:41] PR snapd#3225: cmd/snap-update-ns: add actual implementation [07:42] pstolowski: if not just comment and I'll do my best [07:42] I could also use a 2nd review on (trivial) https://github.com/snapcore/snapd/pull/3234 [07:42] PR snapd#3234: overlord/snapstate/backend,interfaces/mount: move ns management code [07:42] zyga, sure, will do in a moment [07:53] * zyga -> breakfast [07:53] day of code reviews [07:58] PR snapd#3250 opened: store: fix TestDownloadSyncFails for go1.8 [08:01] ogra_, how is the initrd for a kernel snap built? where does snapcraft pull the sources from? [08:19] PR snapd#3251 opened: packaging: cleanup how built-using is generated [08:22] re [08:27] mvo: i'm getting an error with econnreset on all the platforms http://paste.ubuntu.com/24471739/ the timeout makes the execution stop, i'll retrigger all of them removing this test [08:30] fgimenez: uh, strange [08:30] fgimenez: strange because this was/is working ok in qemu and linode, wonder whats the difference with external [08:34] mvo: let me run it isolated with -debug on amd64 to seee what's going on [08:36] pstolowski: some comments on https://github.com/snapcore/snapd/pull/3217#pullrequestreview-35302363 [08:36] PR snapd#3217: snap: support for snap tasks --last= [08:36] thanks [08:39] Chipaca: hey, I see lots of completion failures on Debian: https://github.com/snapcore/snapd/pull/3150 [08:39] PR snapd#3150: In-snap bash tab completion [08:44] pedronis: if you could have a look at 3241 that would be great. it misses tests but I would love to get a early sanity check of the idea. we may need it soon to help CE [08:44] mvo: sorry for my naive question but in https://github.com/snapcore/snapd/pull/3241#discussion_r113743668 does that mean the process could work offline? [08:44] PR snapd#3241: snap: make `snap prepare-image` read .assert files for local snaps [08:44] zyga_: not quite [08:44] zyga_: it could with some more work [08:45] aha, so this would just prime the rest of the code to do the right thing [08:45] ? [08:45] zyga_: right now the way it works is that snap prepare-image gets some data from the store and then later fetches the assertions from the store [08:45] zyga_: my branch just makes the code fish out most of the store info from the assertion. however it will still later contact the store [08:46] zyga_: exactly [08:46] mvo: understood, thanks! [08:46] zyga_: this is a bit of a quick fix for CE, I think we can do it fully offline with a bit more work [08:46] I just never looked at that code [08:46] mvo: one thing that I think would be interesting is a way to reproduce an image build given a manifest (with offline snaps and assertions) [08:47] mvo: but no ad-hoc design on IRC :) [08:47] zyga_: yeah, I think with full offline capability this would be almost there [08:48] zyga_: we disussed that when discussing improving prepare-image (I have an email thread with Gustavo that I need to port to the forum), he wasn't too keen on having a manifest on top of model assertions [08:48] though [08:48] pedronis: manifest could only capture a snapshot of what the model describes [08:48] pedronis: not as a way to create other things [08:49] pedronis: I think that it will be a common request from CE, to be able to reproduce image builds exactly, at specific revisions [08:49] zyga_: well a manifest is not enough (given how the store works atm and rules), you really need to cache what prepare-image used [08:51] pedronis: yes, I was thinkin about a full cache [08:51] pedronis: with all the snaps and assertions [08:51] ok [08:52] it's all a bit odd because if you don't change anything then it's pointless [08:52] you get the image you already got once === chihchun is now known as chihchun_afk [09:11] Son_Goku: hey, how are you doing === chihchun_afk is now known as chihchun [09:19] pstolowski: https://github.com/snapcore/snapd/pull/3119 has two reviews [09:19] PR snapd#3119: interfaces: API additions for interface hooks [09:19] pstolowski: can we merge it? [09:19] zyga, no, I'd like niemeyer's +1 [09:19] PR snapd#3252 opened: tests,cmd/snap-confine: port older snapd-discard-ns tests [09:19] pstolowski: ok [09:20] ooh, el reg article mentioning snaps [09:20] Chipaca: can you share the URL please [09:20] * Chipaca clutches it close to his chest [09:20] NO [09:20] aww :-( [09:20] zyga— http://www.theregister.co.uk/2017/04/28/snap_flatpacks_the_future_of_desktop_linux/ [09:22] shame it sounds like they've only actually tried flatpaks [09:22] ¯\_(ツ)_/¯ [09:26] "the kernel is not going to be a flatpak any time soon" [09:26] yeah, because it's a snap already [09:26] abeato, during the core snap build .... [09:26] mvo: about the error on the boards using the external backend it turns out that now the files under /home/gopath/src/github.com/snapcore/snapd belong to the user created with console-conf and not to the "test" user, not sure why, a week ago this wasn't happen, i'll continue digging [09:28] abeato, https://github.com/snapcore/core/blob/master/live-build/hooks/25-create-generic-initrd.chroot [09:29] ogra_, are not initramfs images built with the kernel? [09:29] I mean, when building the kernel snaps [09:29] abeato, nope, it is generic ... [09:30] abeato, long term we want two initrds, a generic one with all scripts shipped in core and a specific one with only /lib/modules|firmware that the kernel ships [09:30] fgimenez: thank you [09:31] abeato, currently the snapcraft kernel plugin still adds the modules to the generic one at build time though, downloading the core snap and pulling it from there [09:31] ogra_, so how does it work to add those kernel specific files in the initrd? I see there is a "kernel-initrd-modules" list in kernel snapcraftyaml files [09:32] abeato, depends, do you need something specific the official kernels work slightly different (not using the kernel plugin actually) [09:33] zyga, the kernel as a snap is pretty useless from a multi-distro point of view [09:33] if it is for your build, the kernel plugin simply re-packs it ... for official kernels we include whats needed in the /etc/initramfs-tools/modules file at build time [09:33] Son_Goku: I didn't mean the corss distro POV, snapd itself is cross distro as it aims to liberalize what a distribution is [09:33] abeato, all managed via the initramfs-tools-ubuntu-core package from the PPA [09:34] Son_Goku: and who knows, maybe we can teach snapd to install kernels on classic :) [09:34] ogra_, no, just wondering how things are built, thanks for the info [09:34] Son_Goku: it's just software [09:34] * Son_Goku wonders what problem is being solved at that point [09:34] ogra_, ok so we can add things for android-boot in that package as you suggested [09:35] abeato, the official kernels actually use the linux debs from the archive, install them in a chroot and copy the deb contents into the snap ... [09:35] Son_Goku: common packaging environment, I could boot fedora kernels and you could boot ubuntu kernels without any repackaging :) [09:35] abeato, right, just another script [09:35] zyga: if the Linux world wanted that, we would have had a common packaging environment already [09:35] abeato, and a simple switch on the kernel cmdline to pick that instead of the default for booting recovery [09:35] God knows people tried [09:35] Son_Goku: that's totally untrue ;) [09:36] ogra_, why this difference between official kernels and what the kernel pluin does? Which approach should I take to build for a new board? [09:36] Son_Goku: the linux world cannot want anything as there are always polarized opinions [09:36] at least we're down to ~4 systems instead of ~20 [09:36] Son_Goku: I'd say that technically it wasn't done correctly before [09:36] Son_Goku: but it's not about desire at all [09:36] abeato, for secureboot we need the kernel binary signed by the archive key, using the existing deb is the easiest way for that [09:36] zyga, addressed your comments to #3217 [09:37] Son_Goku: just add hrw and any other person into the "linux world" and they will disagree ;) [09:37] pstolowski: thanks! looking [09:37] zyga: that's unfair [09:37] hrw complains about everything :) [09:37] Son_Goku: see [09:37] ogra_, got it, so if secureboot is not involved, it would be better to use kernel plugin, right? [09:37] Son_Goku: "linux world" is not a measure of anything [09:38] zyga: If I *really* wanted to, i could boot Ubuntu kernels on Fedora [09:38] Son_Goku: but the effort is non-trivial [09:38] yes [09:38] but snaps don't make that better [09:38] Son_Goku: if you had any kenrel in a snap you could just do it casually [09:38] they make the process more opaque [09:38] zyga, looking at your 3225, sorry it took so long [09:38] * Chipaca sharpens his Stick of Poking and pokes at spread [09:38] Son_Goku: I don't understand how [09:38] fundamentally, the kernel has to know a lot about what it's being built for [09:38] abeato, for everything but our default set of boards (for which we theoretically also have classic images and want to use the same kernel in both)... i.e. amd64, i386, rpi2/3 and dragonboard, you should use the kernel plugin [09:39] and things like rewriting how the version is constructed, what subsystems are enabled, what monkeypatching was done, etc. affect how it will work on a given userland [09:39] ogra_, I see, thanks! [09:39] abeato, really depends if you want/need to build from source or can actually use an existing deb based kernel [09:39] Son_Goku: I'm not sure I agree [09:39] the kernel influences the userland and vice versa, so changing expectations can have odd effects [09:39] Son_Goku: if that were the case we didn't have generic x86 kernels [09:40] I'm not talking about at the hardware level [09:40] abeato, i.e. for the beaglebone community image i also use the linux-generic kernel simply because it fully supports the borad [09:40] I'm talking purely software interfaces [09:40] ogra_, source-based in this case [09:40] and some of them are non-obvious ones [09:40] right, there you want the kernel plugin [09:40] for example, Ubuntu rewrites the version and bludgeons the upstream kernel version out of the kernel version [09:40] Fedora doesn't [09:40] great [09:41] Son_Goku: so what/ [09:41] if tooling in a Fedora userland expects to check based on the upstream kernel version, that might have unexpected side effects with an Ubuntu kernel [09:41] Son_Goku: sure but that's very unlikely [09:41] is it? [09:42] Son_Goku: and totally irrelevant, every distributor did something similar for whatever reasons [09:42] Son_Goku: yes, it's a nitpick, totally [09:42] but the point is that expectations are set based on a distribution convention [09:42] what expectations? [09:42] the version string, please! [09:42] hey, I know of plenty of scripts and things that check it [09:43] I personally had to write one for blacklisting bad kernel versions [09:43] will a system stop booting because of those scripts? [09:43] amyway [09:43] no, but maybe something will fire that shouldn't or something [09:43] that was not the point [09:43] (that's always possible anyway because FOSS has terrible QA) [09:43] the point is that snaps have this already [09:43] but another thing is that Ubuntu kernels are unsigned and Fedora ones are [09:44] so a Fedora boot system wouldn't boot an Ubuntu kernel anyway [09:44] on UEFI [09:44] you are looking for problems, not solutions IMO [09:44] and ubuntu kernels are signed too AFAIK [09:44] there are two sets [09:44] as far as I knew, on Ubuntu, only shim and grub are signed [09:44] maybe even only shim [09:45] I know in Fedora, we sign everything up to the kernel [09:45] Son_Goku: no, the kernel is signed too [09:45] hm, then does Ubuntu's kernel just not do signature checking for kernel modules? [09:46] Son_Goku: it does [09:46] Son_Goku: you remember 14.04 stuff [09:46] ah [09:46] so this changed after then [09:46] Son_Goku: FYI I had to sign vmware modules to load them on my machine [09:46] ah, so that's good [09:46] Son_Goku: then enroll the key in UEFI [09:46] yeah, I know that process well :P [09:47] mvo: artful-amd64? [09:48] zyga: anyway, I need to take a nap, and then I'll come back and start rolling 2.25 [09:48] I've been up for too long [09:49] Son_Goku: rest well :-) [09:49] zyga: oh yes! [09:49] Son_Goku: I'm cleaning up stuff in anticipation for 2.25 [09:49] mvo: what is artful? :D [09:49] Son_Goku: rest well indeed [09:49] mvo: what is that test about? [09:49] zyga: its for 17.10 [09:49] mvo: I hate live-build, so much [09:49] aaah [09:49] debugging that makes me want to murder things [09:49] zyga: its failing left and right [09:49] 17.10 is artful something [09:49] I forgot [09:49] Son_Goku: everybody hates it [09:50] Son_Goku: I would really like to kill it with fire [09:50] mvo: we were trying to use it for a project but it's horrible [09:50] so now I've moved onto fixing the Ubuntu support in KIWI and build things that way [09:50] Son_Goku: :/ maybe we can chat after your nap, there must be better ways [09:51] it's not brain-dead and doesn't make me feel murderous [09:51] mvo: sure [09:51] zyga: snapd tests are broken in 17.10, i.e. with go1.8 this is why I added the test machinery for it [09:51] zyga: Artful Aardvark [09:51] Son_Goku: aaaaaaaaaaa-distribution [09:52] zyga: to avoid getting a gazillion FTBFS mails when we try to build there [09:52] plus I'm sure somewhere is a distro that already moved to go1.8 too [09:52] mvo: are you on artful already? [09:52] mvo: well to be honest I would like us to use 1.8 everywhere ;) [09:53] * zyga spends all of today in github.com/notifications and pull requests [09:53] pedronis: yes, +100 on 1.8 [09:53] goodies all around [09:53] (because go doesn't care at mantaining old releases) [09:53] Son_Goku: would fedora accept 1.8 toolchain for fc25? [09:54] pedronis: look at it this way - adding artful tests is the first step towards this goal [09:54] Son_Goku: e.g. if we wanted to build snapd with a more recent version [09:54] also the go snap \o/ [09:55] mvo: go snap is an interesting concept but for building classic packages we need to do some heavy lifting to allow building a package with snap as a dependency [09:55] mvo: did you autopkgtests or something else? (me is not paying attention, also I have started to have strong mixed feelings about ever slower tests yesterday) [09:55] s/did you/did you add/ [09:55] mwhudson: I love your go snap (I think its yours, right?) [09:55] pedronis: autopkgtest [09:55] pedronis: so things will run in parallel at least [09:56] pedronis: but we should definitely brainstorm during the sprint [09:56] ok [09:56] pedronis: about the tests [09:56] yes, they get pretty terrible, especially around release [09:56] making your life not so fun I think [09:56] pedronis: did you see my earlier question about 3241? [09:56] pedronis: absolutely, plus travis seems to take a nap in between sometimes too [09:57] pedronis: i.e. we hit some limits there too it seems [09:57] yes [09:57] a mixture of things [09:57] but we need to do something, I think I was starting to have stockholm syndrom but the pain is real [09:58] lol [09:58] * mvo hugs pedronis [09:58] mvo: what's the question 3241, I think I missed it? [09:58] *about [09:58] pedronis: if something hurts we should do it more so it stops and we fix and warts, I agree that tests are pretty unreliable now but the only answer is to fix them and get better infra if needed [09:59] well, I don't think the issue is just unreliable [09:59] pedronis: I was wondering if you could have a quick look at #3241 as a sanity check. if the approach looks ok-ish, I will write more tests etc. mostly asking because this allows us to move forward with CE [09:59] they are also too slow for something we run for each commit in the universe [09:59] I think [09:59] morphis_: I'll re-review /usr/lib/linux/vmlinuz-4.8.0-46-generic.efi.signature [09:59] er [09:59] slow+unreliable = the worst [09:59] copy-paste fail [09:59] yeah, its tricky to find the right balance [09:59] https://github.com/snapcore/snapd/pull/3222 [09:59] PR snapd#3222: many: fix test cases to work with different DistroLibExecDir [10:00] morphis_: just let me know when you are don with it [10:00] mvo: I'll try to look after lunch, I also need to do some sprint prep (both travel but also writing some gdoc) [10:00] pedronis: I wonder if it would make sense for spread to retry individual failing tests (say up to 5 times) [10:00] but we'd have to fix travis time limit [10:00] pedronis: thank you [10:00] and really work on making tests better for a cycle [10:01] zyga: I just fixed the failing unit test [10:01] morphis_: thanks [10:02] ENOSERVERS [10:02] throw more $$$ at linode [10:02] in samsung we had a very nice system [10:02] since any avereage office had easily about a thousand laptops/desktops for devlopers [10:03] and they were mostly idle all day (editing = low cpu) [10:03] they were all wired to gigantic distributed task system for compiling software [10:03] so when you hit build, it'd really use 100s of machines around you for fantastic speedups [10:03] I wonder if we could tap into our machines at home, to send tasks and distribute them among ephemeral test VMs [10:03] so travis and linode are not that critical [10:07] mvo: question about go 1.8, why did you have to switch to DeepContains? [10:07] because of the pointer? [10:08] zyga: I don't know [10:08] mvo: aha, ok [10:09] zyga: looking at this rightnow, but wanted to see if that is the only remaining failure [10:09] mvo: yeah, testing on 1.8 is important, I wonder what else awaits us === chihchun is now known as chihchun_afk [10:11] PR snapd#3253 opened: cmd/snap-confine/spread-tests: discard useless --version test [10:13] why is econnreset failing all the time? [10:17] ogra_, did you remember to update core-build repo? it looks like some files are still missing [10:18] abeato, i am waiting for a second review for the tests PR, i wanted to use your change as an exercise for it once that landed === chihchun_afk is now known as chihchun [10:18] ogra_, ok, please let me know when I can propose the MP [10:19] oh, i have it locally already no need for yopu to do anything [10:19] great :) [10:20] is there anything urgent ? [10:20] no no [10:20] k [10:20] just wanted to make sure this is not lost [10:20] nah [10:20] * ogra_ loses keys and lighters, but rarely code ;) === fnordahl_ is now known as fnordahl [10:21] lol [11:17] PR core#36 opened: drop remaining dpkg binary (LP: #1686106) [11:20] PR snapd#3254 opened: tests: re-enable and moderninze /media sharing test [11:30] how to fix "cannot create user data directory: $path : Permission denied" [11:30] robje— hey. Where are you getting this? [11:30] robje: ls -ld ~/snap/ [11:30] robje: is that owned by root? [11:31] robje: or is the home directory somewhere unusual? [11:31] zyga— ooh, maybe you should ask these same questions on #1686852 [11:31] Bug #1686852: Can only run hello snap as root [11:31] :-) [11:31] heh, i was about to paste the same :P [11:31] all files and dirs owned by my user and group. [11:31] thanks, asking! [11:31] home is on NFS [11:32] aha [11:32] robje: ok, can you please pastebin `dmesg | grep DENIED` [11:32] robje: and `snap version` [11:32] "somewhere unusual" :) [11:32] and snap is a symlink to local storage [11:32] people still use NFS <3 [11:32] [ 1250.129323] audit: type=1400 audit(1493378981.364:156): apparmor="DENIED" operation="sendmsg" profile="/usr/lib/snapd/snap-confine" pid=24796 comm="snap-confine" laddr=172.28.4.67 lport=1003 faddr=172.28.4.7 fport=2049 family="inet" sock_type="stream" protocol=6 requested_mask="send" denied_mask="send" [11:32] [ 1250.129473] audit: type=1400 audit(1493378981.364:157): apparmor="DENIED" operation="mkdir" profile="/usr/lib/snapd/snap-confine" name="/home/r.epping/" pid=24796 comm="snap-confine" requested_mask="c" denied_mask="c" fsuid=2124 ouid=2124 [11:33] robje: are you using NFS? [11:33] oh [11:33] you have a dot in your username? [11:33] zyga, he said so above :) [11:33] robje: aha [11:33] robje: that's unsupported, let me find you a bug report with a lot of details [11:33] yes I have a dot in my username [11:33] robje: unfortunately there's a kernel limitation here [11:33] robje: you can kind of work around this but I'm not sure if that's even possible now [11:34] https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1662552 [11:34] Bug #1662552: snaps don't work with NFS home [11:35] Chipaca, a second checkmark on https://github.com/snapcore/core/pull/36 would be appreciated (one line change) [11:35] PR core#36: drop remaining dpkg binary (LP: #1686106) [11:35] robje: have a look at that report, let me know if you think I can help you somehow [11:35] ogra_— what was it that had made us leave the binary last time? [11:35] robje: the root issue is nfs leaks to various parts of LSM so confinement cannot be the same [11:35] robje: the differences in where user directories are are easier to work around [11:35] but [11:36] zyga— he doesn't have /snap in nfs [11:36] his home is [11:36] but /snap is a symlink [11:36] now I know _that_ won't work :-) but [11:36] Chipaca: NFS leaks everyhwere [11:36] would it work to change it to be a bind mount? [11:36] Chipaca: if you have something on NFS your process will be blocked by seccomp on sendmsg [11:36] Chipaca, no idea ... i think it was for some code in 15.04 [11:36] Chipaca: apart from that apparmor doesn't look at symlinks but at actual files [11:37] Chipaca: so some things may need tuning [11:37] Chipaca $home/snap is a symlink to /local/snap [11:37] Chipaca: and lastly snap-confine's appramor profile is special [11:37] i.e. on local hdd [11:37] zyga— 💩 [11:37] Chipaca: all in all it's not trivial to fix everything [11:37] Chipaca, iirc some snap code used to call dpkg --compare-versions [11:37] Chipaca: I wish someone had fixed the kernel side [11:37] Chipaca: before that I can really do little :/ [11:38] Chipaca: (or allow all network traffic to everything by default, but even then snapd doesn't control the apparmor profile of snap-confine) [11:38] robje— right, but as zyga says as soon as there's a nfs element in the path that needs to be traversed to get to wherever, you sendmsg, and that's blocked [11:40] yay, time travellers! [11:40] PR core#36 closed: drop remaining dpkg binary (LP: #1686106) [11:41] ogra_: ? [11:41] zyga, i was commenting on Chipaca's PR comment :) [11:41]