[00:09] Question: I'm trying to build a kernel snap from Kernel Source 4.5 and I need to build custom kernel modules for some drivers. I have successfully compiled the kernel on my build machine, built the headers, and compiled the drivers, but how do I do all of this using Snapcraft? === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk === xnox_ is now known as xnox === Michaela is now known as Mikaela === stgraber_ is now known as stgraber === chihchunl is now known as chihchun === clemensv_ is now known as clemensv === mjl_ is now known as mjl [04:12] ...helllooo... [04:51] wililupy: it's night time for a lot of the snappy people [04:51] yeah, I figured. I'm having issues building a kernel snap for a vendor, and was hoping to get some guidance. [06:49] good morning === SaMnCo-laptop is now known as SaMnCo-desktop === chihchun is now known as chihchun_afk === chihchun_afk is now known as chihchun [11:04] jdstrand: ping [11:32] sudo is broken now in classic? [11:32] (classic)ubuntu@localhost:~$ sudo apt-get install snapcraft bzr [11:32] sudo: no tty present and no askpass program specified [11:32] guess related to https://bugs.launchpad.net/snappy/+bug/1564316 [11:32] Launchpad bug 1564316 in Snappy "classic dist-upgrade fails on sudo and tzdata..." [Critical,New] [11:32] e.g. to the upgrade that came in [11:34] https://bugs.launchpad.net/snappy/+bug/1564369 [11:34] Launchpad bug 1564369 in Snappy "sudo broken in latest classic" [Undecided,New] [11:36] mdeslaur: ^ [11:36] anything you can think of being wrong with your sudo upload? [11:38] asac: not off hand, no. needs investigating [11:38] its hard blocker on developing snappy [11:38] or on snappy [11:38] anyway, lets see if one of snappy folks gets on i [11:38] t [11:38] mdeslaur: no complains from normal ubuntuf olks? [11:39] * asac doesnt have rthe latest sudo yet on xenial desktop and wonders if i should rather not upgrade [11:39] asac: nope, works fine on regular desktop [11:40] ok let me upgrade [11:40] :) [11:40] lets hope you are right [11:40] mdeslaur: were there any snappy changes in your merge? [11:40] or in the version that we had before :) [11:41] there's nothing snappy-specific in the sudo package, no [11:41] ok [11:41] odd [11:41] do you get any apparmor denials? [11:41] lets wait for snappy crew to come in [11:42] * ogra_ sees the same on a rpi here [11:43] what's the output of "tty" [11:44] (classic)ubuntu@localhost:~$ tty [11:44] not a tty [11:44] (classic)ubuntu@localhost:~$ [11:44] well, there you go [11:44] well, that was guessable from the error :P [11:44] and downgrading sudo works? [11:45] did you get a new glibc too? [11:45] mdeslaur: fyi, 'snappy shell classic' does not run under apparmor (it is just a command to get into an ubuntu chroot) [11:45] * ogra_ only just did "snappy enable-classic" ... this was a fresh install [11:45] you have to ask asac for an upgrade log [11:45] bug #1564316 has the upgrade log [11:45] bug 1564316 in Snappy "classic dist-upgrade fails on sudo and tzdata..." [Critical,New] https://launchpad.net/bugs/1564316 [11:46] ah [11:46] * jdstrand notes he hasn't tried any of this yet [11:46] I wonder if it is the pt_chown removal [11:47] well, i wonder why it mounts /etc/sudoers at all [11:47] we cnanot downgrade [11:47] i had to install fresh [11:47] because dist-upgrade failed [11:47] because sudoers cannot be written on snappy [11:48] ogra_: it does that so you can sudo apt-get install [11:48] yeah, it shouldnt bind mount it [11:48] we could have a copy [11:48] sure [11:48] think thats mvos work [11:48] why ? [11:48] just said... think we could create a copy :) [11:48] the relevant info isnt in there [11:48] on snappy [11:49] so i remember initially in classic you couldnt sudo apt-get install [11:49] then mvo landed something [11:49] maybe it was this :) [11:49] * jdstrand is unable to enable classic [11:49] the relevant file is /etc/sudoers.d/90-cloud-init-users [11:49] jdstrand: it works well [11:49] Get https://images.linuxcontainers.org/meta/1.0/index-system: dial tcp 192.99.34.219:443: i/o timeout [11:49] sudo snappy enable-classic [11:49] :) [11:49] oh we still use those images [11:49] heh [11:49] not /etc/sudoers [11:49] heh, I know, except not :) [11:49] thast a critical bug too [11:50] yes :( [11:50] lets put bets on how likely this is still in when we release :) [11:50] anyway [11:50] well, we need an arm64 image at least [11:50] jdstrand: just try again [11:50] * jdstrand has tried 8 times [11:50] its amateur server that probably will come back [11:50] :P [11:50] * jdstrand keeps trying [11:50] (sorry for being ironic) [11:50] ah, the 9th time's the charm [11:51] see [11:51] CDN [11:51] i am sure though its your internet net [11:51] :P [11:51] cool thanks for trying [11:51] I can confirm the bug [11:52] great... that makes it three :) [11:52] sudo umount /writable/classic/etc/sudoers [11:52] sudo mount --bind /etc/sudoers.d /writable/classic/etc/sudoers.d [11:52] then try again [11:52] that works? [11:52] should work [11:52] the upgrade or the sudo? [11:52] ok [11:52] sudo [11:52] upgrade would work too i guess [11:52] yeah [11:53] or at least less likely to conflict aka break [11:53] I can confirm ogra_'s commands fix it [11:54] it is still not a tty, but sudo works [11:54] ogra_: added it as workaround giving credits to you [11:54] ogra_: https://bugs.launchpad.net/snappy/+bug/1564369 [11:54] Launchpad bug 1564369 in Snappy "sudo broken in latest classic" [Critical,New] [11:54] huh, that's odd [11:54] think now we can wait for mvo to come back unless you guys know what to do [11:54] mdeslaur, nah, not odd [11:54] mdeslaur, /etc/sudoers lives in a squashfs and then gets bind mounted into the container [11:54] bah [11:54] so it is ro underneath [11:54] i cannot save the workaround [11:55] launchpad!!! [11:55] posting as comment instead then [11:55] also not working :( [11:55] re-login ? [11:55] why? i used lp a few minutes ago [11:55] WORKAROUND (credits to ogra): [11:55] ubuntu@localhost:~$ sudo umount /writable/classic/etc/sudoers [11:55] ubuntu@localhost:~$ sudo mount --bind /etc/sudoers.d /writable/classic/etc/sudoers.d [11:56] heh [11:56] ogra_: that makes sense for 'sudo apt-get upgrade' in the chroot failing, but not sure how it explains sudo not working in a fresh 'enable-classic' [11:56] well needed to paste it somewhere so it doesnt get lost :) [11:56] jdstrand, yeah, thats indeed a bit weird, especially the tty bit [11:57] but tty returns 'not a tty' before and after your workaround [11:57] that doesn't seem to be the issue... [11:57] what's in the file in the sudoers.d directory? [11:57] o/ [11:57] ffox is king of lp [11:57] mdeslaur: two files [11:57] mdeslaur: with ogra's bind mount, it is empty [11:58] shouldnt [11:58] sorry, I lied [11:58] * ogra_ waits for apt-get install snapcraft to finish [11:58] I wasn't root do didn't get bash completion [11:58] mdeslaur: [11:58] root@localhost:/home/ubuntu# cat /etc/sudoers.d/10-ubuntu-core-secure-path [11:58] Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snaps/bin" [11:58] root@localhost:/home/ubuntu# cat /etc/sudoers.d/90-cloud-init-users [11:58] # Created by cloud-init v. 0.7.7 on Thu, 31 Mar 2016 11:23:15 +0000 [11:58] # User rules for ubuntu [11:58] ubuntu ALL=(ALL) NOPASSWD:ALL [11:58] and README [11:58] yay upgrading xenial proper also failed ;) [11:58] ah, no the NOPASSWD alleviated the need for a tty [11:59] cups-browsed :0 [11:59] oh, indeed ! [11:59] not sudo... so no panic :) [11:59] but why did this change? [11:59] where did we change something? [11:59] before we had sudoers bindmounted [11:59] it didn't change, it just never properly handled a sudo package upgrade properly [12:00] wait [12:00] tahts not true [12:00] i install fresh classic [12:00] which just downloads a prebuilt image and unpacks [12:00] no uptgrade etc. [12:00] and then i got the sudo problem [12:00] upgrading wasnt possible at all [12:00] so its not upgrade issue [12:00] yeah, that is what I was saying [12:00] at least not an upgrade on my side [12:00] yeah, there must still be something wrt tty [12:00] something changed for sure [12:00] somewhere :) [12:00] hrm [12:01] I wonder if 10-ubuntu-core-secure-path isn't being copied into place with enable-classic or something [12:01] so then you need a password [12:01] remove the bind mount and check from outsuide the container ;) [12:01] maybe mr. graber though hacked up his container tarball ... just so we stop using those unsupported things [12:02] ubuntu@localhost:~$ sudo ls /writable/classic/etc/sudoers.d [12:02] README [12:02] but thats not the issue [12:02] yeah [12:02] there is something wonky about ttys [12:02] ogra_: I think that explains it [12:02] (classic)ubuntu@localhost:~$ tty [12:02] not a tty [12:02] * ogra_ has a non upgraded container, gimme a minute [12:03] ogra_: we aren't bind mounting /etc/sudoers.d so 90-cloud-init-users isn't getting in there [12:03] ubuntu@localhost:~$ mount|grep sudoers [12:03] /dev/mmcblk0p2 on /etc/sudoers.d type ext4 (rw,relatime,data=ordered) [12:03] /dev/loop0 on /writable/classic/etc/sudoers type squashfs (ro,relatime) [12:03] thats with the old classic container [12:04] ogra_: but, why is 90-cloud-init-users there at all on my system... [12:04] ubuntu@localhost:~$ sudo ls /writable/classic/etc/sudoers.d/ [12:04] README [12:04] ubuntu@localhost:~$ snappy shell classic [12:04] ... [12:04] (classic)ubuntu@localhost:~$ tty [12:04] not a tty [12:04] like, why is the ubuntu user not supposed to enter a password? [12:04] jdstrand: we use cloud-init still no? [12:04] (classic)ubuntu@localhost:~$ sudo ls [12:04] [sudo] password for ubuntu: [12:04] ... [12:04] so here it works [12:04] jdstrand: i think we never ask for password in snappy for sudo? [12:04] asked [12:04] despite not having a tty [12:04] hmm [12:05] not true indeed [12:05] asac: sure, but haven't we always required a password for sudo? [12:05] so in classic we dont [12:05] in classic we obviously did [12:05] I'm talking outside the classic shell [12:05] outside we dont [12:05] ogra_: we asked for pass? [12:05] yeah [12:05] oh :) [12:05] see above [12:05] * asac old [12:05] thats a container from about a week ago [12:05] not upgraded [12:06] so what is expected experience? i assume we want same behaviour for sudo in and outside container? [12:06] but even there tty says there is no tty [12:06] e.g. either pass or not? [12:06] meh, even 15.04 has 90-cloud-init-users [12:06] inside the classic container ? [12:06] * ogra_ has never tried classic on 15.04 [12:07] there isnt classic [12:07] right [12:07] on 15.04 [12:07] cloud-init is our first boot tool [12:07] i thought i missed something [12:07] it has always been there [12:07] ogra_: not what I meant. I got distracted by the fact that there is no password prompt for sudo at all in snappy [12:07] sadly, yeah :PO [12:07] jdstrand, thats a cloud init config thing [12:08] ogra_: is that something with udf options? [12:08] nope, something with the cloud init defaults [12:08] cause jeez, I have two live systems with this [12:08] /var/lib/cloud/seed/nocloud-net/user-data and /var/lib/cloud/seed/nocloud-net/meta-data are pre-created [12:09] and iirc theer is an option you can set in user-data to make it ask [12:09] * jdstrand adds this as a todo to discuss with mvo [12:09] * jdstrand edits this file on his live systems [12:10] ogra_: ok, so, back to asac's issue [12:10] the ubuntu user is in the sudp group by default so dropping the 90-cloud-init-users would be fine [12:10] ogra_: I guess something in earlier image worked and something in latest doesn't [12:11] ogra_: I'm on 16.04+20160321.17-27 on the host here [12:11] if I enable-classic and shell classic, I have the no tty issue [12:11] you could try sideloading a newer ubuntu-core (breaks upgrades though) [12:12] ogra_: your bind mount papers over the underlying issue I think [12:12] it does [12:12] but at least it allows people to keep working, so that's good :) [12:12] ogra_: I can say that /usr/lib/pt_chown does not exist in the container [12:12] * jdstrand wishes he had an earlier classic [12:13] wow [12:13] /usr/lib/pt_chown is not supposed to exist btw [12:13] looking at the cloud-init docs it seems thats actually a default [12:13] ogra_: right, I knew that is what they set [12:13] how awful [12:13] I am a lot less sure that is the correct default for snappy [12:14] ogra_: I forget their rationale. smoser could certainly remember [12:14] he's on vacation :) [12:15] ugh [12:15] but i guess there is a reason and there were discussions ... so i wont bother :) [12:16] that is why I want to bring it up with mvo rather than filing a bug, doing the mailing list, etc [12:16] but it's a trap! [12:16] :) [12:16] hah [12:16] I have to know I need to edit that file [12:17] I don't think the snappy docs mention this anywhere [12:17] jdstrand: hey [12:17] you could boot with cloud-init=disabled on the kernel cmdline ;) [12:18] zyga: hey [12:18] (though then you end up without ssh keys ... iirc thats the only thing we need cloud-init for nowadays) [12:18] jdstrand: I posted some cleanups and seccomp [12:18] jdstrand: and I'm about to post dbus [12:18] zyga: I was actually going to respond to your ping and then all that ^ came up at the same time [12:18] jdstrand: no worries :) [12:18] jdstrand: I already forgot what I wanted then [12:18] zyga: yeah, I was going through the emails for that [12:22] ogra_: ah, so, interesting. I happened to have a vm with a previous enable-classic that hasn't been upgraded [12:22] (classic)ubuntu@snappy-20160324-amd64:~$ tty [12:22] not a tty [12:22] (classic)ubuntu@snappy-20160324-amd64:~$ sudo uptime [12:22] [sudo] password for ubuntu: [12:22] 12:21:46 up 0 min, 1 user, load average: 0.43, 0.11, 0.04 [12:22] yeah [12:22] thats what i said above [12:22] so, no tty but it still works [12:22] password prompting works that is [12:23] the bind mounts are all the same on both [12:23] there is no /usr/lib/pt_chown there either, so that should be unrelated [12:23] so nothing changed wrt /dev/pts or so [12:25] I can confirm that [12:25] /dev/ptmx is 0666 and nothing in /dev/pts [12:26] I suspect we actually want to mount /dev/pts/ptmx though [12:26] yeah [12:27] ubuntu@localhost:~$ sudo mount --bind /dev/pts /writable/classic/dev/pts [12:27] ubuntu@localhost:~$ snappy shell classic [12:27] ... [12:27] (classic)ubuntu@localhost:~$ sudo ls [12:27] [sudo] password for ubuntu: [12:27] ... [12:28] ubuntu@localhost:~$ sudo umount /writable/classic/dev/pts [12:28] ubuntu@localhost:~$ snappy shell classic [12:28] (classic)ubuntu@localhost:~$ sudo ls [12:28] sudo: no tty present and no askpass program specified [12:28] so yes, that fixes it properly [12:28] yeah, so bind mounting /dev/pts /writable/classic/dev/pts 'solves' the tty issue (I suspect we actually want a devpts newinstance,ptmxmode=0666,gid=5 mount instead of a bind mount though [12:28] ) [12:29] in any case something inside the snappy enable-classic chain ... [12:29] yeah [12:29] ok, I'm wandering off to look at zyga's PRs [12:29] jdstrand: cool :) [12:30] jdstrand: ah, I wanted to ask about dbus and developer mode [12:30] jdstrand: do we have plans to support anything similar for dbus? [12:31] zyga: I'll first say that I'm fairly in the dark wrt all things developer mode. All I know about it is that it is supposed to be per-snap complain mode for the security sandbox [12:31] jdstrand: yes, that's accurate so far [12:31] zyga: with that said, I think it is sufficient to have apparmor in complain mode for dbus [12:31] jdstrand: ok [12:31] ie, let's not fiddle with dbus bus policy [12:32] jdstrand: we don't have any dbus snippets yet (nothing uses it) but I suspect we'll need something fairly soon [12:33] zyga: I get pinged by the bluez, nm, pulse, mir, etc guys all the time and been telling them once this interfaces (and really the state engine) stuff all lands, we can circle back to them [12:33] zyga: so very soon you and I will have an opportunity to check out the dbus interface [12:36] zyga: I just looked at https://dbus.freedesktop.org/doc/dbus-daemon.1.html. dbus bus policy doesn't (currently) have a concept of complain mode. it is either deny or allow [12:37] ok, that's good to know, thansk [12:37] I'll put this into the code as a note [12:37] zyga: so don't do anything for developer mode and dbus. I suspect a dbus service author is going to be just fine with tuning the bus policy as needed so long as we have apparmor in complain mode [12:38] perhaps some day if it becomes a requirement dbus can be modified to allow but log (ie, support complain) [12:38] jdstrand: I have one more issue (udev) but I wrote it down and we can discuss it when reviweing the patches [12:42] zyga: I had some questions regarding udev, cgroups and the launcher that can probably wait until then. there is an interesting intersection of apparmor, udev and the launcher that we need to be careful about [12:43] zyga: cause as it stands now, the launcher won't setup a cgroup if /var/lib/apparmor/clicks/%s.json.additional doesn't have a particular string [12:43] _clicks_? [12:43] yes [12:43] that's not good [12:43] given the time frame, do we have hands on the launcher? [12:44] my hands are on the launcher all the time lately [12:44] I'm not sure what you are referring to though [12:44] I'll put it another way [12:44] ah, I wasn't sure, I recall you were transitioning but were still busy as a manager [12:45] I'm fine with updating the launcher, but it is going to need to make a choice to apply a cgroup or not [12:45] zyga: I am functioning as interim manager. people have given me some space, but it is more like I have 1.5 jobs atm [12:48] zyga: anyway, historically, as you know, constantly recompiling the apparmor profile because a usb device came up differently or was unplugged, etc, etc was not viable. so we came up with the idea to add devices to an app-specific cgroup (fine) [12:48] yes [12:49] zyga: the idea was then, when a device was assigned to the snap, apply lenient apparmor policy for /dev, and then just let device cgroups handle the mediation (also fine) [12:49] right [12:49] zyga: so, then docker came along and broke when it ws run under a cgroup (iirc) [12:50] so the idea was, have the launcher conditionally setup the cgroup *if* it detected the lenient apparmor policy [12:50] that lenient policy was applied to /var/lib/apparmor/clicks/%s.json.additional [12:50] so the launcher would look in there [12:51] ah [12:51] man, [12:51] I think what we could do instead is look in the udev db to see if any devices are assigned [12:51] and make that the conditional [12:51] I'm inclined to agree [12:51] but I'd like to sit down and write down variou things we need to and can do and see if it all adds up [12:52] sure [12:52] cause the thing is, you'll want an apparmor snippet for the lenient /dev policy on device assignment since the launcher isn't going to modify policy [12:53] device grant* [12:53] and on revoke, remove that [12:53] zyga: I'll add a card for my to adjust snappy_udev_setup_required() to check udev for assignments [12:54] s/my/me/ [12:54] zyga: but I'll wait to act on it til I hear back from you [12:54] I have several MPs for the launcher that are in play, so I can keep moving forward [12:55] this area is admittedly too twisty, but whenever we get apparmor profile composition, we can get rid of this [12:56] (there is a bug on this btw) [12:57] that is almost certainly not a 16.10 thing [12:57] so we'll need to get this right [12:57] zyga: ^ [12:57] * jdstrand makes sure profile composition is in the planning doc tyhicks and I are putting together [12:59] jdstrand: when you say it is not a 16.10 thing, do you mean it needs to be done for 16.04 or that it will take more time than 16.10? [13:00] zyga: I mean I doubt it will even be done for 17.04 (yes, *17*) [13:01] we need to live with this udev/cgroup stuff for a while yet [13:01] ouch [13:01] ok [13:01] what is the preferred way on 16.04 to do initialization (think add user to extrausers, add ssh public keys, etc.) for a snap? [13:04] jdstrand: you should join our standups [13:04] jdstrand: perhaps not today (too late) but I think it would be useful === _bjf is now known as bjf [13:09] if now is too late, then every day is too late [13:09] I started very early today. my normal start of day is 9 minutes ago [13:11] josepht, you cant [13:12] Good morning [13:12] jdstrand: we just started, I think you could still join reguarly [13:13] should webdm start on boot? if not, how do I start it, and what port does it run on? [13:13] (and you shouldnt, everything inside a snap will run as root or if it is a user executable binary, as the user that execs it) [13:13] popey: snappy install webdm, then it should be on webdm.local:4200 [13:13] popey, it should (but there seem to be issues with platforms that use WLAN) ... port is 4200 [13:14] aha! [13:15] hm, all wired here. works on my pi 2 but not pi3, your image ogra_ [13:15] didn't start it seems [13:17] I get no updates on that pi3 image either, wondering if I should expect any? [13:18] zyga: fyi, https://bugs.launchpad.net/ubuntu/+source/ubuntu-core-launcher/+bug/1564401 [13:18] Launchpad bug 1564401 in ubuntu-core-launcher (Ubuntu) "launcher does not apply cgroups on 16.04" [High,Triaged] [13:19] jdstrand: since you already look at that area [13:19] jdstrand: how would it work if two separate apps/snapps wanted to get a device in their cgroup? [13:20] popey, the one from my people.u.c ? [13:20] jdstrand: the rules that we'd make today, from what I know about udev, would overwrite one another [13:20] zyga: I assigned that to me already [13:20] popey, i think that still has sideloaded bits ... which would prevent automatic upgrades [13:20] jdstrand: I might be getting this wrong [13:21] jdstrand: but just something raised my eyebrow yesterday [13:21] zyga: I'm not a udev expert, but would hope that the rules were written such that they would be cumulative. ie, a device has multiple tags. so a device is tagged with foo and bar, but not baz [13:21] jdstrand: well, now there are no rules [13:21] ogra_: yeah, it's all listed as sideloaded [13:21] zyga: I suggest asking pitti about that part (he came up with this snappy tagging scheme) [13:21] jdstrand: but as soon as we want to write some, the simplistic snippets might require parsing/processing to really express this [13:21] jdstrand: ok, noted [13:22] zyga: that said, I think for 15.04 the decision was a device could only be assigned to a single snap [13:22] jdstrand: mmmmmm [13:22] jdstrand: that's good to know [13:22] jdstrand: anyway, I've got this written down and I'll explore this further with udev code soon [13:22] zyga: we need mvo and pitti to comment-- I was only on the periphery of that and reviewed the code [13:23] popey, right, so you could only update it manually ... (or re-build a fresh image witgh all bits from the store) [13:23] is there a plan to make a working image? [13:24]