[01:34] #blackarch === chihchun_afk is now known as chihchun [08:01] good morning [08:01] Pharaoh_Atem: looking :/ [08:01] PR snapd#3248 closed: tests: fail early in the spread suite if trying to run it inside a container [08:01] aha, not great [08:01] Pharaoh_Atem: I'll fix that in a sec, thanks for reporting [08:02] hey zyga_ ! [08:02] pstolowski: hello [08:05] Pharaoh_Atem: offtipic, can I propose rawhide packaging for inclusion into packaging/ [08:05] Pharaoh_Atem: it will be useful for CI soon === zyga_ is now known as zyga [08:15] zyga: we can't yet [08:15] morphis: because of patches/ [08:15] zyga: we still miss changes to get snap-confine building [08:15] morphis: we can add the actual patches too you know [08:15] zyga: but I have things locally and will submit PRs soon [08:15] aha ok [08:15] * zyga pushed the fix for shellcheck [08:15] now where's my github token :/ [08:16] unless we have initial CI I would like to held the .spec file back [08:16] as only then it has value inside the tree [08:17] zyga: but if you have a good idea of how we can solve the problem http://pkgs.fedoraproject.org/cgit/rpms/snapd.git/tree/0001-cmd-use-libtool-for-the-internal-library.patch solves, that would be awesome :-) [08:19] Pharaoh_Atem: https://github.com/snapcore/snapd/pull/3258 [08:19] PR snapd#3258: cmd/snap-confine/tests: fix shellcheck on recently added files [08:19] PR snapd#3258 opened: cmd/snap-confine/tests: fix shellcheck on recently added files [08:21] pstolowski: this needs a 2nd trivial review https://github.com/snapcore/snapd/pull/3234 [08:21] PR snapd#3234: overlord/snapstate/backend,interfaces/mount: move ns management code [08:21] ok [08:25] morphis, Pharaoh_Atem: is that shellcheck fix something you can easily carry in the tree? [08:25] (as a pathc) [08:25] patch [08:25] zyga: I think Pharaoh_Atem is fine as long as we have a PR or the PR merged upstream [08:26] PR snapd#3230 closed: tests: extend network-control spread test to cope with network namespaces [08:29] zyga: do you have a raspbian system at hand? [08:29] morphis— out of curiosity, for what? [08:29] Chipaca: snapd packages :-) [08:30] morphis: yes [08:30] well [08:30] on a pi1 [08:30] but I can (probably) move that to a pi2/3 easily [08:30] zyga: hm, would be nice to see the error message on the pi1 [08:31] error message about? [08:32] zyga: that we don't have a suitable core snap for the pi1 [08:32] I guess it will try to load the armhf one but not sure [08:32] PR snapd#3234 closed: overlord/snapstate/backend,interfaces/mount: move ns management code [08:33] morphis: I think it will just fail, let me try [08:33] pstolowski: thanks! [08:33] zyga: yeah [08:33] pi1 is so slow... [08:34] :-) [08:34] zyga— remember snap 1 on the bbb? [08:34] or weren't you around for that [08:34] snappy* [08:34] Chipaca: snap 1 as in 15.04 snapd? [08:34] snappy --help [08:34] zyga, yw [08:34] <30 seconds waiting> [08:34] Chipaca: I recall it was slow :D [08:34] was that in python or all go already? [08:35] snappy was python [08:35] heh [08:35] nice :D [08:35] and the slow was the main driver for the move to go [08:35] * zyga would love to see python code of snappy [08:35] zyga— no, no you wouldn't [08:35] but it's in git :-) [08:35] zyga— this was pre-overlord, so the fs was the transaction layer [08:36] was is terrible? [08:36] aaaaha :D [08:36] remember the 16.04 push to make the insanity go away? [08:36] morphis: let me update first, I'll install snapd shortly [08:38] zyga: let me upload the repo first [08:47] morphis: which repo is that? [08:49] zyga: deb https://mm.gravedo.de/raspbian/ jessie main [08:50] morphis: fyi, raspbian already has snapd [08:50] 500 http://archive.raspbian.org/raspbian stretch/main armhf Packages [08:51] zyga: stretch does, yes [08:51] 2.21-2 [08:51] zyga: that is currently in rc [08:51] so everyone out there is still using raspbian based on debian 8 [08:51] which doesn't have it [08:51] and a stable raspbian based on debian 9 is still months away [08:57] zyga: what you officially get from the pi foundation is jessie: https://www.raspberrypi.org/downloads/raspbian/ [09:00] zyga: are you running stretch then on your pi? === chihchun is now known as chihchun_afk [09:05] morphis: yes [09:05] morphis: well, not stretch, but raspbian + jessie/wheezy/stretch raspbian archives [09:06] sounds hacky :-) [09:12] zyga: however, if you add the repository you should get snapd 2.24 === chihchun_afk is now known as chihchun [09:15] zyga, have you seen the failures in master? https://forum.snapcraft.io/t/tests-broken-in-master/457 [09:16] pstolowski: no, looking now [09:16] pstolowski: can you please edit your post to use formatting [09:18] zyga_: once I did some more testing I will make a post in the forum to call for wider testing [09:20] morphis: ^ did you see that failure? update test hangs on debian [09:21] zyga_: which failure? [09:24] pstolowski: you have a link to the test run on in travis? [09:24] zyga, done; i thought I did by using ` , but apparently it's not enough [09:25] zyga, https://travis-ci.org/snapcore/snapd/builds/227867834?utm_source=email&utm_medium=notification [09:25] pstolowski: I think you need ``` [09:26] i'm not sure it's from the same travis run, but failed the same I think [09:27] pstolowski: so... [09:27] pstolowski: I think we rejected one fix that may be the cause here [09:27] it got stuck on Run configure hook of "core" snap if present [09:27] zyga, pstolowski: so this is running 2.21 and installing the core snap from stable [09:29] zyga, pstolowski: sounds very muhc like https://bugs.launchpad.net/snappy/+bug/1674193 [09:29] Bug #1674193: core snap's configuration hangs on debian | openSUSE | mainline kernel [09:31] morphis, indeed it's an upgrade from old version. but I wonder why it's failing now, i don't think it's a new test? [09:31] pstolowski: did we release a new core snap to stable recently? [09:31] or rather, any new conditions [09:32] dunno [09:32] zyga_: any idea? [09:32] morphis: yes [09:32] * zyga_ looks for the PR [09:33] https://github.com/snapcore/snapd/pull/3145 [09:33] PR snapd#3145: overlord/ifacestate: fix auto-connect during core snap transition [09:33] zyga_, pstolowski: ok, can reproduce the same problem on my debian install [09:34] look at https://github.com/snapcore/snapd/pull/3145/commits/71ce65eb95b3210515a8c81f4737e5c3d9bd18fb [09:34] so we rejected that PR [09:34] let me try to recall the rationale [09:34] https://github.com/snapcore/snapd/pull/3145#discussion_r110766622 [09:34] PR snapd#3145: overlord/ifacestate: fix auto-connect during core snap transition [09:34] zyga, looking at your last comment, you had a simpler variant merged? [09:35] so the question is, why did the slot not auto-connect after restarting into new snapd? [09:35] zyga: with a core snap from edge the problem doesn't exist [09:36] a $ snap install core fails and a $ snap install --edge core works [09:36] morphis: how about edge? [09:36] er [09:36] beta [09:36] zyga: what do we have currentl in stable? [09:36] morphis: no idea [09:36] morphis: the version field is useless [09:36] wonderful .. [09:37] ogra_: hey [09:37] zyga_, yo [09:37] ogra_: is the snapcrat feature that would let us give real and sensible verison available now [09:37] shall we take this conversation to https://forum.snapcraft.io/t/tests-broken-in-master/457? [09:37] ogra_: so that "snap info core" shows the snapd version there? [09:37] zyga_, not sure if it landed with the last drop [09:38] zyga_: beta works too [09:38] zyga_, if it did i have to do a few changes first ... the bug wasnt closed but perhaps thats just an oversight from sergio [09:38] ogra_: aha, thanks [09:38] morphis: ok, then when 2.25 is released this should be fixed [09:38] zyga: right [09:38] however that wont help us to get CI back working on master [09:39] zyga_: unless we do a $ snap install --beta core for debian during the test run [09:39] morphis: I think that's a reasonable compromise for now [09:39] let me do a PR [09:39] is federico working today? I'd like to know his opinion [09:39] morphis: please do, thanks! [09:41] pstolowski: yeah, let's summarize on the forum [09:41] we can paste this converstation [09:43] zyga_, pstolowski: https://github.com/snapcore/snapd/pull/3259 [09:43] PR snapd#3259: tests/upgrade: force install core snap from beta for debian [09:43] pstolowski: can you summarize the discussion in the forum? [09:44] PR snapd#3259 opened: tests/upgrade: force install core snap from beta for debian [09:49] morphis, will do [09:52] morphis: hmm [09:52] morphis: so one question [09:52] morphis: after 2.25 is out it won't change anything in debian [09:52] morphis: because debian is frozen, right? [09:52] morphis: so I'm trying to understand one thing there [09:52] morphis: the test installs some debs [09:52] morphis: are they not the same as snapd in edge? [09:52] morphis: (except for the name of the plug?) [09:53] morphis: what I'm trying to say is that perhaps, if we get this rigth, the change is not needed, no need to special-case debian [09:53] morphis: I think some code may be missing from master [10:00] morphis: master contains code that renames core-support to core-support-plug [10:00] morphis: so it should work [10:01] why? [10:01] better asked, why -plug as suffix? [10:01] morphis, zyga_ summarized the discussion in the forum, let me know if i missed anything [10:01] morphis: to keep plug and slot names distinct [10:02] zyga_: isn't is already imminent for a plug that it is a plug? [10:05] ogra_: in re yesterday's question about test script, I don't know [10:05] maybe? [10:05] the shellcheck thing has always been there, though [10:06] it passed in 2.24 [10:06] zyga_, morphis: https://paste.fedoraproject.org/paste/NmsFYMd2WEStihK1ATvizV5M1UNdIGYhyRLivL9gydE= [10:06] shellcheck test fails on 2.25 [10:06] Son_Goku, to late ... fix is alöready pending [10:06] :) [10:06] ogra_: is there an active PR? [10:06] (zyga works in his sleep :P ) [10:07] https://github.com/snapcore/snapd/pull/3258 [10:07] PR snapd#3258: cmd/snap-confine/tests: fix shellcheck on recently added files [10:07] excellent :D [10:07] well... not so excellent [10:07] what the hell happened to spread/travis? [10:08] oh, welp [10:08] Son_Goku: from what I experienced so far there are differences between shellcheck on Ubuntu and others [10:08] also the version plays a role [10:08] yeah, shellcheck runs are disabled on F24 and older [10:08] the 14.04 shellcheck is pretyt crappy ... not sure what versioon that was thopugh [10:09] because shellcheck < 0.4.4 doesn't actually support -x [10:09] my trees all install a 16.04 chroot when doing shellcheck [10:09] Fedora 24 and EPEL 7 have shellcheck 0.3.x, which can't actually run the tests :/ [10:09] is pedronis around today ? [10:10] manually upgrading a pi3 gets me "[/] Setup snap "core" (1829) security profiles (phase 2)" ... forever .... shell prompt never returns [10:11] (pretty much like described in one of the forum threads) [10:18] wut [10:18] this PR somehow breaks Fedora builds [10:18] Son_Goku: fixed [10:18] Son_Goku: sorry, the patch had a broken file inside by accident [10:19] take 4 [10:21] good, gobuild is fixed [10:21] now we're at C build [10:21] shellcheck passed... good [10:23] Son_Goku: yeah, sorry about the earlier patch [10:24] * zyga_ feels like he has cold :/ [10:24] zyga_, dude ... i tend to test stuff before commenting ... shellcheck didnt disagree ;) [10:24] but leave it ugly if you feel like ;) [10:25] ogra_: I think it may be different version then, I did fix all the things shellcheck reported [10:25] ogra_: I tested on F26 [10:25] ah [10:25] I wanted to have a patch that would be clean for fedora packaging [10:25] well, newer ubuntu versions might spill a warning there [10:25] btw, I think shellcheck would disagree about $FOO/"whatever" but I may be worng [10:26] warning? [10:26] yeah [10:26] about extra quoting? [10:26] zyga_: I think shellcheck would be fine with "$FOO/whatever", though [10:27] about possible interpretation as: "$TMP/$(basename " $0 ")" [10:27] well, for inner quotes, shouldn't you use a different quote for the inner one? [10:27] anyway, if it doesnt complain now, leave it [10:27] like single quotes for inner and double for outer [10:28] technically just quoting $0 is enough [10:29] Son_Goku: no, that's actually correct [10:29] Son_Goku: $() gives new quoting context [10:29] zyga_: ah [10:29] shell is confusing :) [10:29] Son_Goku: "$(echo "This is $(echo "ok")")" [10:29] yes [10:31] well, I'm about ready to commit my 2.25 package [10:31] +1 [10:31] unless there's any other patches I need besides PR3222 and PR3258 (though I don't need the latter) [10:31] Son_Goku: not sure, I don't know of any [10:32] well, I guess that's what testing updates is for :) [10:32] I've also created a snap-mgmt script that will be run on %preun on final removal [10:32] based on the snapd.postrm script [10:32] so that will need specific testing [10:36] zyga_, morphis: building: https://koji.fedoraproject.org/koji/taskinfo?taskID=19367859 [10:36] Son_Goku: awesome! [10:37] Son_Goku: thank you [10:38] building on all Fedora branches now [10:39] zyga_: there are some advantages to building for an upstream distro that moves fast :) [10:45] I hate armv7hl builds :( [10:45] they need to fully move over to the VMs on aarch64 hosts [10:45] those are so much faster... [10:57] Chipaca: hey [10:58] zyga_— hi [10:58] Chipaca: can you please review https://github.com/snapcore/snapd/pull/3259 [10:58] PR snapd#3259: tests/upgrade: force install core snap from beta for debian [10:58] Chipaca: master is currently broken because of htis [10:58] Chipaca: howerver plase note what I said above: [10:59] Chipaca: actually let me add that to the forum [10:59] zyga_— you said a lot of things above. Which is the note? [10:59] https://forum.snapcraft.io/t/tests-broken-in-master/457/6 [10:59] I want to fix master but I'm not sure this is the actual fix [11:00] and that we're not missing anything [11:06] zyga_, morphis: https://bodhi.fedoraproject.org/updates/FEDORA-2017-866e9643a8 (F24), https://bodhi.fedoraproject.org/updates/FEDORA-2017-2e4459fa03 (F25), https://bodhi.fedoraproject.org/updates/FEDORA-2017-74f7c7df46 (F26) [11:10] morphis, zyga_: please specifically test final removal of snapd *after* upgrading or installing v2.25 [11:10] source script: https://src.fedoraproject.org/cgit/rpms/snapd.git/tree/snap-mgmt.sh [11:11] install step: https://src.fedoraproject.org/cgit/rpms/snapd.git/tree/snapd.spec#n434 [11:11] scriptlet execution: https://src.fedoraproject.org/cgit/rpms/snapd.git/tree/snapd.spec#n568 [11:12] Son_Goku: ack, will do [11:13] Son_Goku: will do! [11:14] this is specifically for solving https://bugzilla.redhat.com/show_bug.cgi?id=1444422, so that's what this aims to fix [11:17] * Son_Goku hopes the patch load doesn't get any bigger on snapd... [11:17] I'm sort of cheating by using PR diffs, but the number of patches being applied to snapd is quite high now :( [11:28] fwiw i think we agreed that we should let you remove core if it was the last snap left, but it wasn't prioritised [11:29] Chipaca: yes, I think we agreed [11:29] (but my memory is bad so don't trust me alone on this) [11:29] Chipaca: I think it should be easy, right? [11:29] Chipaca: no other snap installs in progress [11:29] Chipaca: and no other snaps [11:30] zyga_— i think it wasn't super easy because of how isolated the bit that checks for core was, but i could be misremembering [11:30] in any case, it still isn't prioritised :-) [11:30] Chipaca: we could catch the context at the time the op is made [11:30] and pass a flag like "ok-to-remove-core' [11:32] zyga_— that sounds like the wrong way to do it [11:33] * Son_Goku grumbles about crappy hotel Wi-Fi [11:51] lunch! [12:01] good idea [12:04] zyga_, https://bugs.launchpad.net/snapd/+bug/1687608 ... you asked about more info about this in https://forum.snapcraft.io/t/device-gets-bricked-after-trying-to-upgrade-core-snap/453 .... i can reproduce it fine here if you want me to collect data [12:04] Bug #1687608: running "snap refresh core" on an UbuntuCore device locks you out until reboot === palasso_ is now known as palasso [12:11] re [12:11] ogra_: looking [12:12] zyga_: you had time to give snapd on raspbian a try? [12:12] ogra_: aha, can you tell me what is in snap changes? [12:12] morphis: no, I left the apt-get update && apt-get dist-upgrade -y running in the background [12:12] let's see [12:13] zyga_, http://paste.ubuntu.com/24498251/ [12:14] morphis, zyga_: could one of you guys create a topic on the forums announcing the snapd 2.25 testing for Fedora once it syncs out to updates-testing? [12:14] zyga_, note that my SD is worn out, thus change 13 failed (but rolled byck fine) ... change 14 is the actual one showing the issue [12:14] *back [12:15] interesting [12:15] I wonder why change 13 failed [12:15] how is your SD a factor? [12:16] zyga_, because uboot.env was trashed (Sd worn out) [12:16] aah [12:16] 2017-05-02T09:40:50Z ERROR cannot finish core installation, there was a rollback across reboot [12:16] Son_Goku: sure [12:16] right, ignore that one [12:16] it rolled back fine [12:16] ogra_: ok so after rollback what did you do? [12:17] zyga_, snap refresh core [12:17] 2017-05-02T10:04:41Z INFO Waiting for restart... [12:17] I'm trying to grok the log there [12:17] it was waiting for restart over and over [12:17] which also works fine but does never finish the "Setup snap "core" (1829) security profiles (phase 2)" bit [12:17] yeah [12:17] I think we have a bug there [12:17] let me look [12:17] yes, we do [12:17] it used to give me the console back before ... now it doesnt anymore [12:19] weird [12:19] so [12:19] we log that message when snapd itself saves in the state that it is waiting to restart [12:20] yeah, the message was there before ... but didnt spin forever [12:20] and that is not even saved in the state [12:20] it's just saved in memory [12:20] so we really need to restart [12:20] (though I suspect that it may be broken if we restart snapd rather then the machine) [12:20] the security profiles thing is saved in memory ? [12:20] ogra_: no [12:20] the "restarting" flag [12:21] the restarting flag isnt the issue [12:21] the task just does nothing until that flag is cleaned [12:21] so what is? [12:21] the issue is "Setup snap "core" (1829) security profiles (phase 2)" never finishing [12:21] or at least never returning the console access [12:21] ogra_: ... [12:21] ogra_: did you look at the code? [12:21] no [12:21] ogra_: the restaring flag guards that code from running [12:22] ogra_: that's why I mentioned it [12:22] well, the code obviously runs :) [12:22] ogra_: yes and bails out instantly because of that [12:22] http://paste.ubuntu.com/24498125/ shows even the spinner rotating between line 60 and 78 [12:23] yeah, the spinner is client side query [12:23] so... [12:23] ah, so you mean the reboot flag kicks in to early ? [12:23] no, I don't think it does [12:23] * zyga_ thinks [12:23] so [12:23] it's just this [12:23] we set the flag [12:23] and ask the system to reboot [12:24] right [12:24] but the reboot is not instant, it will be a while before it happens [12:24] which obviously works as expected [12:24] then lots of time passes [12:24] and then we reboot [12:24] and then it works OK [12:24] but the UX is not making it clear what is going on [12:24] that message has been there before since day one ... but it always left me interacting with the tty ... now it doesnt anymore and the spinner goes on [12:25] ogra_: you can ctrl-c and exit [12:25] ogra_: but I suspect shutdown may set nologin [12:25] before the spinner stopped at some point [12:25] no, ctrl-c is ignored [12:25] ogra_: yes but there was a bug there (hence phase 2) [12:26] morphis: no luck with your repo [12:26] morphis: apt-get doesn't like it [12:26] i think before it acvtually printed the "[/] Setup snap "core" (1829) security profiles (phase 2)" ... finished it and only then the reboot message showed up [12:27] at least thats what i remember [12:27] now both are there at the same time and the "[/] Setup snap "core" (1829) security profiles (phase 2)" seems to hold the console access [12:27] zyga_: what does apt-get say? [12:27] it will most likely complain because if the unknown key [12:28] E: Failed to fetch https://mm.gravedo.de/raspbian/dists/jessie/InRelease [12:29] raspbian !? [12:29] zyga_: that file exists, can you give me the full output? [12:29] ogra_: yeah [12:31] morphis: there's nothing else reall [12:31] can that even run our armhf binaries ? [12:31] let me pastebin [12:31] http://pastebin.ubuntu.com/24498297/ [12:31] (given it is a re-compile of the whole archive for v6) [12:31] ogra_: these packages are rebuild for raspbian [12:31] ah, k [12:31] zyga_: there we go: E: The method driver /usr/lib/apt/methods/https could not be found. [12:32] zyga_: and it gives you the solution in the next line [12:34] apt-transport-https isnt a default in raspbian [12:34] ? [12:34] * ogra_ would expect it to be seeded in all deb based distros nowadays ... [12:35] oh [12:35] sorry [12:35] ogra_: me too [12:42] 01 16:56:05 localhost.localdomain /usr/lib/snapd/snapd[897]: helpers.go:173: cannot connect plug "core-support" from snap "core", no such plug [12:42] May 01 16:56:05 localhost.localdomain /usr/lib/snapd/snapd[897]: helpers.go:173: cannot connect plug "network-bind" from snap "core", no such plug [12:42] hmm [12:42] i thought we fixed that one [12:43] Good days [12:43] The forum is going down and back up for a quick configuration change [12:44] * ogra_ wonders if his drafted answer will survive that :) [12:44] ogra_: Yes, it saves while typing [12:44] ah, cool then [12:46] It's back up already [12:47] morphis: installing [12:47] niemeyer, and worked fine :) [12:51] morphis: installed :) [12:51] morphis: trying to install stuff [12:51] zyga_: good [12:51] morphis: well, it downloads core now [12:51] zyga_: hm [12:51] morphis: which is terrbile, we should print architecture that snapd knows about in "snap version" [12:51] zyga_, UH OH ! so hitting ctrl-c like 5 times actually gets me the console back .... it seems just inconsistent in accepting ctrl-c after all! [12:52] still though ... i did never need to actually hit ctrl-c at all before [12:52] zyga_: which architecture does the kernel report? [12:53] niemeyer: you have a minute to snapshot Spread-08 as a new fedora-25-64 image? [12:56] morphis: it runs the configure hook now [12:56] morphis: armv6l [12:56] morphis: core has installed! [12:56] morphis: Yeah [12:56] morphis: Is it ready? [12:56] zyga_— snapd prints its architecture on startup (not sure of the context) [12:57] Chipaca: thanks, so snapd on arm6l thinks it is arctually armhf [12:57] so it all misbehaves [12:58] well, we don't support arm6l inside snapd so ¯\_(ツ)_/¯ [12:58] niemeyer: it is, down to 1022M, the minimal size we can get to [12:58] are we doing a standup today? [12:58] well, and we dont really have any v6 in the archive ... so your build-deps will be armhf, your libc will be at build time etc etc [12:58] morphis: Okay, working on it [12:58] Chipaca, zyga_: we shouldn't then download the armhf snap on arm6l [12:59] niemeyer: thanks! [12:59] Chipaca, good question ... who is around ? you, zyga_, me and pstolowski ? [12:59] yeah [12:59] o/ [12:59] we could meet asnd rant about the guys in montreal ;) [12:59] and people not reviewing branches, also [12:59] morphis: runtime.GOARCH is "arm" [13:00] yeah [13:00] Chipaca: I'm in the room now [13:00] Chipaca: let's hang out for 5 min [13:01] zyga_: I see, that actually is our problem then .. [13:01] zyga_: what does /proc/cpuinfo give you on the pi1? [13:04] morphis: http://paste.debian.net/930351/ [13:06] zyga: uname -m reports arm6l? [13:06] yes [13:06] good [13:08] Morning folks! I don't have access to my canonical account anymore since I don't work there anymore, but I can't use my preferred namespace "ralsina" for my new personal account. Is there any way to get it back? [13:14] morphis: All done.. please remember to test the update task in the spread-images project so we're able to update it easily in the future [13:15] niemeyer: thanks! [13:21] morphis: suggestion to use uname data to konw the real architecture [13:21] zyga_, note that an armhf kernel could drive a v6 userspace though [13:22] (technically ... not sure that is done anywhere) [13:22] ogra_: yeah perhaps, for now let's just close one big gap though [13:22] yeah, unlikely that you hit that in the real-world [13:24] hmm, well ... except probably in a raspbian chroot running on top of a debian installation ... [13:25] (for testing or building or some such) [13:37] PR snapd#3260 opened: WIP: Implement --user instance for snapd [13:53] niemeyer, zyga_: time to review https://github.com/snapcore/snapd/pull/3222 again? [13:53] PR snapd#3222: many: fix test cases to work with different DistroLibExecDir [13:54] PR snapd#3261 opened: snap-confine: init the ENTRY variable, coverity is unhappy otherwise === daniel is now known as Guest47234 === Guest47234 is now known as Odd_Bloke [14:29] Facu: our fake servers in snapcraft turned into an ugly beast. We need at least an url router. Would it make sense to use flask? or maybe there's something lighter. [14:30] elopio, can't you route urls using something static? like apache? [14:32] I have an app that crashes with a segfault when run in a snap sandbox. Is there any way I can run gdb to debug this? [14:36] Facu: wouldn't it be worst to add apache as a dependency than flask? I want something like this, but it's not in xenial :( https://github.com/pretenders/pretenders [14:39] elopio, understand; to just re-route probably flask is even too big, maybe a smaller web framework would also do the job, like pyramid; but if you already are proficient with flask, go for it [14:42] stupid question: what is the wdl-nextcloud snap for? === chihchun is now known as chihchun_afk [14:44] pachulo, perhaps kyrofa knows ... i'd just go with the nextcloud snap from nextcloud [14:45] pachulo, i thinnk wdl means "western digital" and is tied to the netxcloud box [14:46] ah, ok, that makes sense! Thanks ogra_ ! [14:49] Facu: I'll take a look at pyramid. Thanks. [14:52] hi, does anyone know the state of mir-kiosk stuff(gui apps on ubuntu-core), is it still developed/maintained? [14:52] pachulo, ogra_: no idea. It doesn't seem to be available in the stable channel-- how did you find it? Last I heard the upstream nextcloud snap was still shipped on the nextcloud box [14:52] kyrofa, snap find nextcloud on a pi3 shows it to me [14:52] ogra_, ah, so armhf then [14:52] yeah [14:52] That always gets me :P [14:53] stop using dead arches, arm is the future! [14:53] ;) [14:54] ogra_ does all his development work on a pinebook [14:55] haha, not really :) [14:57] jdstrand: hello! r875 of the tools is now in production [15:05] roadmr: that was fast. thanks! :) [15:06] roadmr: fyi, the review-tools fork of crt and its edge snap is not bad [15:07] roadmr: I'm not ready to say 'integrate this into the snapcraft.io store in lieu of crt', but I wouldn't mind feed back from people playing with it [15:07] feedback even [15:07] nice! [15:13] jdstrand: hey [15:14] jdstrand: I'm working with a user that has xenial with 4.4 kernel and hits an apparmor denial for snap-confine [15:14] jdstrand: I'm quite puzzled by what's going on [15:14] what is the denial? [15:15] https://github.com/lxc/lxd/issues/3267 [15:15] jdstrand: a downgrade to 2.24 helps [15:15] until a reboot [15:15] when it is broken again [15:16] Hi! [15:16] I did a diff of snap-confine in 2.24 and I don't see anything relevant [15:16] olive: hey! [15:16] jdstrand: olive here is affected by this issue [15:16] jdstrand: my quick idea is that snapd 2.24 doens't use snap-confine from core yet, i'm checking that now [15:16] jdstrand: the actual denial though is very odd; IMO it is allowed by the profile [15:17] [11120.777073] audit: type=1400 audit(1493729112.057:3410): apparmor="DENIED" operation="mount" info="failed srcname match" error=-13 profile="/usr/lib/snapd/snap-confine" name="/var/lib/lxd/" pid=31726 comm="snap-confine" srcname="/var/snap/lxd/common/lxd/" flags="rw, rbind" [15:17] this is one denial [15:17] (actually this one feels wrong, I saw a different one earlier) [15:17] and this one is also something unexpected [15:18] olive: is there anything in /var/lib/snapd/mount/snap.lxd.fstab [15:18] zyga_: there is no rule for that in 2.25 [15:18] mount options=(rw rbind nodev nosuid noexec) /var/lib/snapd/hostfs/var/lib/lxd/ -> /var/lib/lxd/, [15:18] jdstrand: right, I saw a different denial earlier that matched this rule you just pasted [15:18] the srcname in the denial is /var/snap/lxd/common/lxd/ [15:18] (saw == on irc) [15:19] zyga_: not better after reboot [15:19] olive: ok, thank you for checking [15:19] snapd 2.25 [15:20] jdstrand: so olive here is on 2.25 with no snap-confine installed (I was worried because there was a .dpkg-bak file in /etc/apparmor.d/) [15:20] /var/lib/snapd/mount/snap.lxd.fstab: No such file or directory [15:20] * zyga_ is puzzled [15:20] ok [15:20] olive: can you do this please: [15:20] export SNAP_CONFINE_DEBUG=yes [15:20] and then start lxd if you can [15:20] though [15:21] you can run anything else [15:21] even hello-world snap [15:21] as the denial should be generic [15:22] http://paste.ubuntu.com/24499042/ [15:23] zyga_: it seems snap-confine is doing something wrong. why is it trying to use /var/snap/lxd/common/lxd/ as the srcname instead of /var/lib/snapd/hostfs/var/lib/lxd? [15:23] jdstrand: no idea yet [15:24] jdstrand: but very fishy [15:24] olive: great, can you couple that with 'dmesg | grep DENIED' [15:24] olive: get the last lines [15:24] olive: it should be just one, paste it here directlyu [15:24] /var/snap/lxd/common/lxd/ is for the lxd snap. /var/lib/snapd/hostfs/var/lib/lxd is the core snap. it seems to be picking the wrong thing [15:24] [ 384.578585] audit: type=1400 audit(1493738612.509:148): apparmor="DENIED" operation="mount" info="failed srcname match" error=-13 profile="/usr/lib/snapd/snap-confine" name="/var/lib/lxd/" pid=7625 comm="snap-confine" srcname="/var/snap/lxd/common/lxd/" flags="rw, rbind" [15:24] [ 384.690504] audit: type=1400 audit(1493738612.621:149): apparmor="DENIED" operation="mount" info="failed srcname match" error=-13 profile="/usr/lib/snapd/snap-confine" name="/var/lib/lxd/" pid=7631 comm="snap-confine" srcname="/var/snap/lxd/common/lxd/" flags="rw, rbind" [15:25] jdstrand: note that snap-confine debug log doens't mention anything related to lxd common at all [15:25] * zyga_ looks at the source [15:26] olive: does /var/lib/snapd/hostfs/var/lib/lxd exist? is it a symlink? [15:26] jdstrand: that mount has hard-coded arguments, I doubt we're doing anything worng [15:26] jdstrand: note, hostfs is just inside snaps [15:26] olive: can you please `ls -ld /var/lib/lxd` [15:26] jdstrand: a symlink the is likely the cause [15:27] jdstrand: sorry was otp! oh so review-tools? you say it's a fork? where does that live? [15:27] ls: cannot access '/var/lib/snapd/hostfs/var/lib/lxd': No such file or directory [15:27] # ls -ld /var/lib/lxd [15:27] lrwxrwxrwx 1 root root 25 Feb 1 22:45 /var/lib/lxd -> /var/snap/lxd/common/lxd/ [15:27] aha [15:27] olive: perfect [15:27] olive: so two questions [15:27] olive: did you do that manually? [15:28] no. [15:28] zyga_: 'hostfs is just inside snaps'. huh? it has to exist at whatever point snap-confine is running to mount it. I guess you mean within its mount namespace [15:28] olive: and did it really work before? [15:28] yes :) [15:28] roadmr: https://launchpad.net/review-tools [15:28] hahaha [15:28] jdstrand: yes, I mean that the hostfs directory as you specified it is always empty from a typical unconfined shell [15:28] * ogra_ looks forward to jdstrand's answer in https://forum.snapcraft.io/t/startx-as-a-regular-user/460 [15:29] olive: so that symlink won't work unfortunately, let me think about it for a sec [15:29] (i think the user asked here before and was toold "no Xorg on UbuntuCore" ... but seems he tried anyway (or it is another person) [15:29] olive: is that directory empty? [15:29] olive: I mean /var/lib/lxd [15:30] yes, just this symlink [15:30] oh.wait. [15:30] olive: right, but if you follow the symlink, is /var/snap/lxd/common/lxd empty? [15:30] when I cd /var/lib/lxd and ls... it is not empty. o_O [15:31] stgraber: ^^ is that symlink anything you are familiar with? (symlink from /var/lib/lxd -> /var/snap/lxd/common/lxd) [15:31] olive: ok, as a quick work-around do this: [15:31] ok sorry, I'm tired, of course, it is a symlink... :( [15:31] zyga_: nope [15:31] zyga_: not something we do or would ever recommend someone do [15:32] zyga_: http://paste.ubuntu.com/24499072/ [15:32] zyga_: folks should just "export LXD_DIR=/var/lib/snap/common/lxd/" if they want tools outside of the snap talking to the snap LXD [15:33] it isn't something that would be allowed by the profile anyway [15:33] olive: can you try stgraber's advice, remove that symlink please [15:33] I promise you that I have not tweaked myself. I only followed the stgrabber article;) [15:33] ie, the snap couldn't create that symlink [15:33] jdstrand: I'll patch snap-confine to be smarter [15:34] jdstrand: it should not belly-up fail on that [15:34] I wonder if we could kill the LXD quirk now [15:34] the lxd snap would have to (ab)use its lxd-support interface to load and/or change to a profile that allowed that write and perform the symlink [15:34] jdstrand: the lxd snap never uses /var/lib/lxd [15:35] stgraber: right, not saying you are doing that. saying you'd know if you did cause you'd have to go through hoops [15:35] I remove the /var/lib/lxd link ? [15:35] olive: yeah [15:35] olive: yes, please [15:36] olive: curious, did you ever try to run lxd inside of a container launched from the lxd snap? [15:36] jdstrand: I'm pretty sure this bind-mount was added for the juju, conjure-up or similar snaps that wanted to talk to the host LXD [15:36] stgraber: ah, perhaps that is it [15:36] ok now snapd is running Active: active (running) since Tue 2017-05-02 17:35:10 CEST; 5s ago [15:36] jdstrand: I don't think so [15:36] * jdstrand is wondering if a container was given write access to the host's filesystem and then did that [15:37] olive: did you use juju or conjure-up like stgraber said? [15:37] jdstrand: note that 2.25 has reassociate fix, maybe what is going on is that snap-confine breaks out of the container? [15:37] jdstrand: though I bet lxd uses process namespace [15:37] jdstrand: so probably not that [15:37] never tried juju in the server. [15:37] in this* server [15:37] zyga_: it might be the reassociate stuff [15:38] conjure-up either [15:38] I think making snapd more robust in the face of this is a fine fix, but the root cause needs to be found [15:39] * jdstrand notes he also uses the lxd snap, is on 17.04 with snapd 2.25 and does not have this issue [15:40] zyga_: thanks for the comment, completely forgot to mention that [15:41] thank you everybody. specially zyga_ and stgraber :) [15:41] and jdstrand [15:41] (and sabdfl) [15:42] jdstrand: trying to reason about this, I don't think is could be the reassociate fix, after all, it should not change anything in this case; [15:42] olive: you are welcome! [15:42] stgraber: all lxd containers use the PID namespace, right? [15:44] jdstrand: and in the pastebin from SNAP_CONFINE_DEBUG log above we see: DEBUG: re-associating is not required [15:46] something created that symlink [15:47] the only thing I can think of is a container that had /var(/...) mounted inside [15:47] and then it did it [15:48] or as stgraber said, some tool trying to be smart and getting something wrong (but those tools weren't run) [15:49] Found a way to get gdb running. Is there any reason why QFontDatabase::findFont should segfault when calling into libQt5XcbQpa? [15:50] For instance a plug that should give access to fonts? [15:51] shouldnt the desktop-qt part provide that ? [15:51] jdstrand: I think that we don't bind mount all of /var, right? [15:52] jdstrand: so that cannot be done from inside a snap [15:52] dragly: no, sorry :/ [15:52] dragly: there are no fonts in the core snap [15:52] why would there [15:52] dragly: and host fonts are not accessible [15:52] the snapcraft desktop parts install fonts inside the snap [15:53] that should just work, provded you used the right part in your snapcraft.yaml [15:53] ogra_: we wanted to offer sharing of classic fonts [15:53] ogra_: e.g. run your favourite editor with locally-installed font [15:53] zyga_, right, i'm talking about today [15:54] * zyga_ nods [15:54] there definitely no reason for the above error if you include the right bits [15:56] dragly, https://github.com/ubuntu/snappy-playpen/blob/master/qcomicbook/snapcraft.yaml as an example [15:57] (see the "after: [desktop-qt5]" there) [15:58] zyga_: lxd is in a position to do its own reassociate. not saying it does, it just can. I'm grasping at how it could have happened [16:00] jdstrand: aha, that's true [16:00] jdstrand: still I don't think it is reassociate that was re-enabled that is at fault here [16:05] ogra_: thanks. I'm using a custom Qt install, but it might help to install the default Qt packages and use those to have all the necessary pieces set up and ready. I'll look up what after does. [16:18] PR snapd#3262 opened: cmd/snap-confine: aggregate operations holding global lock [16:26] morphis: hey [16:26] zyga_: hey! [16:26] morphis: do you have a working qmeu image I can use for debian? [16:26] morphis: I'm still debugging https://github.com/snapcore/snapd/pull/3259 [16:26] PR snapd#3259: tests/upgrade: force install core snap from beta for debian [16:26] morphis: I think there's a deeper problem there [16:26] morphis: I can work on linode but I'd prefer a local image [16:27] zyga_: was looking into that too but it looks to me like those failing bits are not coming from the debian part .. [16:27] zyga_: I suspect we need to keep an eye on this. there is no reproducer and other lxd snap users are not complaining. if/when they do, we can request a reproducer/etc [16:29] zyga_: but let me check if I have my dirty script to create a debian qemu image still, didn't had time to clean them up yet [16:31] morphis: much appreciated! [16:31] zyga_: take http://saimei.ftp.acc.umu.se/cdimage/openstack/testing/debian-testing-openstack-amd64.qcow2 and use https://gist.github.com/morphis/1e181e60b3803f8a72952c580fad9a21 [16:31] zyga_: $ ./adt-buildvm-ubuntu-cloud -r sid --image=$PWD/debian-testing-openstack-amd64.qcow2 --verbose [16:32] zyga_: save the gist as adt-buildvm-ubuntu-cloud [16:32] thanks [16:32] zyga_: the script is a hacked variant of the ubuntu variant [16:33] but should give you an image with debian:debian to login [16:35] morphis: I have it, thank you! [16:39] zyga_: however it looks more like the tests are a bit racy [16:39] linode:ubuntu-16.04-32:tests/main/interfaces-openvswitch is the one which fails now [16:46] I am getting a "No such file or directory: [...]/parts/qml/install/etc/xdg/qtchooser/snappy-qt5.conf" when using desktop-qt5 [16:52] morphis: gee, that iso image clones forever [16:52] wgets [16:53] I get modem speeds on it [16:53] zyga: https://cdimage.debian.org/cdimage/openstack/ [16:53] try a different mirror thne [16:53] thnx === JanC is now known as Guest15747 [17:23] is there a way of telling `snapcraft cleanbuild` to use a different release of ubuntu for a local build? I think I need to use a more-recent version than 16.04 for some of my build dependencies to be satisfied [17:24] specifically I need a more recent vala than 0.34.0: checking whether /usr/bin/valac is at least version 0.34.0... no [17:42] diddledan: No. Instead, make "vala" a separate part. Compile your compiler. [17:43] zyga_: you found anything? [17:43] zyga_: looks like all tests for https://github.com/snapcore/snapd/pull/3259 passed now [17:43] PR snapd#3259: tests/upgrade: force install core snap from beta for debian [17:43] morphis: still downloading the image, currently at 25% [17:43] zyga_: even on another mirror? [17:44] yes, maybe something elsewhere on the network [17:44] morphis: mvo just said he's looking too [17:58] zyga: ok [18:00] jdstrand: is my explanation in https://github.com/snapcore/snapd/pull/3252 sufficient? [18:00] PR snapd#3252: tests,cmd/snap-confine: port older snapd-discard-ns tests [18:21] I'm going through the running nodejs as a service demo and am running into this issue. https://github.com/canonical-websites/tutorials.ubuntu.com/issues/70 does anyone have a workaround other than to run services in devmode? I'm wanting to convert some code that's using systemd to snaps and it doesn't seem like it will work right now. [18:26] cratliff: can you just build it entirely, not with snap try [18:26] cratliff: whatever the issue it will likely function correctly then [18:26] cratliff: I suspect you are on encrypted home folder or something else [18:26] cratliff: can you tell me where the [...] part that you removed in the issue was pointing to? [18:29] it finally downloaded [18:29] * zyga_ debugs debian [18:29] Sorry, I'm not aware of removing anything, I'm not sure what you are referring to. I've not attempted to build it without try yet, let me see. [18:33] aha [18:33] cratliff: sorry, I assumed you were reported of the issue there [18:33] cratliff: just build the snap and install it [18:33] cratliff: as for snap-try, there are still some issues [18:33] PR snapcraft#1294 opened: plugin: Add plugin for meson build system [18:34] I think I just figured out a simple way to handle it actually [18:39] I installed the server with --dangerous, not --devmode. It is working, since I used dangerous it should show that it is running without issue contained, right? [18:40] cratliff: not sure really, does snap list say it is devmode? [18:40] no, it does not. [18:41] cratliff: great, then you are good :) [18:41] cool, thanks for the help. [18:44] cratliff: good luck [18:56] finally [18:56] geez [18:56] forever [19:04] PR snapd#3263 opened: snap: make `snap prepare-image --extra-snaps` derive side info [19:06] and it doesn't boot :/ [19:06] I think I will resort to regular debian image to deubg this [19:06] PR snapd#3241 closed: snap: make `snap prepare-image` read .assert files for local snaps [19:08] ok, so it gets to login prompt [19:08] but debian/debian doesn't work [19:08] * zyga_ looks at the script again === alvaro is now known as fede2 [21:44] how do I tell snapcraft to keep the lxc container using cleanbuild when the build fails so that I can try to figure out why I'm getting a permission denied error when running an autotools build when it tries to execute ./configure [21:49] ignore that, I'm gonna do it a different way - run an lxc container manually and call snapcraft directly within that without using cleanbuild [21:53] ok, so it only dies in cleanbuild :-/ === alvaro is now known as fede2