| === sergiusens_ is now known as sergiusens | ||
| === chihchun_afk is now known as chihchun | ||
| === chihchun is now known as chihchun_afk | ||
| === chihchun_afk is now known as chihchun | ||
| === iMadper` is now known as iMadper | ||
| mborzecki | morning | 05:08 |
|---|---|---|
| zyga | good morning! | 05:38 |
| Son_Goku | gah | 05:45 |
| Son_Goku | why am I still awake... | 05:45 |
| mborzecki | zyga: hey | 05:50 |
| zyga | Hey | 05:55 |
| zyga | It’s much colder today | 05:55 |
| mborzecki | zyga: and expecting rain in the afternoon as well | 05:56 |
| zyga | Yeah, the “summer” is over | 05:56 |
| zyga | Afk, walk | 05:57 |
| mborzecki | i have #5073 open, but it's failing on linode, specifically ubuntu-16.04-32 and opensuse-42.2, looks like the linode images are gone and they do not boot anymore | 06:48 |
| mup | PR #5073: set up journal streams in user session application autostart (2.32) <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5073> | 06:48 |
| mborzecki | master is using gce | 06:49 |
| === chihchun is now known as chihchun_afk | ||
| mborzecki | according to travis relase/2.32 branch is failing | 06:54 |
| zyga | Hmmm | 06:55 |
| zyga | Did we drop lindoe already? | 06:55 |
| mborzecki | it looks like that only some images were dropped | 06:56 |
| mborzecki | 16.04-32 and openssue, this is what's failing | 06:56 |
| mborzecki | zyga: see the dial timeouts https://api.travis-ci.org/v3/job/368660863/log.txt | 06:56 |
| mborzecki | and it keeps repeating | 06:56 |
| zyga | Hmmm | 06:57 |
| mborzecki | i've restarted 5073 numerous times now | 06:57 |
| === chihchun_afk is now known as chihchun | ||
| kalikiana | moin moin | 07:05 |
| === pstolowski|afk is now known as pstolowski | ||
| pstolowski | morning | 07:06 |
| zyga | back from walk | 07:11 |
| mborzecki | kalikiana: pstolowski: hey | 07:15 |
| mborzecki | pstolowski: trivial review https://github.com/snapcore/snapd/pull/5078 :) | 07:15 |
| mup | PR #5078: interfaces/builtin, daemon: cleanup mocked builtin interfaces in daemon tests <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5078> | 07:15 |
| mborzecki | pstolowski: ta | 07:18 |
| mup | PR snapd#5078 closed: interfaces/builtin, daemon: cleanup mocked builtin interfaces in daemon tests <Created by bboozzoo> <Merged by stolowski> <https://github.com/snapcore/snapd/pull/5078> | 07:18 |
| Chipaca | mo'in all | 07:56 |
| Chipaca | mborzecki: what am I missing for a +1 from you on #5066? | 07:57 |
| mup | PR #5066: overlord/snapshotstate/backend: introducing the snapshot backend <Created by chipaca> <https://github.com/snapcore/snapd/pull/5066> | 07:57 |
| pstolowski | morning Chipaca | 07:59 |
| mborzecki | Chipaca: i'll go over it again once i'm done with the watchdog, iirc i left some more comments, but those might have been for the overlord pieces that were later pulled | 07:59 |
| * zyga is stuck on the apparmor issue :/ | 08:03 | |
| Chipaca | zyga: stuck how? | 08:04 |
| zyga | there's something wrong with my logic but I cannot see it yet | 08:05 |
| * zyga tries to summarise the problem | 08:05 | |
| zyga | when pawel tried to use a layout that had deep directory like /usr/a/b/c/d | 08:05 |
| zyga | things were broken because our permissions were too strict and prevent us from making /usr writable | 08:06 |
| zyga | I fixed that | 08:06 |
| zyga | now what I find is that we make /usr writable but things subsequently fail to create /usr/a/b/c/d in a way I don't understand | 08:06 |
| zyga | it seems that after making most of that path we end up with EROFS | 08:07 |
| zyga | but.. how? | 08:07 |
| zyga | http://paste.ubuntu.com/p/h8QkcWWQSy/ | 08:07 |
| zyga | on line 1120 we are done with making /bin writable | 08:07 |
| zyga | (after writing a loot of things to a tmpfs mounted over /bin | 08:08 |
| zyga | now on line 1141 we fail with EROFS | 08:08 |
| Chipaca | is it meant to be mounting everything in /bin ? | 08:08 |
| zyga | yes | 08:08 |
| Chipaca | or is that just your test | 08:08 |
| zyga | we re-create /bin in a writable place | 08:09 |
| zyga | this is the writable mimic logic | 08:09 |
| zyga | rbind /bin -> /tmp/.snap/bin | 08:09 |
| zyga | mount -t tmpfs none /bin | 08:09 |
| zyga | touch /bin/a && mount --bind /tmp/.snap/bin/a /bin/a | 08:09 |
| zyga | (then all the way till we "mimic" everything over | 08:09 |
| zyga | then mount /tmp/.snap/bin to clean up | 08:10 |
| zyga | this works so far | 08:10 |
| zyga | what is odd is next we try to mkdir /bin/very/weird/place | 08:10 |
| zyga | and this works by opening / then /bin/ then /bin/very/ and so on | 08:10 |
| zyga | (mkdirat'ing them as we go) | 08:10 |
| zyga | owowow | 08:11 |
| zyga | MAN | 08:11 |
| zyga | how did I miss this | 08:11 |
| Chipaca | zyga: because you don't have a sock puppet to talk to | 08:11 |
| zyga | the error is for /snap/test-snapd-layout/x1/bin/very/weird/place | 08:11 |
| zyga | not for /bin/very/weird/place | 08:11 |
| zyga | the bug is completely different!!! | 08:11 |
| * zyga is dumb | 08:11 | |
| * zyga hugs Chipaca | 08:11 | |
| zyga | thank you John | 08:11 |
| * Chipaca tries to make his eyes wobble like a good sock puppet would | 08:12 | |
| zyga | I see what the problem is now | 08:13 |
| zyga | it's my silly test | 08:13 |
| zyga | the bug is fixed now | 08:13 |
| * zyga wonders if there's an emoji for <man holding hand on forehead> | 08:13 | |
| Chipaca | zyga: U+1F926 | 08:15 |
| Chipaca | zyga: (not quite, but close enough imo) | 08:15 |
| zyga | haha, thank you :-) | 08:16 |
| zyga | this will be my motto of the day | 08:16 |
| zyga | I was looking at this issue all Friday :/ | 08:16 |
| Chipaca | zyga: unicode 9 so not in xenial ¯\_(ツ)_/¯ | 08:16 |
| zyga | fixed, fingers crossed | 08:20 |
| mup | PR snapcraft#2095 opened: lxd: proper error classes for container errors <Created by kalikiana> <https://github.com/snapcore/snapcraft/pull/2095> | 09:05 |
| jamesh | zyga: so, is there anything that needs to be done before https://github.com/snapcore/snapd/pull/3963 can be merged? | 09:07 |
| mup | PR #3963: cmd/snap-confine: add support for per-user mounts <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/3963> | 09:07 |
| jamesh | it got a tick from jdstrand | 09:07 |
| zyga | I asked mborzecki or Chipaca to have a quick look just in case we missed something | 09:07 |
| zyga | but otherwise no | 09:07 |
| popeycore | grrr. installing a snap blew my x session up again :| | 09:08 |
| zyga | I plan to merge it today | 09:08 |
| popeycore | it blows up just at the point when it prints that its connecting the opengl interface | 09:08 |
| jamesh | zyga: cool. As written, it should have no effect when there isn't a .user-fstab file for a snap | 09:08 |
| popeycore | Wimpress: getting coffee while my machine updates/reboots. back shortly :) | 09:12 |
| Wimpress | popeycore: OK. | 09:13 |
| mborzecki | any clue why NOTIFY_SOCKET set by system is @/org/freedesktop/systemd1/no | 09:24 |
| mborzecki | tify/3197004134137770178 on 14.04? | 09:24 |
| zyga | mborzecki: what is is normally? | 09:31 |
| mborzecki | zyga: /run/systemd/notify | 09:31 |
| zyga | 14.04 has older systemd, maybe that just was the thing at the time | 09:31 |
| mborzecki | hmm service-watchdog assumed that the communication happens on a unix socket with filesystem path, not an abstract one | 09:36 |
| mborzecki | then apparmor rules probably need to change too | 09:36 |
| zyga | hmm | 09:39 |
| zyga | apparmor_parser spins at 99%CPU | 09:39 |
| mborzecki | zyga: does `unix (connect, receive, send) type=stream peer=(label=unconfined,addr="@/org/freedesktop/systemd1/notify*")` look ok to you? | 09:39 |
| zyga | I think that's nog great | 09:39 |
| zyga | mborzecki: yes though I honestly don't recall the syntax | 09:39 |
| zyga | don't forget the trailing , | 09:39 |
| mborzecki | aah | 09:39 |
| mborzecki | thx | 09:39 |
| * Chipaca goes to see a man about a dog^W^W my back | 09:43 | |
| zyga | https://github.com/bartzaalberg/snaptastic | 09:45 |
| zyga | interesting | 09:46 |
| * zyga wrote a profile that makes apparmor_parser run for a very very long time | 09:49 | |
| popey | diddledan: fyi, i am backing up and deletingr repos from snapcrafters that are broken / never been published | 09:49 |
| diddledan | right-oh - any I should take a look at to get working? | 09:49 |
| popey | I put them in dropbox snapcrafters_deleted | 09:50 |
| diddledan | coolbeans | 09:50 |
| popey | the only one that would be nice to fix is wire | 09:50 |
| diddledan | yup | 09:50 |
| popey | it crashes, not looked at it for a while | 09:50 |
| diddledan | it might be fixed by the work jdstrand did on converting seccomp kills to EPERMs | 09:50 |
| popey | possibly, yes. | 09:51 |
| diddledan | also the electron/chrome-style peer to peer thingy works now | 09:51 |
| diddledan | (which fixed webtorrent!) | 09:51 |
| popey | yay | 09:51 |
| diddledan | not many files then : "Alan Pope just added 365 files"... | 09:52 |
| diddledan | https://usercontent.irccloud-cdn.com/file/iybhSeuM/image.png | 09:53 |
| mborzecki | zyga: intersting, 14.04 manpage for systemd mentions /run/systemd/notify as notification socket, yet snapd got something else | 09:53 |
| popey | :D soz | 09:53 |
| zyga | mborzecki: hmm | 09:53 |
| zyga | dunno :) | 09:53 |
| mborzecki | heh and /run/systemd/notify is not there in the system | 09:53 |
| pedronis | the systemd enablement in 14.04 was a bit of an adventure, not sure the manpages were updated | 10:12 |
| zyga | so I can get my apparmor changes to work | 10:16 |
| zyga | but with some manual tweaking | 10:16 |
| zyga | there's something wrong with an expression apparently | 10:17 |
| mup | PR snapcraft#2096 opened: tests: don't use os_release and repo from snapcraft <Created by kalikiana> <https://github.com/snapcore/snapcraft/pull/2096> | 10:18 |
| mborzecki | runnig systemd spread again, hopefully all of it works this time | 10:19 |
| mborzecki | s/systemd/watchdog/ | 10:19 |
| mborzecki | also tiny default in Result property of services, when killed by watchdog i've seen it to be signal, or watchdog (recent sytemd) or core-dump (since it got SIGABRT) | 10:20 |
| zyga | OMG | 10:24 |
| zyga | so weird | 10:24 |
| zyga | I mean | 10:24 |
| zyga | apparmor parser and apparmor | 10:24 |
| zyga | ok, fixed one more thing and running spread | 10:31 |
| zyga | * matches files but not directories | 10:31 |
| zyga | so to match anything in the /some/path | 10:31 |
| zyga | you need /some/path/{*,*/} | 10:31 |
| zyga | weird, huh? | 10:31 |
| zyga | ** matches anything at all so it will match /some/path/dir/evil | 10:31 |
| zyga | mborzecki, Chipaca: can you please review the user mounts PR, it is the oldest one around | 10:52 |
| zyga | I'd like to land it but I think it deserves one more look | 10:52 |
| cjwatson | ~/wg 54 | 10:54 |
| cjwatson | sorry | 10:54 |
| mborzecki | hmm NOTIFY_SOCKET on ubuntu-14.04 changes with each reboot, @/org/freedesktop/systemd1/notify/13334051644891137417, the last element looks like a timestamp, meaning we cannot just put the path in apparmor rule :/ | 11:07 |
| zyga | well | 11:08 |
| zyga | we can just put @/org/freedesktop/systemd1/notify/* | 11:08 |
| zyga | , | 11:08 |
| mborzecki | yeah, just have the fact that it becomes a special case | 11:10 |
| Kone | Hi, can someone explain/link how i can set start parameters for an app that is installed as a snap-package? | 11:29 |
| mup | PR snapd#5079 opened: tests: update the bionic image to the latest one uploaded <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5079> | 11:29 |
| === pstolowski is now known as pstolowski|lunch | ||
| mborzecki | well, ubuntu 14.04 systemd watchdog does not appear to work at all | 11:40 |
| zyga | mborzecki: maybe it is compiled out partially? | 11:42 |
| mborzecki | no clue, the service is not pinging systemd and it's not getting killed | 11:43 |
| * cachio afk | 11:45 | |
| mborzecki | fwiw it works on 16.04+ | 11:47 |
| jdstrand | if the systemd-notify denial is @/org/freedesktop/systemd1/notify/13334051644891137417, you'll need a unix rule for that. eg: unix (receive, send) type=??? peer=(addr=@/org/freedesktop/systemd1/notify/[0-9]&, label=unconfined), | 12:00 |
| jdstrand | err | 12:00 |
| zyga | type=??? | 12:00 |
| jdstrand | ...addr=@/org/freedesktop/systemd1/notify/[0-9]*... | 12:00 |
| jdstrand | I was going to say, I don't have the whole denial so can't say what it should be exactly | 12:01 |
| jdstrand | I suspect type=stream, but I don't know | 12:01 |
| zyga | ah, I get it now | 12:02 |
| zyga | jdstrand: btw, I got that apparmor thing to work | 12:02 |
| zyga | from last week | 12:02 |
| zyga | just iterating to simplify now | 12:02 |
| zyga | with spread passing and everything | 12:02 |
| jdstrand | zyga: as for your CPU pegging profile, perhaps file a bug and try to write a rule that works better | 12:02 |
| zyga | it was missing no-simplify-expr | 12:02 |
| zyga | it's sad that is on | 12:02 |
| zyga | it's pretty much broken IMO | 12:02 |
| jdstrand | zyga: aiui, its utility is profile dependent. idr the details, but jjohansen would | 12:04 |
| jdstrand | zyga: please test the rule on armhf though | 12:04 |
| jdstrand | s/rule/profile/ | 12:04 |
| zyga | jdstrand: ack, will do | 12:05 |
| jdstrand | zyga: we shouldn't have to worry about the performance of certain rules, but the reality is that some things compile faster than others | 12:06 |
| zyga | the current profile compiles very quickly but I will test it on arm as well | 12:07 |
| mup | PR snapd#5080 opened: many: support 'system' nickname in interfaces <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5080> | 12:08 |
| mborzecki | off to pick up the kids from school | 12:08 |
| mborzecki | jdstrand: i've updated the interface in https://github.com/snapcore/snapd/pull/4504/commits/6206e555fdbcd94680aa98103cc1aa50b120b840 | 12:10 |
| mup | PR #4504: snap, wrappers: systemd WatchdogSec support <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/4504> | 12:10 |
| === seb128_ is now known as seb128 | ||
| === pstolowski|lunch is now known as pstolowski | ||
| zyga | jdstrand: hey, I published working fix for the apparmor + layouts issue from last week https://github.com/snapcore/snapd/pull/5074 | 13:26 |
| mup | PR #5074: many: allow mimics in parents of layout <Blocked> <Created by zyga> <https://github.com/snapcore/snapd/pull/5074> | 13:26 |
| zyga | jdstrand: I marked the branch as blocked so please don't review it | 13:26 |
| zyga | jdstrand: I will split it into a smaller chunk that I will ask you to review | 13:27 |
| zyga | jdstrand: I also simplified the "xmas tree" code into a useful generic function that splits a path into read/write rules | 13:28 |
| * zyga loves this week (starts well) | 13:34 | |
| zyga | but hates this evening (rain for the rest of the week starts today) | 13:34 |
| zyga | no more biking :( | 13:34 |
| zyga | biking in the rain when it is cold is an acquired taste I lack | 13:34 |
| mup | PR snapcraft#2097 opened: packaging: include changelog for setup.py's version detection <bug> <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2097> | 13:37 |
| mup | PR snapcraft#2098 opened: lxd: wait for on-going refreshes to finish <Created by kalikiana> <https://github.com/snapcore/snapcraft/pull/2098> | 14:10 |
| mup | Issue snapcraft#2099 opened: Issues building snap from my favorite and best GUI RSS reader, Feedreader <Created by sirredbeard> <https://github.com/snapcore/snapcraft/issue/2099> | 14:13 |
| ads20000 | Wimpress: could you get in touch with Discord about this (I think that's the right action here)? :) https://forum.snapcraft.io/t/discord-ptrace-apparmor-denials/5099/5?u=ads20000 | 14:13 |
| ads20000 | or popey | 14:13 |
| matlock | sergiusens I tried your advice on my snapcraft post, didn't sort it out, and then the forum blocked my post for spam for linking to github, so here is the status of the issue https://github.com/snapcore/snapcraft/issues/2099 | 14:22 |
| matlock | sergiusens I am running into 3 different issues trying 3 different approaches to build this snap, you suggested I just let cmake plugin handle everything, but if you will see #1 of my issues is that cmake is not properly handling nested cmake builds, thanks in advance for your assistance | 14:24 |
| zyga | jdstrand: the chop tree PR: https://github.com/snapcore/snapd/pull/5081 | 14:25 |
| mup | PR #5081: interfaces/apparmor: add chopTree <Created by zyga> <https://github.com/snapcore/snapd/pull/5081> | 14:25 |
| mup | PR snapd#5081 opened: interfaces/apparmor: add chopTree <Created by zyga> <https://github.com/snapcore/snapd/pull/5081> | 14:25 |
| Chipaca | pedronis: niemeyer: #1766248 fwiw | 14:30 |
| mup | Bug #1766248: snap install fails with 'cannot update disabled snap "core"' after 'snap refresh core' <snapd:New> <https://launchpad.net/bugs/1766248> | 14:30 |
| cpaelzer | where would I look for snap install logs? | 14:48 |
| cpaelzer | we are looking at a case wondereing why the install hooks did not run | 14:49 |
| cpaelzer | Saviq: ^^ that is for the libvirt.conf in multipass | 14:49 |
| kyrofa | cpaelzer, `snap changes` and `snap change` is the best I know of | 15:00 |
| Saviq | cpaelzer: `snap change --last=install` has «Run install hook of "multipass" snap if present» here | 15:02 |
| Saviq | nothing more | 15:02 |
| Chipaca | kyrofa: Saviq: 'snap change' is an alias of 'snap tasks' fwiw | 15:03 |
| kyrofa | Chipaca, i.e. snap tasks is the newer, shinier one? | 15:03 |
| kyrofa | Is there also a `snap task`? | 15:04 |
| Chipaca | kyrofa: we thought it better reflected what the thing does, yeah | 15:04 |
| Chipaca | kyrofa: NO. | 15:04 |
| kyrofa | Hahaha | 15:04 |
| Saviq | cpaelzer: mystery solved - install hook only runs on (d'uh) installs - not refreshes | 15:21 |
| cpaelzer | yep | 15:21 |
| cpaelzer | that matches what I just posted to the forum | 15:21 |
| cpaelzer | soemthing you can resolve on the next version | 15:21 |
| cpaelzer | glad I could help to spot that | 15:21 |
| cpaelzer | Saviq: thanks | 15:21 |
| Saviq | yup, will be fixed before we release | 15:21 |
| Saviq | cpaelzer: thanks! | 15:22 |
| ogra_ | hmm | 15:23 |
| ogra_ | https://forum.snapcraft.io/t/notepadqq-classic-confinement-request/5024 | 15:23 |
| ogra_ | does "snap install --classic" for a strict confined snap suddenly turn it into a classic snap ? | 15:23 |
| ogra_ | or is that guy seeing a bug ? | 15:24 |
| * ogra_ thought that wasnt possible at all | 15:24 | |
| Chipaca | ogra_: you can install strict snaps with --classic | 15:29 |
| cachio | niemeyer, hey, so I need a # SPREAD_GOOGLE_KEY to add to the .travis.yml for the spread-cron branches | 15:29 |
| Chipaca | ogra_: if your snap is built appropriately, it'll even work | 15:29 |
| cachio | niemeyer, not sure if this is the only one think we need | 15:29 |
| cachio | and the other issue that you mention is already fixed | 15:30 |
| cachio | the issue with the elopio keys that were deprecated on travis | 15:30 |
| niemeyer | cachio: Ack | 15:40 |
| jjohansen | zyga: yes simpify-expr is sub-optimal and needs to be rewritten, but on average it actually sitll improves most compiles. It isn't so important any more as other parts of the pipeline have been rewritten. | 15:45 |
| jjohansen | Fixing it is on the long to do list, it just always seems there are other more important things | 15:46 |
| zyga | jjohansen: I'd say it should be off by default, I haven't read the source to say what it does but it was spinning for 10m+ on my beefy box | 15:46 |
| jjohansen | zyga: I haven't seen it do that | 15:46 |
| jjohansen | can you shoot me the profile that triggered it | 15:47 |
| zyga | yes, let me re-create it | 15:47 |
| === chihchun is now known as chihchun_afk | ||
| mup | PR snapcraft#2100 opened: build_providers: new build provider using multipass <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2100> | 15:59 |
| zyga | jjohansen: http://paste.ubuntu.com/p/hPcBjdnSwd/ | 16:03 |
| Facu | question, if I install a snap as "classic", the process runing from it should be able to see the whole system, right? | 16:33 |
| nacc | Facu: yes | 16:33 |
| zyga | Facu: yes | 16:35 |
| Facu | nacc, zyga, it's strange... with sparkiegeek we're seeing that Pythons inside the snap are "installed" in a way that don't have access to system libraries... http://linkode.org/#iWFk2Akf4EU6owgbpH4hu1 | 16:39 |
| zyga | Facu: what I mean is that there's no confinement preventing that, what happens is that for classic snaps the PATH (or PYTHONPATH) is probably altered by snapcraft itself (snapd doesn't touch that) so the snaps "prefers" to see the things it shipped. | 16:40 |
| zyga | As classic snap can always see what is on the system if it wants to | 16:40 |
| nacc | Facu: right, you should check the PYTHONPATH from the snapped python | 16:40 |
| nacc | Facu: and e.g., you might need to munge the environment if you need access to the 'host' env | 16:41 |
| Facu | nacc, how can I check that? | 16:41 |
| Facu | nacc, it's something in the installed snap, or in snapcraft when being built? | 16:41 |
| nacc | Facu: something like `snap run --shell <snap>; python (or python3).` And at the interpreter, examimen sys.path ? | 16:41 |
| nacc | Facu: both :) | 16:41 |
| nacc | Facu: so you're a classic snap that wants to use python package from the host? | 16:42 |
| Facu | zyga, nacc, this "the snap prefer to see the things it shipped" may be useful for snapping apps done in Python, not so much for Python itself (as it's the case of pypy3) or fades, whose purpose is automate virtualenv usage | 16:42 |
| Facu | nacc, yeap | 16:43 |
| zyga | Facu: my point is that snapd doesn't touch that | 16:43 |
| zyga | it's snapcraft or the snap that needs to be investigated | 16:43 |
| zyga | snapd doesn't constrain anything | 16:43 |
| zyga | or doesn't change PATH | 16:43 |
| nacc | Facu: right, what zyga is saying | 16:43 |
| Facu | zyga, perfect! now I'm trying to understand how to fix this :) | 16:43 |
| zyga | Facu: look at the generated launcher inside your prime/ directory | 16:43 |
| zyga | it probably sets some environment | 16:43 |
| nacc | Facu: so it's by virtue of how snaps are built, that results in their runtime behavior | 16:43 |
| sparkiegeek | Facu: I think it's gonna need some twisting and poking of snapcraft to e.g. ditch the wrapper | 16:44 |
| nacc | Facu: so, tbh, we build our own python in our snap -- because you get xenial's python of course, in all envs. Which may or may not work in all host envs. | 16:44 |
| nacc | Facu: and because of things like you're seeing :) | 16:44 |
| zyga | AFAIR there's an option to disable the wrappers | 16:44 |
| zyga | but perhaps I'm wrong | 16:44 |
| zyga | I rarely use snapcraft and typically construct my test snaps by hand | 16:45 |
| nacc | but zyga is special that way :) | 16:45 |
| nacc | and seems to not be dogfooding :-P | 16:45 |
| zyga | well | 16:45 |
| Facu | jajaja | 16:45 |
| zyga | I'm dogfooding things snapcraft doesn't support yet :) | 16:45 |
| zyga | but yeah | 16:45 |
| nacc | heh | 16:45 |
| zyga | I actually would like a simpler snapcraft | 16:45 |
| zyga | that is less magic now | 16:45 |
| zyga | but that's a story for another day | 16:45 |
| nacc | zyga: +100 | 16:45 |
| nacc | zyga: it's 'smarter' now, but that has made it more fragile, it seems (and it's hard to know, IMO, what is changing each release from a snap owner perspective) | 16:46 |
| Facu | zyga, with "generated launcher inside prime dir" you meant this exec? prime/command-fades.wrapper | 16:47 |
| nacc | Facu: yeah, i think so | 16:47 |
| Facu | exec "$SNAP/usr/bin/python3" "$SNAP/bin/fades" "$@" | 16:47 |
| Facu | no PATH mangling | 16:47 |
| zyga | can you run snap run --shell that thing | 16:47 |
| zyga | and explore $PATH and $PYTHONPATH | 16:47 |
| nacc | that implies there' | 16:47 |
| zyga | and run your $SNAP/usr/bin/python3 | 16:48 |
| zyga | and explore that too | 16:48 |
| nacc | *there's a python3 in your snap, separate from the one in the core snap? | 16:48 |
| zyga | if you are building python from source it is doing one more thing | 16:48 |
| nacc | Facu: are you ? | 16:48 |
| zyga | when prefix is set to /foo it doesn't look in /usr | 16:48 |
| zyga | just into /foo | 16:48 |
| zyga | so that may be a side effect of your build | 16:48 |
| nacc | yeah, that's something we have done in git-ubuntu (on purpose) | 16:48 |
| Facu | nacc, I don't put (on purpose at least) a different python in my snap... what I'm packaging is just a lib/script that needs Python to run | 16:49 |
| Facu | it has a plugin: python | 16:50 |
| Facu | (because it needs Python to be run, right?) | 16:50 |
| nacc | oh maybe the plugin pulls it in; but i thought it didn't ... | 16:50 |
| nacc | Facu: i thought it used the core snap's python (again, i might be wrong) | 16:50 |
| Facu | nacc, it would be great if I can get that! the snap itself (the binary) will be much smaller | 16:51 |
| sparkiegeek | Facu: FTR, using the python inside the snap, from that wrapper, then manually adding /usr/lib/python3/dist-packages to sys.path does the right thing | 16:51 |
| nacc | sparkiegeek: right, that makes sense | 16:51 |
| nacc | sparkiegeek: i assume your code is defensive enough for when an expected dist-package is not present, btw ? :) | 16:51 |
| sparkiegeek | Facu, nacc: AIUI the python plugin always ships its own python because of the shenanigans that it does to Python to get the import path right | 16:51 |
| Facu | zyga, nacc, I'm not sure how to use "snap run --shell" | 16:52 |
| nacc | sparkiegeek: ah ... that could be | 16:52 |
| nacc | sparkiegeek: it did not, at some point :) | 16:52 |
| nacc | Facu: if you ran it, `snap run --shell <snap name>` | 16:52 |
| sparkiegeek | Facu: snap run --shell fades | 16:52 |
| nacc | it will put in a shell that is the same environemnt as your snap runs in | 16:52 |
| nacc | `env | grep SNAP` to see what is set | 16:52 |
| nacc | cd $SNAP to switch to the snap''s directory, etc. | 16:52 |
| Facu | it doesn't, just ends | 16:53 |
| nacc | Facu: what doesn't, what ends? | 16:53 |
| nacc | Facu: the shell won't say anything | 16:53 |
| nacc | Facu: it literally spawned a new shell process | 16:53 |
| zyga | are you inside the snap run --shell shell by any chance? | 16:53 |
| Facu | oh! | 16:53 |
| Facu | no prompt change or anything? | 16:53 |
| nacc | Facu: right :) | 16:53 |
| Facu | pff | 16:53 |
| zyga | I think we tried but it was not working well | 16:53 |
| zyga | or is not in stable yet | 16:54 |
| zyga | I forgot | 16:54 |
| nacc | yeah, it's not something that is easy to do, in some sense, because what the prompt will be is sensitive to the snap's env now | 16:54 |
| sparkiegeek | well, you might have a classic snap that wants to do something with $PS1 :) | 16:54 |
| nacc | right | 16:54 |
| nacc | or even to what 'bash' is :) | 16:54 |
| Facu | nacc, zyga, snap env: http://linkode.org/#Txhr9JNzC4iueywyfA3bx3 | 16:55 |
| nacc | Facu: what is sys.path by default (run `python3` then import sys\nprint(sys.env) or so | 16:55 |
| nacc | Facu: from what i can tell it's what sparkiegeek already said, though, the system paths are not in PYTHONPATH (sys.path at runtime) | 16:56 |
| sparkiegeek | $ $SNAP/usr/bin/python3 -c 'import site; print(site.PREFIXES)' | 16:57 |
| sparkiegeek | ['/snap/fades/169/usr', '/snap/fades/169/usr'] | 16:57 |
| sparkiegeek | I think that's the key point | 16:57 |
| Facu | being inside the snap shell (confirmed) it looks that python3 is from the system? | 16:57 |
| Facu | $ which python3 | 16:57 |
| sparkiegeek | Facu: $SNAP/usr/bin/python3 is what the wrapper uses | 16:57 |
| nacc | Facu: right, so PATH isn't changed for classic snaps (iirc) | 16:57 |
| nacc | Facu: unless you manually change it | 16:57 |
| Facu | sparkiegeek, nacc, this looks like the responsible: SNAP/usr/lib/python3.5/sitecustomize.py | 17:01 |
| Facu | forgot the $ | 17:01 |
| * sparkiegeek nods | 17:01 | |
| sparkiegeek | sergiusens: ^ that's deliberate right? | 17:01 |
| * zyga -> break | 17:02 | |
| nacc | sparkiegeek: Facu: so what we do in our yaml is in the environment: field for the app, we set PYTHONPATH to something specific (in our case, internal to the snap). You could just as easily set it to something outside the snap, e.g. the host's python env) | 17:02 |
| nacc | sparkiegeek: but i believe you are right, that is a deliberate thing (not 100%, i'm not a snapcraft developer :) | 17:03 |
| sparkiegeek | nacc: trying to determine the target host's python env at "compile"-time is fraught with error though :) | 17:03 |
| Facu | nacc, but me, as a developer doing stuff in snapcraft, I don't know the host's python env for the all users running my snap | 17:04 |
| nacc | you can do, iirc | 17:04 |
| nacc | ORIG_PYTHONPATH: "$PYTHONPATH" | 17:04 |
| nacc | in your environemnt yaml | 17:04 |
| nacc | then you'd need to have your own wrapper | 17:04 |
| nacc | now, i'm not sure if that's actually necessary, i'd wait for sergiusens :) | 17:05 |
| nacc | but i believe it would work | 17:05 |
| sparkiegeek | right, but once you go for 'have your own wrapper', the whole problem goes away AFAICS | 17:05 |
| sergiusens_ | sparkiegeek: not following, what is deliberate, using sitecustomize instead of PYTHONPATH? If that is the question, yes; there are multiple reasons, one is, using a staged python to build other parts, another a python application calling into another python application | 17:05 |
| nacc | sparkiegeek: yeah :) | 17:05 |
| mup | PR snapd#5079 closed: tests: update the bionic image to the latest one uploaded <Created by sergiocazzolato> <Merged by zyga> <https://github.com/snapcore/snapd/pull/5079> | 17:05 |
| sergiusens_ | like snapcraft being a python application, if PYTHONPATH were set and we called into bzr (which is also python2) a mess will occur | 17:06 |
| Facu | sergiusens_, we may have wandered badly around the original problem, maybe you have another POV on the original problem and may focus us better | 17:06 |
| sparkiegeek | sergiusens_: basically the issue we're seeing is that a classic snap is not using the system python interpreter, and doesn't have the system python's environment | 17:06 |
| === sergiusens_ is now known as sergiusens | ||
| Facu | sparkiegeek, which, written that way, may be a feature :/ | 17:07 |
| sparkiegeek | Facu: well... *classic* | 17:07 |
| sparkiegeek | sergiusens: so Facu makes 'fades' which allows you to maintain virtualenvs. It has a --system-site-packages flag, which is broken, because the 'system' that it's seeing is the one inside the snap, which ain't all that helpful | 17:07 |
| sergiusens | sparkiegeek: oh, yeah, the python plugin is meant to use an in_snap python; you'd need to roll your own for other things, then again, just `dump` | 17:08 |
| Facu | sergiusens, "dump"? | 17:08 |
| sergiusens | what if I `apt purge python`? will fades work? | 17:08 |
| sparkiegeek | Facu: https://docs.snapcraft.io/reference/plugins/dump | 17:09 |
| sergiusens | Facu: and https://docs.snapcraft.io/build-snaps/pre-built | 17:10 |
| zyga | pstolowski: I found one more interesting bug in layouts thanks to you. :-) we don’t preserve permissions on directories mimicked as writable | 17:10 |
| zyga | And some software complains when certain things get world writable | 17:10 |
| zyga | I will fix it next | 17:10 |
| Facu | sergiusens, sparkiegeek, ubuntu core already brings a Python, right? it's Py 3? does it have access to system's libs? may I use *that* from the snap? | 17:11 |
| sergiusens | Facu: yes, you can, but not if it is classic as there will be ABI breakages when using that on a newer system such as 18.04 | 17:12 |
| Facu | sergiusens, well, I should leave "classicness" at some point | 17:13 |
| sparkiegeek | Facu, sergiusens: right - that, plus it has similar issues | 17:13 |
| sparkiegeek | ['', '/snap/core/current/usr/lib/python35.zip', '/snap/core/current/usr/lib/python3.5', '/snap/core/current/usr/lib/python3.5/plat-x86_64-linux-gnu', '/snap/core/current/usr/lib/python3.5/lib-dynload', '/snap/core/current/usr/local/lib/python3.5/dist-packages', '/snap/core/current/usr/lib/python3/dist-packages'] | 17:13 |
| sergiusens | it should have the same issues with different paths | 17:13 |
| sparkiegeek | that's sys.path from the python in core | 17:13 |
| sparkiegeek | sergiusens: indeed | 17:14 |
| sparkiegeek | I mean, that's expected | 17:14 |
| Facu | sparkiegeek, I'm not sure if we have a clean good solution for this :/ | 17:21 |
| Facu | sparkiegeek, not even a reasonable hack | 17:22 |
| Facu | sparkiegeek, it will be great if there's a workaround a user (you) could do | 17:22 |
| sparkiegeek | Facu: well I think nacc's suggestion of ORIG_PYTHONPATH might be reasonable | 17:22 |
| nacc | sparkiegeek: Facu: the advantage of it is that it makes some sense that only the snap-generator knows about this nuance, and then the app wrapper itself | 17:23 |
| Facu | sparkiegeek, but my PYTHONPATH doesn't include system libs | 17:23 |
| nacc | oh it's your sys.path in the host environment, probably | 17:24 |
| sparkiegeek | Facu: sure, I mean it doesn't *have* to be exactly like that but could be something like ' run python from the host, grab it's sys.path, add them to the one inside the snap' | 17:24 |
| nacc | so your wrapper, could, technically do what sparkiegeek is suggesting | 17:25 |
| Facu | sparkiegeek, and that at "snap install" time, right? | 17:25 |
| sparkiegeek | Facu: no, that would be a wrapper for the command, which then invokes fades | 17:25 |
| Facu | ah, everytime it runs? | 17:25 |
| sparkiegeek | yes | 17:25 |
| nacc | Facu: right it's part of your (snapped) app's definition | 17:26 |
| nacc | which in this case, I think, makes sense | 17:26 |
| nacc | and your app code wouldn't need to change at all, hopefully, as it's all in the yaml and the wrapper | 17:26 |
| Facu | nacc, sparkiegeek, something like the following? export PYTHONPATH=`python3 -c "import sys, os; print(os.pathsep.join(path for path in sys.path if path.startswith('/usr')))"` | 17:33 |
| nacc | Facu: why do you only want what is in /usr ? | 17:35 |
| nacc | Facu: e.g., what if user is using some special python installation by default | 17:35 |
| Facu | nacc, makes sense; I should just filter out the empty one | 17:37 |
| nacc | Facu: yeah, i think that's sensible | 17:37 |
| kyrofa | zyga, if I install a snap and get "error: cannot install snap file: snap is unusable due to missing files; contact developer" | 17:41 |
| kyrofa | zyga, is there a way for me to determine what exactly is missing? | 17:41 |
| zyga | Yes | 17:41 |
| zyga | Just apps are inspected | 17:42 |
| kyrofa | zyga, oh, it was seeing if the app actually exists? | 17:42 |
| zyga | I think there is some more places where you can run a command and see | 17:42 |
| zyga | But I’m AFK now, chipaca would know | 17:42 |
| zyga | Check your declared app command | 17:43 |
| kyrofa | zyga, yep, that was it, thank you! Some nice feedback in that error (what the actual issue is) would be helpful | 17:44 |
| mup | PR snapcraft#2101 opened: errors: implement an always option to send to sentry <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2101> | 17:44 |
| Facu | sergiusens, nacc, sparkiegeek, do you know about any doc to how provide wrapper changes/replacement? | 17:45 |
| nacc | Facu: i did it with trial and error, but basiclaly in your app defintion, you chang ethe command: to point at some script you install into your snap | 17:47 |
| Facu | nacc, currently the "command" points to a Python script called "bin/fades"... it looks it's building the wrapper around that... you say that if I change it to "bin/fades_wrapper_for_snaps.sh" it will not build a wrapper around it? | 17:51 |
| * Facu tries | 17:51 | |
| nacc | Facu: well, it will, but the wrapper will call your script | 17:51 |
| nacc | Facu: (iirc) -- and then your script calls bin/fades | 17:51 |
| Facu | nacc, snapcraft crashes badly :/ http://linkode.org/#5b4uN7fV4HG2rbCTBkxtU3 | 18:02 |
| nacc | Facu: you have to be careful | 18:02 |
| nacc | 'python3' in your snap wrapper is running in the context of the snap | 18:02 |
| nacc | ah that's one reason not to do it in the wrapper i suppose | 18:03 |
| Facu | nacc, what if I do /usr/bin/python3 | 18:03 |
| Facu | ? | 18:03 |
| nacc | Facu: right, i'd try that | 18:04 |
| Facu | in any case, you say that's why it crashes now? | 18:04 |
| nacc | Facu: i'm not sure :/ -- but i'd use absolute paths for everything in your wrapper | 18:04 |
| Facu | absolute path fixed, snapcraft still crashing | 18:04 |
| nacc | Facu: hrm, it seems to think there is no pip installed in the part | 18:05 |
| nacc | (iiuc) | 18:06 |
| pstolowski | zyga: wow, amazing.... | 18:08 |
| niemeyer | Chipaca: #5066 finally reviewed.. | 18:09 |
| mup | PR #5066: overlord/snapshotstate/backend: introducing the snapshot backend <Created by chipaca> <https://github.com/snapcore/snapd/pull/5066> | 18:09 |
| Facu | mmm... snapcraft just crashes, without any change in the project | 18:11 |
| nacc | Facu: i was going to ask that next | 18:11 |
| Facu | ja! my bad! I was using it where I played with PYTHONPATH | 18:11 |
| nacc | Facu: :) | 18:11 |
| * Facu slappes himself | 18:11 | |
| Chipaca | kyrofa: hello | 18:56 |
| Chipaca | kyrofa: I'm past my EOD, but, when you get that 'cannot be installed' thing, the log will have the details | 18:56 |
| Chipaca | kyrofa: the 'journalctl -u snapd' log | 18:56 |
| kyrofa | Chipaca, ah, good to know, okay, thank you :) | 18:57 |
| Chipaca | kyrofa: it's not just the apps | 18:57 |
| Chipaca | it also checks that all of meta is sane | 18:57 |
| Chipaca | (and things that count as 'not sane' include vim scratch files for example) | 18:57 |
| Chipaca | kyrofa: let me know what you find | 18:58 |
| mup | PR snapcraft#2102 opened: tests: use a common cache for the integration tests <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2102> | 19:30 |
| mup | PR snapcraft#2103 opened: package: make use of snapcraftctl snapcraft features <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2103> | 21:33 |
| mup | PR snapcraft#2097 closed: packaging: include changelog for setup.py's version detection <bug> <Created by sergiusens> <Merged by kyrofa> <https://github.com/snapcore/snapcraft/pull/2097> | 21:51 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
