[02:02] <vivus> Hello all.
[02:03] <vivus> Are snaps basically LXC containers running different applications?
[03:46] <xnox> vivus, and so much more. some of the underlying security and confinment technology is similar, but that's only the basic building blocks. Cause there is a lot of mediation and support around installing/updating/upgrading snaps, allowing/disallowing snaps to do things on the host OS, and to/with other snaps to connect to each other. and the core systems, ship everything as the snap including bootloaders and kernel.
[03:47] <xnox> vivus, and that's definately so much more than just "containers" and "applications"
[03:47] <vivus> xnox: so how do I make my snap-installed IDE see my development environment such that I get code auto-complete, etc. ?
[03:49] <xnox> vivus, usually i would expect it to connect with a home plug to access user's home.
[03:49] <xnox> vivus, are you creating the IDE snap? or are you just using some IDE snap created by somebody else?
[03:49] <vivus> xnox: so I still need to install lots of dependencies on the main system then?
[03:49] <xnox> no, none.
[03:49] <xnox> well depends what you build.
[03:50] <vivus> its mostly hypothetical at this stage. Im seeing if I should switch from LXC to snaps
[03:50] <xnox> vivus, play around with examples to understand what snaps are. they are more like android/iphone apps then chroots/containers/vms.
[03:51] <xnox> since the snap app itself is read-only typically, but can have data in strict ways on the host system.
[03:51] <xnox> so e.g. there is golang provided as a snap, which gives one compiler.
[03:51] <vivus> xnox: by lots of dependencies I mean dev-environment deps. For example, let's say I install PyCharm. Now I want to use PyCharm with autocomplete for my Python environments. Where do I install the Python environment stuff?
[03:52] <xnox> typically in a pyenv in like ~/code
[03:52] <vivus> is the pyenv a snap also?
[03:52] <xnox> no
[03:52] <vivus> so that's on the host?
[03:52] <xnox> yeah... cause you'd need it read-write, don't you?
[03:53] <vivus> that seems like going a bit backwards for me then. all my dev stuff are isolated in LXC containers, because of the bloat they create on the host.
[03:53] <vivus> can i attach and modify a snap container?
[03:54] <xnox> right but pycharm itself could be a snap. and all of its dependencies would be inside the snap and do not leak on the host at all.
[03:54] <xnox> and you can have multiple things co-installed which use same deps, but different versions, without any conflicts on the host.
[03:54] <vivus> yeah but say I want to install a python package that has lots of system dependencies, like Pillow, then my host would still accumulate system-deps just for Pillow
[03:55] <xnox> this is generic development stuff. you would not be installing those as snaps typically, no. you would use some form of confinement.
[03:55] <xnox> but you would not need to install it on the host into /usr
[03:56] <xnox> vivus, and e.g. Pillow -> i'd be installing that in a virtual env, in project directory in a reproducible way. https://docs.python.org/3/tutorial/venv.html
[03:57] <xnox> cause with python, one typically creates a virtual-env which contains this project dependencies only, and develops / runs / tests code there.
[03:57] <vivus> xnox: yeah, but Pillow and some other Py packages will sometimes need system-deps to compile
[03:57] <xnox> true
[03:58] <vivus> now say you have to do this for Python, and then maybe Node too. Creates lots of bloat
[03:58] <vivus> The other day I had to install mono and I used isolation to specifically avoid the bloat
[03:58] <xnox> maybe you want to look at snapcraft
[03:59] <vivus> *on the host
[03:59] <xnox> that's tooling to build snaps, with isolation, which has a lot of tooling to use lxd containers to build all the layers/dependenices of your app, including your app, and bundle everything as a snap.
[03:59] <vivus> Perhaps I can create the snaps myself. but they are immutable right? I cannot `sudo lxc-attach -n snapcontainer` right?
[04:00] <xnox> but that's mostly for "from scratch build & deploy" scenario. not meant for interactive coding/iterations.
[04:00] <xnox> vivus, snap filesystem structure, is a bind-mounted squashfs -> so no, attaching into its mountspace will not gain you much.
[04:00] <xnox> and paths and prefixes may look odd to you
[04:01] <xnox> vivus, what's your actual problem and why did you start looking at snaps? =)
[04:02] <xnox> vivus, are lxd containers not working out for you? (i find $ lxc launch ubuntu:cosmic etc so much nicer than old lxc-* commands)
[04:02] <xnox> vivus, have you used lxd already?
[04:08] <vivus> xnox: ill reply in 5. just afk
[04:14] <vivus> xnox: the distro I use doesn't have first-class support for LXD, but I've been using LXC for a few years now, so that's not the pain-point. I was hoping I'd get to use some type of containerized desktop IDEs that can help me do code auto-complete, etc.
[04:15] <vivus> right now what I use is Vim
[04:15] <vivus> and I kind of prefer mouse interaction at times
[04:19] <Doctor_Nick> if anyone could help me out with this in this post, i would greatly appreciate it, I need to kick this out the door soon: https://forum.snapcraft.io/t/getting-a-part-build-artifact-into-another-parts-build-before-pull/7082
[05:04] <mborzecki> morning
[05:40] <zyga> o/
[05:40] <zyga> good morning
[05:44] <mvo> hey zyga
[05:44] <zyga> hey, how are you doing?
[05:48] <mvo> zyga: doing well, thank you
[05:48] <mvo> zyga: how are you?
[05:52] <mborzecki> zyga: mvo: hey
[05:58] <mvo> mborzecki: hey hey
[06:22] <mborzecki> zyga: pushed the changes to https://github.com/snapcore/snapd/pull/5713
[06:23] <zyga> looking
[06:24] <zyga> there is a conflict with your other patch in master now
[06:24] <mborzecki> zyga: i'm really bad at naming things, feel free to suggest something different than ExpandSnapMountVariables()
[06:24] <zyga> ExpandSnapVariables(perspectiveHost,perspectiveSnap) # maybe?
[06:24] <zyga> that is maybe a flag rather than a new function
[06:25] <zyga> because then anyone using it must face the fact that perspective matters
[06:25] <zyga> but I didn't read all so just quick reaction
[06:25] <mborzecki> idk, i'm open for alternatives
[06:26] <zyga> let's use the word perspective for that across the tree
[06:26] <zyga> though not sure if it is sufficient:
[06:27] <zyga> there's the outside perspective (host perspective?)
[06:27] <zyga> there is the inside perspective for strictly and devmode confined snaps
[06:27] <zyga> (perhaps for classic snaps the inside perspective should just error or be the same as outside perspective)
[06:27] <zyga> mvo: btw: did we decide to do something about "snap install classic"
[06:27] <zyga> mvo: I had some ideas on that
[06:28] <zyga> mborzecki: but then there's the inside perspective of instance snaps, what should is say for $SNAP - /snap/foo/42 or /snap/foo_bar/42
[06:28] <mborzecki> zyga: hm that sounds good
[06:28] <zyga> mborzecki: if the instance aware path is rare we could do ExpandSnapVariables(innerPerspective | instanceAware)
[06:28] <mvo> zyga: snap install classic is on my todo
[06:29] <zyga> mborzecki: but normally just pass innerPerspective
[06:29] <mvo> zyga: funny enough it will work, it will just be the wrong classic
[06:29] <zyga> mborzecki: from what I guess there's just three places that need to be instanceAware
[06:29] <mborzecki> zyga: and it has an upside that i could push this as a pr outside of 5713
[06:29] <mvo> zyga: it will pull in "core"
[06:29] <zyga> mvo: right :)
[06:29] <zyga> mvo: remember that on core18 all the snaps use pivot_root
[06:29] <zyga> so yeah :)
[06:33] <mborzecki> zyga: naming aside, please take a look at the rest of the changes and the spread test and i'll start looking into this change with perspective flag
[06:36] <zyga> I
[06:36] <zyga> OK
[06:50] <zyga> the instance layout test is very nice
[06:56] <mborzecki> zyga: do you think i could add something to the content interface test?
[06:56] <zyga> reading it now
[06:58] <abeato> mvo, hey, would it be possible to make sure https://forum.snapcraft.io/t/pulling-network-online-target-as-prerequisite-target-slows-down-starting-services/6063 does not happen on UC18?
[07:00] <mborzecki> zyga: omg, found some bad merge
[07:01] <mvo> abeato: indeed, sorry, it is still on my TODO let me move it up
[07:01] <zyga> mmm? where?
[07:01] <mvo> mborzecki: yeah, tell us more
[07:01] <mborzecki> mvo: my code :) no worries
[07:01] <mvo> puhhh :)
[07:01] <abeato> mvo, awesome, thanks a lot
[07:01] <mborzecki> zyga: in info.go, ExpandSnapMountVariables, around SNAP_DATA
[07:02] <zyga> I'm not there yet
[07:02] <mborzecki> hm wonder why it was picked up by vet on travis, but not locally, iirc go test is supposed to run vet now too?
[07:04] <zyga> is it? I was never using go vet automatically, it's always manual
[07:06] <mborzecki> zyga: https://golang.org/doc/go1.10#test-vet
[07:06] <zyga> ah,go 1.10!
[07:07] <zyga> so I was using it without being aware of it
[07:07] <mborzecki> they mention high confidence checks though, idk how unreachable code is not among those
[07:08] <zyga> I'd love unused code detector
[07:08] <mborzecki> zyga: try gometalinter
[07:08] <zyga> e.g. public functions that are not used
[07:08] <mborzecki> there's deadcode checker
[07:09] <zyga> yeah but $standard vs $toolbox
[07:09] <zyga> I agree I'm just lamenting it is not a standard issue feature
[07:10] <mborzecki> btw. deadcode actually picks up some stuff in the project, just didn't open any PRs yet
[07:14] <pstolowski> morning
[07:15] <zyga> mborzecki: https://github.com/snapcore/snapd/pull/5713/files#r213925025
[07:15] <zyga> the path discussion summarized
[07:15] <zyga> hey pawel
[07:15] <mvo> hey pstolowski
[07:15] <mborzecki> pstolowski: cześć
[07:19] <mborzecki> zyga: wonder if that's a bit too many flags
[07:19] <zyga> mborzecki: four?
[07:19] <zyga> really we need the two flags for perspective + one flag that "flips" the natural instance awareness
[07:19] <zyga> so three
[07:20] <zyga> but the reality is that I think we just need it
[07:25] <mborzecki> zyga: i'm taking yagni angle here, all the use cases are inside mount ns now https://paste.ubuntu.com/p/RZF98hg8hd/
[07:26] <zyga> snapshots need the outside path
[07:26] <zyga> inside you need both the instance and non-instance paths
[07:26] <zyga> (or we can just craft the instance paths with instance key manually
[07:26] <zyga> +1 on yagni though, if you can get it to work
[07:26] <zyga> we can add the override flag if needed
[07:28] <mborzecki> zyga: i'd expect snapshots to use snap.Info methods, those do instance-specific paths
[07:28] <zyga> interesting
[07:28] <zyga> yeah
[07:30] <mborzecki> zyga: to recap, the snap.Info.SomeSuchDir() return all instance path, then the package level helpers snap.MountDir() et al take a snapName, so you can do snap.MountDir(info.SnapName(), info.Revision)
[08:10] <mborzecki> zyga: hm need to update that symlink path in layouts test
[08:10] <zyga> ah, right
[08:10] <zyga> +1
[08:15] <mborzecki> zyga: actually something broke there with /etc/demo, i'm looking into this right now, will ping you if i need a hand
[08:15] <zyga> ok
[08:15] <zyga> what was the error?
[08:16] <mborzecki> /etc/demo/writable is not created in $SNAP_COMMON for some reason
[08:16] <mborzecki> zyga: the fstab looks ok to me https://paste.ubuntu.com/p/wMw8hrWZmq/
[08:17] <zyga> mmm
[08:17] <zyga> ok
[08:18] <mborzecki> zyga: i can see /etc/demo/writable inside snap mount ns, but it's not in $SNAP_COMMON/etc/demo for some reason (and not in /var/snap/test-snapd-layout_foo/common either)
[08:18] <zyga> is it because /etc is the real /etc and it is writable and then you don't get a private-namespace copy?
[08:18] <zyga> aka, trespassing bug
[08:19] <zyga> maybe we can revive my branch for that
[08:19] <mborzecki> zyga: mountinfo inside, https://paste.ubuntu.com/p/FXrp8Cmymc/
[08:19] <mborzecki> zyga: hm it worked before fwiw
[08:20] <zyga> 894 737 8:1 /var/snap/test-snapd-layout/common/etc/demo /etc/demo rw,relatime master:1 - ext4 /dev/sda1 rw,data=ordered
[08:20] <zyga> so /etc/demo is /var/snap/test-snapd-layout/common/etc/demo
[08:22] <zyga> anyway, I'll revive it
[08:22] <zyga> I need to land htat
[08:23] <mborzecki> zyga: hmm https://paste.ubuntu.com/p/3MXW6fz5RC/ what if it's not? :)
[08:24] <zyga> and on the host :D
[08:24] <zyga> because this really tells you what happens
[08:24] <zyga> ext4 filesystem on /dev/sds1, the slice at /var/snap/test-snapd-layout/common/etc/demo is at /etc/demo
[08:25] <zyga> 977 764 8:1 /var/snap/test-snapd-layout_foo /var/snap/test-snapd-layout rw,relatime master:1 - ext4 /dev/sda1 rw,data=ordered
[08:25] <zyga> this hides it from your namespace (it's later in the sequence)
[08:25] <zyga> (sequence matters)
[08:25] <zyga> but just check on the host
[08:25] <zyga> am I right?
[08:27] <mborzecki> hm on the host, /etc/demo is there, /var/snap/test-snapd-layout_foo/common/etc/demo/writable is not, but /var/snap/test-snapd-layout/common/etc/demo/writable is and contains what was written by the _foo instance :/
[08:27] <mborzecki> zyga: ^^
[08:27] <zyga> right
[08:27] <zyga> that's what I expected
[08:27] <zyga> I mean, mountinfo doesn't lie
[08:27] <zyga> it's just non-obvious what the outcome is
[08:29] <mborzecki> zyga: heh, ok, i'll just leave a comment in the test that this part blows up too
[08:39] <zyga> mborzecki: I'm looking at my trespassing fix onw
[08:39] <zyga> the part I got stuck on is something you could help me with
[08:39] <zyga> if you want we can HO today and I'll walk you through it
[08:39] <zyga> maybe the solution will become obvious as we d oit
[08:39] <MattJ> Is there a way I could grant a snap read-only access to /etc/ssl without going all the way and changing it to a classic one? Is there an interface I'm missing? If not, is there a potential for one to be added?
[08:40] <zyga> MattJ: let me look
[08:40] <mborzecki> zyga: ok, sure
[08:41] <zyga> so
[08:41] <zyga> MattJ: the network interface does that
[08:42] <zyga> but you need to know that /etc/ssl is not from the host
[08:42] <zyga> it is the one from the base snap of your app
[08:42] <zyga> so you cannot add ssl certs by adding them to your host
[08:43] <MattJ> Ah, ok :/
[08:44] <MattJ> Thanks - do you see any easy way around this or should I give up and switch to classic confinement?
[08:44] <zyga> it depends on what you want to do
[08:44] <zyga> you can add ssl certs from your snap to your snap's runtime
[08:44] <zyga> that is
[08:44] <MattJ> Another thing I thought was if I could just run one command without confinement to import the certs to the data path or something
[08:44] <zyga> any given snap can add certs for itself
[08:45] <MattJ> Well it's a server process, but it needs valid certificates for the domain it is serving
[08:45] <MattJ> Most people these days will already have these via Let's Encrypt/etc.
[08:45] <zyga> mmm
[08:45] <zyga> where do you need that cert to be exactly?
[08:46] <MattJ> I need it anywhere that can be read by the server process launched in the snap
[08:46] <zyga> copy it to $SNAP_COMMON
[08:46] <zyga> and have your snap read it from there
[08:46] <MattJ> I guess I could provide a script in the snap that does this, and the user could execute it manually
[08:48] <MattJ> Thanks for the thoughts :)
[08:50] <Chipaca> MattJ: a script can detect if it's being run or being sourced, and error if it's being run, with an error along the lines of 'you need to source this'
[08:51] <Chipaca> MattJ: in bash, that is: check whether ${BASH_SOURCE[0]} is $0
[08:51] <MattJ> This would probably need to be actually run as root, from cron
[08:51] <MattJ> But thanks for the tip
[08:51] <Chipaca> MattJ: systemd timers ftw
[08:53] <MattJ> Sure, though that can't be installed by the snap... but I guess it could be installed by this script, hmm :)
[08:56] <Chipaca> MattJ: OTOH you're not the only person asking us to be able to add certificates
[08:57] <MattJ> I think it's easier for web-based apps that may be the only thing on the system, they can bind to port 80 and include certbot in the snap
[08:57] <Chipaca> so maybe we need to think about a more general solution
[08:58] <Chipaca> MattJ: if your snap was installed on a core device, where /etc/ssl is readonly, what would your users do?
[08:59] <MattJ> Probably install some Let's Encrypt/certbot snap
[09:00] <MattJ> (I would like the snap to work on a core device for sure, so I would also like a solution for that - but I haven't got that far yet)
[09:01] <Chipaca> MattJ: so from your POV getting the cert is never your snap's concern, it always needs to come from outside
[09:02] <Chipaca> MattJ: why would the copying need to be put on a timer? because presumably the certs get updated periodically in some other place and the whole tree needs to be in sync?
[09:03] <t1mp> should the app version number always be "hard-coded" in the snapcraft.yaml file?
[09:03] <t1mp> I'd like to get it from my_python_package.__version__ instead
[09:04] <mborzecki> zyga: i'm looking at the log from test-snapd-layout_foo, https://paste.ubuntu.com/p/j5WSWJtyJH/ if the mount changes are pefroemd in the order they are logged in line 178 then this will be incorrect, won't it?
[09:04] <mborzecki> zyga: i mean i see it's doing all the layout stuff before parallel instance setup, although fstab lists parallel instance pieces first
[09:05] <zyga> hmmm
[09:05] <mborzecki> zyga: before it worked because layouts used instance-specific paths and the instance -> snap bind mount would not confclit
[09:05]  * zyga looks
[09:06] <zyga> aha
[09:06] <zyga> interesting
[09:06] <zyga> hmmm
[09:06] <zyga> right
[09:06] <zyga> we sort them in a specific way
[09:07]  * zyga thinks
[09:07] <mborzecki> hm parallel-instance then should come first?
[09:07] <Chipaca> t1mp: you can tell snapcraft to grab the version from elsewhere
[09:07] <zyga> lines 162-190 of the log
[09:07] <zyga> not sure if that's correct
[09:07] <mborzecki> zyga: we have x-snapd-origin we could use
[09:07] <MattJ> Chipaca, Let's Encrypt certificates are only valid for 3 months, and are typically updated more often than that
[09:07] <zyga> yes but it's super super tricky what happens
[09:07]  * mborzecki looking
[09:08] <zyga> the rationale for the ordering is specific
[09:08] <zyga> I was thinking if we need to do batching
[09:08] <MattJ> Chipaca, so either the snap needs constant access to them, or they need to be imported periodically by something with access to them
[09:08] <Chipaca> t1mp: look up "snapcraftctl set-version", i think that'll lead you to the how
[09:08] <mborzecki> zyga: like parallel instances first, then layouts, then the rest (if needed?)
[09:09] <zyga> perhaps
[09:09] <Chipaca> MattJ: yeah, that's what i thought
[09:09] <zyga> but it's all non-trivial
[09:09] <mborzecki> zyga: computing changes would be tricky
[09:09] <zyga> not sure if this remains correct
[09:09] <t1mp> Chipaca: thanks, reading.
[09:11] <zyga> mborzecki: what if we do a stable sort
[09:11] <zyga> and sort by (target, source)
[09:11] <zyga> not just by target
[09:11] <zyga> I think what is happening that is wrong is
[09:12] <zyga> we order things by where they appear
[09:12] <mborzecki> zyga:  i can try that
[09:12] <zyga> ignoring the what they bring aspect
[09:12] <zyga> and what they bring is also relevant
[09:12] <zyga> we need to correctly order the desired state
[09:12] <zyga> but then the question becomes;
[09:12] <zyga> is this breaking any invariants the current code has
[09:13] <zyga> will it undo correctly
[09:13] <mborzecki> zyga: i'll do some experimentation and we can later meet in HO
[09:15] <mborzecki> zyga: heh, finny that it worked before by accident :)
[09:15] <mborzecki> and i was like, 'oh hey it works, so nice' ;)
[09:19] <zyga> mborzecki: I need to go to the post office
[09:19] <zyga> mborzecki: we can HO after that
[09:19] <zyga> ok?
[09:19] <mborzecki> sure
[09:19] <zyga> brb
[09:21] <mborzecki> niemeyer: hi, when you're around could you take a look at https://github.com/snapcore/snapd/pull/5735 ?
[09:22] <niemeyer> mborzecki: Looking
[09:23] <mborzecki> niemeyer: thanks
[09:29] <niemeyer> mborzecki: Reviewed, np
[09:34] <Chipaca> t1mp: this: https://forum.snapcraft.io/t/extracting-information-from-sources-in-snapcraft-parts/4642
[09:42] <t1mp> Chipaca: thanks.
[09:42] <t1mp> I see there is also a version-script property in the yaml file, but not a lot of docs on that yet.
[09:45] <zyga> re
[09:47] <mborzecki> zyga: i'm playing with this naiive change https://paste.ubuntu.com/p/K54npKZzFg/
[09:49] <zyga> sure but think about the consequences as well
[09:50] <mvo> zyga: do you think 5307 needs another look by jamie ? or can we squash it?
[09:50] <mborzecki> zyga: just verifying that moving those mounts to be firs fixes the issue at hand
[09:58] <zyga> mvo: looking
[09:59] <zyga> mvo: ideally jamie would look but he's been busy
[09:59] <zyga> also gustavo asked for +1 from jamie
[10:10] <mborzecki> zyga: the tests are passing now (expectedly)
[10:18] <t1mp> Chipaca: version-script looks like it will do what I want, but still the 'version' property is required. I assume that version-script will override that (but it is not documented what will happen). I guess I'll have to try it
[10:18] <niemeyer> WOAH
[10:18] <niemeyer> There's no a "Resolve conversation" button in GH..
[10:18] <niemeyer> So simple and so useful
[10:19] <pstolowski> pedronis: hey, wrt your comment about revert & Sequence after currentIndex, what're the semantics of Sequence?
[10:19] <niemeyer> And "Show resolved" and "Hide resolved".. OMG.. my day just got better
[10:21] <Chipaca> niemeyer: nice
[10:22] <niemeyer> https://usercontent.irccloud-cdn.com/file/uMovdf2m/image.png
[10:24] <mvo> xnox: that reminds me, can I do an sru for https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1778936 or do you plan to do one anyway where you could include this change?
[10:29] <ogra> mvo, hmm, you said SNAP_ARCH_TRIPLET was not implemented ... this one is very curious though ... :
[10:29] <ogra> $ snap run --shell qemu-virgil -c "/usr/bin/env"|grep SNAPCRAFT
[10:29] <ogra> SNAPCRAFT_ARCH_TRIPLET=x86_64-linux-gnu
[10:29] <ogra> seems the snapcraft var is actually inside the installed snap
[10:29] <ogra> (whihc is great but smells like it should not be like that)
[10:31] <mvo> ogra: I did not follow what snapcraft was doing there
[10:31] <ogra> mvo, well, somehow SNAPCRAFT_ARCH_TRIPLET ends up in my runtime env
[10:32] <ogra> that snap has no wrappers or anything so this is very curious
[10:33] <ogra> i mean, i appreciate that ... but it doesnt feel right that a SNAPCRAFT var ends up in my installed snap package
[10:34] <ogra> $ grep SNAPCRAFT /snap/qemu-virgil/current/command-qemu-virgil.wrapper
[10:34] <ogra> $
[10:34] <ogra> it is definitely not in the default wrapper
[10:34] <ogra> so i wonder where it comes from
[10:34] <ogra> gar
[10:35] <ogra> ignore that ... somthing exported it into the shell i'm in
[10:35] <ogra> (which is also curious ... should snap run --shell not wipe my env ? )
[10:42] <Chipaca> ogra: nope, only remove some of it
[10:51] <ogra> ah
[10:53] <mborzecki> pstolowski: left you some comments on #5680
[10:54] <pstolowski> mborzecki: ty!
[11:37] <zyga> hmm, if a test panics the panic breaks the test
[11:37] <zyga> but if a test panics and teardown uses c.Assert() that fails then the assertion error is show but the panic is gone
[11:38] <zyga> is this expected?
[11:58] <Chipaca> grr grr grr integration tests grr
[11:58]  * Chipaca ~> lunch
[12:11] <zyga> mborzecki: hey
[12:11] <zyga> do you want to HO?
[12:11] <mborzecki> sec
[12:11] <zyga> I'm progressing on the trespassing fix, I rebased it and solved a blocker I had the last time
[12:11] <zyga> now going through tests that ... well, need some love since syscalls changed
[12:12] <zyga> but it's mostly inserting a syscall in a chain and going on
[12:12] <mborzecki> zyga: ok, ready
[12:12] <zyga> standup?
[12:37] <pedronis> mvo: can #5583 be reviewed again?  I might not get to it today tough
[12:46] <ogra> hmm, an install hook should also run on upgrades, right ?
[12:47] <mvo> pedronis: yes, I updated it ~1h ago
[12:48] <pedronis> ok
[12:48] <mvo> pedronis: it has a minimal wait and tracks the idle connections now
[12:48] <mvo> s/tracks/checks/
[12:49] <pedronis> mvo: likely I will look at it tomorrow, trying to getting started with something today (but had a bit of a cut up day by errands so far)
[12:49] <pstolowski> ogra: no
[12:49] <mvo> pedronis: no worries, its not urgent
[12:49] <pstolowski> ogra: on upgrades we run pre-refresh and post-refresh hook sinstead
[12:49] <pstolowski> *instead
[12:52] <ogra> pstolowski, hmm, so i need two scripts if i want something to work on install as well as on upgrades ?
[12:52] <ogra> hooks/install and hooks/*-refresh ?
[12:53] <zyga> jdstrand: hey, there's a pr that is blocked on you (the one about /mnt)
[12:53] <zyga> jdstrand: do you think you will have time to review it today
[12:53] <pstolowski> ogra: yes
[12:53] <jdstrand> I got through some yesterday. will try to get to that one
[12:53] <ogra> boo
[12:54] <ogra> ok
[12:54] <zyga> jdstrand: thank you!
[12:54] <pstolowski> ogra: if the logic is the same you can move it to a helper.. or maybe a symlink will work (not sure)
[12:54] <ogra> well, i#ll just cp it ...
[13:33] <zyga> I must say I love our standups
[13:33] <mborzecki> guys, any clue why i'm not able to edit https://forum.snapcraft.io/t/security-policy-and-sandboxing/554 ?
[13:34] <zyga> it's not a wiki
[13:34] <zyga> the irony there hurts ;)
[13:35] <zyga> oh boy!!
[13:36] <zyga> I used 1TB of data this month
[13:36] <zyga> and I actually hit my cap
[13:36] <zyga> wow
[13:36] <zyga> (LTE cap)
[13:36] <zyga> I need better visibility into this, who is using this data ....
[13:37] <mborzecki> nasty neighbours :)
[13:38] <mborzecki> zyga: hm i can edit this https://forum.snapcraft.io/t/the-snap-format/698 but not the other one
[14:33] <axino> hi
[14:33] <axino> I have a snap containing some rust code, it compiles fine on amd64/i386 but not on arm64/armhf
[14:34] <axino> error is "error[E0658]: non-reference pattern used to match a reference (see issue #42640)"
[14:34] <axino> has anyone seen this ?
[14:34] <axino> https://launchpad.net/~build.snapcraft.io/+snap/17fc91e19f932bd8a75175923b533019-xenial builds are here
[14:39] <Chipaca> axino: is this specific to the snap?
[14:40] <axino> Chipaca: I can't answer that
[14:41] <Chipaca> axino: why not?
[14:41] <axino> Chipaca: I haven't tried to build sentry outside of the snap context
[14:41] <axino> Chipaca: or maybe I misunderstood your question
[14:42] <Chipaca> axino: the error does not seem to be about snaps at all
[14:43] <Chipaca> axino: I'm not sure how we could help
[14:43] <axino> Chipaca: maintainers of multi-arch rust snaps might have seen this behaviour
[14:46] <axino> I guess I'll try #rust
[14:56] <Chipaca> axino: if you don't have luck there, try a topic on the forum
[14:56] <axino> Chipaca: yup I will - thanks
[15:17] <Chipaca> mup: ping
[15:17] <Chipaca> mup: how much chug would a moose chug … wait
[15:41] <popey> jdstrand: I'm getting a snap suddenly getting rejected by the store, which was fine previously
[15:41] <popey> jdstrand: https://dashboard.snapcraft.io/snaps/scummvm/revisions/466/
[15:42] <popey> "unknown keys in snap/manifest.yaml: snapcraft-os-release-id,snapcraft-os-release-version-id,snapcraft-version lint-snap-v2_snap_manifest "
[15:43] <jdstrand> popey: tools needs an update. thanks!
[15:43] <jdstrand> popey: can you request a manual review?
[15:43] <jdstrand> popey: we'll get that fixed fast
[15:43] <popey> done, thanks
[15:48]  * zyga goes for a bike for 1..2 hours
[15:48] <jdstrand> popey: can you do the same for 466 (i386)?
[15:49] <popey> jdstrand: done
[15:49] <zyga> jdstrand: I'm making good progress on the trespassing bug (it was unblocked today), I'll bug you about that tomorrow
[15:49] <jdstrand> zyga: great. fyi, I'm off tomorrow/Monday (will send email later)
[15:50] <zyga> ah
[15:50] <zyga> ok, then on Tuesday
[15:50] <zyga> can you try to look at https://github.com/snapcore/snapd/pull/5307 - you're the last +1 needed :)
[15:50] <zyga> (regardless, I'm heading out)
[15:51] <jdstrand> zyga: yes, we talked about that already today :P
[15:51] <zyga> I know, sorry :|
[15:53] <diddledan> snapd 2.35 is installing the .snap files as root:root 600 - previous snapds installed root:root 644
[15:53] <diddledan> caused an error that I've documented in the cft for snapcraft 2.43: https://forum.snapcraft.io/t/call-for-testing-snapcraft-2-43/7024/5
[15:53] <popey> i had this the other day too
[16:00] <Chipaca> diddledan: popey: that is known and fixed on master fwiw
[16:01] <Chipaca> master of snapcraft that is
[16:01] <diddledan> how can snapcraft, running as me, decide to override root-only access?
[16:05]  * diddledan eyes it suspiciously
[16:11] <Chipaca> diddledan: say again?
[16:11] <Chipaca> diddledan: it doesn't
[16:11] <Chipaca> diddledan: it no longer tries to copy those files into the container
[16:12] <diddledan> you said it was fixed in snapcraft, but snapcraft can't override the root-only access
[16:12] <diddledan> aah
[16:12] <diddledan> gotcha
[16:12]  * diddledan removes suspicious eyes
[16:12] <Chipaca> diddledan: in the future we'll bring the caching back by adding api to snapd to stream the snaps
[16:12]  * diddledan inserts googley eyes instead
[16:12]  * Chipaca whacks diddledan over the head just watch the googley eyes bobble
[16:13]  * ogra steals diddledan's goggley eyes and replaces them with xeyes for nostalgic reasons
[16:13] <diddledan> https://www.youtube.com/watch?v=BBFqGHgCFiY
[16:13]  * popey runs wayland on diddledan  to break the xeyes
[16:13] <diddledan> haha
[16:14]  * ogra secretly shoves XWayland underneath popey's injection ... just to notice the xeyes became stills due to securit reasons
[16:14] <ogra> +y
[16:15] <Chipaca> clearly the way to have it  work again is to make diddledan be all eyes
[16:15] <jdstrand> roadmr: hi! can you pull r1123 of the review-tools. this should be considered fairly urgent since anything using a new snapcraft will end up in manual review
[16:16] <jdstrand> roadmr: ideally done today since I'm off tomorrow/Monday
[16:16] <ogra> http://www.davidmelling.co.uk/blog/wp-content/uploads/2012/05/monster5-e1337951522979-1024x602.jpg
[16:16] <jdstrand> roadmr: fyi, https://paste.ubuntu.com/p/553Jg578yF/
[16:17] <jdstrand> roadmr: (r1123 changed sr_common.py)
[16:17] <jdstrand> popey: that has your fix ^
[16:17] <popey> yay
[16:17] <popey> ta
[16:19] <Chipaca> fair warning, a lot of our argentine colleagues might be a bit distracted these days
[16:19] <ogra> we should just pay them in euros then
[16:20] <ogra> (or is there more than the crazy inflation ?)
[16:20] <Chipaca> ogra: it's the chaos and turmoil that with this
[16:21] <ogra> yeah
[16:38] <Chipaca> could I get a review or two of https://github.com/snapcore/snapd/pull/5744 ? reasonably straightforward (might even qualify as Simple)
[16:39] <Chipaca> tests are green despite what github says (at least  here)
[16:39] <sergiusens> niemeyer, kyrofa: wrt out conversation from Monday, are we good on s/templates/extensions/ ?
[16:40] <niemeyer> So far it sounds good
[16:45] <sergiusens> niemeyer: great, we will rename them
[16:52] <roadmr> jdstrand: hi! I'll put it in the pipeline
[17:10] <popey> jdstrand: i take it you've seen the review tools themselves got rejected? :)
[17:11] <popey> (also, lol)
[17:11] <roadmr> haha
[17:11] <mvo> heh
[17:17] <zyga> Returned
[17:17] <zyga> TIL police does not arrive even after 25 minutes of waiting
[17:26] <ogra> oh gosh ... thats a lot of additional apt sources in that "16.04 snap missing" thread !
[17:26] <ogra> ... what could possibly go wrong ...
[17:34] <roadmr> zyga: why did you call police? 😱
[17:54] <zyga> roadmr: some drunk assholes were harassing a young couple
[17:54] <roadmr> oh not nice :(
[18:04] <ogra> jdstrand, hmm ... i just had three weird rejects of a plain package rebuild ... " unknown keys in snap/manifest.yaml: snapcraft-os-release-id,snapcraft-os-release-version-id,snapcraft-version lint-snap-v2_snap_manifest"
[18:06] <ogra> jdstrand, https://dashboard.snapcraft.io/snaps/qemu-virgil/revisions/30/ (and 31, 32) in case you want to take a look ... it is built using build.snapcraft.io and there were no fancy changes or anything ... the former build went through for all arches
[18:08] <ogra> jdstrand, oh, ignore me, just saw the backlog ... i guess it will magically fix itself
[18:12] <Caelum> zyga: I'm getting that exact fail on my tw server with 'osc build'
[18:12] <zyga> Caelum: hey
[18:12] <zyga> Caelum: I talked to some golang people from suse
[18:13] <zyga> Caelum: apart from some issues in test suite (unicode vs ascii output) I'm re-working the use of golang helpers
[18:13] <zyga> Caelum: tl;dr; version is that people that make golang helpers recommend not to use golang helpers
[18:13] <Caelum> zyga: so you found the issue, was it a segfault or something like that?
[18:14] <Caelum> zyga: yeah figures
[18:14] <zyga> Caelum: ish, I have not confirmed it on pure i586 system
[18:14] <Caelum> zyga: the go build stuff is very weird to be honest
[18:14] <zyga> "go build" is very nice but %go_build is horrible
[18:29] <Caelum> zyga: honestly I've never yet built snapd with the regular tools, the patches I ran through osc
[18:29] <Caelum> zyga: but I'm going to try again now
[18:39] <zyga> Caelum: actually it's pretty easy, just go build :)
[18:39] <zyga> Caelum: anyway, I'm reading now, but I will push updates to my home branch
[18:42] <Caelum> zyga: looks like ./run-checks is all I really need
[18:49] <Caelum> zyga: looks like fixing things for gentoo is going to be non-trivial, already ran into a needed dependency that gentoo doesn't have
[19:14] <jdstrand> roadmr: thanks for putting it in the pipeline-- do you have an eta?
[19:16] <roadmr> jdstrand: I'll do my best to roll out today but no promises; if not, it'd be until Monday/Tuesday
[19:16] <jdstrand> roadmr: ok, thanks
[19:17] <jdstrand> roadmr: Monday/Tuesday is going to be a problem. basically, all snapcraft.io/LP builds are getting blocked
[19:18] <jdstrand> roadmr: I apologize for this. I wasn't aware of the changes
[19:19] <roadmr> :(
[19:19] <roadmr> I'll do my best to get this out today
[19:19] <popey> sorry, but +1 to getting this fixed asap. We're already getting mails about stuff failing
[19:35] <cjwatson> Do we need to revert LP's snapcraft?
[19:35] <cjwatson> (Is it a snapcraft change that caused this?  I have no idea TBH.)
[19:58] <jdstrand> cjwatson: yes, snapcraft
[19:58] <jdstrand> cjwatson: but lets hold off on the revert and see what roadmr can do
[19:59] <roadmr> jdstrand, cjwatson : if it's easyish to revert, it might be faster than what I can do :(
[19:59] <jdstrand> I don't personally have a preference, but I don't know what snapcraft is fixing
[20:00] <roadmr> jdstrand: at this point the store release pipeline involves a CI run which might take 2 hours, then requesting the rollout which, assuming someone from IS is around to process and I nag them hard enough, might take another hour. But it's already 4 PM for me so that puts me past EOD (my son is very strict in enforcing EOD)
[20:00] <roadmr> jdstrand: we put a veto on Friday releases since last Friday's broke the world :D so if not done today, that's why it would have to wait for Monday. Monday is Labor Day in some countries so coverage will be low as well
[20:00] <roadmr> (I'm off on Monday for one, and so are you :D)
[20:00] <cjwatson> roadmr: unless I'm missing a quicker way, we'd do it by building a reverted version with a higher version number and put it in the snappy-dev/ubuntu/tools PPA
[20:01] <jdstrand> cjwatson: would it be possible to revert until next Monday/Tuseday when roadmr pings us that the fix is in place?
[20:01] <cjwatson> That doesn't require (or even really benefit from) an LP person being available, which is a good thing since I'm about to walk out the door
[20:01] <cjwatson> Anyone in ~snappy-dev could do it
[20:01] <jdstrand> I'm in snappy-dev
[20:02] <cjwatson> This works because LP puts "deb http://ppa.launchpad.net/snappy-dev/tools/ubuntu %(series)s main" (with the appropriate series substituted for "%(series)s", but it's usually xenial and in particular it's xenial for all build.snapcraft.io builds) in the sources.list for every snap build
[20:02] <cjwatson> For exactly this sort of reason
[20:02] <pedronis> how did this snapcraft got throught anyway? should have it broken some store integration tests?
[20:03] <jdstrand> cjwatson: https://launchpad.net/~snappy-dev/+archive/ubuntu/tools seems to have ancient stuff...
[20:03] <cjwatson> Yes, it hasn't been actively used for a while
[20:03] <cjwatson> But it's still in sources.list
[20:03] <jdstrand> cjwatson: oh, you're saying take the last xenial from -updates and shove it there
[20:04] <jdstrand> (with proper versioning, etc)
[20:04] <cjwatson> It needs to have a higher version number than anything else in the snap build's sources.list, but yes
[20:04] <jdstrand> sure
[20:04] <cjwatson> I mean you should probably alert the snapcraft team that you're doing this
[20:04] <cjwatson> o hai
[20:04] <roadmr> once that thing is in the PPA, how does Launchpad pick it up? magic?
[20:05] <jdstrand> pedronis: that will be my next question. snapcraft should definitely have some tests that run snaps it produces through the review tools
[20:05] <sergiusens> cjwatson, roadmr: what's up?
[20:05] <cjwatson> roadmr: snap builds are dispatched to builders with a sources.list that includes that PPA
[20:05] <cjwatson> so not particularly magic
[20:05] <roadmr> cjwatson: it is magic! https://www.youtube.com/watch?v=tYh94XTJTDU
[20:06] <cjwatson> sergiusens: latest snapcraft apparently breaks the store until a new click-reviewers-tools is rolled out, which may not be until Monday depending on how things go.  we were planning to have jdstrand drop a reverted version into ppa:snappy-dev/ubuntu/tools until such time as that happens, so that e.g. BSI users don't suffer in the meantime
[20:06] <jdstrand> sergiusens: the review-tools got grumpy over some added manifest.yaml keys. I've added them (and one could argue they shouldn't do that) but atm LP/build.s.io builds are failing review
[20:06] <sergiusens> jdstrand: heh, those are the manifest keys you asked for :-) the irony :-P
[20:06] <jdstrand> sergiusens: the problem is even though they are fixed, they can't get to prod fast enough
[20:07] <jdstrand> sergiusens: indeed. I seem to be paying the price for that request
[20:07] <pedronis> something seems still broken process wise
[20:07] <jdstrand> pedronis: I mentioned that
[20:07] <sergiusens> this hassle just also proves that my calls for testing are useless
[20:07] <sergiusens> should probably just stop doing them
[20:07] <jdstrand> 15:05 < jdstrand> pedronis: that will be my next question. snapcraft should definitely have some tests that run snaps it produces through the review-tools
[20:08] <cjwatson> jdstrand: anyway, as mentioned, I'm about to go out.  all clear on what needs to be done if you do go the revert route?
[20:08] <jdstrand> that could be part of SRU, autopkgtest, travis, whatever
[20:08] <roadmr> thanks cjwatson
[20:08] <pedronis> sergiusens: this would have broken any  good enough integration test with the store, unless I'm missing something, or those don't run making a manifest
[20:08] <pedronis> so we are missing that
[20:08] <jdstrand> cjwatson: yes. you said xenial is all I need to fix?
[20:08] <cjwatson> jdstrand: I'll have my phone with me so feel free to SMS the number in the directory if you get stuck
[20:08] <sergiusens> pedronis: we use the staging store
[20:08] <sergiusens> pedronis: explicitly told not to use the prod store for any testing
[20:08] <cjwatson> jdstrand: for BSI, yes.  for other LP builds, that'll still catch nearly everything so it should be fine
[20:08] <jdstrand> cjwatson: I'll be fine. thanks for the offer
[20:09] <roadmr> sergiusens, pedronis : this should have broken even with the staging store, since the reviewer-tools that support this only landed on staging today after jdstrand brought the problem up
[20:09] <pedronis> indeed, still not understanding
[20:09] <pedronis> are we not checking enough? not testing the manifest case?
[20:10] <roadmr> I wonder if the store pre-release tests  would have caught them; but we don't run those when snapcraft changes, only when we're about to do a prod release
[20:10] <sergiusens> roadmr: it should of broken all your travis runs too fwiw, as this functionality is in since late May
[20:10] <jdstrand> cjwatson: on monday/whenever, I guess we could simply remove the packages from the ppa, then all is good cause builders are ephemeral?
[20:10] <cjwatson> jdstrand: yes
[20:10] <roadmr> sergiusens: oh! and it hasn't broken anything!
[20:10] <jdstrand> cool
[20:10]  * jdstrand updates the ppa
[20:10] <jdstrand> cjwatson: thanks
[20:10] <pedronis> roadmr: sounds either that we don't check that the test go past review tools or the tests are generating manifests
[20:10] <sergiusens> roadmr: oh, this is missing coverage about generating manifests for those tests.
[20:10] <jdstrand> cjwatson: enjoy your evening :)
[20:10] <cjwatson> I plan to.  thanks
[20:11] <jdstrand> hehe
[20:11] <cjwatson> good luck
[20:11] <roadmr> thanks!
[20:11]  * roadmr needs to step out for a bit but will read backscroll later
[20:16]  * cachio afk
[20:50] <hunterk> ok, reading the scrollback, I'm not the only person with the 'unknown keys' build failure, and that's something the snapcraft folks will sort out on their end?
[20:50] <ogra> yes
[21:04] <hunterk> kk, thanks :)
[21:06] <LargePrime> hi,  can i get a link or keyword to enable write areas in a snap?
[21:06] <LargePrime> am tryint to have a snap packager to fix his snap
[21:06] <LargePrime> please ping
[21:09] <popey> LargePrime: snaps have writable areas. $SNAP_USER_DATA and $SNAP_USER_COMMON - see https://docs.snapcraft.io/reference/env  for details
[21:09] <popey> LargePrime: which snap out of interest?
[21:09] <LargePrime> popey, https://github.com/diddlesnaps/starruler2/issues/2
[21:10] <LargePrime> star ruler 2 has lots of mods and modability.  and they seem mostly borked with current ackage
[21:11] <popey> one possibility that i don't know if diddledan has tried, is you can copy the moddable stuff out to a writable area on first launch
[21:12] <popey> I did this with one or two snaps
[21:12] <popey> can end up wasting space though
[21:12] <popey> the other option is to patch upstream to look in multiple places
[21:12] <diddledan> I've not had a change to look into it yet
[21:12] <diddledan> chance*
[21:12] <LargePrime> popey, would it be a small ask for you to reply to the issue?
[21:12] <LargePrime> ther he is
[21:13] <LargePrime> haha
[21:13] <diddledan> afaict it requires shoving stuff into the binaries directory
[21:13] <popey> ahh
[21:13] <popey> that's unfortunate
[21:13] <diddledan> aye
[21:13] <popey> how big is the binaries directory?
[21:13] <LargePrime> diddledan, also we cannot add portraits
[21:13] <popey> can that be copied out and then launched?
[21:14]  * diddledan goes looksee
[21:15] <diddledan> 858MB
[21:16] <diddledan> most of that in ./data which is probably one of the places that people want to fiddle
[21:16] <diddledan> there's also ./maps, and ./mods
[21:17] <diddledan> ./mods we can probably sync across, and put a symlink in place instead
[21:20] <popey> that might work
[21:20] <popey> what happens when they have traditionally packaged ones
[21:23] <popey> like, if you install from a deb (is there a deb?)?
[21:23] <diddledan> there is no deb that I know of
[21:25] <popey> is there a mac dmg or somehting
[21:25] <popey> Or do you just unpack a zip and run it in place?
[21:26] <diddledan> for the snap it's compiled from sauce
[21:26] <diddledan> I know not how it's distributed elsewhere :-)
[21:38] <jdstrand> roadmr (cc cjwatson): I uploaded 2.43+0really2.42.1 to the ppa. it's building
[21:38] <roadmr> jdstrand: I love the version number :)
[21:38] <jdstrand> roadmr: yeah, Sergio is uploaded 2.43.1 on Monday, so it should all be good
[21:38] <jdstrand> uploading*
[21:39] <roadmr> nice
[21:39] <roadmr> jdstrand: meanwhile the dashboard rollout process toils along :/ I'm still waiting for the CI run to complete.
[21:39] <roadmr> should finish in the next 40 minutes or so \o/ *sob*
[22:23] <diddledan> if a snap with layouts passes manual review can anybody install it? i.e. do users still need to opt-into layouts experiment for a manually approved store snap?
[22:41] <jdstrand> jeez it takes snapcraft two hours to build
[22:41]  * jdstrand taps fingers
[22:41] <jdstrand> (in LP)
[23:23] <jdstrand> Get:11 http://ppa.launchpad.net/snappy-dev/tools/ubuntu xenial/main ppc64el snapcraft all 2.43+0really2.42.1 [199 kB]
[23:30] <cjwatson> good good
[23:30] <jdstrand> fyi, things should start passing again
[23:30] <jdstrand> see https://forum.snapcraft.io/t/builds-failing-automated-review/7112/3
[23:30] <jdstrand> my review-tools upload grabbed the downgraded snapcraft, built and reviewed successfully
[23:30] <jdstrand> cjwatson: thanks for the tip :)
[23:34] <cjwatson> np