[00:09] <mup> Bug #1758465 opened: snapd doesn't upgrade on bionic when a local installed snap mount is failing <Snappy:New> <https://launchpad.net/bugs/1758465>
[00:59] <mup> PR snapd#4858 closed: strutil, cmd/snap: drop strutil.WordWrap, first pass at replacement <Created by chipaca> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/4858>
[04:20] <mup> Bug #1741486 changed: failed snap try leaves snap symlink around <Snappy:Expired> <https://launchpad.net/bugs/1741486>
[07:04] <mup> PR snapd#4882 closed: snap: make `snap run` look at the system-key for security profiles <Critical> <Squash-merge> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4882>
[07:12] <mup> PR snapd#4924 opened: snap: make `snap run` look at the system-key for security profiles (#… <Created by mvo5> <https://github.com/snapcore/snapd/pull/4924>
[07:44] <mup> PR core#86 opened: hooks: create compat copy of /lib/udev/snappy-app-dev <Created by mvo5> <https://github.com/snapcore/core/pull/86>
[07:45] <mup> PR core#86 closed: hooks: create compat copy of /lib/udev/snappy-app-dev <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/core/pull/86>
[08:28] <pedronis> mvo: zyga: I'm a bit confused, it seems  account-control expects files to be already present in /var/lib/extrausers  but in our tests they are just because of a hack
[08:32] <mup> PR snapd#4924 closed: snap: make `snap run` look at the system-key for security profiles (2.32) <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4924>
[08:33] <pedronis> if I tweak the hack I get:    useradd: cannot open /var/lib/extrausers/passwd
[08:37] <pedronis> it's not a confinement issue btw,     useradd --extrausers alice  as root also gives the same
[08:59] <mup> PR snapd#4925 opened: tests: fix snap-run tests when snapd is not running <Created by mvo5> <https://github.com/snapcore/snapd/pull/4925>
[09:00] <mup> PR snapd#4926 opened: tests: fix snap-run tests when snapd is not running <Created by mvo5> <https://github.com/snapcore/snapd/pull/4926>
[10:10] <thresh> hello.  how do I debug why stage-packages: pulls in packages that I don't see installed when I do apt-get install for that list?
[10:11] <thresh> I see some packages that should not be there
[10:12] <thresh> found --debug, let me see if that helps...
[10:16] <mborzecki> zyga: hi, you around?
[10:17] <mborzecki> zyga: there's a hackathon at GOG office in May, want to show them how make snaps? https://www.gog.com/hackathon
[10:19] <mup> PR snapd#4925 closed: tests: fix snap-run tests when snapd is not running (2.32) <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4925>
[10:28] <zyga> Hi mborzecki
[10:28] <zyga> Ooohhh
[10:28] <zyga> Thanks
[10:29] <zyga> Wanna come?
[10:36] <mborzecki> zyga: i'm interested for sure, don't know yet if i will be able to go, probably will know more mid-april, registration closes on may 7, so we still have some time
[10:36] <bulld> hi
[10:37] <zyga> We could snap gog games
[10:37] <bulld> am building my app with qt5.7.1 and later am going to snap pack it, how can i use my local qt5.7.1 toolkit to build in snapcraft file
[10:37] <zyga> I’ll register us back home
[10:38] <zyga> bulld: hey, I’m not sure but it’s possible for sure.
[10:38] <zyga> Did you check on the forum? Maybe someone did this already
[10:39] <bulld> i build qt5.7.1's webengine myself with codecs support so i need it to build/ship the binary am using in snap package
[10:39] <thresh> so I've ran two snapcraft --debug prime runs, and it seems like the behaviour is inconsistent: https://thre.sh/stuff/snapcraft-different-packages-staged.diff.txt
[10:39] <bulld> okay @zyga i will check
[10:39] <thresh> any idea how to make sure snapcraft works the same between different runs?
[10:40]  * zyga is a snapd hacker and builds snaps less frequently than others
[10:40] <thresh> e.g. you can see it fetches libvlc5 in one case and not in the other
[10:40] <zyga> And when I do I build weird snaps that use new features
[10:40] <bulld> @zyga :)
[10:41] <zyga> Speaking of which, I have some cool snap demo to publish
[10:41] <zyga> But first family time with my daughter :-)
[10:42] <bulld> hmm :)
[10:55] <bulld> my snap installation is broken somehow cannot install any snap package local or from store. getting some apparmor errors
[10:55] <bulld> am using ubuntu 16.04
[10:57] <mup> PR snapcraft#2024 closed: errors: improve the UX for sending error data <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2024>
[11:02] <bulld> i already posted details of errors am getting in snapcraft forums https://forum.snapcraft.io/t/unable-to-install-snap-from-store-or-loallay/4629
[11:08] <zyga> bulld: can you give me more details
[11:08] <zyga> Snap version
[11:08] <zyga> And the errors you are seeing
[11:09] <bulld> i provided everything in the forum link in my previous msg
[11:09] <bulld> https://forum.snapcraft.io/t/unable-to-install-snap-from-store-or-loallay/4629
[11:10] <zyga> Ok
[11:10] <zyga> Thanks
[11:22] <zyga> I’ll check that later today
[11:27] <bulld> okay ty
[11:55] <mup> PR snapd#4927 opened: release 2.32 <Created by mvo5> <https://github.com/snapcore/snapd/pull/4927>
[12:02] <pedronis> zyga: I think  I know how to disentangle the etc mess,  was confused for a bit by writable-paths (surprise)
[12:02] <zyga> Ha
[12:02] <zyga> I sent a PR that looks for the damage
[12:02] <zyga> There are some tests that need fixes
[12:03] <pedronis> zyga: well, it's no the tests
[12:04] <pedronis> is the prepare stuff that does very strange stuff
[12:04] <pedronis> (in hindsight)
[12:04] <pedronis> zyga: we basically run the tests with /etc/passwd  being a bind mount of /var/lib/extrausers/passwd
[12:04] <pedronis> that makes little sense conceptually
[12:05] <zyga> Why do we do that’s
[12:05] <zyga> Note that would not break the mimic code
[12:05] <pedronis> zyga: well we do need to bind mount something on /etc/passwd
[12:05] <zyga> It is this with combination of some unfortunate rm -rf
[12:05] <pedronis> afair
[12:05] <pedronis> sorry
[12:05] <pedronis> afaiu
[12:06] <pedronis> but making it /var/lib/extrausers/passwd
[12:06] <pedronis> adds to the confusion
[12:06] <pedronis> because then some other code will modify it
[12:06] <pedronis> zyga: it doesn't seem to be a rm -rf , more some cases of sed -i
[12:06] <pedronis> but I bit unclear exactly how (I don't get that effect playing locally with sed -i)
[12:07] <zyga> Interesting
[12:07] <zyga> I think sed would not cause that
[12:07] <zyga> Kernel code indicates that dentry must be removed
[12:07] <zyga> So more like the parent directory
[12:07] <pedronis> zyga: anyway  having etc/passwd [12:07] <pedronis> conceptually
[12:07] <zyga> I agree
[12:08] <pedronis> zyga: I think we need to bind mount /etc/passwd   because we need to put there the spread root user
[12:08] <pedronis> afaiu
[12:08] <zyga> Cannot we use extra users for that?
[12:08] <pedronis> zyga: no, that's the problem
[12:08] <pedronis> root cannot be in extrausers
[12:08] <pedronis> because the one in /etc/passwed wins
[12:09] <zyga> Ah, I see
[12:09] <pedronis> so we need a bind mount
[12:09] <pedronis> but no reason to do this bind mount
[12:09] <pedronis> (is super confusing in various ways)
[12:09] <pedronis> also it means that when we test extrausers stuff
[12:09] <pedronis> (which is important)
[12:09] <zyga> Well
[12:09] <pedronis> we really test nonsense
[12:09] <zyga> At the very least I know how to improve mimic code
[12:10] <zyga> But our test environment is in a very broken state
[12:10] <pedronis> zyga: anywy I can propose a change that doesn't have any /etc  //deleted mounts at least
[12:10] <pedronis> anymore
[12:10] <pedronis> and also untangle a bit that confusion
[12:10] <pedronis> I don't know if it's enough to not confused layout
[12:10] <pedronis> but is a step
[12:10] <zyga> Because all those //deleted mount points are broken
[12:10] <pedronis> yea
[12:10] <zyga> I will check kernel code
[12:11] <pedronis> no more of those
[12:11] <pedronis> with my change
[12:11] <pedronis> I mean for etc
[12:11] <zyga> But I’m worried that such deleted mount points cannot be the target of a mount call
[12:11] <pedronis> we have some for try snaps afaict
[12:11] <zyga> Because they no longer exist conceptually
[12:11] <pedronis> zyga: ?
[12:11] <pedronis> zyga: so?
[12:11] <zyga> It’s a detached inode
[12:11] <pedronis> zyga: as I said I fixed the //deleted bit
[12:11] <zyga> So mount with that as a target will always fail
[12:11] <zyga> So what can we
[12:11] <zyga> Do?
[12:12] <pedronis> is that a problem?
[12:12] <zyga> Yes
[12:12] <zyga> The mimic is broker
[12:12] <zyga> Broker
[12:12] <pedronis> I mean do you think it will happen in practice?
[12:12] <zyga> Broken
[12:12] <zyga> No
[12:12] <zyga> But we must handle that in tests
[12:12] <pedronis> ?
[12:12] <zyga> In reality it will not happen
[12:12] <pedronis> if there are no  //deleted
[12:12] <pedronis> anymore
[12:12] <pedronis> there's nothing to handle
[12:13] <pedronis> I made changes such that there are no more //deleted
[12:13] <pedronis> I think I said that a couple fo times
[12:13] <zyga> But I don’t want to fail tests 30% of the time because of that
[12:14] <zyga> Oh
[12:14] <pedronis> ?
[12:14] <zyga> Cool
[12:14] <zyga> Can you merge my patch from a later pr that detects dangling //deleted please?
[12:14] <pedronis> zyga: I used your patch
[12:14] <pedronis> as a baseline
[12:14] <pedronis> otherwise how would I know
[12:14] <pedronis> that there no //deleted
[12:15] <zyga> Ah perfect
[12:15] <pedronis> to be clear I'm talking about /etc here
[12:15] <pedronis> there are some tests using try
[12:15] <pedronis> that live //deleted around
[12:15] <pedronis> not sure it's an issue or not
[12:15] <zyga> I see
[12:15] <zyga> I can review that next week and try to eliminate them
[12:17]  * zyga is very much afk
[13:12] <mborzecki> updated snapd to 2.32 in AUR
[15:07] <mup> PR snapd#4922 closed: overlord/configstate: change how ssh is stopped/started (#4912) <Created by mvo5> <Merged by zyga> <https://github.com/snapcore/snapd/pull/4922>
[15:27] <pedronis> ok, my change seems to do what I wanted (there was a further subtle detail, but I think I have a fix)
[15:54] <mup> PR snapd#4928 opened: tests: disentangle etc vs extrausers <Created by pedronis> <https://github.com/snapcore/snapd/pull/4928>
[15:56] <pedronis> zyga: ^
[15:56] <zyga> Thank you. I will return home soon and review it
[15:57] <zyga> Did you find this by reading the strace from last night?
[15:59] <pedronis> zyga: no, I just assumed  that /etc/group  being //deleted would be the source of the problem?
[15:59] <pedronis> zyga: anyway I tried to run those account tets and layout together and layout passes
[15:59] <pedronis> with this change
[16:03] <zyga> Thank you so much!
[16:04] <zyga> My question about the strace was motivated by wanting to see if there was anything else fishy there
[16:04] <zyga> I will review it in the evening
[16:05] <pedronis> zyga: anyway as I explain in the PR, this is a saner change because what we did would make our /var/lib//extrausers not very realistic
[16:06] <pedronis> because /etc/passwd would also contain the stuff
[16:06] <pedronis> etc
[16:16] <pedronis> zyga: rechecked, on master if one runs ubuntu-core-create-user just before layout  the latter fails with the /etc issue
[16:17] <zyga> Great find! But I also managed to fail it by running just layout with -repeat 2
[16:18] <pedronis> that is stranger
[16:19] <zyga> But that was against core 16
[16:19] <zyga> So I thought the suite restore is buggy
[16:19] <pedronis> this is a core only problem btw
[16:19] <pedronis> we don't do this strange things on classic
[16:19] <zyga> It would fail on 2nd try
[16:20] <zyga> I hope so. I think we saw layout failures in classic too
[16:20] <zyga> But perhaps older codebase
[16:20] <pedronis> well it might be that layout breaks layout
[16:20] <pedronis> in some cases
[16:20] <pedronis> are the  threspassing fixes merged?
[16:21] <zyga> No but they cannot affect this
[16:22] <pedronis> also linode and google are different because reuse vs not
[16:22] <zyga> Trespassing bug means we leave empty regular file, directory or symlink
[16:22] <pedronis> that also changed
[16:22] <zyga> Ah, good point
[17:04] <pedronis> grumble
[17:22] <mup> PR snapd#4926 closed: tests: fix snap-run tests when snapd is not running <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4926>
[17:22] <pedronis> zyga: yes, that was my grumble, classic test is failing
[17:22] <zyga> pedronis: I reviewed https://github.com/snapcore/snapd/pull/4928
[17:22] <mup> PR #4928: tests: disentangle etc vs extrausers in core tests <Created by pedronis> <https://github.com/snapcore/snapd/pull/4928>
[17:23] <zyga> ha
[17:23] <pedronis> I think I know what to do
[17:23] <pedronis> just annoying
[17:23] <pedronis> I still need to create extrausers
[17:23] <zyga> what is your plan?
[17:23] <pedronis> just not bind it somewhere
[17:23] <zyga> aha
[17:23] <pedronis> zyga: basically we need the "test" user there
[17:23] <pedronis> I'm testing the fix right now locally
[17:23] <zyga> if we need it for just one test, perhaps we could do it there
[17:23] <zyga> we can bind mount over /etc/passwd
[17:23] <zyga> and restore that correctly
[17:24] <pedronis> it doesn't need to be in /etc/passwd
[17:24] <pedronis> it really needs to be in extrausers
[17:24] <pedronis> because classic knows that's were core puts users
[17:24] <pedronis> so it's the thing it copies inside
[17:24] <zyga> ah, I see
[17:25] <pedronis> as I said, I think I have a fix that should keep the rest of the sanity
[17:25] <pedronis> trying it now
[17:25] <zyga> cool, thank you
[17:25] <zyga> I will play with the kernel, I have an idea on how to improve the error reporting for my sanity, at least
[17:25] <zyga> just add some logging in places where kernel returns error on mount
[17:26] <pedronis> zyga: fwiw   is not sed -i that was ripping apart /var/extrausers/foo,  it's useradd itself
[17:26] <pedronis> I think
[17:26] <zyga> I wrote a similar patch for pivot_root which has half a dozen reasons for EINVAL
[17:26] <pedronis> it does atomic writes with renames
[17:26] <pedronis> of stuff there
[17:26] <zyga> ah, that's sensible, it would get a new inode and the old one would become //deleted
[17:27] <zyga> kernel is "fun"
[17:34] <pedronis> zyga: I pushed the fix
[17:34] <zyga> lookng
[17:38] <zyga> let
[17:39] <zyga> let's see if it passes now
[18:09] <pedronis> zyga: passed, I'll need a 2nd review on monday
[18:09] <zyga> sounds very good
[18:27] <cachio> zyga, is it gonna be a new 2.32?
[18:27] <cachio> I am running beta validation for the current one
[18:27] <zyga> cachio: no, I don't think so
[18:27] <cachio> zyga, ah, ok, good news
[18:36] <pedronis> it's  a test only fix, we probably want it in 2.32.x  if likely we need to do that, but should affect current 2.32 testing
[18:36] <pedronis> *but shouldn't