[03:21] PR snapcraft#2642 closed: remote-build: detect early build errors [05:20] morning [05:50] Hey mborzecki [05:50] zyga: hey hey [05:51] Check out https://github.com/snapcore/snapd/pull/7172 [05:51] Magic :-) [05:51] PR #7172: tests: work around classic snap affecting the host [05:51] zyga: looking [05:56] Interesting behavior [05:59] zyga: it'd be nice to pinpoint the reason [05:59] As soon as I get to the office [06:00] I spoke with Jamie and we have some ideas [06:16] hmmm [06:16] "newinstance" literally does nothing at all [06:17] eedf265aa003b4781de24cfed40a655a664457e6 [06:17] devpts: Make each mount of devpts an independent filesystem. [06:17] this option is from 2016 [06:17] it seems that before devpts was really magically shared no matter where it is mounted [06:19] mborzecki: so the working theory is gone [06:20] zyga: hah :P [06:24] mborzecki: something like this: [06:24] https://www.irccloud.com/pastebin/JdL7fdQS/ [06:27] zyga: we know it'll happen each time you run classic snap though? so it might be better to error when it does not, in case it's a regression or sth [06:27] so you want more tests to fail? :) [06:28] I'm happy if whoever fixes this adjusts workaround [06:34] zyga: more like a failure accounted for :P but not a blocker [06:34] zyga: let me +1 it [06:34] I'm testing the change [06:34] hey mvo [06:35] mborzecki: I'll likely fix the original issue and adjust this test if that makes you feel better [06:35] zyga: so classic snap mounts it's own devpts then? [06:36] mborzecki: don't even get me started, that code is horrid [06:36] yes [06:36] let me show you [06:36] zyga: hey, good morning [06:36] haha [06:36] and hey mborzecki [06:36] https://github.com/snapcore/classic-snap/blob/master/bin/classic#L67 [06:37] feel sorry for whoever will have to explain all meanings of classic [06:37] mborzecki: it uses systemd-run [06:37] mborzecki: so, presumably, that's running on the host, right? [06:37] it's a init-spawned service? [06:37] mborzecki: what is curious [06:38] if you experiment on your machine [06:38] is that there's only one /dev/pts mounted [06:38] it seems that whatever the kernel did affects the existing /dev/pts, not a newly mounted one [06:38] it's not a new mount, it seems [06:39] zyga: --scope makes the process a child of systemd-run [06:40] mborzecki: do you know if systemd-run does reparenting to the initial mount namespace? [06:40] I'll experiment some more in a moment, just need to get breakfast and take the dog out [06:40] mborzecki: classic is not loved much, though [06:40] that script cries for fixes [06:40] even shellcheck pass [06:42] zyga: perhaps if it's a normal unit, it wouldn't make sense with --scope [06:43] mborzecki: pushed a small update [06:43] mvo: wanna review this craze? [06:44] zyga: sure [06:45] https://github.com/snapcore/snapd/pull/7172 [06:45] PR #7172: tests: work around classic snap affecting the host [06:45] thank you! [06:45] mvo: morning :) [06:50] mborzecki: on the up side, we now found this :) [06:50] I have more coming, core is a bug rich environment [06:58] guys, just a heads up [06:58] I noticed some unit tests hang [06:58] we may have a racy deadlock or something like that [06:58] I restarted about half a dozen tests like that yesterday [07:03] * zyga spawns more tests and goes for a break [07:03] I work way too long lately, need to take a walk and have breakfast [07:53] back [07:53] no breakfast yet but I feel much better [07:53] walk + shower :) [07:55] PR snapd#7172 closed: tests: work around classic snap affecting the host [08:04] mvo: did you see the python-apt + snapd build regression? [08:04] https://www.irccloud.com/pastebin/AcGQjDFA/ [08:05] it was reported by Dimitri [08:06] zyga, hey, was testing the change to mount /lib/firmware, but it is not happening for me - besides bind-mounting snap-confine, is there anything else I should consider? [08:13] zyga, nm, reinstalling the snap made the trick (plus reloading the snap-confine apparmor profile) [08:15] zyga: looking [08:28] mo'in, mo'in [08:31] abeato: Due to the place where this change happens it requires the namespace to be discarded [08:31] Hey Chipaca [08:55] zyga: this looks like a bug in python-apt in eoan [09:09] re [09:09] * zyga made a coffee [09:13] juliank: hey, do you already know about https://paste.ubuntu.com/p/DH8YZj8Gfd/ in python-apt 1.9 - fetch_source (and probably fetch_binary) is unhappy. I can try to look into it later today if you are busy [09:25] * zyga chases another issue in core tests [09:26] on the up side, fewer things fail now, it takes a while to get to a broken one [09:26] zyga: nice [09:27] mvo: little by little, it's not green yet though [09:35] PR snapd#7167 closed: cmd/Makefile.am: support building with the go snap [09:36] brb, need to help my wife [09:45] re [09:56] PR snapd#7173 opened: Support Tegra display drivers in x11/opengl interfaces [10:02] ogra@acheron:~/Devel/branches/snapd:master$ git pull [10:02] ... [10:02] 1578 files changed, 153607 insertions(+), 22623 deletions(-) [10:02] ... [10:02] hmm, i should perhaps do that more often ... [10:03] abeato: reviewed [10:03] ogra: wow [10:03] * zyga fixed another core test bug [10:06] zyga, thanks [10:24] PR snapd#7174 opened: allow setting start_x=1 which enables the CSI interface for the RPi c… [10:26] thanks zyga [10:27] thank you ogra [10:28] hm, what's even the purpose of snap.PlaceInfo [10:56] Chipaca: there must be a place for it [10:56] * zyga hides [11:01] hehe [11:04] ogra: hmm ogra@ubuntu.com (https://api.launchpad.net/1.0/~ogra) has NOT signed the CLA [11:04] our CLA checker going bonkers? [11:05] Chipaca: do you have any clue what could be happening here? ^^ [11:06] wat [11:06] mborzecki: dunno. Is there a commit by ogra in the PR in question? [11:07] Chipaca: yup, From: Oliver Grawert checks out [11:08] mborzecki: hadn't we landed a change to make it pull the whole repo? [11:08] mborzecki: i can't see it in the .travis.yml though [11:09] mborzecki: there should be a git: depth: false in the CLA section [11:09] Chipaca: it was reverted apparently [11:10] mborzecki: anyway, that would/should fix it :-/ [11:22] mvo: yes I have a fix in progress [11:23] Both hardcoded md5 so kind of good they broke [11:39] mborzecki, Chipaca, sorry, was melting^Wlunching ... do i need to do anything ? [11:39] ogra: sweat [11:39] at yur command ! [11:39] *your [11:39] 'yur' worked well in context [11:40] ogra: I don't think you need to do anything, no [11:40] ok [11:40] ogra: the CLA checker has a bunch of heuristics to work around the fact that we can't ask launchpad for all your emails :-) [11:40] ogra: nor can we ask it about your group membership [11:40] bah [11:41] we can't ask it whether you're a member of a private team [11:41] geez ... how silly ... given you could screen-scrape them from my LP page [11:41] ogra: no you can't [11:41] that's the point :-) [11:41] ah, right, private [11:41] silly security ! [11:41] always in the way [11:56] * zyga chases systemd/snapd mount bug :/ [11:56] man that test is very har [11:56] dd [11:58] google:ubuntu-core-16-64:tests/main/ubuntu-core-gadget-config-defaults:ssh_common [11:58] this test is leaking a mount for a test snap [11:58] the test snap cannot be removed (gadget requirement) [11:58] the code hand crafts the removal [11:58] but the removal depends on the mount unit being okay [11:58] but it isnt [11:58] daemon-reload just a moment before _probably_ corrupts systemd state [11:59] I'm debugging but iteration is slow [12:19] juliank: thanks! [12:20] Chipaca: if ogra signes with @canonical.com things are fine, no? [12:21] mvo: yes [12:21] mvo: or @mozilla.com :-) [12:21] mvo: or if he signs the CLA [12:21] s/signs/commits/ [12:22] mvo: or with an email address in the commit history [12:22] mvo: the latter is already done, so it should work if it doesn't do --depth=50 [12:40] DOH [12:40] man [12:40] that's sooooo frustrating [12:40] solved the bug [12:41] mvo, mborzecki: can we use systemd-escape in all tests or is it absent in 14.04? [12:41] zyga: don't remember [12:42] wow, it's standup coffee time [12:44] PR snapd#7175 opened: gadget: effective structure role fallback, extra tests [12:46] fun bug [12:47] and there are more of it! [12:50] running again [12:50] :) [12:57] https://github.com/snapcore/snapd/pull/7176 [12:57] mborzecki, Chipaca: ^ [12:57] PR #7176: tests: correctly escape mount unit path [12:58] PR snapd#7176 opened: tests: correctly escape mount unit path [12:58] though I think I've missed a nice pun chance there [12:59] better now [12:59] :D [13:06] @popey you're better at words - can you try to explain what I failed to please?: https://forum.snapcraft.io/t/vlc-cve-a-good-oportunity-for-snappy/12440 [13:07] * popey_ looks [13:07] thanks :-) [13:08] note that this CVE is also very controversial ... vlc denies it is their issue blaming it on a distro lib) [13:08] even more reason to steer clear :-) [13:16] mvo, have you seen last nights discussion about classic snap vs using lxd ? i wonder if we should sunset classic for core18 and onwards and point people to use lxd containers [13:16] * ogra must admit he hasnt used classic in ages ... [13:23] ogra: I did not see that but it sounds great, I think lxd is really the better choice in many ways, only for real corner cases (like low level hw access) the real one might be needed [13:24] yeah and there you could simply download an ubuntu-base tarball and chroot [13:24] 180/277 core tests passed without leaking any mount entries [13:27] doh - I take that back [13:27] wrong branch :D [13:37] mvo: I didn't want to take more time at the standup but we have one more general issue [13:37] mvo: some snaps leave stuff on the system when we "remove" them [13:37] classic is one [13:37] lxd is another [13:37] I'll file a bug on lxd to make sure this is tracked [13:46] zyga: ok [13:47] mborzecki, mvo: https://github.com/snapcore/snapd/pull/7177 [13:47] another simple one [13:47] PR #7177: tests: remove test-snapd-curl [13:48] PR snapd#7177 opened: tests: remove test-snapd-curl [13:51] sorry, force pushed because there are two tests like that [13:52] I'll dust my cleanup-tool patch [13:52] this will be the biggest win for reliability and speed [13:52] test prepare/restore won't be this "magic" anymore [14:07] PR snapd#7178 opened: tests: remove test-snapd-snapctl-core18 in restore [14:07] PR snapd#7179 opened: tests: remove installed test snap [14:07] PR snapd#7180 opened: tests: remove installed snap in the restore section [14:37] cachio: https://github.com/snapcore/snapd/pull/7171#pullrequestreview-267225535 [14:37] PR #7171: tests: improve how the system is restored when the upgrade-from-2.15 test fails [14:39] * zyga runs another pass [14:39] and spawns core18 to see what happens there [14:39] next up pass on more than main suite [14:39] I will land the test fix PRs when they become green [14:39] zyga, thanks [14:41] * zyga takes a break for an hour [14:44] ogra: what's up with the CLA test? [14:47] zyga, dunno, Chipaca said i dont need to do anything (i asked above) [14:47] is this happening in one PR, or everywhere? [14:48] zyga: ^? [14:52] Setting up snapd (2.39.2+18.04) ... [14:52] Installing new version of config file /etc/apparmor.d/usr.lib.snapd.snap-confine.real ... [14:52] md5sum: /etc/apparmor.d/usr.lib.snapd.snap-confine: No such file or directory [14:52] ^ don't like that [14:52] juliank: oh? where is that coming from [14:52] perhaps the dpkg conffile helpers? [14:52] mvo: ^ help [14:53] I just upgraded: Unpacking snapd (2.39.2+18.04) over (2.38+18.04) ... [14:53] Chipaca: in one PR [14:53] juliank: looks like a bug, let me look [14:53] zyga: that pr (or another one that lands and then is merged) needs to update .travis.yml [14:54] juliank: I see it, let me fix that for the next update [14:54] Chipaca: what needs to happen in travis.yml? [14:54] ah md5sum /etc/apparmor.d/usr.lib.snapd.snap-confine is just hardcoded in an upgrade check w/o checking if the file exists [14:54] mborzecki: there should be a git: depth: false in the CLA section [14:54] zyga: ^ [14:54] though there is a 2>/dev/null redirect [14:54] but at the wrong part of the pipe :D [14:54] md5sum /etc/apparmor.d/usr.lib.snapd.snap-confine | cut -f1 -d' ' 2>/dev/null [14:55] juliank: the redirect is on cut, I think [14:55] should be [14:55] yeah [14:55] md5sum /etc/apparmor.d/usr.lib.snapd.snap-confine 2>/dev/null | cut -f1 -d' ' [14:55] - if test "$(md5sum /etc/apparmor.d/usr.lib.snapd.snap-confine | cut -f1 -d' ' 2>/dev/null)" = "2a38d40fe662f46fedd0aefbe78f23e9"; then [14:55] + if test -f /etc/apparmor.d/usr.lib.snapd.snap-confine && test "$(md5sum /etc/apparmor.d/usr.lib.snapd.snap-confine | cut -f1 -d' ')" = "2a38d40fe662f46fedd0aefbe78f23e9"; then [14:55] or like this [14:55] a bit more verbose but perhaps more to the point [14:56] right? [14:56] yeah [14:56] I'll send a PR [14:56] thanks for reporting this [14:59] mvo: https://github.com/snapcore/snapd/pull/7181 [14:59] before you leave please [14:59] PR #7181: packaging/debian: don't md5sum absent files [14:59] juliank: ^ [14:59] thanks zyga [14:59] PR snapd#7181 opened: packaging/debian: don't md5sum absent files [14:59] core16 main is clean, running all suite now [15:00] * cachio lunch [15:00] core18 looks ok after 50% of the pass [15:00] I think that's all of them :) [15:05] Bug #1838040 opened: delete user data thunderbird [15:14] Bug #1838040 changed: delete user data thunderbird [15:20] some unit tests are hanging in travis [15:20] 10 minute timeout reached [15:23] $ snap info mojo [15:23] error: no snap found for "mojo/" [15:23] ^ what's that? [15:23] is there a mojo directory? [15:23] Oh now it works, this was happening while snap install mojo --classic was running [15:23] snap info can be used to look at unpacked snaps on disk [15:23] ah but yes, there is a mojo dir too [15:24] right, that's why [15:30] zyga: re https://answers.launchpad.net/snappy/+question/682352 if they removed the snap, the data is gone from disk [15:30] Chipaca: oh, no snapshot? [15:30] but right [15:30] zyga: (if new enough they'll have snapshots) [15:30] can you comment about the snapshot aspect [15:31] but if they removed snapd the snapshot data is gone as well [15:31] all core16 tests are mountinfo clean [15:31] that's all folks :) [15:31] zyga: huzzah [15:31] zyga: happy weekend [15:32] nah, I'm rewriting a helper tool from shell into python [15:32] it will be healthier for what it will do next [15:32] I want to fix more test bugs [15:32] I'm off to the gym, then I'll review ijohnson's PR when I get back while dinner cooks [15:32] test bugs drive me crazy [15:32] * Chipaca hugs zyga [15:32] Chipaca: have fun, :) [15:32] zyga: you were crazy before we had anything to do with it [15:32] hahaha [15:32] thanks Chipaca [15:32] I'll take that as a compliment :) [15:32] zyga: :-) [16:16] zyga: thank you [16:18] PR snapd#7176 closed: tests: correctly escape mount unit path [16:18] lxd leakes more state on fedora [16:18] but I'll fix that later, need to run an errand now [16:19] two more failures from fedora: lxd cgroup state and binfmt_misc being mounted by something traversing the automount point [16:33] aaargh [16:33] I have a snap in a wedged state [16:33] installed: (11381) 0B disabled,broken [16:33] i can't refresh, install, enable it [16:33] - Setup snap "lxd" (11381) security profiles (cannot find installed snap "lxd" at revision 11381: missing file /snap/lxd/11381/meta/snap.yaml) [16:34] it's right /snap/lxd is empty [16:51] * popey_ starts https://discuss.linuxcontainers.org/t/lxc-snap-wedged-unable-to-refresh-install-enable/5343 [17:02] * diddledan points at popey's lxd and laughs :-p [17:02] https://media.giphy.com/media/Q8OOs80Hb5Bj1qNA1d/giphy.gif [17:03] this one's better https://media.giphy.com/media/cO39srN2EUIRaVqaVq/giphy.gif [17:03] PR snapd#7179 closed: tests: remove installed test snap [17:25] popey_: are you running 5.2? [17:50] * ijohnson lunches back in hour or two [18:01] * cachio afk [18:23] PR snapd#7180 closed: tests: remove installed snap in the restore section [18:42] ijohnson: you mind if i push a patch to your PR? [18:42] how rude [18:42] ijohnson: otherwise I'll pastebin a diff [18:42] :-p [18:42] ijohnson: even if I push the patch it's still a suggestion :-) [18:43] it's just that it's easier to explain in code [18:43] anyway, i'm going to carry on ignoring diddledan and go make dinner [18:43] github totally needs the ability to file a PR against someone's PR [18:45] ah, and i thought the ability to make dinner for you [18:50] diddledan: you can file a PR against someone's branch that's PR'ed though [18:50] true, tho it's more complex by being a repo that you didn't "fork" from on the github ui [19:06] Chipaca: go for it [19:08] ijohnson: did I understand pedronis right that the values are always strings? [19:08] ijohnson: meaning that the list in the verbose test is bogus? [19:13] i could check the code myself but i'd have to stop stuffing my face [19:17] PR snapd#7177 closed: tests: remove test-snapd-curl [19:17] PR snapd#7178 closed: tests: remove test-snapd-snapctl-core18 in restore [19:28] ijohnson: i pushed, but i know it breaks a test [19:28] ijohnson: but the tests were broken before so i'm feeling only 90% sorry [19:28] now i need to go walk the dog (otherwise i would've spent more time on this) [19:28] (dog has almost given up in a huff :-( ) [19:31] ijohnson: the test that was failing before is about the command not being categorised. Put it under 'other', beside 'known', at least for now [19:31] we're still under 80 chars so it's ok [19:31] * Chipaca hitches the dog up to the air con [19:32] PR snapd#7181 closed: packaging/debian: don't md5sum absent files [19:35] PR snapd#7182 opened: tests: remove core18 if installed by tests [19:38] I think that's correct Chipaca, that's what I understood pedronis to mean [19:39] thanks for the patch, will review it, and no worries about the tests I'll clean those up [19:53] PR snapcraft#2645 opened: file utils: better error for NotADirectory [19:55] zyga: you are quick