[05:19] morning [05:55] good morning [05:55] mborzecki: last week of school [05:55] any "holiday" plans? [05:56] yeah, i'm taking 2 weeks off in the beginning on july [05:56] and your kids/ [05:56] we're going away for a while together [05:56] nice [05:58] good morning mvo [05:58] hey zyga [05:58] zyga: good morning [05:58] damn, my emacs setup fell apart :( [05:58] mvo: hey [05:58] mborzecki: good morning to you as wlel [05:58] mvo: when does school finish for your family? [05:58] well [05:58] mborzecki: what happened with your emacs? [05:59] zyga: updated spacemacs and not it complains that there's no gocode, godef et al, probaly PATH messed up, or something special about zsh env files zshenv/zshrc [05:59] zyga: this is the last week - but its different for the different regions in germany, some finish later [06:04] yay, turns out i had to fix the spacemacs init file only :) [06:06] PR snapd#5330 closed: snapstate: support restarting snapd from the snapd snap on core18 [06:07] getting a second review for 5324 would be great! it has one review already [06:13] mvo: looking now [06:14] mvo: we should discuss what to do today, should I finish the work on system or is that premature and it needs more review and discussion [06:15] zyga: a good question, I would love to see it but maybe worthwhile to wait until the meeting after the standup [06:16] mvo: sounds good, I have other things I can finish meanwhile [06:16] mvo: if you merge it with your WIP branch and disable tests in ifacestate you can perhaps run spread and see how "far" it can go [06:17] zyga: cool idea, with 5324 I think I can almost propose a spread PR, I think there is just one trivial change for core18 left to get very basic spread working [06:17] zyga: and yeah, will be fun to run the entire thing :) [06:18] zyga: part of the challenge will be to review the tests to see which really need core and which just pull it in for no good reason [06:18] currently core is pre-installed before tests run [06:18] so most tests just blindly assume core [06:19] but yeah, it will be interesting to review tests to see what we ought to get [06:19] * mvo nods [06:19] I also wonder if we want to have tests/main be tests/main-on-core [06:19] that is, shall we specialise some suites for a given core [06:19] I don't know honestly [06:33] zyga: quick question about 5250 - do you think it needs an extra review? it has two +1 now, so we can (squash)merge it I think. I'm a little bit nervous about it but it looks very good and solves a real problem [06:34] hmm mhmm it's a tricky one [06:34] maybe ask someone from foundation that knows udev for a last look [06:34] zyga: yeah, I have the same feeling, tricky [06:34] zyga: good idea! [06:35] mvo: or, just ask lennart [06:35] xnox: if we needed to ask a udev expert in foundations for a review, what would be your recommendation? we would love to get a high level review of #5250 [06:36] PR #5250: interfaces/udev,misc: only trigger udev events on input subsystem as needed

[06:36] zyga: *cough* you are joking, right? [06:36] no, really [06:36] just ask a direct question, "is this and this sequence of udev calls" a safe thing to do [06:59] mvo: reviewed === pstolowski|afk is now known as pstolowski [07:03] mornings [07:05] zyga: yay, thank you [07:05] pstolowski: good morning [07:06] pstolowski: hey [07:08] zyga: I like your idea, very nice, will do that now [07:12] zyga: hm, with the change the glob becomes a bit useless, right? snap-confine.core.111* basicly? but no issue as we just write a single file anyway [07:12] zyga: right? [07:12] hmm, let me look [07:14] yeah, the glob would just match that one fine [07:14] *file [07:14] zyga: the change makes the test much nicer to read [07:14] zyga: good stuff [07:14] zyga: i.e. all this crazy "strings.Replace("/", ".") etc is gone [07:14] yeah [07:15] it's more obvious what is going on [07:17] zyga: hm, but the glob needs to be something else or than snap-confine.core.111* because .111* also matches .1111 which we don't want [07:17] the glob doesn't need to use * [07:17] zyga: aha, nice [07:17] it can be literally the name of the file you want to create [07:17] the glob is there in case we want to "own" a namespace [07:17] since (AFAIK, correct me if I'm wrong) we also have a bit of code that cleans those up when a snap is removed [07:17] ... or do we? [07:18] zyga: yeah, we clean those up [07:18] zyga: and we have tests for that [07:28] https://paste.ubuntu.com/p/j3g5RqxFPZ/ iirc Chipaca saw something similar [07:29] error: e-book-client-error-quark[2] Conflicting UIDs found in added contacts, hmmm [07:29] address book persists between runs? [07:31] mvo: are you sure we can use * for revision? [07:31] this means that a setup for snapd 123 will remove profiles for snapd.122 [07:32] as the system will find a file matching the glob that is not described by the content map [07:32] zyga: yes, that is was we are currently doing [07:33] zyga: its not a regression but its not great either [07:33] mmm, I see [07:33] let's do it and see [07:33] zyga: its not ideal, we need to think how to improve it [07:34] zyga: I was also thinking it might be racy because we update the current symlink and then when snap run runs it will not yet have profiles for snap-confine [07:34] zyga: but fortunately the system-key fixes this, i..e we have code in snap run that waits for profile re-generation on system key mismatch [07:35] I agree, ultimately I think the profile ought to be removed only when the snap is removed [07:35] since for various reasons it can persist longer, even while the snap is inactive [07:36] zyga: +1 [07:36] zyga: I will do a followup with this [07:47] * zyga breakfast [08:01] moin moin [08:01] good day to you sir, on this lovely morning [08:01] finally got a test run that failed with econnreset with the new extra debug yesterday [08:01] if anybody's interested [08:02] :) [08:02] also didn't get more than a couple of hours of sleep last night, so i'm not seeing straight right now [08:03] Chipaca: I am interessted [08:03] mvo: I'll send you my paypal address [08:04] mvo: https://api.travis-ci.org/v3/job/393403309/log.txt [08:04] i also have a local copy of that if it goes away [08:05] Chipaca: the latest theory of cachio is that this happens when there is a "snap userd" leftover from previous tests [08:06] Chipaca: then "kill $(pidof snap)" fails [08:06] maybe killall instead of kill? [08:06] I mean, that might be one source of failures [08:06] but it's not this one [08:06] this one is definitely the 'no match' error [08:06] what I would really like is my measure patches to spread [08:06] so that we could put up walls around each test to ensure it doesn't leak [08:07] Chipaca: aha, looking [08:10] hm #5353 looking good so far, 5 runs, all but one green, that one was unrelated though (eds thing i pasted above) [08:10] #5335 [08:11] PR #5335: tests: fix for the download of the big snap [08:14] Chipaca: hm, this log does not (yet) contain the output of "ls -lh test-snapd-huge_*", right? or am I just missing it? [08:15] mvo: it does not contain output of that sort, no [08:15] mvo: where was that added? [08:15] Chipaca: thats slightly sad, it would be nice to know if the download simply finished, this is not logged yet, is it? [08:16] mvo: it does contain logs that say what it did though [08:16] mvo: yes it does say when it's finished [08:16] let me get that [08:17] Chipaca: is where the partial file output got added [08:17] Chipaca: https://github.com/snapcore/snapd/pull/5333 [08:17] PR #5333: tests: show status of the partial test-snapd-huge snap in econnreset test [08:17] mvo: that's on master, and i merged that master into thsi pr [08:17] mvo: so maybe i was wrong and it does include the ls [08:17] didn't see it tho [08:17] Chipaca: I don't see it [08:18] Chipaca: it would be good to know if the download finished successfully, I think this is what happend [08:20] Chipaca: oh nooooo [08:20] Chipaca: we have two debug: | in the test now [08:20] mvo: given it doesn't start downloading the assertion until the download is finished, I'd say you are correct [08:20] Chipaca: must be a double merge, this is why the debug output is missing [08:21] Chipaca: let me push a PR [08:22] mvo: I might add to that PR :) but yes [08:22] mvo: in any case: assertion download doesn't start unless snap download succeeded [08:23] so, fastly is _too_ good :) [08:23] * Chipaca wants to make it clear that this is not a complaint [08:23] Chipaca: haha [08:23] it's downloading 500MB in less than .1 seconds [08:23] Chipaca: maybe - or iptables is really slow appying the filter [08:23] Chipaca: I push something that adds more timestamps [08:24] Chipaca: we could make another episode of "speed" with this [08:24] mvo: I think it's time to add a slow writer debug thing [08:24] e.g. have a SNAPD_DEBUG_HTTP value of 8 mean 'be slow' [08:24] hmmm [08:25] PR snapd#5336 opened: tests: remove double debug: | entry in tests and add more checks [08:26] Chipaca: lets see if the times add up. I saw 50mb/sec download speed in tests so far but that means (even if it becomes more speedy) that the download will take ~10sec, even if its actually only 5sec but still a bit strange. so I would love to see logs [08:27] Chipaca: maybe we can even add a log in httpclient "fetched %q with size %v in %v seconds" or something? [08:27] Chipaca: anyway, a fun test issue to track down :) [08:28] ❝fun❞ [08:28] mvo: I asked this last week but I think I missed the response if any [08:28] what is inside that 1GB snap? [08:28] is it 1GB of random data? [08:29] zyga: I think so [08:29] zyga: I don't remember, let me check [08:29] what's the name? [08:29] zyga: I think /dev/urandom because otherwise it compresses too well [08:29] zyga: test-snapd-huge [08:30] thanks [08:30] fetching now [08:30] hm 536MB [08:30] I"m getting ~8MB/s [08:30] is it an electron app? :) [08:31] mborzecki: just the splash screen featuring 4K video [08:31] zyga: haha wish that technology exited in the 90s :P [08:32] offtopic, I recall my Samsung times when the Samsung website, when viewed in Korea, would display a 1080P full screen feed as the _background_ [08:32] the huge snap is actually 0.5GB as mborzecki noted [08:34] hmm my tiny user service for tmux was supposed to remain on logout, i guess it didn't :/ [08:38] zyga: so 0.5gb is not huge enough ;) [08:40] PR snapd#5324 closed: snap: run snap-confine from the re-exec location [08:41] mvo: should I go ahead with the limiting writer? [08:41] PR core18#27 closed: run-snapd-from-snap: start snapd after seeding [08:42] that is, having httputil's logger client slow things down for us [08:44] Chipaca: I would say lets wait a tiny bit to see what logs with timestamps show, i.e. I'm keen to learn what is going on, if e.g. ipstables takes 10s to run. or if it runs and then just does nothing for whatever reason to the running tcp stream [08:45] mvo: [08:45] 2018/06/18 08:44:37.358695 logger.go:82: DEBUG: < "HTTP/1.1 200 OK\r\nContent-Length: 536875008\r\nAccept-Ranges: bytes\r\nAccept-Ranges: bytes\r\nAge: 43918\r\nCache-Control: max-age=86400\r\nConnection: keep-alive\r\nContent-Disposition: attachment; filename=YVl7fwoQCPqOE2PGt5wxZnhu1lSJhvki_1.snap\r\nContent-Type: application/octet-stream\r\nDate: Mon, 18 Jun 2018 08:44:37 GMT\r\nEtag: cee7a99d-0a0d-4981-b2d6-563c667c332c\r\nLast-Modified: Sun, 17 [08:45] Jun 2018 20:32:12 GMT\r\nServer: TwistedWeb/14.0.2\r\nStrict-Transport-Security: max-age=2592000\r\nVary: Accept-Encoding\r\nVia: 1.1 varnish\r\nVia: 1.1 varnish\r\nX-Bzr-Revision-Number: 7463\r\nX-Cache: MISS, HIT\r\nX-Cache-Hits: 0, 0\r\nX-Request-Id: WybFTH8AAQEAAC3@nDwAAAB61\r\nX-Served-By: cache-lcy19224-LCY, cache-iad2645-IAD\r\nX-Timer: S1529311477.354131,VS0,VE1\r\n\r\n" [ 58ms; 9.26GB/s] [08:45] ^^^ 9 YIGAWATTS [08:45] Chipaca: wuuuut? [08:45] Chipaca: thats crazy [08:45] mvo: so, er, that's faster than we'll see if we blink [08:45] also, hot damn, i'm moving to wherever this is :) [08:46] Chipaca: sounds like we need a limiting writer :) [08:46] (it's probably in the USA so heck no) [08:46] Chipaca: well, I'm sure this is fastly inside GCE [08:46] Chipaca: but even then, *amazing* [08:46] mvo: here's hoping it's not fastly from somebody's garage [08:46] Chipaca: I will rent that garage [08:46] Chipaca: I'm sure they make apples there [08:47] Chipaca: anyway, nice find! that is amazing [08:47] I'll push a PR that adds those two bits of info, as it's useful to have [08:47] hm, maybe together with the limiting writer [08:47] Chipaca: aha, I was wondering where it comes from [08:47] Chipaca: did you try this in a gce machine? [08:47] mvo: yes, this is in gce [08:48] this is not my home network [08:48] i don't even get that speed from the switch [08:48] Chipaca: so "snap download test-snapd-huge" in a fresh machine? [08:48] mvo: yes [08:48] Chipaca: amazing [08:48] Chipaca: thanks, I think we have the answer then to the conundrum [08:48] mvo: it's s till up if you want me to try anything further [08:49] * mvo hugs Chipaca and hands him the hero-of-the-day medal [08:49] Chipaca: any chance our CDN is in google? [08:49] so two box spawn on the same rack because google updated the Borg with some AI [08:49] Chipaca: you could download some other big snaps just for fun [08:49] and then then RDMA some memory across [08:49] Chipaca: firefox, some electron apps [08:50] zyga: fastly.cdn.snapcraft.io is at least 3 hops away, on a different ip range [08:50] wow [08:50] then wow :) [08:50] zyga: still 18ms end to end [08:50] mtr thinks it's 5 hops [08:51] (18 ms ping, that is) [08:57] mmmmmm, maybe i was too hasty [08:57] mvo: ^ [08:57] i'm printing the response before reading the body :-( [08:57] * Chipaca needs a brown paper bag, now [08:57] Chipaca: so google doesn't have transwarp internet links? [08:58] zyga: disappointed [08:59] drat [08:59] Chipaca: heh :) [08:59] shoudl've used my head [08:59] Chipaca: I think you mentioned earlier you didn't sleep enough, I would hand you a cup of tea now in a real office :) [09:00] 'time' says snap download takes ~15s [09:00] Chipaca: hm, that *should* be plenty of time [09:00] Chipaca: same if you run it again? [09:00] Chipaca: I mean after "rm *.snap" [09:00] mvo: if I run it again, _without_ removing the snap, it takes 7s [09:01] Chipaca: hm, hm, even that should be enough [09:01] * mvo scratches his head [09:01] tcpdump [09:01] and see what happens [09:01] it takes ~3s to heck the hash [09:02] wow [09:02] HDD IO slow? [09:02] are we checking the hash in-flight? [09:02] zyga: not for the 'already there' case [09:02] zyga: when downloading, yes [09:02] mhm [09:09] * zyga zeroes on another layout fix [09:09] brace for a simple but boring review (large diff) [09:09] well, *test* diff [09:09] actual diff not large :) [09:15] 2018/06/18 09:15:36.223438 store.go:1546: DEBUG: Download succeeded in 5.264175609s ( 102MB/s). [09:15] mvo: ^ a little bit more believable [09:17] Chipaca: yeah, thank you [09:17] Chipaca: still really fast [09:17] i still want that 9GB/s world tho [09:19] heh, indeed [09:35] this is an interesting failure [09:35] https://travis-ci.org/snapcore/snapd/builds/393533806 [09:35] "network availability confirmed" [09:36] then bzzzt and silence [09:38] mvo, zyga, did you guys see the last post on https://forum.snapcraft.io/t/missing-information-in-the-documentation/5969 [09:38] that looks very weird [09:39] (given he is root ... ) [09:39] I replied [09:39] thx [09:39] but I won't look for a while, need to focus [09:39] ogra_: looks like they installed lxd, have running vms, and trying to 'apt remove' everything without uninstalling lxd first [09:40] ah ! [09:40] on the other hand that's just a guess, because they're not telling us what they did [09:40] yeah, it would have helped if he has kept the package names in the paste [09:40] they purposely have edited the output of some stuff, and are hiding or altering info without saying [09:40] *had [09:40] so it's not worth my time to debug further [09:40] * Chipaca might be particularly short of patience today [09:41] also, what, the 'apt remove snapd' produced no output at all? [09:41] it reads like a bad work of fiction [09:42] :shrug: [09:46] PR snapd#5337 opened: store: log a nice clear "download succeeded" message [09:49] Chipaca: would love to see those YB/s downloads [09:50] mborzecki: from quantity.go: // zetta and yotta are me being pedantic: maxuint64 is ~18EB [10:15] PR snapd#5337 closed: store: log a nice clear "download succeeded" message [10:20] mvo, niemeyer hey, I have a school meeting today, perhaps I'll arrive a bit late to the standup [10:21] cachio: ok [10:30] PR snapd#5338 opened: interfaces: move assertions around for better failure line number [10:32] pstolowski: ^ quick ack please [10:33] +1 [10:33] thnx [10:40] opensuse failed twice on zypper network operation [10:43] mborzecki: small conflicts in https://github.com/snapcore/snapd/pull/5314 [10:43] PR #5314: many: rename snap.Info.Name() to snap.Info.InstanceName(), leave parallel-install TODOs [10:50] PR snapd#5339 opened: tests: initial core18 spread image building [10:58] mvo: ^ reviewed [10:59] 2nd review for simple https://github.com/snapcore/snapd/pull/5315 [10:59] PR #5315: cmd/snap-update-ns: introduce mimicRequired helper [11:00] PR snapd#5338 closed: interfaces: move assertions around for better failure line number [11:01] Wimpress: Skype segfaults on startup on my Ubuntu 18.04 Wayland install but there's no clear place to report the bug given in the `contact` field of `snap info skype`? [11:01] *segfaults GNOME Shell [11:02] ads20000: does it work under x? [11:02] (so it plonks me back to the login screen) [11:02] I'll give it a go :) [11:02] I'm going to be a git and say if an app crashes your wayland (or x11), it's a wayland (or x11) bug [11:08] popey: seems to be OK on X, was crashed back to GDM once but I wonder if it's the Polari Flatpak... journalctl didn't seem too clear on that one [11:08] ads20000: you can file a bug now you have it running :) [11:08] ads20000: I had the same symptom with skype on wayland, and no Polari involved [11:08] Help -> Report a problem. [11:09] ah nice thanks popey! :) [11:09] np :) [11:09] but I agree with Chipaca that this is surely a gnome-shell/wayland/something bug (either exclusively, or in addition) [11:10] ah right I can't see those messages since I was logged out [11:10] until they're logged in a couple of hours... [11:10] pstolowski: maybe you again, pretty please: https://github.com/snapcore/snapd/pulls/zyga [11:10] https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1760487 but expired for lack of requested info [11:11] the first one there [11:11] mimic required, green [11:11] Bug #1760487: Shell crashes when I start skype under Wayland session

[11:11] * tomwardill -. lnch [11:12] ads20000: maybe you could follow the instructions in the bug above? I do have a crash from a few days ago but am not completely sure that it was from trying to start skype and I don't want to nuke my session right now ... [11:12] zyga: woah, that was quick :) [11:13] here's my log https://paste.ubuntu.com/p/K8KK65ftbk/ submitted via the form in Skype :) [11:14] reporting to Skype isn't the same as reporting the bug in Xwayland, though [11:14] you should hopefully have a .crash file in /var/crash/ that you can report as per the bug above [11:14] (it may not be a skype bug at all ...) [11:15] cjwatson: on it :) [11:15] great [11:25] PR snapd#5340 opened: cifs-mount-control interface (discussed in snapcraft.io forum post 5689) [11:27] cjwatson: please can you mark yourself as affected by https://bugs.launchpad.net/bugs/1777425 ? I've linked back to it in the other bug report :) [11:27] Bug #1777425: Skype snap segfaults GNOME Shell on Wayland

[11:28] ads20000: done [11:31] ty [11:31] PR snapd#5315 closed: cmd/snap-update-ns: introduce mimicRequired helper === Son_Goku is now known as Conan_Kudo === Conan_Kudo is now known as Son_Goku [11:35] that skype taking down wayland bug is awfully like the slack thing on fedora [11:35] pedronis: hey, added a comment to your gadget PR [11:35] PR snapd#5336 closed: tests: remove double debug: | entry in tests and add more checks [11:49] pstolowski: they are a sort of auto-connection, we can also store the gadget flag in conns, if it will help in the future keeping this under control [11:51] Wimpress: Hi! Need review on https://github.com/snapcrafters/android-studio/pull/28 -- its mostly inspired by your similar pull request for sublime. [11:51] PR snapcrafters/android-studio#28: Embed autodownloads [11:51] PR snapd#5341 opened: cmd/libsnap-confine-private: intoduce helpers for validating snap instance name and instance key [11:52] zyga: ^^ [11:52] yeah, looking [11:53] queued, I'll do it soon [11:55] zyga: thanks [11:55] any PRs in need of a 2nd review? [12:01] mborzecki: I don't think so, though I'm working on one more [12:01] (at least not mine) [12:01] PR snapd#5081 closed: interfaces/apparmor: add chopTree [12:01]