| Son_Goku | zyga, I need karma for these updates: | 02:14 |
|---|---|---|
| Son_Goku | F28: https://bodhi.fedoraproject.org/updates/FEDORA-2019-2d75eaae81 | 02:14 |
| Son_Goku | EL7: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-be55f1a139 | 02:15 |
| Son_Goku | F29: https://bodhi.fedoraproject.org/updates/FEDORA-2019-a93e7637d2 | 02:15 |
| mup | PR snapd#6697 opened: interfaces/daemon_notify: add {net,sys}_admin capabilities, update spread test <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/6697> | 02:30 |
| === chihchun_afk is now known as chihchun | ||
| mborzecki | morning | 05:16 |
| mborzecki | google:ubuntu-18.10-64:tests/main/refresh-app-awareness failing randomly? | 05:19 |
| zyga | Good morning | 05:28 |
| zyga | Oh | 05:28 |
| zyga | How? | 05:28 |
| zyga | I am afk with the dog | 05:28 |
| mborzecki | zyga: hey | 05:36 |
| zyga | Back | 05:44 |
| zyga | mborzecki: how are you doing | 05:45 |
| zyga | mborzecki: I'll test neal's package update | 05:46 |
| mborzecki | zyga: found some weird things about snap[d] binaries with mvo on friday | 05:50 |
| zyga | what did you find? | 05:50 |
| mborzecki | zyga: we don't know why that's happening yet, but armhf binaries coming from core snap would attempt to perform mmap2() with ridiculously large sizes | 05:51 |
| mborzecki | zyga: afaik that's the initial allocations that go runtime attempts | 05:51 |
| zyga | anonymous mapping? | 05:51 |
| zyga | go's new GC | 05:51 |
| mborzecki | yes, MAP_ANONYMOUS, and some were done with | MAP_FIXED | 05:52 |
| zyga | oh | 05:52 |
| zyga | that is interesting | 05:52 |
| mborzecki | for instance, the `snap` comand would try to alocate 150MB with MAP_ANONYMOUS | MAP_FIXED, and then another 800M with MAP_ANONYMOUS | 05:52 |
| zyga | fixed implies they use address-based zones | 05:52 |
| mborzecki | i was like wtf? | 05:52 |
| zyga | funny | 05:52 |
| zyga | I was reading a book about this over weekend | 05:53 |
| zyga | https://www.amazon.de/Garbage-Collection-Handbook-Automatic-Management/dp/1420082795/ref=sr_1_1?__mk_pl_PL=ÅMÅŽÕÑ&keywords=garbage+collection&qid=1554702790&s=gateway&sr=8-1 | 05:53 |
| zyga | I recommend it | 05:53 |
| zyga | that's the new garbage collector in place | 05:53 |
| zyga | since it's just address space it's okay | 05:53 |
| zyga | but pehraps it has some side effects | 05:53 |
| mborzecki | fwiw, i think it's exhausting system wide overcommit limits | 05:53 |
| mborzecki | zyga: and we found some bug reports about that for 32bit arches | 05:54 |
| zyga | mmm | 05:54 |
| zyga | so | 05:54 |
| zyga | google doesn't really care about 32bit | 05:55 |
| mborzecki | zyga: oh, and another thing, i dropped a snap binary built by me (although cross compiled), but it had none of the crazy mmap() calls | 05:55 |
| zyga | mmmm | 05:55 |
| zyga | which go versions were involved? | 05:55 |
| mborzecki | zyga: theoretically same go version as in xenial (1.10.4), though i got mine from golang.org | 05:55 |
| zyga | what kind of patches do we see in ubuntu? | 05:56 |
| mborzecki | zyga: haven't looked at that yet, we stopped at ~11pm on friday :P | 05:56 |
| zyga | mborzecki: good results! | 05:56 |
| zyga | mborzecki: I'm eager to learn more as you discover what's changed | 05:56 |
| zyga | mborzecki: one hint: | 05:56 |
| zyga | look at golang's source code to see if the new garbage collector is uncoditional and see if there are any knobs (environment?) to control it | 05:57 |
| zyga | perhaps we cannot reverse course on how go behaves but we can tweak it enough to be sufficient for our needs | 05:57 |
| zyga | though I strongly worry about the long term viability of 32bit snapd | 05:57 |
| zyga | unless we change our model | 05:57 |
| zyga | (e.g. snapd doesn't run unless API calls are in place) | 05:57 |
| zyga | (and snapd has more smaller binaries for particular tasks) | 05:58 |
| zyga | (e.g. only daemon runs and dispatches to new binaries for tasks) | 05:58 |
| mborzecki | zyga: some pages i have open since friday: https://github.com/golang/go/issues/21044 https://github.com/golang/go/issues/28114 https://github.com/golang/go/commit/2b415549b813ba36caafa34fc34d72e47ee8335c | 05:59 |
| mborzecki | zyga: idk, but armhf is 32bit, rpi3 is still running 32 userspace, unless you forfeit the gpu and run 64bit instead | 06:00 |
| mborzecki | zyga: oh, and if anyone uses ubuntu core on atom, those are 32bit too | 06:00 |
| zyga | mborzecki: sure, I know we will be stuck with 32bit systems for al ong time | 06:00 |
| mborzecki | zyga: btw. on friday we got to a state, where it wasn't possible to run any snap command :P, you'd type snap list and got runtime: out of memory :P | 06:01 |
| zyga | rust baby ;-) | 06:02 |
| zyga | I'd love a rust dispatcher for tasks with go backends to run stuff we have already | 06:02 |
| zyga | snapd taking 3MB of ram while idle | 06:02 |
| zyga | we could call it "project schizo" | 06:03 |
| mborzecki | heh | 06:04 |
| mborzecki | zyga: fwiw, if any, i think it's a bug somewhere in the compiler or runtime | 06:04 |
| zyga | bug as in undesired behavior? | 06:04 |
| zyga | perhaps | 06:04 |
| mborzecki | zyga: the binaries that i built would ask for 2M not 150M | 06:04 |
| zyga | I doubt it is an actual algorithm bug | 06:05 |
| zyga | I wonder if cross compiling is a factor | 06:05 |
| zyga | perhaps that is the bug | 06:05 |
| zyga | compile natively on arm perhaps? | 06:05 |
| mborzecki | yeah, idk, that's why i'm setting up my rpi | 06:05 |
| zyga | like our builders do | 06:05 |
| zyga | ok :) | 06:05 |
| mborzecki | btw. mvo mentioned when he passed mem=256 on kernel command rpi classic wouldn't boot, wonder why | 06:06 |
| zyga | memory split for gpu? | 06:08 |
| zyga | I think though that modern systems struggle at that amount | 06:08 |
| zyga | (128MB) | 06:08 |
| mborzecki | i should buy a new case for my rpi, one that exposes the gpio header | 06:17 |
| zyga | mborzecki: just find one you like and I'll print you a copy | 06:18 |
| zyga | I have some spare official cases | 06:18 |
| zyga | those expose the gpio header but not in a useful way for serial | 06:18 |
| zyga | hmm, I have issues with my fedora system :/ | 06:38 |
| mvo | zyga: hey, good morning | 06:40 |
| zyga | good morning :) | 06:40 |
| mborzecki | mvo: hey | 06:49 |
| mvo | hey mborzecki | 06:50 |
| zyga | "a start job is running for /dev/mapper/fedora" | 06:50 |
| zyga | eh | 06:50 |
| mvo | mborzecki: anything new in armhf land :) ? | 06:50 |
| zyga | I think my disk got corrupt somehow | 06:50 |
| mborzecki | mvo: not yet, pulled down rpi image, flashed, and nothing on the serial port :/ maybe my uart adapter dones't work anymore | 06:51 |
| zyga | mborzecki: screen? | 06:51 |
| mvo | mborzecki: meh, ok | 06:51 |
| zyga | mborzecki: how did you wire the serial? | 06:51 |
| mborzecki | zyga: same as usual, idk i dragged it some meetups recently, maybe something got loose | 06:52 |
| zyga | I gave up on my fedora system | 06:56 |
| zyga | it's odd because it can boot to recovery | 06:56 |
| zyga | but not to normal mode | 06:56 |
| zyga | I will try some more later | 06:56 |
| * zyga breaks for quick breakfast and then back to RPs | 06:56 | |
| zyga | PRs | 06:56 |
| pstolowski | morning | 07:10 |
| zyga | back from breakfast | 07:12 |
| zyga | hey pawel | 07:12 |
| mvo | ppisati: can I run a 64bit pi3 kernel on a 32bit userland, what do I need to do to get this working? tracking down a nasty go bug and I want to replicate our buildd setup as closely as possible | 07:49 |
| mvo | sil2100: do you know if we have a machine to native build on arm that is close to a buildd? i.e. 64bit kernel, 32bit userland ? I tried the armhf porter box but for some reason I don't get access there | 07:50 |
| pedronis | mvo: mborzecki: hi, is this that you are observing: https://github.com/golang/go/issues/28081 ? | 07:50 |
| mborzecki | pedronis: maybe, we looked at that on friday too | 07:51 |
| mborzecki | pedronis: if you scroll back, i liked some other issues too | 07:51 |
| mvo | pedronis: we are still a bit in the dark, the worst part it is not reproducible on a pi2 (neither core nor classic) | 07:53 |
| mvo | pedronis: and it goes away when cross building on amd64 and scp to arm even with the same toolchain (or we think its the same toolchain, we need to 100% replicate I think) | 07:53 |
| mvo | pedronis: some (very hard to read notes) https://gist.github.com/mvo5/f3a4e330f8bf32bd514d9a038665f591 - mborzecki has a similar stakc of notes | 07:54 |
| mvo | pedronis: my theory right now is that the allocator tries to map a way to large chunk of "mheap_arena_used" but its totally unclear why - it may be a red herring but its definitely wrong and we don't see it anywhere else | 07:55 |
| pedronis | mvo: well they tell they are allocating much larger chunks | 07:55 |
| pedronis | 64mb vs 2mb | 07:55 |
| mborzecki | pedronis: fwiw, there's a request for 800M at some point | 07:55 |
| pedronis | ah | 07:56 |
| mborzecki | in the notes, ther's a strace log, first mmap requests 115MB (well, large but ok), and then the next requests 800M+ | 07:56 |
| mborzecki | and that's when even snap commands would keep failing | 07:56 |
| pedronis | mborzecki: that's on the device ? | 07:57 |
| mborzecki | pedronis: yes | 07:57 |
| pedronis | but we don't see it on rpi2 | 07:58 |
| pedronis | (so far) | 07:58 |
| mvo | mborzecki, pedronis AIUI the big problem is MAP_FIXED here | 07:58 |
| sil2100 | mvo: hm hm, did you have access to the porter boxes previously? | 08:04 |
| pedronis | mborzecki: are we seeing the same allocation (800M+) on rpi2 and they succeed ? or we not seeing them instead? | 08:08 |
| mvo | sil2100: do we have a arm porter box? maybe I was doing it wrong | 08:16 |
| mborzecki | wow, my rpi3 does not even boot anymore | 08:30 |
| === chihchun is now known as chihchun_afk | ||
| mvo | mborzecki: did you add mem=256M the the kernel commandline ? I tried that and nothing worked anymore | 08:49 |
| mvo | mborzecki: as in - I would not even get kernel messages that it initializes correctly | 08:50 |
| mborzecki | mvo: i can't boot any of the rpi images on my rpi3, only raspbian works | 08:50 |
| mvo | mborzecki: woah, thats strange | 08:50 |
| zyga | mborzecki: do you have the 3B+ model? | 08:50 |
| zyga | that is not supported yet officially AFAIR | 08:50 |
| zyga | for that reason my 3B+ is also running raspbian | 08:51 |
| zyga | mborzecki: note, if you need a 3B I can arrange on in ~1 hour | 08:51 |
| zyga | exposed over the network for you | 08:51 |
| mborzecki | zyga: i used a core image before on this device | 08:51 |
| ppisati | mvo: if you don't use uboot, copy the arm64 kernel in the vfat partition and call it kernel8.img, remove kernel7.img, and have a fairly recent firmware, it should work | 08:51 |
| zyga | mborzecki: oh | 08:51 |
| zyga | that's new | 08:51 |
| mborzecki | i remembed i did some tweaking, but don't recall what exactly that was | 08:52 |
| mvo | ppisati: thank you! | 08:55 |
| === chihchun_afk is now known as chihchun | ||
| zyga | mvo, sil2100: https://github.com/snapcore/core18/pull/126 | 09:14 |
| mup | PR core18#126: hooks: reduce snapd skeleton directories <Created by zyga> <https://github.com/snapcore/core18/pull/126> | 09:14 |
| mup | PR core18#126 opened: hooks: reduce snapd skeleton directories <Created by zyga> <https://github.com/snapcore/core18/pull/126> | 09:15 |
| === cpaelzer__ is now known as cpaelzer | ||
| === chihchun is now known as chihchun_afk | ||
| === chihchun_afk is now known as chihchun | ||
| zyga | mborzecki: question | 10:22 |
| zyga | mborzecki: when I mkdir a thing from snap-confine | 10:22 |
| zyga | mborzecki: should I worry about selinux label? | 10:22 |
| zyga | mborzecki: specifically, can you review this patch https://github.com/snapcore/snapd/pull/6642/commits/44a0f5283bbf753151e45c6d3fdf7482bf6c2430 | 10:24 |
| mup | PR #6642: cmd/snap-confine: prevent cwd restore permission bypass <Created by zyga> <https://github.com/snapcore/snapd/pull/6642> | 10:24 |
| zyga | er | 10:24 |
| zyga | https://github.com/snapcore/snapd/pull/6642/commits/47018f084e66965ef51a67bc80c8820be8b75893 | 10:24 |
| mborzecki | afaict it should be fine, /var/lib/snapd is snappy_var_t, everything created under that dir will get the same label | 10:29 |
| mup | PR snapd#6698 opened: overlord/ifacestate: introduce HotplugKey type use short key in change summaries <Created by stolowski> <https://github.com/snapcore/snapd/pull/6698> | 10:37 |
| zyga | pstolowski, mborzecki: thank you for the reviews for https://github.com/snapcore/snapd/pull/6643 | 11:29 |
| mup | PR #6643: tests: deny ioctl - TIOCSTI with garbage in high bits <Created by zyga> <https://github.com/snapcore/snapd/pull/6643> | 11:29 |
| zyga | I updated the branch in response to the review | 11:30 |
| zyga | I would appreciate a 2nd look | 11:30 |
| * zyga switched to investigate google:ubuntu-16.04-64:tests/main/refresh-app-awareness | 11:31 | |
| mborzecki | looks like we may indeed have some memory problems with go 1.7 - 1.10, https://paste.ubuntu.com/p/FqczCC22WN/ | 11:32 |
| mborzecki | that's a request that actually counts towards overcommit, go.16 requested 1M, 1.7-1.10 requests 17M, 1.11 requests 4M | 11:33 |
| pstolowski | np, zyga: will do later today! | 11:33 |
| mborzecki | oh, and that's for the minimal program which does virtually nothing | 11:34 |
| pstolowski | mborzecki: ouch | 11:35 |
| zyga | in case anyone notices regressions in refresh-app-awareness, please let me know | 11:41 |
| zyga | Chipaca: hello sir, would you mind attempting to review https://github.com/snapcore/snapd/pull/6690 today? | 11:43 |
| mup | PR #6690: overlord/snapstate: inhibit refresh for up to a week <Created by zyga> <https://github.com/snapcore/snapd/pull/6690> | 11:43 |
| Chipaca | zyga: I would not | 11:43 |
| Chipaca | zyga: but, note today I am not doing too good | 11:44 |
| zyga | I wish you to get better soon, I understand | 11:44 |
| pedronis | pstolowski: I did pass a on #6662 | 11:50 |
| mup | PR #6662: overlord/snapstate,snapshotstate: create snapshot on snap removal (3/4) <Created by stolowski> <https://github.com/snapcore/snapd/pull/6662> | 11:50 |
| pedronis | zyga: hi, have you seen the incoming comments on the new tmp logic ? | 11:50 |
| pstolowski | pedronis: ty, also for the remarks on 4/4! | 11:51 |
| pedronis | pstolowski: np | 11:53 |
| zyga | pedronis: hey, perhaps not, where are they? | 11:54 |
| pedronis | zyga: https://github.com/snapcore/snapd/pull/6614#discussion_r272974008 | 11:55 |
| mup | PR #6614: cmd/snap-confine: use fixed private tmp directory <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/6614> | 11:55 |
| zyga | ah, interesting, I didn't notice that | 11:56 |
| zyga | thank you | 11:56 |
| pedronis | zyga: I will look at 6690 today | 11:56 |
| zyga | thank you! | 11:57 |
| zyga | I will check if there are any other comments on closed branches | 11:57 |
| zyga | my github.com/notifications is rather long | 11:57 |
| === chihchun is now known as chihchun_afk | ||
| mup | PR snapd#6699 opened: daemon: add RootOnly flag to commands <Simple 😃> <Created by chipaca> <https://github.com/snapcore/snapd/pull/6699> | 12:05 |
| zyga | pstolowski: reviewed https://github.com/snapcore/snapd/pull/6698#pullrequestreview-223799068 | 12:05 |
| mup | PR #6698: overlord/ifacestate: introduce HotplugKey type use short key in change summaries <Created by stolowski> <https://github.com/snapcore/snapd/pull/6698> | 12:05 |
| zyga | Chipaca: is https://github.com/snapcore/snapd/pull/6699/files driven by the suse audit? | 12:07 |
| mup | PR #6699: daemon: add RootOnly flag to commands <Simple 😃> <Created by chipaca> <https://github.com/snapcore/snapd/pull/6699> | 12:07 |
| Chipaca | zyga: no, it's one of the changes to users raised last week | 12:07 |
| Chipaca | there's another two coming | 12:07 |
| Chipaca | (one of which needs agreement before i do it) | 12:07 |
| zyga | so it's unrelated to the security review we received? | 12:07 |
| Chipaca | maybe even three | 12:07 |
| Chipaca | zyga: it changes nothing security-wise beyond perhaps making the logic easier to follow | 12:08 |
| zyga | understood, thank you | 12:08 |
| Chipaca | (which is a win) | 12:08 |
| Chipaca | zyga: one of the things coming is to not have the user endpoints when not needed/supported, which might impact the sec review | 12:08 |
| zyga | thank you | 12:09 |
| Chipaca | zyga: another change is moving the user stuff to api_user | 12:09 |
| Chipaca | zyga: a further change that i'd like to do but it's arguably v3 material is moving all the user actions to /v2/users | 12:09 |
| zyga | Chipaca: I reviewed the auth code | 12:12 |
| === ricab is now known as ricab|lunch | ||
| * zyga -> small break | 12:28 | |
| pedronis | Chipaca: hi, what's the status of #6653 ? | 12:33 |
| mup | PR #6653: tests: try to be a little bit invariant <Created by chipaca> <https://github.com/snapcore/snapd/pull/6653> | 12:33 |
| Chipaca | pedronis: need to fix it | 12:34 |
| * zyga reads tests/main/security-private-tmp and is puzzled | 12:43 | |
| zyga | looking at it now | 12:43 |
| === chrisccoulson_ is now known as chrisccoulson | ||
| === chihchun_afk is now known as chihchun | ||
| pedronis | Chipaca: standup? | 13:07 |
| Chipaca | pedronis: ye | 13:07 |
| zyga | ah, I think I understand what the problem is | 13:19 |
| zyga | with refresh awareness thing | 13:19 |
| === ddstreet_away is now known as ddstreet | ||
| zyga | re, partially | 14:24 |
| zyga | mborzecki: I'm fixing that bug you ran into in the morning | 15:18 |
| zyga | just still in partial capacity away from the office | 15:18 |
| Chipaca | what is /usr/lib/sysimage/rpm/Basenames and why do i hate it | 15:19 |
| zyga | whaat? | 15:19 |
| zyga | no idea, which system is that on? | 15:19 |
| Chipaca | zyga: opensuse-15.0-64 | 15:44 |
| pstolowski | zyga: have you seen my general meta-question re #6643? | 16:14 |
| mup | PR #6643: tests: deny ioctl - TIOCSTI with garbage in high bits <Created by zyga> <https://github.com/snapcore/snapd/pull/6643> | 16:14 |
| Chipaca | hrmph, it looks like core18 spread tests are rather messy wrt leaving core behind | 17:02 |
| === pstolowski is now known as pstolowski|afk | ||
| Chipaca | EOD for me | 17:22 |
| Chipaca | o/ | 17:22 |
| zyga | pstolowski|afk: not yet, looking now | 17:22 |
| zyga | pstolowski|afk: responded | 17:24 |
| pstolowski|afk | zyga: ty | 17:24 |
| * zyga resumes work | 17:25 | |
| zyga | sorry, took a nap after dinner | 17:25 |
| zyga | hmm | 17:43 |
| zyga | core18 prepare hanging ... ? https://www.irccloud.com/pastebin/MyVsgbcc/ | 17:43 |
| zyga | hmm | 17:47 |
| zyga | but the system has seeded | 17:47 |
| zyga | changes from the system https://www.irccloud.com/pastebin/loXhyaB3/ | 17:47 |
| zyga | I think I fixed that now | 17:58 |
| zyga | another round of testing... | 17:58 |
| zyga | almost there | 18:59 |
| om26er | is there a way to tell multipass through snapcraft to use more cores when packing a snap ? currently it default to 2, I would like it to use all available threads | 19:12 |
| om26er | zyga Hey! Were you able to find a fix for https://bugs.launchpad.net/snapd/+bug/1819318 ? | 19:23 |
| mup | Bug #1819318: no interfaces when installing only core18 <snapd:In Progress by zyga> <https://launchpad.net/bugs/1819318> | 19:23 |
| om26er | So we have a server software as a snap and it won't build on 16.04 because a package it depends on, requires a version of python that is not in 16.04 | 19:24 |
| om26er | If we go with base: core18 then the above issue is problematic as Ubuntu Server on many VPSs doesn't come with core preinstalled | 19:25 |
| zyga | and ... got it | 19:28 |
| zyga | om26er: yes, but it's not near yet | 19:28 |
| zyga | om26er: we need to install core in such cases | 19:28 |
| zyga | I spent some time researching this but it's the closes solution for now | 19:28 |
| * zyga just understood a pair of bugs | 19:29 | |
| om26er | that's kind of the workaround I have already been using, even before discovering ogra's bug ;-) | 19:30 |
| om26er | will see if we can update our docs... | 19:30 |
| zyga | it will get properly fixed by snapd.snap | 19:30 |
| zyga | but for now... | 19:30 |
| zyga | nothing else | 19:30 |
| om26er | re: the python plugin in snapcraft, is there a way to set the `source` to a specific branch ? | 19:56 |
| zyga | GAAH | 20:19 |
| roadmr | /o\ | 20:22 |
| mup | PR snapcraft#2524 closed: Snapcraft try <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2524> | 20:35 |
| mup | PR snapcraft#2519 closed: packaging: use local patchelf <Created by cmatsuoka> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2519> | 20:38 |
| mup | PR snapcraft#2525 opened: catkin plugin: check stage-snaps for ROS dependencies <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2525> | 20:38 |
| mup | PR # closed: core-build#11, core-build#22, core-build#26, core-build#37 | 20:39 |
| mup | PR # opened: core-build#11, core-build#22, core-build#26, core-build#37 | 20:40 |
| mup | PR snapcraft#2526 opened: Release changelog for 3.4 <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2526> | 20:44 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!