=== dgadomski_ is now known as dgadomski
=== henrix_ is now known as henrix
[10:01] <caribou> Hello, is allocated hugetlb pages accounted for in the outpu of free ?
[10:05] <apw> caribou, i would expect it to be
[10:05] <caribou> apw: that's what I thought
[10:06] <caribou> apw: got two kernel panics caused by OOM; 118 Gb of memory is blocked by hugepages out of 125Gb
[11:24] <apw> caribou, hmmm, what kernel version is that ?
[11:43] <caribou> apw: 4.4.0-34-generic #53~14.04.1-Ubuntu
[12:07] <apw> caribou, i've not heard of anything like that, but it would be worth checking the reserves to see how much is being held 
[12:09] <caribou> apw: I'm reading the code to understand what parameters triggers the OOM
[12:15] <arges> caribou: who allocated and locked the hugepages?
[12:16] <caribou> arges: they're used for QEMU & OpenVswitch
[12:17] <caribou> arges: they leave 8Gb out of 125Gb for the O/S & the rest goes to hugepages; my suspiscion is that they were close to the OOM threshold & something came in to push them over the edge
[12:18] <caribou> since the OOM threshold is calculated on the basis of 125Gb
[12:19] <arges> caribou: did you get a crash dump and use kmem
[12:19] <arges> did you/ were you able to
[12:20] <caribou> arges: yep, go two of them & waiting for another one from a stable system
[12:21] <caribou> arges: oh, & I found something funny : an 'lshw' process using 650Mb of RSS, stucked in doing do_huge_pmd_anonymous_page
[12:21] <arges> maybe there are a lot of pages : )
=== JanC is now known as Guest23252
=== JanC_ is now known as JanC
[15:09] <smoser> hey. i need some help. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1618572
[15:09] <ubot5> Launchpad bug 1618572 in linux (Ubuntu) "cloud-init failing to configure sources and additional repository" [Undecided,Incomplete]
[15:10] <smoser> it appears to me that this is regression in overlayfs coming in xenial kernel
[15:10] <apw> smoser, what makes you think it is the overlayfs ?
[15:14] <_ami_> wow, finally someone speaks! 
[15:18] <smoser> apw, i do.
[15:19] <smoser> i could not reproduce it on a simple overlay mount
[15:19] <smoser> but i cuold reproduce it in 2 ways (apt-key add and a python program)
[15:20] <smoser> i started with a trusty system
[15:21] <smoser> and installed each of the linux-virtual-lts-utopic kernels, booted into them with overlay root and tried recreate
[15:21] <smoser> the failure starts in xenial kernel i'm pretty sure.
[15:22] <smoser> apw, if you need more info on how to recreate i can put it together.
[15:43] <tdaitx> apw, infinity: hey there! it seems the linux powerpc/ppc64el asm/ioctls.h is missing a include for asm-generic/ioctls.h (it's there for the other archs)
[15:44] <tdaitx> and I just noticed that the same happens for termbits.h
=== neoark is now known as Guest3396