[09:03] hi there [09:04] quick question about running the cloudimages on an openstack platform and specificaly io performance differences between stock xenial and stock trusty with a xenial-lts kernel installed through apt [09:04] are there differences in the kernel options? [09:06] the xenial cloud image with kernel 4.4.0-24-generic has almost 2 times better IO performance than the trusty cloud image with the xenial-lts 4.4.0-34-generic kernel [09:06] which i find surprising as i would expect the same performance [12:56] jsheeren: just a theory, but the xenial kernel is built with a newer gcc [12:58] 5.4.0 vs 4.8.4 [12:58] better/newer userspace too? (depending on how the testing is done....) [12:58] that too [12:59] so the xenial kernel for xenial is built with 5.4.0, the xenial kernel for trusty is built with 4.8.4? [13:00] yep [13:01] jsheeren, the package for each release is built in the target release build chroot. So all of toolchain for -lts kernels is from the target distro. Everything including things like binutils, glibc, compilers, etc. [13:02] the backport kernels are just that -> new source code, new package name, compiled on an old release. [13:02] just to add hardware compatibility, but not to enhance performance, etc? [13:02] it's not a .deb / binary copy. (that would result in naming clash) [13:03] is it doable to upgrade the gcc on trusty to the version on xenial? or are there too many dependencies that will bork? [13:03] jsheeren, there are multiple reasons and multiple usecases as to why hwe kernels were started to be provided. It's a mix of hardware compat / performance / (new)security features [13:03] no no no no [13:03] that would break the ABI [13:03] on c++ [13:03] ok [13:04] i mean kernel is special, cause it's stand-alone / freestanding code. [13:04] but we would not do that.... Upgrade to xenial perhaps? =) [13:04] it is shiny =) [13:05] it's got the "new car" scent [13:06] xnox: yeah if we run a do-release-upgrade on trusty so we end up with a xenial, the performance issue is the same [13:06] we've looked at a lot of stuff [13:06] using the same hypervisor, same openstack flavor, etc [13:06] a bit stumped at the moment on why we are seeing the IO performance difference that is so large at the moment [13:08] we compared the sysctl settings as well but did not see any differences that would impact IO [13:08] jsheeren, your findings would be interesting. To rule things out, you /could/ *warning not supported* enable xenail repository on trusty, pin it down below trusty with apt-pinning, install/upgrade generic kernel from the xenial repository with e.g. apt install linux-generic/xenial, see what happens. [13:08] ah yeah, could try that [13:08] this way you will rule out if the $userspace in xenial affecting benchmarks, or indeed the 4.4 kernel compiled on xenial, when running trusty userspace, is better. [13:08] we tried copying the files from a xenial to a trusty but that did not go so well ... heh [13:09] yeah.... partial apt upgrade to xenial to get new kernel from xenial rather than via hwe-stack is *safer*. Still a frankenstein =) [13:10] https://gist.github.com/beci/2a2091f282042ed20cda this is a no go then? === Elimin8r is now known as Elimin8er [22:49] I take it that the 9134 version is the latest.... Quite a step change from the last one?... [22:49] phillw@piglet:~$ ls -lt /boot/abi* [22:49] -rw-r--r-- 1 root root 1241623 Aug 16 23:23 /boot/abi-4.4.0-9134-generic [22:49] -rw-r--r-- 1 root root 1241623 Jul 27 22:28 /boot/abi-4.4.0-34-generic [22:49] Kindest Regards, [22:49] Phill.