[04:46] <jjohansen> hallyn: can I throw your tested by on the patch
[04:48] <hallyn> jjohansen: yup!
[05:56] <alkisg> Good morning all
[05:59] <alkisg> So currently I'm at: problem verified between vanilla 4.15rc4 and 4.15rc5, and I've also done two "git bisect good" steps after that. Each "git bisect" step takes me 1 hour, so I might hopefully finish it today...
[07:00] <f_g> alkisg: welcome to the world of kernel bisecting, where progress can be slow and tiresome :) at least your reproducer is easy and fast - I've had bisects where the test was "do the following non-trivially automatable steps in several VMs, some of which run Windows", with the chance of triggering the problem being about 30%.
[07:01] <f_g> so each build took ~10 minutes, then rebooting/kexecing, then preparing the VMs, then running the test (that took about 20 minutes), then repeat the preparation and test at least 5 times unless the issue triggered
[07:01] <f_g> that was a fun week and a half ;)
[07:01] <alkisg> Meh... well, at least I can do a few things while I'm building the kernel... but I'd really appreciate a monster PC right now, that could build it in under half an hour...
[07:01] <thresh> :S
[07:02] <alkisg> One of the builds needed 88 mins because I was doing some other work simultaneously :(
[07:14] <alkisg> Testing time :) (unfortunately I can only reproduce it on my work pc, not any spare ones!)
[07:20] <alkisg> 3rd bisect => good. I'm afraid the problem will be the pti patches that came right before rc5.
[07:28] <f_g> alkisg: three good beside rc4? or including?
[07:33] <alkisg> f_g: rc4 good, rc5 bad, then git bisect good 3 times
[07:35] <alkisg> $ git bisect good
[07:35] <alkisg> Bisecting: 20 revisions left to test after this (roughly 4 steps)
[07:35] <alkisg> [ed1bbc40a0d10e0c5c74fe7bdc6298295cf40255] x86/cpu_entry_area: Move it to a separate unit
[07:37] <f_g> alkisg: sounds like you should be done soon, and that it's like one of the PTI changes.
[07:37] <alkisg> well, in 4 hours... :(
[07:39] <f_g> out of curiosity, did you test -rc8 as well? if not, you might want to add that to your next reboot cycle..
[07:40] <alkisg> Yes, that was one of my very early tests, but only using the mainline .deb, not with compiling
[07:41] <f_g> nevermind then
[07:41] <alkisg> (I also tested the daily deb)
[09:36] <alkisg> git status ==> HEAD detached at ed1bbc4, You are currently bisecting, 
[09:36] <alkisg> ...and yet that gave me a kernel named 4.14? Is that possible?
[09:36] <alkisg> (while bisecting between 4.15rc4 and 4.15rc5)
[09:38] <alkisg> In bisection step 3, I got this: linux-image-4.15.0-rc4-custom_4.15.0-rc4-custom-6_i386.deb
[09:38] <alkisg> Then moving forward (git bisect good), I got this: linux-image-4.14.0-custom_4.14.0-custom-7_i386.deb
[09:39] <alkisg> ...can that be because some older commit got merged later on, or does it sound like I did something wrong?
[09:41] <alkisg> Anyway continuing with: $ git bisect good/ Bisecting: 10 revisions left to test after this (roughly 3 steps)/[9c294ec08408ed90c0f2d994a7979366675e3734] Merge tag 'powerpc-4.15-5' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
[10:07] <f_g> alkisg: yes, that is usually an artifact of bisecting into some merged branch which hasn't been rebased prior to getting merged. the git DAG in linux-stable looks rather interesting :P
[10:13] <alkisg> ty! I got worried for my sanity there for a while :D
[11:27] <alkisg> $ git bisect good / Bisecting: 5 revisions left to test after this (roughly 3 steps) / [24e3a7fb60a9187e5df90e5fa655ffc94b9c4f77] libnvdimm, btt: Fix an incompatibility in the log layout
[15:52] <alkisg> Yey for grub-reboot, it allows to continue this remotely :)
[15:54] <alkisg> $ git bisect good/ Bisecting: 2 revisions left to test after this (roughly 2 steps) /[f6c4fd506cb626e4346aa81688f255e593a7c5a0] x86/cpu_entry_area: Prevent wraparound in setup_cpu_entry_area_ptes() on 32bit
[16:01] <alkisg> So, I'm guessing I'll find that the problem was https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=caf9a82657b313106aae8f4a35936c116a152299, the pti preparation patches.
[16:01] <alkisg> What next? File a bug report in Ubuntu? Or directly upstream? Or both?
[16:02] <alkisg> Or "blame" that commit on github?
[16:04] <alkisg> Also, would I be able to test the latest version without that huge patch? I'm not sure it can be cleanly un-applied in the latest version...
[16:23] <mattsm> do any scripts / automation exist to copy a ubuntu kernel to a remote machine and boot into it?/
[16:37] <alkisg> mattsm: I don't know, but it shouldn't be hard to do this with ssh/scp/grub-reboot...
[16:39] <mattsm> alkisg yea i was just curious... for grub-reboot I'd need to add an entry for it is all i suppose
[16:39] <alkisg> mattsm: normally, installing a kernel add the grub entr
[16:40] <alkisg> grub-reboot then just selects that for the next reboot
[16:40] <mattsm> well, i need to copy kernel + modules and add grub entry... and try it out
[16:41] <mattsm> was just hoping for a script that copied stuff to a target machine existed
[16:41] <alkisg> mattsm: ah, you're not using a .deb package that does all that automatically?
[16:43] <mattsm> alkisg im attempting to rebuild a kernel on a remote machine and copy it over
[16:58] <TJ-> mattsm: you can also reboot into another kernel using kexec - but best to test it on the actual target hardware in person first before relying on it for remote reboot :)
[16:58] <mattsm> TJ-: i've got remote power reset, so that would be OK
[17:01] <TJ-> mattsm: in which case after rsync-ing the /boot/vmlinuz-$VERSION, /boot/initrd.img-$VERSION and /lib/modules/$VERSION/* you can do "sudo kexec -l /boot/vmlinuz-$VERSION --initrd /boot/initrd.img-$VERSION --reuse-cmdline" (that  loads them into memory) you can do either "sudo systemctl reboot" or for a more abrupt switch "sudo kexec -e"
[17:01] <mattsm> TJ-: that sounds like it will work pretty good, also it seems like the kernel build instructions do in fact build a deb as well
[17:02] <TJ-> Although recently I've found kexec seems to be broken on every hardware and kernel version combination I've tried
[17:03] <mattsm> a major downside
[17:04] <TJ-> mattsm: if you're doing "fakeroot debian/rules binary" yes that generates the packages in the parent directory
[17:04] <mattsm> does anyone know where the zesty kernel configs are?
[17:05] <mattsm> TJ-: yea im playing with that
[17:05] <mattsm> i suppose that will put the config somewhere?
[17:05] <TJ-> mattsm: they're combined at build-time from fragments (-common + -flavour specific)
[17:05] <TJ-> mattsm: the generated config will be in the build directory as .config 
[17:06] <mattsm> TJ- nice, i  think i have a zesty kernel building
[17:12] <OliverO> Hi everyone!
[17:12] <OliverO> Any idea why v4.14.14 (released 2 days ago) build does not show up in http://kernel.ubuntu.com/~kernel-ppa/mainline?
[17:43] <apw> alk
[17:43] <chiluk> OliverO it could be getting held up by requisite changes to gcc in order to enable retpoline.  If I was the kteam that's what I would expect.
=== ben_r_ is now known as ben_r
[17:52] <mattsm> What gcc version does Ubuntu intent to use for building zesty kernel?
[17:55] <mattsm> and is there a way to tell fakeroot which version to use?
[18:25] <tomreyn> !zesty | mattsm 
[18:25] <ubot5> mattsm: Ubuntu 17.04 (Zesty Zapus) was the 26th release of Ubuntu. Support ended on January 13th, 2018. See !eol and !eolupgrade
[18:25] <tomreyn> just in case you're not aware of the EOL status
[18:26] <tomreyn> other than that, to actually answer your question, my guess would be: the gcc version available in zesty.
[18:27] <mattsm> hmm which tree is in the next version?
[18:27] <tomreyn> !info gcc artful
[18:27] <ubot5> gcc (source: gcc-defaults (1.173ubuntu1)): GNU C compiler. In component main, is optional. Version 4:7.2.0-1ubuntu1 (artful), package size 5 kB, installed size 64 kB
[18:28] <mattsm> !artful | mattsm
[18:28] <ubot5> mattsm, please see my private message
[18:29] <tomreyn> note my guess can be very wrong, i'm not sure how / where default kernel image packages are built.
[18:30] <mattsm> you made it clear to me i was confused as well
[18:30] <mattsm> i wanted the next kernel version, so bionic beaver...?
[18:30] <tomreyn> but i would indeed expect that building them with the gcc version of the same ubuntu release will work without an issue
[18:31] <tomreyn> bionic is going to be 18.04 lts
[18:31] <mattsm> right
[18:31] <tomreyn> zesty was 17.04, artful is 17.10
[18:31] <tomreyn> so 'next' can mean many things ;)
[18:38] <mattsm> well, there is an ubuntu-bionic tree in the same place
[18:38] <alkisg> $ git bisect bad / Bisecting: 0 revisions left to test after this (roughly 1 step) / [613e396bc0d4c7604fba23256644e78454c68cf6] init: Invoke init_espfix_bsp() from mm_init()
[18:39] <alkisg> At least it's NOT the pti preparation patches :)
[18:56] <mattsm> Does anyone know if there is a wiki for getting changes into an Ubuntu kernel release....
[19:11] <TJ-> mattsm: bug report, plus email to the kernel-team mailing list
[19:13] <mattsm> TJ- perfect, thanks
[19:56] <alkisg> $ git bisect bad / Bisecting: 0 revisions left to test after this (roughly 0 steps) / [92a0f81d89571e3e8759366e050ee05cc545ef99] x86/cpu_entry_area: Move it out of the fixmap
[19:57] <alkisg> ...so I guess I'll file this to bugzilla.kernel.org...
[20:20] <alkisg> https://bugzilla.kernel.org/show_bug.cgi?id=198529
[20:20] <ubot5> bugzilla.kernel.org bug 198529 in x86-64 "Reboot on kernel load due to 92a0f81d" [High,New]
[20:21] <TJ-> alkisg: best to send an email to the maintainer and sub-system mailing list (using "scripts/get_maintainer.pl  path/to/source/file.c" )
[20:22] <alkisg> TJ-: after reporting the bug, bugzilla said it notified his mail
[20:22] <alkisg> Is that enough, or should I also send it to the sub system mailing list?
[20:22] <TJ-> alkisg: oh, cool, i've not noticed it do that for my reports!
[20:22] <alkisg> Email sent to:            bugsfx@gmail.com,          tglx@linutronix.de
[20:22] <alkisg> That second mail there is the committer
[20:23] <TJ-> alkisg: yeah, that's Gleixner :)
[20:24] <alkisg> I saw a follow up commit where fixes a breakage in 32bit, and he blames "the moron that wrote the code", and I think that's again himself... sounds like he has a sense of humor :D
[20:28] <TJ-> alkisg: he signed off under protest on a commit for 4.14 that fixed a regression for nvidia that I reported with " Despised-by: Thomas Gleixner <tglx@linutronix.de>" (commit 87df26175e67)
[20:29] <alkisg> Haha