[06:48] Hi, which kernel is 19.10 mostly likely going to ship with? We want to prepare some out-of-tree drivers before release... [06:54] alkisg: 5.3 is planned [06:54] Thank you tjaalton :) [07:36] hey alkisg tjaalton [07:36] Heya lotuspsychje [07:37] tjaalton: i been able to grab a dmesg on that intel flickering with a kernel param Bug #1838644 [07:37] bug 1838644 in linux-hwe (Ubuntu) "Booting into desktop results in flickering" [Undecided,Incomplete] https://launchpad.net/bugs/1838644 [07:37] lotuspsychje: hi, tested -12 yet? [07:38] tjaalton: no, did you own build or from mainline? can you link plz? [07:38] same link [07:38] https://aaltoset.kapsi.fi/bisect/ [07:38] ok lets test [07:39] fifo underruns would match the expected symptom [07:41] I'm going to be afk for a bit.. but will check once back [07:42] tjaalton: Rootbox 5.0.0-rc1 #12 working : ) [07:52] really? [07:52] it's basically 5.0-rc1 [07:52] now you can bisect using mainline rc builds while I'm gone [07:53] back in 1,5h or so [07:53] tjaalton: wich versions [07:53] 5.0-rcN [07:53] can you link plz [07:53] same mainline repo https://kernel.ubuntu.com/~kernel-ppa/mainline [07:53] tnx [07:55] rc1==>rc8 right [07:58] lotuspsychje, obviously you should bisect it, -rc4 first or whatever [07:59] apw: sorry im not really a dev, bisect means? [07:59] not a technical term, it is a search algorithm [07:59] divide-and-conquor [08:00] can you explain a bit plz? [08:00] if you think the answer is between 1 and 10, and you can tell if your guess is too low [08:00] you would try 5 [08:00] if the guess was too low you try 7.5, too high 2.5 [08:00] and so on [08:00] apw: we are looking for the bottleneck kernel version and guess into the middle? [08:00] you are doing the equivalent here [08:00] so always take a half way, eliminating half [08:02] for you you know 5.0-rc1 is good i think, and a 5.0 ubuntu kernel is bad [08:04] so likely i would test v5.0 itself next, to eliminate the ubuntu delta [08:04] then if that 5.0 fails (and you may have tested that already), v5.0-rc4 or something [08:04] apw: so lets say i test rc5, how would i know if upper or lower is bad/good? [08:07] lotuspsychje, you know which is good, you are testing for something, like display issues right ? [08:07] so you said -rc0 does not have 'the problem' and ubuntu has 'the problem' [08:07] apw: correct, intel gpu gives flickering at desktop boot [08:08] lotuspsychje: imagine you've got (to make it easy) 12 commits between the known-bad and later known-good. A bisect will build #6 first. if you report that works (good) it'll then build #3, if you report that bad, it'll then build #5, if that is good it'll build #4, if that is good you know it contains the fix. if it is bad, you know #5 contained the fix [08:09] right [08:09] that is your test to say which way to go [08:09] if 5.0-final is good then we know ubuntu delta is faulty [08:10] if 5.0-final is bad we know the breaking commit is before 5.0 so try something later than good and before bad [08:11] every kernel test i should update my bug with? [08:26] apw TJ- 5.0.0-050000rc4-generic is bad [08:26] lotuspsychje: someone in #kubuntu reported same today: https://paste.ubuntu.com/p/JcrbHxq2hn/ [08:26] sadly they have gone [08:26] RikMills: can you check my bug if its related to the new KDE guy that replyed? Bug #1838644 [08:26] bug 1838644 in linux-hwe (Ubuntu) "Booting into desktop results in flickering" [Undecided,Incomplete] https://launchpad.net/bugs/1838644 [08:27] lotuspsychje: no, as I said, they left the channel before I looked. I only have the log [08:28] I was just FYI [08:28] lotuspsychje, [08:28] lotuspsychje, lower then [08:28] apw: ok [08:28] always closing the gap between good and bad [08:29] lotuspsychje: for what it is worth I examined the Ubuntu cherry-picked patches since 5.0.0-18.19 I think it was, comparing them with where they came from in the later kernel, and I got an indication a related patch that should have been cherry-picked was missing, but I ran out of time in analysing it in more detail to figure out which [08:30] RikMills: did you read #12 on my bug? [08:30] thats also a carl [08:31] lotuspsychje: oh, I sees. can't confirm, but seems likely [08:32] RikMills: ok tnx for the notice [08:54] apw: rc1 rc2 rc3 rc4 are bad [08:54] now what? [08:55] lotuspsychje, huh ... -rc1 was reported good earlier in this channel, so now i am confused [08:55] lotuspsychje, or more specifically the thing you tested which was labelled 'basically' -rc1 by tjaalton was good [08:55] apw: that was an own built one version from tjaalton #12 [08:56] lotuspsychje, can you re-test that one to confirm it still works; if so that say -rc1.. has the fix [08:56] sure [08:58] Linux Rootbox 5.0.0-rc1 #12 SMP Thu Aug 1 19:38:58 EEST 2019 x86_64 x86_64 x86_64 GNU/Linux build from tjaalton confirmed still working [08:59] tjaalton, ^ it seems my mainline 5.0-rc1 is bad and that is good ... so we need to know how they differ [08:59] lotuspsychje, we need to t-jaalton now to know what to try, i assume he will be around later [09:00] allright, tnx for your time apw [09:00] lotuspsychje, np [09:37] lotuspsychje: verify that you had -modules installed for the mainline builds, if not then i915.ko isn't available and you can't hit the bug [09:37] you can also check with 'lsmod| grep i915' [09:59] tjaalton: yeah i installed headers, modules and image on all tests [10:01] weird [10:02] tjaalton: did you check the dmesg with drm.debug=0x1e log_buf_len=4M maybe some clues there? [10:02] it just verifies the bug [10:02] ok [10:04] tjaalton: that carl guy from my bug, also has intel 620 graphics like me [10:04] on my NUC its intel 650 and there it doesnt happen [10:05] it's not the gpu but the panel [10:05] ah [10:05] your nuc doesn't have one [10:05] correct [10:05] is it a 4k unit? [10:05] resolution [10:06] Resolution: 1920x1080@60.01hz [10:06] ok [10:23] so I don't know what to do now [10:24] tjaalton: im currently browsing older bugs on underrun errors [10:25] so far i didnt have customers calling for issues yet [10:26] I wonder if your testing of my -12 was just incomplete, meaning that maybe it'd start to flicker later or something [10:27] tjaalton: from my testing, i can reproduce pretty similar, aka, few seconds i see desktop, then the flickering starts heavy, cant do anything anymore in between [10:28] maybe double check if you have i915 loaded with -12 [10:29] i am https://paste.ubuntu.com/p/4W83KkgT6M/ [10:30] yep [10:31] tjaalton: im browsing this currently, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1550779 [10:31] Ubuntu bug 1550779 in linux (Ubuntu) "[drm:intel_cpu_fifo_underrun_irq_handler [i915]] *ERROR* CPU pipe A FIFO underrun" [Medium,Confirmed] [10:32] as they also mentioning flickering [10:40] and jeremy31 found just this: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824216 [10:40] Ubuntu bug 1824216 in linux (Ubuntu Disco) "Linux 5.0 black screen on boot, display flickers (i915 regression with certain laptop panels)" [Undecided,Fix released] [10:41] that was a revert of a commit [10:45] a regression in 5.0.x stable branch [11:01] lotuspsychje: could you post dmesg without debug output from both -12 and mainline 5.0-rc1 somewhere.. [11:02] trying to search for hints why the other fails while the other doesn't [11:02] on it [11:04] btw, check for bios updates [11:18] RC1: https://paste.ubuntu.com/p/QFJY3jNSHb/ [11:19] #12: https://paste.ubuntu.com/p/hTXK4KmgVS/ [11:20] thanks, I'll have a look.. [11:21] ty tjaalton [11:23] another fact i just found out, pressing backlight Fn+ F8/F9 brightness makes it flicker more [12:04] they look mostly the same, my kernel has a slimmer config but all the framebuffer/drm things look the same on both [12:05] tjaalton, so perhaps a timing issue? the extra weight changing the timing ? [12:06] could you perhaps make your kernel using the other config and see? [12:17] sure [13:14] lotuspsychje: -13 is built, using identical config as mainline rc1 [13:15] on it tjaalton [13:18] apw: I'm using 'make bindeb-pkg' btw, and gcc 7.4.0 from bionic. the mainline builds are built on a newer distro? [13:18] likely so yes [13:18] prolly against disco [13:19] * apw realises he could use the version data to produce multiple, but uggg [13:21] tjaalton: its flickering https://paste.ubuntu.com/p/B3WQDxfwdK/ [13:22] phew [13:24] is that a good phew? [13:24] yes [13:24] yay [13:25] at least we have a baseline now... [13:25] well done tjaalton ; ) [13:26] ngh, looking at the config diff makes me sad [13:26] let's just bisect [13:28] I'll build drm-intel-next-2019-02-02 now, which was what you tested as -11 with the old config [13:28] okay [13:40] lotuspsychje: -14 is uploaded [13:40] lets test [13:43] tjaalton: Linux Rootbox 5.0.0-rc1 #14 SMP Fri Aug 2 16:28:16 EEST 2019 x86_64 x86_64 x86_64 GNU/Linux [13:44] working [13:46] great [13:46] now the actual bisecting can start.. [13:47] good luck [13:47] you're still testing them :P [13:48] apreciating the work you doing mate, its the least i can do [13:48] my part is just mechanics, building kernels [13:48] takes maybe 5min to build [13:49] sure, coffee to the rescue :p === jdstrand_ is now known as jdstrand [13:58] beer [13:58] lol [14:02] ok it took 15min, -15 is available [14:02] lets fire it up [14:04] tjaalton: 5.0.0-rc1+ boot this now? [14:04] yes [14:05] allrighty [14:08] tjaalton: + is flickering [14:08] cool [14:21] lotuspsychje: -16 done [14:21] okay tnx [14:28] tjaalton: 16 flickering [14:28] great [14:28] : ) [14:30] the actual compile takes a minute, then building the package takes 10 [14:30] dont worry mate, im used to idle [14:35] lotuspsychje: -17 uploaded [14:36] allrighty! [14:39] tjaalton: Linux Rootbox 5.0.0-rc1+ #17 working! [14:39] aha [14:48] lotuspsychje: -18 up [14:48] okay [14:52] tjaalton: -18 flickering [14:53] got it [14:57] meanwhile, you could try booting 18 with 'i915.fastboot=1' [14:57] tjaalton: im always installing ubuntu in legacy, will that param have effect? [14:58] dunno [14:58] ok lets try [14:59] could be it's "drm/i915: Try to sanitize bogus DPLL state left over by broken SNB BIOSen" but we have that backported in 5.0 already [14:59] but if bisect points to it, then it needs some other commit(s) to work... [15:00] -19 uploading [15:00] want me to test 18 with param first? [15:01] either way [15:01] hmm maybe yes, if this replaces 18 [15:03] tjaalton: fastboot trick on -18 worked [15:03] ha [15:04] a true head-scratcher... [15:05] lotuspsychje: and no fifo underruns in dmesg? [15:07] https://paste.ubuntu.com/p/4GN5kzzJCq/ [15:08] that would be a no then [15:10] tjaalton: try -19 now? [15:10] lotuspsychje: so, boot the distro kernel with fastboot [15:10] -19 should be bad [15:10] 5.0.0.23 with fastbbot right? [15:10] boot [15:10] yep [15:11] kk [15:11] and then maybe boot 5.1 mainline with fastboot=0 [15:12] to rule this out [15:13] tjaalton: working! 5.0.0-23-generic #24~18.04.1-Ubuntu [15:13] now try suspend&resume :) [15:16] tjaalton: suspend makes it flicker [15:16] bingo [15:17] \o/ [15:18] so, boot 5.1 with fastboot=0 [15:18] maybe add it to /etc/default/grub so that it's always there [15:19] GRUB_CMDLINE_LINUX [15:19] yeah im doing it that way [15:19] cool [15:20] tjaalton: i got a 5.1.0 and 5.1.21 [15:20] boot .21 [15:20] allrighty [15:24] tjaalton: i915.fastboot=0 on 5.1.21 flickering [15:24] marvellous [15:25] apw: recent drm-tip builds have failed [15:25] 07-30 is the latest [15:26] drm-next too [15:26] um, drm-intel-next that is [15:26] lotuspsychje: install this https://kernel.ubuntu.com/~kernel-ppa/mainline/drm-intel-nightly/2019-07-30/ [15:27] allrighty [15:29] if it's broken, then all we can do is wait for the guy at intel responsible for this to return from vacation :P [15:29] tjaalton: boot with fastboot=0 ? [15:29] or bisect where it broke, but I'd say reverting something at this point would risk breaking it for others [15:29] yes [15:30] cross your fingerz [15:32] tjaalton: 5.3 with fastboot=0 working [15:32] try suspend? [15:32] sure [15:33] but that sounds promising [15:33] not flickering [15:33] try latest 5.3-rc then [15:34] rc2? [15:34] yes [15:39] tjaalton: 5.3.0-050300rc2-generic working, but it doesnt wanna suspend [15:40] ok, good. now test 5.2.5 [15:40] ok [15:46] tjaalton: 5.2.5 flickering [15:46] okay [15:47] lotuspsychje: try this one https://kernel.ubuntu.com/~kernel-ppa/mainline/drm-intel-next/2019-06-20/ [15:48] installing [15:49] lotuspsychje: if that works, try the one before that [15:49] on the parent dir [15:49] and so on [15:51] okay [15:51] once we have known good/bad, it's time for another bisect.. [15:52] but I'm about to hop on a swimming pool :P [15:54] tjaalton: 2019-06-20/ 5.2.0 flickering [15:55] oh [15:59] only a handful of commits between that and 5.3-rc2 [16:04] but a new build will take closer to an hour now [16:04] allrighty, lets take a break :p [16:04] 19 commits of which two are suspicious [16:04] but both seem to fix commits not in 5.0 :/ [16:05] ok, swim/beer break [16:50] lotuspsychje: -20 uploaded [16:50] 5.2.0-rc4+ [17:40] lotuspsychje: ping? [17:43] tjaalton: ok lemme test [17:53] tjaalton: 5.2.0 rc4 flickering [17:55] alright === Laney is now known as awayney [18:08] building the next one, seems to take a while [18:08] okay tnx [18:09] tjaalton: i need to still test them all with fastboot=0 right? [18:09] yes [18:09] kk [18:45] lotuspsychje: -21 finally uploaded [18:46] allrighty! [18:51] tjaalton: -21 also flickering [18:53] thanks [18:53] building [18:53] cool [19:41] lotuspsychje: -22 uploaded [19:41] okay [19:45] tjaalton: -22 also flickering [19:46] okie [19:47] tjaalton: was the last one for today for me, ill continue tomorrowz ok [19:47] ok [19:48] I'll push 23 once it's ready, you'll find it in the same place [19:48] allrighty! [19:48] ill test when i wake [19:53] thanks [21:28] I just tried the 5.0.0-23 kernel on my laptop with the Intel UHD 620 graphics and have no flicker