| xnox | 4.4 looks very juicy =0 | 01:30 |
|---|---|---|
| tsimonq2 | phillw: well since i is usually every 3 point releases for Ubuntu, it's probably 4.5 | 02:43 |
| phillw | tsimonq2: I doubt 4.5 for an LTS... but, I'm happy to be proven wrong. 4.3 was the vote, as to if they sway to 4.4... I'm not too sure. | 02:46 |
| tsimonq2 | phillw: where was this voted> | 02:50 |
| phillw | tsimonq2: read the mailing list. the kernel team do not 'vote', it is a technocracy just as the tech board are. | 02:52 |
| mamarley | I asked in here just the other day and was told that 4.4 would be uploaded to Xenial "soon". | 03:00 |
| phillw | mamarley: I've no idea who told you that :) | 03:02 |
| mamarley | Let me check... | 03:02 |
| mamarley | [11.01.16 13:14:44] <apw> mamarley, yep, "soon" waiting on some dkms fallout to be resolved as much as anything | 03:03 |
| mamarley | After I asked: "[11.01.16 13:13:55] <mamarley> Xenial will eventually be getting 4.4, won't it?" | 03:03 |
| phillw | mamarley: The Ubuntu Linux kernel team has announced that the Linux kernel in Ubuntu 16.04 LTS has been upgraded to version 4.4, the latest stable release made available. | 03:04 |
| phillw | you mean that annoucement :) | 03:04 |
| phillw | mamarley: this is pretty breaking news... http://news.softpedia.com/news/ubuntu-16-04-lts-now-based-on-linux-kernel-4-4-lts-498901.shtml | 03:05 |
| mamarley | According to Launchpad, 4.4 hasn't been uploaded yet. Softpedia isn't the most reliable news source... | 03:06 |
| phillw | mamarley: for the admins, this is a major head ache... Trust me. But, in the cycle we are just past alpha 1 testing and alpha 2 is not due until near end of month. It will be interesting to see if ubuntu actually take part in the alpha2 to test the kernel out. | 03:09 |
| pkern | Hi. Did you hear anything about Haswell Intel machines being slow with current 3.19/4.2 kernels (on trusty, but that might not matter)? The problem is cores stuck to 450 MHz. | 10:14 |
| apw | pkern, don't think i have seen a specific result | 10:19 |
| apw | s/result/report/g | 10:20 |
| cking | pkern, not heard of any direct reports on this for Haswell | 10:23 |
| cking | although I am aware of a few users seeing cpus getting stuck at a low freq allegedly because of a microcode update, but we've not figured out if this is a false report as yet | 10:24 |
| g0g0boy | hey all, I have a problem with an update scenario | 10:32 |
| g0g0boy | need assistance | 10:32 |
| g0g0boy | was doing an apt-get update/upgrade and got an error: the following packages have unmet dependencies: | 10:33 |
| apw | g0g0boy, which packages | 10:34 |
| g0g0boy | recomended I run apt-get -f install | 10:34 |
| apw | what series are you running ? do you have -proposed enabled ? | 10:34 |
| g0g0boy | one line is: linux-image-extra-4.2.0-23-generic : Depends: linux-image-4.2.0-23-generic but it is not installed | 10:35 |
| g0g0boy | second line is: linux-image-generic : Depends: linux-image-4.2.0-23-generic but it is not installed | 10:35 |
| TJ- | g0g0boy: did you do "sudo apt-get dist-upgrade" ? | 10:35 |
| g0g0boy | I didn the apt-get -f install which tells me that I have no space in /boot :( forget to clean out my old kernels. | 10:35 |
| TJ- | g0g0boy: ahhh, first try "sudo apt-get autoremove". If that doesn't free up space then do "dpkg -l 'linux-image*' " and identify older kernel versions you no longer need | 10:36 |
| g0g0boy | TJ- same error | 10:36 |
| g0g0boy | TJ- same error again regarding depenacies | 10:37 |
| apw | yep you are in a hole an no mistake, you need to finish the install before it will do anything else | 10:37 |
| g0g0boy | I tried checking and deleting the old kernels to clear sapce but then get the depenacie error error :( | 10:37 |
| g0g0boy | going around in circles it seems | 10:37 |
| apw | you might (as TJ- suggests) be able to request removal of an older kernel by hand | 10:37 |
| apw | g0g0boy, is /boot separate partition ? | 10:38 |
| TJ- | g0g0boy: once you've identified older versions you can remove them with "dpkg --remove linux-{image,headers}-<VERSION>" for each identified version | 10:38 |
| apw | g0g0boy, if so you could simply move the old kernel/initrds out of the way on to /, finish the install and then move htem back | 10:38 |
| TJ- | g0g0boy: once you've removed at least 1 older version there'll be space to fix with "apt-get -f install" | 10:38 |
| g0g0boy | will try usually use command: sudo apt-get purge -y linux-headers-xxxxxxx | 10:38 |
| apw | g0g0boy, likely you need to go under its view and use dpkg like TJ- suggests | 10:39 |
| apw | as apt knows things are going on | 10:39 |
| apw | and wnats to finish them first | 10:39 |
| g0g0boy | okay will try dpkg | 10:40 |
| g0g0boy | will let you know what happens, thanks for the suggestions | 10:40 |
| g0g0boy | so far looks like it is working :) | 10:42 |
| apw | good stuff, TJ- ^ | 10:42 |
| TJ- | apw: we see that all the time - the default installer-created /boot/ is too small these days | 10:43 |
| TJ- | the pain-point is added to because there is no kernel image autoremove function, so GUI users esspecially don't know about autoremove | 10:44 |
| apw | TJ-, is there a bug filed to make that bigger, as it sounds like there should be | 10:46 |
| apw | TJ-, does it not also run autoremove the gui "install updates" i thought it did, which is why the kernles are now marked to keep just the last 3 | 10:47 |
| TJ- | hmm, well, unless something has changed very recently. We see this issue very frequently | 10:47 |
| TJ- | especially with broken boots after upgrades, due to an incomplete initrd.img | 10:48 |
| g0g0boy | TJ-, apw: no luck :( | 10:48 |
| TJ- | g0g0boy: where are you at? have you removed some older kernel images? | 10:48 |
| g0g0boy | I was able to remove some items with dpkg --remove which I thought did something cleard out most of the linux-headers | 10:48 |
| apw | headres are not in /boot | 10:48 |
| g0g0boy | I stil lhave the following left | 10:48 |
| TJ- | g0g0boy: what does "df | grep boot" report? | 10:48 |
| apw | you need to remove linux-image-* for older versions | 10:49 |
| g0g0boy | ::::::/dev/sda1 240972 240619 0 100% /boot | 10:49 |
| g0g0boy | brb | 10:49 |
| g0g0boy | try removing those also | 10:50 |
| TJ- | apw: this is the most relevant I can recall right now: bug 1357093 | 10:51 |
| ubot5 | bug 1357093 in unattended-upgrades (Ubuntu) "Kernels not autoremoving, causing out of space error on LVM or Encrypted installation or on any installation, when /boot partition gets full" [Undecided,In progress] https://launchpad.net/bugs/1357093 | 10:51 |
| g0g0boy | confirmed everything was removed now but still getting errors, come back to you shorlty, Im just reading through some of the output | 10:58 |
| g0g0boy | so I removed all linux-headers and linux-image-xxx so when I check for any kernels nothing appears | 11:02 |
| g0g0boy | still get error when doing apt-get upgrade or dist-upgrade | 11:02 |
| g0g0boy | try doing apt-get -f install and throws out a lot off output | 11:02 |
| g0g0boy | it appears to complain that it cannot find the old linux kernels | 11:03 |
| TJ- | g0g0boy: I'd recommend you /join #ubuntu since that is the correct support channel for these kind of issues | 11:03 |
| g0g0boy | I have the output just dunno where to post it for review | 11:03 |
| g0g0boy | LOL :( ok :) | 11:04 |
| g0g0boy | thanks anyways TJ-, apw | 11:04 |
| TJ- | g0g0boy: waiting for you there :) | 11:04 |
| pkern | cking: Urgh, that's a very good point. | 12:34 |
| pkern | cking: We did indeed push new microcode. | 12:34 |
| cking | pkern, it would be useful to double check that. if you do find an issue with the microcode we can ping intel about it | 12:35 |
| pkern | But we already did confirm that it doesn't happen with 3.13, fwiw. We found it with 3.19 and 4.2. | 12:36 |
| apw | pkern, presumably with the same firmware in place | 12:40 |
| apw | (and cpu firmware if loaded very early in initramfs) | 12:41 |
| pkern | Yeah, we package it up and it's the same. | 12:41 |
| apw | i mean it was the same in each of the tests, 3.13 ok others not | 12:41 |
| pkern | It was pretty clear that >3.13 used a different scaling method. | 12:44 |
| cking | pkern, intel_pstate is now being used, is this specific to just one set of haswells? | 12:45 |
| apw | cking, that feels really familiar, some patch to make one specific model use something differnt in intel_pstate | 12:46 |
| apw | (perhaps it is tooo specific) | 12:46 |
| cking | apw, b27580b05e6f5253228debc60b8ff4a786ff573a ("intel_pstate: Fix BYT frequency reporting") perhaps | 12:48 |
| pkern | So I rebooted into 3.19 with the same firmware package installed and it's broken again. Now I can try to rollback the firmware. | 12:50 |
| pkern | This is a Xeon E5-1650 v3. | 12:51 |
| cking | pkern, sounds like https://bugs.launchpad.net/ubuntu/+source/intel-microcode/+bug/1480349 | 12:51 |
| ubot5 | Launchpad bug 1480349 in intel-microcode (Ubuntu) "Intel Microcode Breaks frequency scaling in Xeon® E5-2687W v3 & E5-1650 v3" [High,Incomplete] | 12:51 |
| pkern | Argh. :) Thank you. I'll try rolling back the microcode update. | 12:52 |
| cking | pkern, if it is, please add your findings to the bug and I will prod Intel again on this one | 12:53 |
| pkern | I'll try reverting to the trusty version. | 12:57 |
| pkern | Yup, microcode. | 13:20 |
| pkern | cking: Ok, updated the bug. I theory I could try to bisect which microcode causes it, but still one'd need to raise it with Intel. \: | 13:29 |
| cking | pkern, a bisect on the microcode would be useful if you can affort the time to do that | 13:30 |
| cking | I'm contacting intel right now about it | 13:31 |
| pkern | I guess I could just try 0x2d on this processor and see what happens. Would fit date-wise. | 13:31 |
| pkern | (I have access to the errata but not when stuff got publically released, so I'd need to cross-ref multiple places.) | 13:32 |
| cking | yep, 0x2d is a good start I think | 13:32 |
| pkern | Ok, so 0x2b was actually even what the CPU was shipped with because there's no update in the package with that sig. | 13:38 |
| cking | ok, that's a good starting point | 13:40 |
| dsmythies | cking, pkern: Often CPU frequencies stuck low with the intel_pstate driver is due to somehow Clock Modulation being involved. You can test, by: 1 - disable the intel_pstate driver, as the acpi-cpufreq driver works fine with clock modulation; 2 - examine the Clock modulation MSR (0x19A) during the issue. | 14:51 |
| dsmythies | apw,rtg: Re: A couple of days ago. Should I enter a bug report about the collateral damage due to the "depends on ZONE_DMA" issue? 11 devices are effected, all are sound cards. (reference: linux/sound/pci/Kconfig). | 14:56 |
| apw | dsmythies, yes please and paste the number here so we can find it in the deluge | 15:04 |
| TJ- | is there any guidance on when 4.4 will land for xenial? Been getting a few questions about testing it recently (I know it's only 5 days since release!) and I see the 4.4 tags in the repo | 15:09 |
| manjo | rtg, can you install gcc 4.9 in tangerine amd64 chroot ? | 15:12 |
| manjo | rtg, the latest xenial gcc has bugs due to linaro merge | 15:12 |
| rtg | manjo, USE THE WILY CHROOT | 15:12 |
| rtg | oops | 15:13 |
| manjo | rtg, arm64 kernels don't boot .. yeah that is a good idea.. I will use wily | 15:13 |
| apw | rtg, :) | 15:13 |
| apw | TJ-, we are in the last throws of getting the worst bits sorted, we are hoping "soon" like next week ish depending | 15:13 |
| rtg | TJ-, I've got a 4.4 kernel staged in unstable | 15:14 |
| manjo | apw, you like the yellin ? | 15:14 |
| apw | (ppa not debian unstable) | 15:14 |
| apw | manjo, heh he only has one hand | 15:14 |
| TJ- | apw: thanks. that's 'sooner' than I expected. I'm running my own builds of v4.4 here, but had a few of the QA testers asking | 15:14 |
| manjo | apw, oops tmi ... | 15:15 |
| apw | they can get a preview in the ppa:canonical-kernel-team/ubuntu/unstable but its not supported | 15:15 |
| apw | as yet obviously | 15:15 |
| apw | manjo, heh no not like that | 15:15 |
| manjo | lol | 15:15 |
| pkern | After having added the 0x2d microcode update my machine didn't even boot. Oh well. | 15:22 |
| apw | man | 15:27 |
| dsmythies | apw, rtg: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1534647 | 16:03 |
| ubot5 | Launchpad bug 1534647 in linux (Ubuntu) "Collateral damage due to kernel configuration change enabling CONFIG_ZONE_DEVICE (Kernel 4.4 amd64)" [Undecided,New] | 16:03 |
| apw | rtg, ^ | 16:03 |
| apw | oh you did that already | 16:03 |
| apw | rtg, i might look at that on monday if you haven't already | 16:03 |
| rtg | apw, I have not researched it yet, so feel free. | 16:04 |
| rtg | apw, I'm going through the bug list to see if there are some that apply to unstable, then I'll upload in a bit. | 16:05 |
| apw | sounds good | 16:05 |
| rtg | gotta keep xnox happy | 16:05 |
| apw | he does get antsy :) | 16:08 |
| xnox | rtg, hm?! =) | 17:13 |
| xnox | rtg, did you see my comment about CONFIG_KVM btw? | 17:13 |
| * xnox checks bug mail and pulls latest kernel git changes | 17:14 | |
| rtg | xnox, yes | 17:14 |
| rtg | only applied to unstable though | 17:14 |
| xnox | rtg, that's ok. no rush at the moment =) | 17:15 |
| xnox | rtg, thanks. | 17:15 |
| rtg | apw, nm, update script is running amok | 17:16 |
| rtg | kamal, sforshee: I need to bounce tangerine and reinstall the xenial chroots | 17:17 |
| sforshee | rtg: ack | 17:19 |
| kamal | rtg, ack | 17:19 |
| rtg | rebuilding xenial chroot on tangerine | 17:30 |
| sil2100 | Hello! I recently merged a new ppp version from Debian and noticed that ppp is blocked in -proposed due to ppp-modules not being built for s390x | 17:58 |
| sil2100 | Is there a reason for ppp-modules not being available for s390x ? | 17:58 |
| rtg | sil2100, checking... | 18:24 |
| rtg | sil2100, started bug #1534780 if you'd like to subscribe. | 18:47 |
| ubot5 | bug 1534780 in linux (Ubuntu Xenial) "Xenial: review s390x module exclusions" [Undecided,In progress] https://launchpad.net/bugs/1534780 | 18:47 |
| TJ- | rtg: apw had a user testing 4.4.0-0.11 from unstable; getting PCI "unable to assign BAR" errors and resulting inoperable device. (4.3.0-5.16 was fine). Testing with "pci=realloc" | 18:51 |
| sil2100 | rtg: thanks! :) | 18:58 |
| rtg | TJ- have them test vanilla mainline in order to make sure its not an Ubuntu patch causing the problem. http://kernel.ubuntu.com/~kernel-ppa/mainline/ | 19:30 |
| TJ- | rtg: will do; there's been a long-standing (17 month) regression in the PCI bridge allocation code | 19:30 |
| rtg | TJ- have you narrowed it down to a committ ? | 19:31 |
| TJ- | I've been having to use mainline kernels with patches and I'm still waiting for YingHai Lu to get them into mainline. there's a mainline bug covering part of it at https://bugzilla.kernel.org/show_bug.cgi?id=81431 | 19:31 |
| ubot5 | bugzilla.kernel.org bug 81431 in PCI "pci=realloc fails to modify bridge windows, causing devices to fail BAR allocation" [High,New] | 19:31 |
| TJ- | rtg: right now the trouble is YingHai has a stack of patches because the rework is so invasive, and I don't think is focused on this stuff. | 19:32 |
| rtg | great :( | 19:32 |
| TJ- | For v4.1 my issue only needed 2 patches from the stack, but now I need the entire stack | 19:32 |
| apw | barf | 19:33 |
| TJ- | I'm currently pulling in git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git and the pci-for-v4.5-next | 19:33 |
| TJ- | I have to rework those patches to apply on 4.4 right now, though | 19:34 |
| TJ- | The user is 'phillw' and he's just posted the most recent kern.log (includes 4.4 and 4.3.x boots) here: http://phillw.net/kern.log | 19:36 |
| phillw | o/ | 19:36 |
| TJ- | The line is "Jan 15 19:27:47 piglet kernel: [ 0.687558] pci 0000:03:00.0: can't claim BAR 6 [mem 0xffff0000-0xffffffff pref]: no compatible bridge window" | 19:36 |
| TJ- | this is with "pci=realloc" too | 19:37 |
| TJ- | phillw: rtg suggests you test the upstream, mainline v4.4.0 build from http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-wily/ | 19:38 |
| TJ- | phillw: did you notice, that last 4.4 boot doesn't have "assign-busses" ? I think you forgot to do "sudo update-grub" after changing /etc/default/grub | 19:39 |
| TJ- | I need to head off to dinner now | 19:39 |
| phillw | TJ-: I'll do that now and reboot... I also want some food... it is 19:40 here and food time is about 17:00 :D | 19:40 |
| phillw | As asked, I have issued sudo update-grub and WiFi still fails, logs are at http://phillw.net/kern.log | 20:01 |
| TJ- | phillw: one last config option to try adding, "nocrs"... as in "pci=realloc,assign-busses,nocrs" then "update-grub" then reboot test | 20:07 |
| phillw | TJ-: okies, can you hang on for a while, it is past 8pm here and I'm now cooking my meal I should have had at 6pm :) | 20:10 |
| TJ- | same here, and yes | 20:15 |
| * tsimonq2 runs a generic kernel fresh out of Linus' git repo | 20:20 | |
| * phillw runs what ubuntu will be using... TJ- food cooked, rebooting with your extra command. | 20:34 | |
| phillw | TJ-: file modified, still no WiFi, logs at http://phillw.net/kern.log | 20:47 |
| TJ- | phillw: that last 4.4 boot configured the wifi device correctly! | 20:49 |
| TJ- | as in, the 1 before this | 20:49 |
| TJ- | phillw: can you show me "dpkg -l 'linux-image*' | pastebinit" | 20:50 |
| phillw | TJ-: no, it did not. I have to reboot to 4.3 to have wifi | 20:52 |
| TJ- | Yes, it did. But, no driver module claimed it | 20:53 |
| phillw | okies.. just installing pastebinit | 20:53 |
| phillw | TJ-: http://paste.ubuntu.com/14510427/ | 20:54 |
| TJ- | phillw: As I thought; your install didn't include "linux-image-extra-4.4.0-0-generic" which contains the ath9k and other drivers | 20:57 |
| phillw | TJ-: so an apt get should resolve it? | 20:58 |
| phillw | TJ-: we are about to find out :) | 21:00 |
| phillw | TJ-: Linux piglet 4.4.0-0-generic #11-Ubuntu SMP Mon Jan 11 16:58:35 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | 21:04 |
| TJ- | phillw: so, that was why the wifi never came up even after the BAR assignment was fixed | 21:04 |
| phillw | so, it is add the ppa and then apt-get linux-image-extra-4.4.0-0-generic, which I assume will let apt know to grab the actual kernel as well :) | 21:05 |
| TJ- | phillw: if you check the dependency "apt-cache depends linux-image-extra-4.4.0-0-generic" that should pull in the -image- and the -headers- | 21:05 |
| phillw | phillw@piglet:~$ apt-cache depends linux-image-extra-4.4.0-0-generic | 21:06 |
| phillw | linux-image-extra-4.4.0-0-generic | 21:06 |
| phillw | Depends: linux-image-4.4.0-0-generic | 21:06 |
| phillw | |Depends: crda | 21:06 |
| phillw | crda:i386 | 21:06 |
| phillw | Depends: wireless-crda | 21:06 |
| phillw | phillw@piglet:~$ | 21:06 |
| phillw | yup :) | 21:06 |
| phillw | So, add the ppa and then issue the image-extra apt to get the system? | 21:07 |
| TJ- | so asking for -extra- does the job. Usually, we have the meta-packages to pull in these dependencies | 21:08 |
| phillw | early days, but the announcement is out and we do get asked! | 21:09 |
| phillw | TJ-: Thanks ever so much for your patience on this. Bits I do recall from cycles ago, others are new to me. I'm happy to answer questions from any of our testers now. | 21:10 |
| TJ- | it would be a good test to remove (but save where you can reuse it) the pci=... additions, update-grub, and see if it works without those. I think you may need the pci=realloc but not the others, but you may get away without it, if the missing module was the core problem | 21:10 |
| phillw | TJ-: back to "" and it has booted fine :) | 21:18 |
| phillw | these kernel modules wil be the death of us all :) | 21:19 |
| TJ- | LOL what a fuss! | 21:20 |
| TJ- | well, at least *you* aren't affected by the PCI bridge window issues then | 21:20 |
| phillw | indeed, just needed to ask for 'extra' and apt would have sorted it all out :D | 21:21 |
| TJ- | I must have missed in the logs where the initial BAR assignment failure was solved later on | 21:22 |
| phillw | I'll have a minion... oops, I mean volunteer, verify that adding the ppa and then the extra system results in a bootable system. tsimonq2 most likely has got 4.4 already, so I will need a volunteer without it installed on bare metal. | 21:24 |
| phillw | and you, as with all on here, are welcome to pop onto #phillw. We are an affable bunch and have testers willing to help out across various areas. | 21:25 |
| tsimonq2 | phillw: no I compile directly from Linus' git repo | 21:39 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!