/srv/irclogs.ubuntu.com/2021/05/15/#ubuntu-server.txt

raddyHello10:06
raddyintl module is not showing up in phpinfo, but shows up in php -m10:06
raddyhave restarted php-fpm and nginx several times10:07
lotuspsychje!crosspost | raddy10:09
ubot3raddy: Please don't ask the same question in multiple Ubuntu channels at the same time. Many helpers are in more than one channel and it's not fair to them or the other people seeking support.10:09
lotuspsychjeok nvm i see you got forwarded10:09
=== denningsrogue682 is now known as denningsrogue68
dn_I have a very odd nvme problem with a new machine (threadripper); I have 7x the same NVME (980 PRO) installed - one of them is roughly 2x (4k random read) faster than the rest. They all seem to connected the same way (PCIe4, x4) but testing with fio shows the difference - and the fast one is also the only one with the expected speed. I'm a bit lost13:47
dn_what I could do/check/verify. It seems like slot dependend, but e.g. lspci -vvv looks identical for all.. (LnkSta: Speed 16GT/s (ok), Width x4 (ok). I'm testing with fio on the raw block device.13:47
tomreynno idea. have you tried physically swapping them, to rule that out?13:53
tomreynhave you tried playing with any related uefi settings? is hot swap enabled (try disabling it)?13:54
tomreyninspect dmesg / systemd journal (if you havenT' already). try uefi upgrades, maybe check speeds with the OS supported by mainboard vendor, get support from mainboard vendor.13:55
tomreyndn_: ^13:56
dn_tomreyn: will try disabling hotswapping and switch slots again - there is an error in dmesg, I forgot about it - but it's an error for the fast device - need a moment to reboot to try disabling hotswap & get the msg13:57
dn_tomreyn: sadly no change, but I got the log entry -> [  103.889739] nvme 0000:43:00.0: AER: aer_status: 0x00000001, aer_mask: 0x0000000014:48
dn_[  103.890764] nvme 0000:43:00.0:    [ 0] RxErr                  (First)14:48
dn_[  103.890767] nvme 0000:43:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID14:48
dn_Also on 43:00.0 is the fastest device .. so not sure what I shall make of it14:48
tomreyndn_: and uefi is up to date?14:51
dn_tomreyn: good question - never checked that, bios is - will google how to check/update14:51
tomreynjournalctl -b | grep DMI:14:51
dn_May 15 14:58:54 brrrmm kernel: DMI: ASUS System Product Name/Pro WS WRX80E-SAGE SE WIFI, BIOS 0405 03/17/202115:02
tomreyndn_: i haven't checked whether that's the latest, but that's a rather current build date, so it could be. i suggested other things you could give a try above, such as swapping NVMEs. since this is a hardware error, it could also be interesting what happens if you just remove this one device.15:05
dn_tomreyn: thank you for the ideas, this really helps. will do that next... because I'm really out of ideas ;-)  it's kinda maeh...15:05
tomreynbut i guess this is really a ##hardware topic at this point.15:05
tomreynalthough, just in case, try different linux (kernel) versions, too15:06
tomreynso far i don't know which ubuntu server version and kernel you've tested with15:07
dn_tomreyn: will do, also a good idea .. it's kinda odd that one disk is fast, I would understand all are slow ... but I don't see any pattern why one is fast; - I just reinstalled 20.04 - will try after fresh install, will update & retry15:07
tomreyn!mainline15:07
ubot3The kernel team supply continuous mainline kernel builds which can be useful for tracking down issues or testing recent changes in the Linux kernel. More information is available at https://wiki.ubuntu.com/Kernel/MainlineBuilds15:07
dn_I also tried archlinux, just to be sure it's not something totaly odd - but both are the same (arch & ubuntu) - exact same behaviour15:08
TJ-dn_: there's quite a lot of intereseting info under /sys/ that might help you deduce a reason/cause15:08
dn_I diffed /sys/block stuff for the devices - I might be blined, but everything like scheduler, queue, readahead seems to be the same15:09
TJ-my guess thought is going to be firmware's configuration of the hardware15:09
dn_the nvme all have the same firmware version, anything else I could check?15:09
TJ-dn_: can you show sus "pastebinit <(  sudo strings /sys/firmware/acpi/tables/DSDT  | grep -i windows | sort )"15:10
dn_https://paste.ubuntu.com/p/Tts87YC52h/ - sweet, never used pastebinit15:12
dn_tomreyn: just for the record, I tried `5.4.0-73-generic` - will now upgrade to latest stable and retry15:12
TJ-dn_: in /theory/ the kernel should say YES to the most recent of those _OSIs /but/ the kernel expects the firmware's ACPI DSDT caller to pass the most recent ISO first. In the event it doesn't then it is possible to the firmware to misconfigure or use a less than optimal config15:21
dn_TJ- got some keywords for me to google, like how change/check etc?15:22
TJ-dn_: with that in mind it may be worth trying a workaround developed some years ago but now mostly not needed, to force Linux to only recognise the most recent _OSI string. See https://iam.tj/prototype/enhancements/Windows-acpi_osi.html15:22
dn_TJ- thanks, will look into it15:29
dn_TJ- might be a stupid question, but which one do I want to use? Windows 2015?15:30
TJ-dn_: yes 'latest' based on intelligent analysis!15:31
dn_I think I also just figured out the reason ..15:31
dn_if so I'll have to cry for a moment15:31
TJ-go on!?15:31
TJ-cables?15:32
TJ-connectors?15:32
dn_I think trim ... but not 100% sure - trim & space used15:32
dn_they are all 2TB disks, all but one has round 40% namespace usage ... so the fast one has 0 usage....; I did a blockdiscard and ns format on one of the slows - that also has now no usage - and it's fast now15:33
dn_I find that a bit unexpected ...15:33
TJ-wow!15:33
TJ-nice find though, and toally left-field15:34
dn_am I stupid/missing something? .. like why!?15:34
dn_so, e.g. for 4k rand write - I get now -> `[r=4446MiB/s][r=1138k IOPS]` ... on the device that was slow before, slow devices do `[r=2731MiB/s][r=699k IOPS]`15:35
TJ-dn_: this thread has some interesting comments and towards the end by 'gerard' is mention of a recent firmware upgrade and Samsung confirming issues. Haven't checked that claim out myself. https://chiaforum.com/t/extremely-slow-results-from-samsung-980-m-2-ssd/900/2615:40
dn_TJ- thanks! - hm I think it's not even possible under linux to update windows... how annoying15:44

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!