[06:33] <alkisg> Could I `dpkg -i xserver-xorg-video-nouveau` from 16.04 to 18.04 to see if it fixes various showstopper issues that I see in ancient nvidia cards? Or would it break due to abi changes?
[06:34] <alkisg> I mean, install the 16.04 package to 18.04
[06:34] <tjaalton> no, you'll find that it won't install
[06:35] <alkisg> Thank you :/
[06:42] <KitsuWhooa> alkisg: which version of X is it?
[06:43] <alkisg> KitsuWhooa: stock 18.04,  xserver-xorg-video-nouveau 1:1.0.15-2
[06:44] <alkisg> The problems I've seen so far are xorg segfault/crashes, and scrambled lines in the physical output which do display fine over vnc though
[06:44] <alkisg> In various old nvidia cards, e.g. tnt2, mx 400, mx 4000...
[06:46] <KitsuWhooa> Hm, 18.04 has 1.19.6
[06:46] <KitsuWhooa> Are the segfaults with acceleration enabled consistent?
[06:46] <alkisg> KitsuWhooa: yes, and they are also consistent with NoAccel defined as well
[06:48] <alkisg> http://termbin.com/haem
[06:49] <alkisg> It boots fine up to lightdm, and crashes on login (where I assume things like compositing and opengl are used)
[06:50] <KitsuWhooa> Ah, it's probably different to what I encountered
[06:50] <alkisg> Maybe; I've seen tens of crashes and I assumed it was the same; maybe I've seen various different ones and haven't realized it yet
[06:50] <KitsuWhooa> I had a similar backtrace with the s3 savage driver with acceleration enabled, and resolved it by re-enabling sigio in X and recompiling it. I never found out why it happened, but it got disabled some time in 1.19 and I only found out by bisecting it
[06:51] <alkisg> (I'm getting reports from various schools with a lot of different clients)
[06:51] <KitsuWhooa> If possible, maybe try enabling HWE on a 16.04 machine and see if that also breaks it
[06:52] <alkisg> Thanks for the sigio pointer, I'll give it a try; I'll also see if I can reproduce it locally to try with 16.04 + hwe
[06:52] <alkisg> Although I think some schools already have that, and none reported the issue
[06:52] <KitsuWhooa> Maybe also install the debug symbols and get a gdb backtrace to see if there is anything interesting there
[06:54] <KitsuWhooa> My (old) school's ubuntu lab still runs the version with gnome2 (pre 12.04). I guess no one bothered updating it :p
[06:55] <alkisg> Some of the schools that I upgraded from 12.04 to 18.04 report that they want to go back :D
[06:55] <alkisg> I'd love to see less features and more stability, but I guess many developers find stability programming = boring :D
[06:56] <KitsuWhooa> I think it's more like it's getting incrementally difficult to support old hardware
[06:56] <alkisg> Oh I see it in programs that aren't related to hardware as well
[06:56] <alkisg> Panels that crash by just switching keyboard layouts "but they're so modern now,they support searching for programs while you type!"
[06:57] <KitsuWhooa> oh :p
[06:58] <alkisg> Btw the problem with the scrambled output was in nvidia 7200 as well; I don't know, is that considered too old/unsupported too?
[06:59] <tjaalton> alkisg: so did you try the git version?
[06:59] <alkisg> tjaalton: I need to reproduce it locally first
[06:59] <alkisg> I'll try to get such a card in my office
[07:00] <tjaalton> just build it and push to clients, it's only five commits on top of 1.0.15 ;)
[07:02] <alkisg> ty, will do
[09:05] <KitsuWhooa> alkisg: I can confirm I get corruption on 18.04.1 mate with nouveau on an MX400
[09:06] <alkisg> KitsuWhooa: thank you :) Btw, changing resolutions some times fixes it
[09:06] <alkisg> I tried with fx5200 locally, it worked on 1024x768, will try some other one now...
[09:07] <KitsuWhooa> https://tasossah.com/CameraPics/P1110844.JPG
[09:07] <KitsuWhooa> this is the ubuntu mate installer screen :p
[09:07] <alkisg> Yeah exactly like that
[09:11] <KitsuWhooa> I'll see what other old nvidia cards I have. I know there's an MX440 somewhere
[09:16] <alkisg> KitsuWhooa: I think it might also help if you click "try ubuntu" instead of "install ubuntu", as it might cause the second bug too, the segfault
[09:16] <KitsuWhooa> I can't see where to click :p
[09:16] <alkisg> Hehe, true, I have vnc there, you don't
[09:16] <KitsuWhooa> it's a live boot over usb1.1 and it's ridiculously slow
[09:16] <KitsuWhooa> so that's not helping either
[09:16] <alkisg> Ouch,network boot would help
[09:19] <KitsuWhooa> I started lightdm manually and I can barely see the firefox icon pinned in mate
[09:19] <KitsuWhooa> so I'm going to say this didn't segfault
[09:20] <KitsuWhooa> And I think netboot would take too much time to set up. I'd also need to find a NIC for this board as it doesn't have an onboard one
[09:20] <KitsuWhooa> and even then, it'd be fast ethernet, so I'm not sure how much better it'd be
[09:22] <KitsuWhooa> alkisg: interesting thing. When I switch to a tty, the desktop renders fine for a bit before switching to a tty
[09:23] <alkisg> KitsuWhooa: if you want help with netbooting, I'm an expert, I could set it up for you in a few minutes, and, 100mbps is a hell of a lot faster than 1.1 usb
[09:24] <alkisg> NIC => ipxe boots almost all of them
[09:24] <KitsuWhooa> That'd be appreciated
[09:25] <KitsuWhooa> give me a bit to see if I can find a PCI NIC to plug in to this board
[09:26] <KitsuWhooa> Right, I found one that looks to have a realtek chipset clone of sorts, and what looks like a boot rom
[09:27] <alkisg> That way you won't even need ipxe then :)
[09:28] <alkisg> So the last notes that I've made for netbooting without our "ltsp" project, are the "automation script" paragraph of this page: https://wiki.ubuntu.com/LiveCDNetboot
[09:28] <alkisg> I.e. you're supposed to mount the cd to the server /cdrom, and just run this command:
[09:28] <alkisg> wget 'http://alkisg.mysch.gr/steki/index.php?action=dlattach;topic=2525.0;attach=1421' -O /tmp/livecd-netboot && sudo sh /tmp/livecd-netboot
[09:29] <KitsuWhooa> one moment, trying to find out how to enable the boot rom in the bios
[09:29] <alkisg> If the boot rom is in the nic itself, you might need to press ctrl+f11 or so when it displays that message
[09:29] <alkisg> I.e. it might not be in the bios
[09:30] <alkisg> The easiest way would be to create a usb stick/floppy/cd with ipxe though
[09:30] <KitsuWhooa> it's not loading the rom at all, and IIRC there's usually a toggle in the bios
[09:30] <alkisg> boot.ipxe.org => images to download
[09:30] <alkisg> The bios usually is for onboard nics. It's the same bios for many boards, so it won't work for pci nics.
[09:31] <alkisg> (well, unless the bios was expecting a realtec onboard nic in other board versions)
[09:34] <KitsuWhooa> yeah looks like I'll be going with ipxe
[09:41] <KitsuWhooa> Προέκυψαν σφάλματα κατά την επεξεργασία του: nfs-kernel-server
[09:41] <KitsuWhooa> I have a feeling this isn't going to work
[09:42] <KitsuWhooa> mount: unknown filesystem type 'rpc_pipefs'
[09:42] <KitsuWhooa> looks like I'm going to be recompiling my kernel
[09:50] <alkisg> Er, nfs-kernel-server can't be installed? Yeah that's not a good sign...
[09:50] <alkisg> You can also boot another pc with the ubuntu mate usb stick, and then run that command, and it will allow you to netboot the older pc
[09:51] <alkisg> i.e. both live server and live client
[09:51] <KitsuWhooa> alkisg: your mysch site doesn't seem to be responding over IPv6
[09:51] <alkisg> (in case your actual setup is strange and you can't install nfs temporarily...)
[09:51] <alkisg> Yeah they're old school :D
[09:51] <KitsuWhooa> I'll do it in a stock 18.04 VM
[09:51] <KitsuWhooa> it has an AAAA record though and it resolves to an address
[09:51] <KitsuWhooa> so it just causes wget to hang
[09:52] <alkisg> I've filed complaints a lot of times, but no solution yet. They also don't support https in their hosting. Lame :/
[09:52] <KitsuWhooa> might want to edit the wiki page to use wget -4 then
[09:52] <KitsuWhooa> so that it forces ipv4
[09:52] <alkisg> Oh I wrote that paragraph 10 years ago, I don't think anyone maintains it
[09:53] <KitsuWhooa> Ah
[09:53] <alkisg> It's full of obsolete information, but I think my script still works
[09:53]  * alkisg loves code that works 10 years later :D
[09:54] <alkisg> 01:00.0 VGA compatible controller [0300]: NVidia / SGS Thomson (Joint Venture) Riva128 [12d2:0018] (rev 10) ==> nah this one loaded vesa, too old, trying another...
[10:02] <KitsuWhooa> I love you
[10:02] <KitsuWhooa> er
[10:02] <KitsuWhooa> lmao
[10:02] <KitsuWhooa> not sure how that got in my X clipboard
[10:02] <alkisg> Haha no worries it's always a good thing to say
[10:02] <KitsuWhooa> Sorry for that. Anyway, I'm waiting for the unattended updates to finish
[10:03] <KitsuWhooa> because it started running dpkg when I booted the VM so I can't use apt
[10:03] <KitsuWhooa> that is true :p
[10:04] <alkisg> I managed to reproduce the segfault with this one: 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation NV5 [Riva TNT2 Model 64 / Model 64 Pro] [10de:002d] (rev 15)
[10:16] <KitsuWhooa> the netboot script seems to fail to detect my IP
[10:16] <KitsuWhooa> oh well
[10:18] <KitsuWhooa> alkisg: how do I get ipxe to boot?
[10:18] <alkisg> alkisg: do you mean how to put it in floppy/cd/usb?
[10:18] <alkisg> or, how does it detect the server ip?
[10:18] <KitsuWhooa> the latter
[10:18] <KitsuWhooa> I got ipxe running on the machine but it just says press a key to reboot
[10:19] <alkisg> The normal netbooting setup there is "a dhcp server somewhere, e.g. in a router, and my script running dnsmasq in proxydhcp mode to only send the boot server ip/boot filename"
[10:19] <alkisg> Is this your use case? Btw, did you put the VM in bridged mode, so that it has all TCP/UDP ports open?
[10:19] <KitsuWhooa> your script is running dnsmasq, yeah
[10:20] <KitsuWhooa> the VM is in bridged mode
[10:20] <KitsuWhooa> it has an IP in the lan like any other device
[10:20] <KitsuWhooa> and doesn't go through any nAT
[10:20] <KitsuWhooa> *NAT
[10:20] <alkisg> Does ipxe get an ip?
[10:20] <alkisg> You can also try ctrl+b there, and then run `autoboot`, and if it fails, run `config` and see what it got from dhcp/proxydhcp
[10:21] <KitsuWhooa> It definitely shows the mac address, but I'm not sure about the IP. I am already in the command prompt so I'll try that
[10:21] <KitsuWhooa> "[...]/tftpboot/pxelinux... no such file or directory"
[10:21] <KitsuWhooa> it does get an ip
[10:21] <KitsuWhooa> and it does talk to the serv er
[10:22] <KitsuWhooa> *server
[10:22] <alkisg> (01:21:51 μμ) KitsuWhooa: "[...]/tftpboot/pxelinux... no such file or directory" => that sounds like some error in the script? Did you see anything wrong when it ran?
[10:23] <KitsuWhooa> https://tasossah.com/txt/netboot_script_log
[10:25] <alkisg> route => not found => yeah that's an issue
[10:25] <alkisg> Old script, not relying on ip
[10:25] <KitsuWhooa> guess I'll install it
[10:26] <alkisg> Btw it should be trying to download pxelinux.0, not pxelinux
[10:26] <KitsuWhooa> I thought that was to detect the ip
[10:26] <KitsuWhooa> route, I mean
[10:26] <alkisg> Can you try to symlink it just for a quick hack?
[10:26] <alkisg> Yeah, probably
[10:26] <KitsuWhooa> Sure, give me a bit to restart it
[10:27] <KitsuWhooa> I don't think I can
[10:27] <KitsuWhooa> I take that back
[10:27] <KitsuWhooa> however, pxelinux.0 is a symlink itself that points to a file that does not exist
[10:28] <KitsuWhooa> it points to /usr/lib/syslinux/pxelinux.0
[10:28] <KitsuWhooa> my guess is it depends on the pxelinux package
[10:28] <alkisg> It installs it, but it moved elsewhere, moment,
[10:28] <alkisg> /usr/lib/PXELINUX/pxelinux.0
[10:29] <KitsuWhooa> ah
[10:29] <alkisg> OK so I guess the 10 year old script needs to be updated with the new pxelinux path, sorry :D
[10:29] <KitsuWhooa> okay, adding those two symlinks got me further
[10:30] <KitsuWhooa> now it's complaining about .c32 files not being found
[10:31] <KitsuWhooa> https://tasossah.com/txt/netboot_script_log_2
[10:31] <alkisg> The new location is in /usr/lib/syslinux/modules/bios/
[10:31] <alkisg> Copy them from there or symlink them or something
[10:33] <KitsuWhooa> Yeah this is really broken
[10:34] <KitsuWhooa> now it fails loading capser/vmlinuz
[10:34] <KitsuWhooa> ...IO error? what
[10:34] <KitsuWhooa> "attempt to access beyond end of device"
[10:35] <alkisg> I wonder if the vmlinuz/initrd symlinks also point to wrong paths
[10:36] <alkisg> Do an ls -lR in the tftp dir and check for broken symlinks
[10:36] <KitsuWhooa> I remounted the image and now it loads vmlinuz but it can't find initrd.lz inside the casper dir
[10:36] <KitsuWhooa> and no, no broken symlinks
[10:36] <alkisg> It might be initrd.gz now or something
[10:36] <KitsuWhooa> there's only an initrd in the image
[10:36] <KitsuWhooa> and I can't modify anything under casper because it's a symlink to /cdrom
[10:37] <alkisg> Ah you modify the kernel etc in pxelinux.cfg/default
[10:37] <alkisg> That's somewhere under tftp, a file that the script generated
[10:37] <KitsuWhooa> I found it
[10:38] <KitsuWhooa> looks like it's booting
[10:38]  * alkisg crosses fingers, took too long already...
[10:39] <KitsuWhooa> "nfs server not responding"
[10:39] <alkisg> Maybe that route part that failed, failed to export to local network only
[10:39] <alkisg> what's /etc/exports like?
[10:39] <KitsuWhooa> I installed route and restarted 
[10:40] <KitsuWhooa> there's /cdrom in there
[10:40] <KitsuWhooa>  /cdrom *(ro,no_subtree_check,no_root_squash)
[10:40] <KitsuWhooa> oh in the script log there's a "job for nfs-server.service cancelled"
[10:40] <alkisg> Sounds good. Try `exportfs -ra` in case it helps,
[10:41] <alkisg> ah
[10:41] <alkisg> try restarting it manually from another tab and see why it fails to start
[10:41] <alkisg> systemctl stop nfs-kernel-server, then start again, journalctl -xe, etc
[10:42] <KitsuWhooa> active (exited)
[10:42] <KitsuWhooa> and there are no errors whatsoever
[10:42] <KitsuWhooa> there are only two lines in the log, starting and started
[10:42] <KitsuWhooa> but it's not running
[10:43] <alkisg> https://serverfault.com/questions/859934/ubuntu-16-04-nfs-kernel-server-wont-start
[10:43] <alkisg> s your Ubuntu server a linux container (lxc) ?  If yes, you need to set something like explained here:  mount fstype=rpc_pipefs, mount fstype=nfsd, 
[10:44] <KitsuWhooa> That happened on my 16.04 desktop because I run a custom kernel that didn't have the filesystem needed
[10:44] <alkisg> I wonder if running in a VM is related there
[10:44] <KitsuWhooa> the 18.04 VM is virtualbox
[10:44] <alkisg> Hmm
[10:44] <KitsuWhooa> so I very much doubt it
[10:44] <KitsuWhooa> it doesn't fail to start it
[10:44] <KitsuWhooa> issuing the start command doesn't throw any errors, I mean
[10:45] <KitsuWhooa> I wonder if the unattended update installed a new kernel and broke things. Let me reboot the VM and redo all the symlinks :p
[10:46] <alkisg> Ah, damn those unattended updates :)
[10:47] <KitsuWhooa> I ended up disabling them, since it's a VM anyway
[10:53] <KitsuWhooa> yeah sch.gr hosting is terrible
[10:53] <KitsuWhooa> I can't even download the script over v4 now :p
[10:54] <alkisg> Ah don't wget it again, one time is more than enough for one day :D
[10:54] <alkisg> Hehe
[10:54] <KitsuWhooa> I rebooted, so it's gone from /tmp
[10:54] <KitsuWhooa> I made the mistake of not saving it to the disk
[10:54] <KitsuWhooa> there we go, it worked on the third attempt
[10:54] <alkisg> This avoided the segfault, I'll try to limit down now: Option "HWCursor" "off" Option "PageFlip" "off" Option "WrappedFB" "on" Option "ShadowFB" "on"
[10:57] <KitsuWhooa> we're back to ipxe not detecting the server
[10:59] <KitsuWhooa> or the NIC
[10:59] <KitsuWhooa> one of the two
[10:59] <alkisg> What's the client output, does it get an ip? autoboot, config etc...
[10:59] <alkisg> ctrl+b before that
[11:01] <KitsuWhooa> okay yeah that was my fault. Needed to reseat the NIC
[11:01] <KitsuWhooa> sorry
[11:01] <alkisg> tjaalton: I didn't get to compiling git yet, but I found out that `Option "PageFlip" "off"` avoids the segfault in
[11:01] <alkisg> 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation NV5 [Riva TNT2 Model 64 / Model 64 Pro] [10de:002d] (rev 15)
[11:10] <KitsuWhooa> alkisg: after the reboot, nfs is working
[11:10] <KitsuWhooa> however the live image fails
[11:11] <KitsuWhooa> goes into emergency mode
[11:11] <KitsuWhooa> a few units fail, and I can't really figure out why
[11:12] <alkisg> Ouch, it sounds like it needs a lot of updating... I'm using ltsp everywhere now so I haven't updated it
[11:12] <alkisg> Sorry about that
[11:12] <KitsuWhooa> Oh well, it's fine
[11:12] <alkisg> I need to go afk for a couple of hours, be back later... :/
[11:12] <KitsuWhooa> sure
[11:12] <KitsuWhooa> thanks again
[11:13] <alkisg> np, thank you too
[11:13] <alkisg> Do check that pageflip option if you get the chance
[11:13] <KitsuWhooa> I'll go through the sch.gr manual and see if it's worth/easy to set up LTSP
[11:13] <alkisg> Nono ignore sch.gr, follow this one (mine again): http://wiki.ltsp.org/wiki/Installation/Ubuntu
[11:14] <KitsuWhooa> Ah, thanks
[11:14] <alkisg> If you have a mate installation, you can make it an ltsp server in about 5 commands and 10 minutes
[11:14] <alkisg> And it gives epoptes=vnc as a bonus
[11:14] <KitsuWhooa> I have a stock 18.04 with gnome in a vm
[11:15] <alkisg> That works too
[11:15] <KitsuWhooa> does that mean the client will try to boot gnome too
[11:15] <KitsuWhooa> ?
[11:15] <KitsuWhooa> I doubt gnome3 will work
[11:15] <alkisg> In the quick "chrootless" setup, yeah
[11:15] <KitsuWhooa> ah
[11:15] <alkisg> But you can choose an xterm session if you prefer
[11:15] <alkisg> Or install mate as well ...
[11:15] <KitsuWhooa> Ah, yeah
[11:15] <KitsuWhooa> thanks
[11:15]  * alkisg really goes for now, bbl
[11:16] <KitsuWhooa> see ya
[11:42] <KitsuWhooa> alkisg: for when you get back, this is what happened when I tried booting the image in a VM to install it and then set up LTSP https://tasossah.com/s/dd9d97babad3.jpg
[12:07] <alkisg> KitsuWhooa: try alt+ctrl+f1, then alt+ctrl+f7
[12:07] <alkisg> *right ctrl, since it's vbox
[12:08] <KitsuWhooa> it's rshift for me, but wow that worked
[12:08] <KitsuWhooa> (I rebound it)
[12:10]  * alkisg searches how to fetch/compile the git version...
[12:11] <KitsuWhooa> My guess would be to git clone the repo and then either install it directly, or use the files from the ubuntu package to make a deb
[12:12] <KitsuWhooa> packages.ubuntu.com usually points to all the necessary resources
[12:12] <alkisg> This one? https://cgit.freedesktop.org/nouveau/xf86-video-nouveau/
[12:12] <alkisg> I only see 3 commits there from 2018, that's a good sign that it will be somewhat easy to bisect it...
[12:13] <KitsuWhooa> I think that's the one, yeah
[12:16] <alkisg> The test client ran glxgears for an hour with pageflip off, so I guess it makes things stable
[12:22] <tjaalton> you assume it's fixed in git..
[12:22] <tjaalton> that's not at all clear
[12:22] <alkisg> tjaalton: I don't assume that at all! I'm assuming it broke in git! :)
[12:23] <tjaalton> ah
[12:23] <alkisg> I.e. i want to bisect and find the commit that broke it, I don't expect to find a commit that fixed it...
[12:23] <tjaalton> it's not necessarily the driver that broke it
[12:23] <KitsuWhooa> it may be X server itself
[12:24] <KitsuWhooa> and a smaller chance the drm driver
[12:24] <alkisg> Ah
[12:24] <tjaalton> 1.0.15 was released in april '17
[12:26] <tjaalton> test cosmic
[12:26] <tjaalton> or just file a bug upstream
[12:26] <tjaalton> against nouveau
[12:26] <alkisg> I'll do both tomorrow morning
[12:27] <alkisg> I haven't reproduced the "scrambled screen" issue locally though, only the segfault
[12:27] <alkisg> I can only see the scrambled screen in vbox
[12:27] <KitsuWhooa> pretty sure that's a different issue
[12:28] <KitsuWhooa> I can't get the scrambled screen to go away by switching to a tty and back with my MX400
[12:28] <alkisg> Gotcha. So, 3 different issues.
[12:28] <KitsuWhooa> and then I got distracted trying to set up netboot/ltsp
[12:28] <alkisg> Hehe
[13:12] <KitsuWhooa> alkisg: I installed ltsp-server-standalone ltsp-client, ran ltsp-update-image, didn't install epoptes, and ran the first ltsp-config dnsmasq line in the wiki
[13:12] <KitsuWhooa> is it supposed to be working now?
[13:19] <alkisg> KitsuWhooa: I believe so, do you get any errors while booting the client?
[13:20] <KitsuWhooa> ipxe says nothing to boot
[13:21] <KitsuWhooa> dnsmasq seems to be running
[13:21] <KitsuWhooa> but ipxe is only seeing my normal gateway
[13:42] <alkisg> try ltsp-config dnsmasq --overwrite; systemctl restart dnsmasq
[13:46] <KitsuWhooa> that was the first thing I did, and I even rebooted
[13:46] <KitsuWhooa> Only different thing I did from the wiki was to not install epoptes or make a user for it
[14:13] <KitsuWhooa> Apparently I just needed to change the subnet in the dnsmasq conf for the proxy
[14:28] <alkisg> It should be autodetected... unless you didn't have an ip when you run ltsp-config
[14:30] <KitsuWhooa> that might have been it
[14:30] <KitsuWhooa> but since it works now, I can also reproduce the bug with an MX440
[14:31] <alkisg> The fuzzy lines or the segfault?
[14:32] <KitsuWhooa> corruption
[14:32] <KitsuWhooa> no segfault
[14:32] <KitsuWhooa> I think the segfault might be exclusive to the TNT2
[14:33] <alkisg> does the pageflip off option fix the corruption?
[14:33] <KitsuWhooa> I haven't tried yet