[09:11] <foobarry> scp and rsync stalls at 64GB through an 80GB file :|
[09:11] <foobarry> consistently
[09:11] <SuperMatt> what is your local filesystem?
[09:12] <foobarry> i was copying to NFS, so i'm about to try local fs
[09:15] <SuperMatt> you'll probably find that NFS can't handle files over 64G
[09:20] <foobarry> maybe in the 90s
[09:21] <SuperMatt> what's the filesystem behind the nfs?
[09:21] <SuperMatt> nfsv3 only supports up to 64G
[09:22] <foobarry> i have to extend my VG to fit it locally :(
[09:22] <foobarry> orly? wow
[09:22] <foobarry> that sucks
[09:23] <SuperMatt> wait, I may have read something wrong
[09:23] <SuperMatt> one sec
[09:23] <SuperMatt> but also, find out what the underlying fs is
[09:23] <foobarry> also i'm using nfs4
[09:23] <foobarry>  /share/apps type nfs4
[09:23] <SuperMatt> "Version 3 clients support larger files (up to 64 bit offsets). Maximum file size depends on the NFS server's local file systems"
[09:24] <SuperMatt> so whatever the filesystem that nfs share is sharing is where the limit is likely to be
[09:24] <foobarry> /dev/mapper/vol1-data on /exports/data type ext4 (rw,relatime,seclabel,data=ordered)
[09:24] <foobarry> no probs there
[09:24] <foobarry> could be a network issue too
[09:24] <foobarry> gonna try local fs as soon as i grow it
[09:24] <SuperMatt> maybe don't use nfs, ssh or rsync, instead try netcat
[09:25] <SuperMatt> https://nakkaya.com/2009/04/15/using-netcat-for-file-transfers/
[09:26] <brobostigon> morning boys and girls.
[09:52] <zmoylan-pi> split the file and reassemble on destination?
[09:53] <foobarry> thats a workaround , not a solution :P
[09:53] <zmoylan-pi> solutions are better but workarounds can be quicker
[09:56] <foobarry> whole_genome_SNVs.tsv.gz                                                                                                                    80%   64GB   0.0KB/s - stalled -
[09:56] <foobarry> yeah its network/router/fw related
[09:56] <foobarry> local fs bad too
[16:59] <diddledan> run away run away will william tell, shove a fork up his bum and he'll run like heck
[17:00]  * zmoylan-pi takes the fermented banana sandwiches away from diddledan