| foobarry | scp and rsync stalls at 64GB through an 80GB file :| | 09:11 |
|---|---|---|
| foobarry | consistently | 09:11 |
| SuperMatt | what is your local filesystem? | 09:11 |
| foobarry | i was copying to NFS, so i'm about to try local fs | 09:12 |
| SuperMatt | you'll probably find that NFS can't handle files over 64G | 09:15 |
| foobarry | maybe in the 90s | 09:20 |
| SuperMatt | what's the filesystem behind the nfs? | 09:21 |
| SuperMatt | nfsv3 only supports up to 64G | 09:21 |
| foobarry | i have to extend my VG to fit it locally :( | 09:22 |
| foobarry | orly? wow | 09:22 |
| foobarry | that sucks | 09:22 |
| SuperMatt | wait, I may have read something wrong | 09:23 |
| SuperMatt | one sec | 09:23 |
| SuperMatt | but also, find out what the underlying fs is | 09:23 |
| foobarry | also i'm using nfs4 | 09:23 |
| foobarry | /share/apps type nfs4 | 09:23 |
| SuperMatt | "Version 3 clients support larger files (up to 64 bit offsets). Maximum file size depends on the NFS server's local file systems" | 09:23 |
| SuperMatt | so whatever the filesystem that nfs share is sharing is where the limit is likely to be | 09:24 |
| foobarry | /dev/mapper/vol1-data on /exports/data type ext4 (rw,relatime,seclabel,data=ordered) | 09:24 |
| foobarry | no probs there | 09:24 |
| foobarry | could be a network issue too | 09:24 |
| foobarry | gonna try local fs as soon as i grow it | 09:24 |
| SuperMatt | maybe don't use nfs, ssh or rsync, instead try netcat | 09:24 |
| SuperMatt | https://nakkaya.com/2009/04/15/using-netcat-for-file-transfers/ | 09:25 |
| brobostigon | morning boys and girls. | 09:26 |
| zmoylan-pi | split the file and reassemble on destination? | 09:52 |
| foobarry | thats a workaround , not a solution :P | 09:53 |
| zmoylan-pi | solutions are better but workarounds can be quicker | 09:53 |
| foobarry | whole_genome_SNVs.tsv.gz 80% 64GB 0.0KB/s - stalled - | 09:56 |
| foobarry | yeah its network/router/fw related | 09:56 |
| foobarry | local fs bad too | 09:56 |
| diddledan | run away run away will william tell, shove a fork up his bum and he'll run like heck | 16:59 |
| * zmoylan-pi takes the fermented banana sandwiches away from diddledan | 17:00 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!