Let’s say we’re trying to copy a \(1\) GiB file from one machine to another over a gigabit ethernet LAN. How long will it take? (Spoiler: it’s quite a bit more than the simple calculation says.)
Our setup contains two machines connected via gigabit ethernet to a router. We’re going to transfer \(1\) GiB (\(2^30\) bytes) between them. A quick calculation tells us that this should take \(8.6\)s, but does it?
We have a few ways of transferring the file; let’s start with the simplest:
# Start `netcat` on one of the machines edge13 % nc -l 12345 > /dev/null # Create a 1 GiB file on the other machine edge12 % dd bs=1024 count=1048576 \ if=/dev/zero of=file.bin 1048576+0 records in 1048576+0 records out 1073741824 bytes (1.1 GB) copied, 12.6983 s, 84.6 MB/s # Transfer the file over via `netcat` edge12 % time nc edge13 12345 < file.bin
I see it takes \(13.27\)s to transfer \(1\) GiB between the two machines with
netcat (average over \(5\) runs). That comes up to \(647.31\) mbps. This is pretty bad, considering the network’s theoretical speed of \(1000\) mbps.
Now, let’s try
# Transfer the file over via `scp` edge12 % time scp file.bin edge13:/dev/null
I see it takes \(15.30\)s to transfer \(1\) GiB between the two machines with
scp (average over \(5\) runs). That comes up to \(561.43\) mbps, which is worse than
netcat. That’s not unexpected, though, considering that
scp runs over
ssh, which must add quite a bit of overhead.
netcat doesn’t seem to be as fast as it can be. Running a pair of very simple programs, I see that \(1\) GiB of data could be transferred in \(11.95\)s. That comes up to \(718.82\) mbps. The size of the receiver’s buffer seems to play a big role in determining the speed of the transfer. Anything smaller than \(1\) KiB, or larger than \(64\) KiB increases the transfer time by full seconds.
The moral of this story is this: when estimating how long a particular file transfer will take, you should guess that it will be at least twice as slow as the network speed would allow.