## 9. On Transfer Speeds

Let’s say we’re trying to copy a $$1$$ GiB file from one machine to another over a gigabit ethernet LAN. How long will it take? (Spoiler: it’s quite a bit more than the simple calculation says.)

Our setup contains two machines connected via gigabit ethernet to a router. We’re going to transfer $$1$$ GiB ($$2^30$$ bytes) between them. A quick calculation tells us that this should take $$8.6$$s, but does it?

We have a few ways of transferring the file; let’s start with the simplest: netcat.

# Start netcat on one of the machines
edge13 % nc -l 12345 > /dev/null

# Create a 1 GiB file on the other machine
edge12 % dd bs=1024 count=1048576 \
if=/dev/zero of=file.bin
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied,
12.6983 s, 84.6 MB/s

# Transfer the file over via netcat
edge12 % time nc edge13 12345 < file.bin

I see it takes $$13.27$$s to transfer $$1$$ GiB between the two machines with netcat (average over $$5$$ runs). That comes up to $$647.31$$ mbps. This is pretty bad, considering the network’s theoretical speed of $$1000$$ mbps.

Now, let’s try scp:

# Transfer the file over via scp
edge12 % time scp file.bin edge13:/dev/null

I see it takes $$15.30$$s to transfer $$1$$ GiB between the two machines with scp (average over $$5$$ runs). That comes up to $$561.43$$ mbps, which is worse than netcat. That’s not unexpected, though, considering that scp runs over ssh, which must add quite a bit of overhead.

Interestingly enough, netcat doesn’t seem to be as fast as it can be. Running a pair of very simple programs, I see that $$1$$ GiB of data could be transferred in $$11.95$$s. That comes up to $$718.82$$ mbps. The size of the receiver’s buffer seems to play a big role in determining the speed of the transfer. Anything smaller than $$1$$ KiB, or larger than $$64$$ KiB increases the transfer time by full seconds.

The moral of this story is this: when estimating how long a particular file transfer will take, you should guess that it will be at least twice as slow as the network speed would allow.