TCP Kernel Tuning: Speeding Up High-Latency Links

Why a high-latency link "won't fill up"

TCP caps how much unacknowledged data can be in flight using its sliding window. The most a link can hold at once is its bandwidth-delay product (BDP):

BDP (bytes) = bandwidth (bytes/sec) × RTT (seconds)

Take a 200 Mbps trans-oceanic path with a 150 ms RTT:

200 Mbps ≈ 25 MB/s
BDP = 25,000,000 × 0.15 ≈ 3.75 MB

If your TCP buffer ceiling is the default few hundred KB, the window can never grow to the BDP, so the pipe sits half-empty no matter how much bandwidth you're paying for.

The parameters that matter

First confirm window scaling is on — it's the prerequisite for large windows and is enabled by default on modern kernels:

sysctl net.ipv4.tcp_window_scaling   # should be 1

Then drop your values into /etc/sysctl.d/99-tcp-tuning.conf (size them to your BDP; here we use an 8 MB ceiling as an example):

# three values: min default max (bytes)
net.ipv4.tcp_rmem = 4096 131072 8388608
net.ipv4.tcp_wmem = 4096 16384 8388608
# hard ceiling for socket buffers
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608
# keep receive-buffer autotuning on
net.ipv4.tcp_moderate_rcvbuf = 1

The key idea: the third value in tcprmem/tcpwmem is the upper bound for autotuning, not a fixed allocation. Set it a little above your BDP and the kernel grows the buffer only as needed. Just make sure rmemmax/wmemmax are at least as large as that ceiling, or it gets clamped.

Pair it with BBR

Switch the congestion control to BBR, which usually beats the default cubic on lossy, high-latency paths:

net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

BBR needs kernel 4.9 or newer — anything from Ubuntu 20.04 / Debian 11 onward qualifies.

Apply and verify

sudo sysctl --system                       # or: sysctl -p <file>
sysctl net.ipv4.tcp_congestion_control     # expect bbr
sysctl net.core.rmem_max                   # confirm the new value

Then run iperf3 between the two ends and compare throughput before and after. Trust the numbers, not the assumption that "tuning made it faster."

Watch out for

• Don't paste in the giant magic numbers from random blogs. These buffers are per connection; on a busy server, × connections can exhaust RAM or trigger the OOM killer. A ceiling of 1–2× your real BDP is plenty.
• This only helps high-BDP paths (high latency or high bandwidth). It does nothing for local/low-latency links, CPU- or disk-bound workloads, or application-level rate limits.
• Both ends need headroom: downloads are gated by the receiver's rmem, uploads by the sender's wmem.
• Change one thing at a time and re-measure. Stacking a dozen settings at once makes a regression impossible to pin down.

Summary

Speeding up a high-latency link comes down to letting the TCP window reach the BDP: estimate the BDP, raise the tcprmem/tcpwmem ceilings and rmemmax/wmemmax, confirm window scaling, add BBR, apply with sysctl --system, and verify with iperf3. Size it to your actual link and skip the cargo-cult configs.