Measuring TCP Pacing and RTT Variance Under BBRv1 vs Cubic

When running high-bandwidth flows over transpacific paths (RTT ~130–155ms), the classical Cubic congestion control algorithm inevitably drives bufferbloat on bottleneck routers with deep buffers. BBRv1 was designed to pace packets at the estimated bottleneck bandwidth while bounding the inflight data to BDP + 1.5 * min_rtt.

Over a four-week benchmark measuring 100 Gbps aggregate transfers across our San Jose to Tokyo transit circuit, we captured packet arrival timings using hardware-timestamped NIC queues. With Cubic, TCP sender bursts led to periodic micro-burst queueing delays averaging 18.4ms above line RTT. Switching the queuing discipline to fq with BBR reduced queue delay jitter to under 1.2ms.

# Kernel pacing parameters
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_notsent_lowat = 16384

However, BBRv1 displayed an aggressive tendency when competing against loss-based flows in shallow-buffered switches, occasionally consuming over 82% of total link capacity due to its cycle of probeRTT draining every 10 seconds. This empirical measurement reinforced our decision to deploy explicit pacing lowat boundaries on our measurement nodes.