Network Latency Optimization

Every millisecond of latency costs money. Amazon found that every 100ms of latency cost them 1% in sales. Google found that an extra 500ms in search page load time dropped traffic by 20%. Network latency is the physics you cannot engineer away — but you can minimize, mask, and optimize around it.

---

Latency Budget

Total request latency = DNS + TCP + TLS + Server + Transfer

Example: API call from New York to Oregon
  DNS resolution:        2ms  (cached)
  TCP handshake:        35ms  (1 round trip × 70ms RTT)
  TLS handshake:        70ms  (2 round trips × 35ms each)
  Server processing:    15ms  (application logic)
  Response transfer:    35ms  (1 round trip + data size)
  ────────────────────────────
  Total:               157ms

Same call with optimization:
  DNS resolution:        0ms  (prefetched)
  TCP handshake:         0ms  (connection reuse - keep-alive)
  TLS handshake:         0ms  (TLS session resumption)
  Server processing:    12ms  (optimized query)
  Response transfer:    35ms  (same physics)
  ────────────────────────────
  Total:                47ms  (70% reduction!)

---

TCP/TLS Optimization

# Nginx: Optimized TCP/TLS configuration

# TCP optimization
tcp_nodelay on;        # Disable Nagle's algorithm (reduce latency)
tcp_nopush on;         # Optimize packet sending (sendfile)
sendfile on;           # Direct file transfer (kernel-level)

# Keep-alive: Reuse TCP connections
keepalive_timeout 65;
keepalive_requests 1000;  # Max requests per connection

# TLS optimization
ssl_session_cache shared:SSL:50m;
ssl_session_timeout 1d;
ssl_session_tickets on;      # TLS session resumption

# TLS 1.3: Only 1 round trip (vs 2 for TLS 1.2)
ssl_protocols TLSv1.3;

# OCSP stapling: Avoid extra round trip for certificate validation
ssl_stapling on;
ssl_stapling_verify on;

# HTTP/2: Multiplexing = multiple requests on one connection
http2 on;

# Early hints: Send preload hints before response is ready
add_header Link "</style.css>; rel=preload; as=style" early;

---

DNS Optimization

<!-- DNS prefetching: Resolve domains before they're needed -->
<link rel="dns-prefetch" href="//api.example.com">
<link rel="dns-prefetch" href="//cdn.example.com">
<link rel="dns-prefetch" href="//analytics.example.com">

<!-- Preconnect: DNS + TCP + TLS ahead of time -->
<link rel="preconnect" href="https://api.example.com">
<link rel="preconnect" href="https://fonts.googleapis.com">

<!-- Preload: Fetch critical resources immediately -->
<link rel="preload" href="/fonts/inter.woff2" as="font" crossorigin>
<link rel="preload" href="/api/v1/initial-data" as="fetch" crossorigin>

---

Edge Computing

Without edge:
  User (Tokyo) → Origin (Oregon)
  Round trip: ~150ms
  
With edge (CDN + edge compute):
  User (Tokyo) → Edge (Tokyo)
  Round trip: ~10ms
  
  Static content: Served from edge cache (0ms compute)
  Dynamic content: Edge function processes at edge (~5ms)
  Personalized: Edge function + origin API call (~50ms total)
  
  ┌──────────┐      ┌──────────┐      ┌──────────┐
  │  User    │─10ms─│  Edge    │─140ms─│  Origin  │
  │ (Tokyo)  │      │ (Tokyo)  │      │ (Oregon) │
  └──────────┘      └──────────┘      └──────────┘
  
  Cache hit: 10ms total ✓
  Edge compute: 15ms total ✓
  Cache miss: 150ms total (same as without edge)

---

Anti-Patterns

Anti-Pattern	Consequence	Fix
New TCP connection per request	Handshake overhead on every call	Connection pooling, keep-alive
TLS 1.2 without session resumption	2 extra round trips per new connection	TLS 1.3 or session tickets
DNS resolution on every request	20-100ms added per request	DNS caching, prefetching
All processing at origin	Users far from origin pay latency tax	Edge computing for latency-sensitive paths
No compression	Large payloads = more transfer time	Brotli/gzip compression

Network latency is governed by the speed of light — you cannot make packets travel faster. But you can make them travel shorter distances (edge), make fewer trips (connection reuse), and send less data (compression).