Backpressure
Signal upstream to slow down instead of queuing unboundedly or dropping requests.
Why
When a downstream service is slow, the options are: queue (memory grows, latency rises unboundedly), drop (requests fail), or push back (tell the caller to slow down). Backpressure is the third option and the only one that degrades gracefully — the system slows rather than breaks. Without it, a slow DB causes every API instance to queue 10k requests, exhaust memory, and OOM-crash, taking the whole service down. With backpressure (HTTP 429, queue depth limits, reactive streams), the API rejects early, stays alive, and recovers when the DB recovers. It's the difference between a slow service and a dead service.
When
For any service that calls a slower downstream or receives bursty traffic. Takes half a day. Skip for internal-only services with gentle traffic.
How
1. Set a max in-flight requests per instance (e.g. 200); reject with 429 + Retry-After above that 2. For downstream calls, use a bounded queue (not unbounded); when full, fail fast 3. Propagate backpressure: if downstream returns 429, return 429 to your caller (don't queue) 4. Use circuit breakers: after N failures, stop calling for a cooldown, then half-open to test 5. For streaming, use reactive streams (backpressure via request(n) signaling) 6. Load-test past capacity: confirm the service returns 429s and stays up, not OOMs
Example
API with DB dependency, DB slows to 2s/query: Without backpressure: API queues 10k requests → memory 4GB → OOM → service down With backpressure: max_in_flight = 200; request 201 → 429 + Retry-After: 5 DB slow → in-flight fills → API returns 429 to new requests Clients back off (Retry-After) → queue drains → DB recovers → 200s resume Service stayed up the whole time, just degraded.
Common Problems
- ⚠Using an unbounded queue 'to be safe' — it's the opposite of safe; it converts a slow downstream into an OOM. Always bound queues.
Best Practices
- ✓Return 429 with a Retry-After header, not 500 — 429 tells the client to back off; 500 tells it something broke.
- ✓Make the in-flight limit configurable per route — a health check shouldn't compete with user requests for a slot.