Scaling Patterns
The architectural shapes that let a system grow — vertical, horizontal, sharding, partitioning — and when each applies.
Why
Scaling is not one thing — vertical (bigger box), horizontal (more boxes), sharding (split by key), read replicas (split read/write) each solve a different bottleneck and each has a different cost. Choosing the wrong pattern wastes money and often doesn't fix the bottleneck (e.g. adding read replicas when the limit is write throughput). Knowing the patterns and their trade-offs lets you pick the cheapest one that actually addresses your limit, and sequence them so you don't shard (the most expensive) before you've exhausted horizontal scaling (the cheapest).
When
When a component hits its limit. Takes half a day to choose + implement. Revisit at each 10x growth.
How
1. Identify the bottleneck: CPU, memory, connections, DB writes, DB reads, storage? 2. Match the pattern to the bottleneck: - CPU/memory bound, stateless → horizontal (add instances) - CPU/memory bound, stateful → vertical (bigger instance) then shard - DB read bound → read replicas - DB write bound → sharding by key, or move to a write-optimized store - Storage bound → partition by time (cold storage) or shard by tenant 3. Apply the cheapest pattern that fixes the bottleneck first 4. Measure before and after; if the bottleneck didn't move, you picked wrong 5. Plan the next limit — what breaks at 10x after this fix?
Example
Bottleneck: DB write throughput at 8k writes/s (Postgres limit on this instance) Options in cost order: 1. Vertical: r5.2xlarge → r5.4xlarge → ~12k writes/s, +$400/mo ← cheapest 2. Connection pooler (pgbouncer): frees write slots → ~10k, +$20/mo 3. Shard by user_id: 2 shards → ~16k writes/s, +$500/mo + migration 4. Move writes to Cassandra: ~50k writes/s, +$2k/mo + rewrite Chosen: pgbouncer (2) + vertical (1) → 15k writes/s for +$420/mo. Sharding deferred until 30k writes/s needed (gives 2 years).
Common Problems
- ⚠Sharding before exhausting vertical + horizontal — sharding is the most expensive scaling (cross-shard queries, rebalancing) and is often premature.
Best Practices
- ✓Always measure the bottleneck before and after scaling — 'we added instances and it feels faster' is not validation.
- ✓Choose a shard key that distributes writes evenly and keeps related data co-located — a bad shard key creates hot shards and cross-shard joins.