Performance Profiling

Measure before optimizing — find the actual bottleneck with a profiler, not a guess.

Why

Optimization without measurement is gambling. Developers 'know' the DB is slow, so they add a cache — but the real bottleneck is a N+1 ORM query the cache doesn't help, and now they have a stale-data bug too. A profiler (CPU, flame graph, APM trace) shows exactly where time goes: 60% in serializeUser, 30% in the DB, 10% in render. You fix serializeUser (a 3-line change) and get a 2x speedup; without the profile you'd have spent a week on the cache. The rule is: profile, find the top item, fix only that, re-profile. Repeat. This is 10x faster than 'optimizing everything that looks slow'.

When

When a user-facing operation is slower than its target (e.g. API > 200ms, page > 2s). Takes 30-60 min to profile + fix. Skip for ops that already meet target.

How

1. Define the target and measure the current state
   - "Checkout API must be < 200ms p95"; measure: it's 850ms p95
   - Use the production trace, not a local synthetic — local has no real data volume

2. Capture a profile of the slow path
   - Node: --cpu-prof flag, then inspect with clinic.js or 0x
   - Browser: Chrome DevTools Performance tab, record the interaction
   - DB: EXPLAIN ANALYZE on the slow query

3. Read the flame graph / trace and find the single biggest bar
   - Ignore everything below 10% — fixing a 2% bottleneck won't move the p95
   - Note the function name and the line

4. Fix only that one bottleneck — one change, one PR
   - Common wins: N+1 → join, sync JSON.parse → streaming, missing index, memoize a pure fn
   - Re-measure: did p95 drop? If not, the fix didn't work — revert and re-profile

5. Stop when you hit the target — don't gold-plate
   - "Fast enough" is a real bar; further optimization has diminishing returns and adds complexity

Example

# Node CPU profile of a slow API
node --cpu-prof --cpu-prof-name=checkout.cpuprofile server.js
# then load checkout.cpuprofile into chrome://inspect → CPU profiler

# 0x flame graph (one command, opens a browser)
npx 0x server.js
# hit the slow endpoint, then Ctrl+C → 0x generates flamegraph.html

# Postgres — find the slow query and its plan
EXPLAIN ANALYZE SELECT * FROM orders o JOIN items i ON i.order_id = o.id WHERE o.user_id = 42;
# Seq Scan on orders (cost=0..8500 rows=10000)  ← missing index!
CREATE INDEX idx_orders_user_id ON orders(user_id);
# re-run EXPLAIN ANALYZE → Index Scan, 8ms

Common Problems

⚠Profiling locally with 10 rows, then 'fixing' a bottleneck that doesn't exist at prod scale (1M rows).
⚠Optimizing 5 things in one PR, seeing a 30% win, and not knowing which change drove it — so it regresses.
⚠Adding a cache before profiling, hiding the real bottleneck (the N+1) and introducing stale-data bugs.
⚠Gold-plating past the target — p95 is 180ms (target 200ms) and you spend a week getting it to 120ms that users can't perceive.

Best Practices

✓Profile production (or a prod-like staging with real data volume) — local profiles lie because the data shape is wrong.
✓Make one optimization per PR and include the before/after p95 in the description so the win is attributable and reviewable.
✓Set an SLO budget (e.g. p95 < 200ms) and alert when you burn 50% of the budget — catch regressions before users notice.
✓Re-profile after every fix; the bottleneck moves, and the second-biggest bar is now the one to fix.