Resource Optimization

Why

When

How

1. Collect 30 days of usage metrics per service
   - CPU and memory usage, p50/p95/p99, per pod
   - Use the metrics server or a monitoring tool (Datadog, Grafana)
   - 30 days captures weekly patterns and occasional spikes

2. Set requests to p95 usage + headroom
   - requests.cpu = p95(cpu) * 1.2   (20% headroom)
   - requests.memory = p95(memory) * 1.3  (30% headroom; memory is less bursty)
   - Requests determine scheduling; setting them too high wastes cluster resources

3. Set limits carefully (or remove them)
   - limits.cpu = p99(cpu) * 2  (allow bursts; CPU throttle is graceful)
   - limits.memory = p99(memory) * 1.5  (OOMKilled is NOT graceful; set high or remove)
   - If OOMKilled has caused incidents → remove memory limit (let the node handle it)

4. Right-size replicas and HPA
   - If p95 CPU is 10% of request → you're over-provisioned; lower request or reduce replicas
   - If the HPA never scales above min → min is too high; lower it
   - If the HPA is always at max → max is too low; raise it (or fix the bottleneck)

5. Validate after right-sizing
   - Deploy the new requests/limits, watch for 1 week
   - No OOMKilled? No CPU throttle? Latency in SLO? → right-size succeeded
   - If issues → revert and re-examine (you used bad data or wrong headroom)

Example

# Right-sizing checkout (30 days of metrics):
#   CPU:    p50 0.2, p95 0.5, p99 0.8  (cores)
#   Memory: p50 200MB, p95 400MB, p99 600MB

# Current (over-provisioned):
resources:
  requests: { cpu: "2", memory: "2Gi" }   # 4x over on CPU, 5x on memory
  limits:   { cpu: "4", memory: "4Gi" }

# Right-sized:
resources:
  requests:
    cpu: "0.6"      # p95(0.5) * 1.2 = 0.6
    memory: "520Mi" # p95(400) * 1.3 = 520
  limits:
    cpu: "1.6"      # p99(0.8) * 2 = 1.6 (CPU throttle is graceful)
    # memory limit removed — OOMKilled is worse than node pressure

# Cost savings:
#   Before: 50 pods × 2 CPU × $50/CPU/mo = $5,000/mo
#   After:  50 pods × 0.6 CPU × $50/CPU/mo = $1,500/mo
#   Savings: $3,500/mo = $42k/year (70% reduction, no headroom loss)

# HPA right-sizing:
#   Current: min 10, max 50. HPA never exceeds 12.
#   → min 5 (lower baseline), max 30 (lower cap; if it hits 30, fix the bottleneck)
#   → Saves 5 pods × 0.6 CPU × $50 = $150/mo more

# Validation (1 week after):
#   OOMKilled: 0  ✅
#   CPU throttle: <0.1%  ✅
#   p99 latency: 450ms (SLO 500ms)  ✅
#   → Right-size succeeded

Common Problems

• ⚠Right-sizing without data — guessing '1 CPU' leaves you over (waste) or under (OOM); use 30 days of p95/p99 metrics.
• ⚠Memory limit too low — OOMKilled during a burst, losing in-flight requests; set high or remove (OOM is worse than node pressure).
• ⚠One-time right-size, never re-checked — usage drifts as code changes; re-right-size quarterly.
• ⚠Right-sizing requests but not replicas — 50 pods at 0.6 CPU each is still 30 CPU; if p95 is 0.5, you need 5 pods, not 50.

Best Practices

• ✓Collect 30 days of p50/p95/p99 CPU and memory per pod; set requests to p95 + 20% headroom, limits to p99 * 2 (or remove memory limit if OOMKilled is risky).
• ✓Right-size replicas and HPA too: if p95 CPU is 10% of request, lower request or reduce replicas; if HPA never exceeds min, lower min.
• ✓Validate after right-sizing: watch 1 week for OOMKilled, CPU throttle, SLO breaches; revert if issues appear.
• ✓Re-right-size quarterly and after major code changes; usage drifts, and a service right-sized 6 months ago may be over-provisioned now.