Skip to content
Back to Quality Assurance

Secret Management

Store, rotate, and deliver secrets (API keys, DB passwords, signing keys) without ever leaking them.

Why

A leaked database password or API key is the most common cause of a cloud account takeover, and the leak almost always happens through a vector that was considered convenient at the time: a .env file committed to git, a secret hardcoded for a quick test, a CI log that printed the environment, a Slack message sharing a key for a pair-programming session. Secret management is about making the secure path the easy path — a secrets manager (Vault, AWS Secrets Manager, Doppler) that injects secrets at runtime, so developers never touch raw secrets and never have a reason to commit them. The cost of a leak is catastrophic (account takeover, data breach, compliance violation), and the cost of prevention is a few hours of setup that pays back forever.

When

Set up the secrets manager before the first secret exists — retrofitting is painful because secrets have already spread to .env files, CI variables, and Slack. Rotate all secrets immediately if any leak is suspected.

How

1. Choose a secrets manager and make it the only source of secrets
   - Cloud: AWS Secrets Manager / Parameter Store, GCP Secret Manager, Azure Key Vault
   - Self-hosted: HashiCorp Vault, Doppler, Infisical
   - The manager is the single source; .env files are for local dev only and gitignored

2. Inject secrets at runtime, never bake them into the image or commit them
   - Container: read from the manager on startup, set as env vars in the process
   - CI: use OIDC federation to assume a role, fetch secrets, never store them in CI vars
   - Local dev: .env.local (gitignored) or doppler run -- npm run dev

3. Scan for leaked secrets continuously
   - gitleaks or TruffleHog on every commit (pre-commit hook) and every PR
   - Enable GitHub secret scanning + push protection (blocks the commit before it lands)
   - If a secret is found in git history, rotate it immediately — do not just delete the file

4. Rotate secrets on a schedule and on every team member departure
   - Database passwords: every 90 days; API keys: every 30-180 days per provider policy
   - On employee departure: rotate all secrets they had access to, within 24h
   - Automate rotation where the manager supports it (AWS RDS auto-rotation)

5. Audit access and never log secrets
   - Log a hash or "***" never the raw value; ensure serializers redact secret fields
   - Audit who accessed which secret and when; alert on anomalous access patterns

Example

# .github/workflows/ci.yml — OIDC federation, no long-lived secrets in CI
name: CI
on: [pull_request]
permissions:
  id-token: write   # required for OIDC
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123:role/ci-deploy
          aws-region: us-east-1
      - name: Fetch secrets at runtime
        run: |
          export DB_PASSWORD=$(aws secretsmanager get-secret-value             --secret-id prod/db-password --query SecretString --output text)
          npm test
      # The DB password never appears in CI logs, never stored as a GitHub secret,
      # and is rotated automatically by AWS every 90 days.

# Pre-commit hook: block secrets before they are committed
# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.0
    hooks:
      - id: gitleaks
        # If this hook fires, the commit is blocked. Rotate the secret if it
        # was ever committed in the past — git history is forever.

Common Problems

  • ⚠Storing secrets as CI provider variables (GitHub Secrets) and treating that as secure — CI variables are visible in logs if printed, accessible to anyone with write access to the repo, and not rotated automatically, so they accumulate and leak over time.
  • ⚠Committing a .env file to git "just for the team to have the local config" — the file contains the production database password, it is now in git history forever, and deleting the file in a later commit does not remove it from history.
  • ⚠Hardcoding a Stripe key in the source for a quick test and forgetting to remove it — the key ships to production, is in the public bundle if it is a frontend key, and is found by an attacker scanning GitHub or the deployed JS.
  • ⚠Not rotating secrets on employee departure — a departing engineer still has the database password in their local .env and can access production after they leave, which is a real insider-threat scenario.

Best Practices

  • ✓Use OIDC federation in CI so no long-lived secrets are stored at the CI provider — the CI job assumes a cloud role via a short-lived token, fetches secrets from the manager, and the credentials expire in minutes.
  • ✓Enable push protection (GitHub, GitLab) so a commit containing a recognized secret pattern is blocked before it lands — this stops the leak at the source, before it reaches history.
  • ✓Automate rotation wherever the secrets manager supports it (AWS RDS auto-rotation, Vault dynamic secrets) — manual rotation is forgotten, and a secret that is never rotated is a secret that is one leak away from a breach.
  • ✓Treat any secret found in git history as compromised and rotate it immediately — deleting the file does not remove it from history, and the secret must be assumed leaked the moment it was committed.