Test Gates

Why

When

How

1. Run unit tests on every PR (fast, 1-5 minutes)
   - npm test (jest, pytest, cargo test)
   - Run only the affected tests if possible (jest --findRelatedTests)
   - Fail the build if any test fails
   - This is the most important gate — it catches logic regressions

2. Run integration tests on every PR (medium, 5-10 minutes)
   - Test the API endpoints with a real or test database
   - Test the DB queries with a real or test database
   - Use a test database (not production) — spin up in CI, tear down after
   - Fail the build if any integration test fails

3. Run e2e tests on every PR or on main (slow, 10-30 minutes)
   - Test the critical user flows (login, checkout, search)
   - Use Playwright, Cypress, or Selenium
   - Run on every PR if fast enough, or only on main if too slow
   - Fail the build if any e2e test fails

4. Enforce coverage thresholds
   - jest --coverage --coverageThreshold='{"global":{"lines":80}}'
   - Fail the build if coverage drops below the threshold
   - This ensures new code is tested
   - Do not set the threshold too high (100% is counterproductive)

5. Parallelize the tests
   - jest --maxWorkers=4 (run 4 test files in parallel)
   - Split the e2e tests across multiple runners
   - This reduces the gate time from 20 minutes to 5 minutes

6. Report the results clearly
   - Show which tests failed, with the error message and stack trace
   - Show the coverage report (which lines are not covered)
   - Use a test reporter that integrates with GitHub (annotations on the PR)

Example

# .github/workflows/ci.yml — test gates
name: CI
on: [pull_request]

jobs:
  # Gate 1: Unit tests (2m) — fast, run on every PR
  unit-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test -- --coverage --maxWorkers=4
        env:
          COVERAGE_THRESHOLD: '{"global":{"lines":80,"branches":70}}'

  # Gate 2: Integration tests (5m) — medium, run on every PR
  integration-test:
    needs: [unit-test]  # wait for unit tests to pass
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: test
        ports: ['5432:5432']
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run test:integration

  # Gate 3: E2E tests (15m) — slow, run on every PR
  e2e-test:
    needs: [integration-test]  # wait for integration tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run build
      - run: npx playwright test
      - uses: actions/upload-artifact@v4
        if: always()  # upload screenshots even on failure
        with:
          name: playwright-report
          path: playwright-report/

# package.json — test scripts
{
  "scripts": {
    "test": "jest",                    // unit tests
    "test:integration": "jest --config jest.integration.config.js",
    "test:e2e": "playwright test"
  }
}

# jest.config.js — coverage threshold
{
  "coverageThreshold": {
    "global": {
      "lines": 80,
      "branches": 70,
      "functions": 80,
      "statements": 80
    }
  }
}

# Branch protection: require all test gates
# Settings > Branches > Branch protection rules:
# - Require status checks: unit-test, integration-test, e2e-test
# - Do not allow bypassing

# Test failure annotation on the PR (GitHub)
# The test reporter shows the failing test as an annotation on the PR:
# ❌ test/user.test.ts > getUser > returns the user with the matching id
#   Expected: { id: 1, name: "Alice" }
#   Received: { id: 1, name: null }
#   at Object.<anonymous> (test/user.test.ts:15:23)

Common Problems

• ⚠Setting the coverage threshold to 100% — developers spend time writing tests for trivial getters and setters to meet the threshold, and the tests provide no value but slow down the suite and the review.
• ⚠Running e2e tests on every PR when they take 30 minutes — the developer waits 30 minutes for feedback, the pipeline is perceived as slow, and the team starts bypassing the e2e gate to unblock PRs.
• ⚠Not spinning up a test database for integration tests — the tests run against the developer database or a shared database, the tests are flaky (data conflicts between PRs), and the gate is unreliable.
• ⚠Not failing the build on coverage drops — the coverage decreases from 80% to 60% as untested code is added, the gate still passes (no threshold), and the codebase accumulates untested code.

Best Practices

• ✓Run unit tests (fast, every PR), integration tests (medium, every PR with a test database), and e2e tests (slow, every PR or main) as separate gates, ordered from fast to slow with needs: dependencies.
• ✓Enforce a coverage threshold (80% lines, 70% branches) that fails the build on drops — this ensures new code is tested, but do not set it to 100% (it is counterproductive and wastes time on trivial tests).
• ✓Parallelize the tests (jest --maxWorkers=4, split e2e across runners) and run only the affected tests when possible — this reduces the gate time from 20 minutes to 5 minutes.
• ✓Spin up a fresh test database in CI for integration tests (not a shared database) and upload artifacts (screenshots, reports) on failure — this makes the tests reliable and the failures debuggable.