Skip to content
Back to Quality Assurance

Review Automation

Automate the repetitive parts of review (lint, test, stale PRs) so reviewers focus on the parts that need human judgment.

Why

Review has two parts: mechanical (style, types, tests, coverage) and judgment (logic, architecture, security, naming). The mechanical part is repetitive, error-prone, and exactly what tools do better — a linter never misses a style violation, a type checker never misses a null dereference, and a coverage tool never miscounts. By automating the mechanical part, the reviewer's time is spent entirely on judgment — the part that requires human insight and that tools cannot do. This makes reviews faster (15 minutes instead of an hour), more valuable (the feedback is about the design, not the formatting), and more consistent (the tool applies the same standard every time, while humans vary). Automation also removes the social friction — nobody enjoys commenting on a missing semicolon, and the reviewer who does is resented. By letting the tool enforce the mechanical standard, the human review is about collaboration and improvement, not policing.

When

Set up automation at project start and add tools as the team identifies repetitive review comments. Review the automation quarterly to add new checks.

How

1. Automate the mechanical checks in CI
   - Linter (ESLint, Biome): style, common bugs, unsafe patterns
   - Formatter (Prettier): no formatting discussions
   - Type checker (TypeScript): no type errors
   - Test coverage: require coverage for new code
   - Dependency scan (Dependabot, Snyk): no known vulnerabilities

2. Automate the PR workflow
   - Auto-assign reviewers based on the files changed (CODEOWNERS)
   - Auto-label PRs (bug, feature, refactor) based on the title or files
   - Auto-close stale PRs after 30 days (stale bot)
   - Auto-merge PRs that pass all checks and are approved (for low-risk changes)

3. Automate the review reminders
   - Ping the reviewer if the PR has no review after 24 hours
   - Ping the author if the review has comments but no response after 48 hours
   - Do not ping on weekends or outside working hours

4. Use AI review tools for the first pass
   - GitHub Copilot, CodeRabbit, or similar for an initial review
   - The AI catches common issues (missing tests, unclear naming)
   - The human reviewer focuses on the issues the AI missed

5. Keep the human in the loop for judgment
   - Automation handles the mechanical, humans handle the judgment
   - Never auto-approve a PR — a human must review the logic
   - The goal is to free the human from the mechanical, not replace them

Example

// .github/CODEOWNERS — auto-assign reviewers
# Global reviewers
*                        @alice @bob

# Frontend changes
src/components/          @carol @dave
src/views/               @carol @dave

# Backend changes
src/api/                 @eve @frank
src/services/            @eve @frank

# Database migrations — require senior review
db/migrations/           @alice @bob

# Security-sensitive paths
src/auth/                @alice @bob
src/payments/            @alice @bob

// .github/workflows/auto-label.yml — auto-label PRs
name: Auto Label
on: [pull_request]
jobs:
  label:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/labeler@v5
        with:
          configuration-path: .github/labeler.yml
// .github/labeler.yml
"bug":
  - any: ['src/**']
    body: 'fix|bug|patch'
"feature":
  - any: ['src/**']
    body: 'feat|add|new'
"refactor":
  - any: ['src/**']
    body: 'refactor|clean|rename'

// .github/workflows/stale.yml — close stale PRs
name: Stale
on:
  schedule:
    - cron: "0 0 * * *"  # daily
jobs:
  stale:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/stale@v9
        with:
          stale-pr-message: "This PR has been open for 30 days with no activity. It will be closed in 7 days unless there is new activity."
          days-before-pr-stale: 30
          days-before-pr-close: 7

// AI review: CodeRabbit or GitHub Copilot
// Automatically reviews the PR and leaves comments like:
// "This function is missing a test. Consider adding a test
//  for the case where the input list is empty."
// "This variable name 'd' is unclear. Consider 'daysUntilExpiry'."

// The human reviewer then focuses on:
// "This has a race condition — two requests can update the same
//  record. Consider using optimistic locking."
// (The AI does not catch this; it requires human judgment.)

Common Problems

  • ⚠Automating so much that the human review becomes a rubber stamp — the CI passes, the AI approves, the human clicks approve, and a logic bug ships because nobody actually read the diff.
  • ⚠Auto-merging PRs that pass CI without human review — a PR that adds a dependency with a typo in the package name passes all checks, auto-merges, and breaks the build in production.
  • ⚠Configuring CODEOWNERS but not keeping it updated — the file points to reviewers who left the team, PRs are assigned to inactive accounts, and the review stalls.
  • ⚠Using AI review as a replacement for human review — the AI catches missing tests and naming, but misses the race condition, the security flaw, and the architectural issue, and the team relies on it for approval.

Best Practices

  • ✓Automate the mechanical checks (lint, format, typecheck, coverage, dependency scan) so the human reviewer focuses on judgment (logic, architecture, security) — this is the division of labor that produces the best reviews.
  • ✓Never auto-approve or auto-merge without human review — automation can catch mechanical issues, but logic, security, and design require human judgment that tools cannot provide.
  • ✓Keep CODEOWNERS updated as the team changes — a stale CODEOWNERS file assigns PRs to inactive reviewers, and the review stalls silently.
  • ✓Use AI review as a first pass to catch common issues, then have a human review for judgment — the AI catches 30% (missing tests, naming), the human catches 70% (logic, architecture, security), and both are needed.