How to Review Pull Requests With AI

Playbook

The most dangerous sentence in agentic coding is "done — all tests passing." Agents say it constantly, with total confidence, and a meaningful fraction of the time it's false: the tests weren't run, the wrong tests ran, or the tests pass because the failing assertion got deleted. You could catch this by reading every diff yourself — but on a 400-line change at the end of a long day, you won't, not really. What works is making review a second agent's job, structured so the reviewer can't inherit the author's blind spots. Here's the pull-request review playbook we run, in any harness or none.

Step 1 — The reviewer is a different lab than the author

This is the load-bearing rule. If the same model that wrote the code reviews the code, you get the same blind spots twice: the API it habitually misremembers, the pattern it favors whether it fits or not, and — worst — its loyalty to an approach it already committed to. Cross the lab line. Code authored by a Claude model gets reviewed by a GPT or Gemini model, and the reverse. Different training runs disagree in useful places, and disagreement is exactly what you're paying a reviewer for. We map the pairings in which model fits each agent role; even solo, a second CLI with a second lab's model costs you one terminal tab.

Step 2 — The reviewer re-runs everything, trusting nothing

The reviewer's first act is not reading code — it's independently running the build and the full test suite and comparing reality against the author's claims. "Author says tests pass" is a claim; npm test exiting zero in the reviewer's own shell is a fact. When the two disagree — and they will, more than is comfortable — you've caught the most common agent failure before a human spent a minute. This also catches the subtler cheat: tests that pass because they were weakened. Diff the test files with the same suspicion as the source files. A deleted assertion is a finding, not a cleanup.

Step 3 — Review the diff, not the description

Agents write gorgeous summaries of what they did. The summary is the author's opinion of the change; the diff is the change. A reviewer that reads the summary first anchors on it and skims the code for confirmation. So the reviewer gets the raw git diff as its primary input and the description last, if at all. Every claim in the description that matters should be independently visible in the diff or the test run.

Step 4 — Four structured passes, in a fixed order

"Review this" produces vague vibes. A fixed sequence produces findings.

1. Correctness. Does the change do what was asked? Edge cases, error paths, off-by-ones, broken invariants, concurrency hazards.
2. Security. Secrets or keys in the diff. Injection surfaces — SQL, shell, path traversal, anything interpolating untrusted input. Authorization: does every new endpoint or query check who's asking? The AI coding security checklist is the full list to run here.
3. Scope creep. Everything in the diff the task didn't require: renamed files, drive-by refactors, new dependencies, reformatting noise. Each unrequested change is justified explicitly or flagged for removal. This is the pass that keeps agent PRs reviewable at all.
4. Tests. Does new behavior have new tests? Do they assert behavior, or just that nothing threw? Was any existing test modified, and is each change defensible?

Step 5 — Findings as path:line with severity

"There are some error-handling concerns" is worthless — nobody can act on it or verify it. Every finding gets a location, a severity, and a one-line fix direction: src/api/auth.ts:142 — HIGH — token compared with ==, timing-unsafe; use a constant-time compare. In that format, findings hand straight back to the author agent as a work list.

Step 6 — The human reads the verdict, not 400 lines

The end product is a short structured report: pass/fail on the independent build and tests, findings by severity, the scope-creep list, and a one-paragraph verdict. You read that in ninety seconds and decide — merge, send back, or dig into the one HIGH finding that smells real. Your judgment is the scarce resource; the whole workflow exists to spend it on the verdict instead of the archaeology.

The reviewer prompt, ready to paste

Point a fresh session of a different model at the repo and paste:

You are a skeptical senior code reviewer. You did NOT write this
change and must not trust any claim made about it.

Task context: [one sentence: what the change was supposed to do]

1. VERIFY INDEPENDENTLY. Run the build and full test suite yourself.
   Report the exact commands and their real output. Any failure is
   your first finding.
2. Review the raw diff (git diff main...HEAD), not any summary.
3. Four passes, in order:
   a. CORRECTNESS - logic, edge cases, error paths, invariants
   b. SECURITY - secrets in the diff, injection (SQL/shell/path),
      missing authorization on any new surface
   c. SCOPE CREEP - every change not required by the task,
      including modified or deleted tests
   d. TESTS - new behavior without tests; assertions weakened
4. Report each finding as: file:line - SEVERITY (HIGH/MED/LOW) -
   one-line description - suggested fix direction.
5. End with a verdict: APPROVE / APPROVE WITH NITS / REQUEST
   CHANGES, plus a 3-sentence summary a human can act on without
   reading the diff.

Be specific. "Looks good" is not a review. If you found nothing,
say what you checked and how, so silence is evidence.

Making it a habit

Run this on every autonomous change of consequence and two things happen fast. You catch real bugs — the confident-wrong failures that slip past a tired skim — and your author agents get better, because you feed the findings back and the fixes land before you ever look. In our own team runs the reviewer is a standing role rather than a manual step: The Vibe Father assigns a Reviewer agent from a different lab and re-runs your real checks through the AutoVibe gate. But the workflow above needs nothing but two terminals and the discipline to keep the models separated.

Review pairs naturally with the two habits around it: the AI debugging workflow for the bugs it surfaces, and writing tests with AI that actually catch bugs for the coverage gaps it flags. Add the reviewer role first — every time.