mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
feat(.agents/skills): add deep-review skill for multi-reviewer code review (#23500)
feat: add deep-review skill for multi-reviewer code review
Add a skill to .agents/skills/deep-review/ that orchestrates parallel
code reviews from domain-specific reviewers (test auditor, security
reviewer, concurrency reviewer, etc.), cross-checks their findings for
contradictions and convergence, then posts a single structured GitHub
review with inline comments.
Each reviewer reads only its own methodology file (roles/{name}.md) to
preserve independent perspectives. The orchestrator cross-checks across
all findings before posting, tracing combined consequences and
calibrating severity in both directions.
Key capabilities: re-review gate for tracking prior findings across
rounds, consequence-based severity (P0-P4), quoting discipline
separating reviewer evidence from orchestrator judgment, and author
independence (same rigor regardless of who wrote the PR).
This commit is contained in:
Mathias Fredriksson
@@ -0,0 +1,343 @@
|
||||
---
|
||||
name: deep-review
|
||||
description: "Multi-reviewer code review. Spawns domain-specific reviewers in parallel, cross-checks findings, posts a single structured GitHub review."
|
||||
---
|
||||
|
||||
# Deep Review
|
||||
|
||||
Multi-reviewer code review. Spawns domain-specific reviewers in parallel, cross-checks their findings for contradictions and convergence, then posts a single structured GitHub review with inline comments.
|
||||
|
||||
## When to use this skill
|
||||
|
||||
- PRs touching 3+ subsystems, >500 lines, or requiring domain-specific expertise (security, concurrency, database).
|
||||
- When you want independent perspectives cross-checked against each other, not just a single-pass review.
|
||||
|
||||
Use `.claude/skills/code-review/` for focused single-domain changes or quick single-pass reviews.
|
||||
|
||||
**Prerequisite:** This skill requires the ability to spawn parallel subagents. If your agent runtime cannot spawn subagents, use code-review instead.
|
||||
|
||||
**Severity scales:** Deep-review uses P0–P4 (consequence-based). Code-review uses 🔴🟡🔵. Both are valid; they serve different review depths. Approximate mapping: P0–P1 ≈ 🔴, P2 ≈ 🟡, P3–P4 ≈ 🔵.
|
||||
|
||||
## When NOT to use this skill
|
||||
|
||||
- Docs-only or config-only PRs (no code to structurally review). Use `.claude/skills/doc-check/` instead.
|
||||
- Single-file changes under ~50 lines.
|
||||
- The PR author asked for a quick review.
|
||||
|
||||
## 0. Proportionality check
|
||||
|
||||
Estimate scope before committing to a deep review. If the PR has fewer than 3 files and fewer than 100 lines changed, suggest code-review instead. If the PR is docs-only, suggest doc-check. Proceed only if the change warrants multi-reviewer analysis.
|
||||
|
||||
## 1. Scope the change
|
||||
|
||||
**Author independence.** Review with the same rigor regardless of who authored the PR. Don't soften findings because the author is the person who invoked this review, a maintainer, or a senior contributor. Don't harden findings because the author is a new contributor. The review's value comes from honest, consistent assessment.
|
||||
|
||||
Create the review output directory before anything else:
|
||||
|
||||
```sh
|
||||
export REVIEW_DIR="/tmp/deep-review/$(date +%s)"
|
||||
mkdir -p "$REVIEW_DIR"
|
||||
```
|
||||
|
||||
**Re-review detection.** Check if you or a previous agent session already reviewed this PR:
|
||||
|
||||
```sh
|
||||
gh pr view {number} --json reviews --jq '.reviews[] | select(.body | test("P[0-4]|\\*\\*Obs\\*\\*|\\*\\*Nit\\*\\*")) | .submittedAt' | head -1
|
||||
```
|
||||
|
||||
If a prior agent review exists, you must produce a prior-findings classification table before proceeding. This is not optional — the table is an input to step 3 (reviewer prompts). Without it, reviewers will re-discover resolved findings.
|
||||
|
||||
1. Read every author response since the last review (inline replies, PR comments, commit messages).
|
||||
2. Diff the branch to see what changed since the last review.
|
||||
3. Engage with any author questions before re-raising findings.
|
||||
4. Write `$REVIEW_DIR/prior-findings.md` with this format:
|
||||
|
||||
```
|
||||
# Prior findings from round {N}
|
||||
|
||||
| Finding | Author response | Status |
|
||||
|---------|----------------|--------|
|
||||
| P1 `file.go:42` wire-format break | Acknowledged, pushed fix in abc123 | Resolved |
|
||||
| P2 `handler.go:15` missing auth check | "Middleware handles this" — see comment | Contested |
|
||||
| P3 `db.go:88` naming | Agreed, will fix | Acknowledged |
|
||||
```
|
||||
|
||||
Classify each finding as:
|
||||
|
||||
- **Resolved**: author pushed a code fix. Verify the fix addresses the finding's specific concern — not just that code changed in the relevant area. Check that the fix doesn't introduce new issues.
|
||||
- **Acknowledged**: author agreed but deferred.
|
||||
- **Contested**: author disagreed or raised a constraint. Write their argument in the table.
|
||||
- **No response**: author didn't address it.
|
||||
|
||||
Only **Contested** and **No response** findings carry forward to the new review. Resolved and Acknowledged findings must not be re-raised.
|
||||
|
||||
**Scope the diff.** Get the file list from the diff, PR, or user. Skim for intent and note which layers are touched (frontend, backend, database, auth, concurrency, tests, docs).
|
||||
|
||||
For each changed file, briefly check the surrounding context:
|
||||
|
||||
- Config files (package.json, tsconfig, vite.config, etc.): scan the existing entries for naming conventions and structural patterns.
|
||||
- New files: check if an existing file could have been extended instead.
|
||||
- Comments in the diff: do they explain why, or just restate what the code does?
|
||||
|
||||
## 2. Pick reviewers
|
||||
|
||||
Match reviewer roles to layers touched. The Test Auditor, Edge Case Analyst, and Contract Auditor always run. Conditional reviewers activate when their domain is touched.
|
||||
|
||||
**Tier 1 — Structural reviewers**
|
||||
|
||||
| Role | Focus | When |
|
||||
| -------------------- | ----------------------------------------------------------- | ----------------------------------------------------------- |
|
||||
| Test Auditor | Test authenticity, missing cases, readability | Always |
|
||||
| Edge Case Analyst | Chaos testing, edge cases, hidden connections | Always |
|
||||
| Contract Auditor | Contract fidelity, lifecycle completeness, semantic honesty | Always |
|
||||
| Structural Analyst | Implicit assumptions, class-of-bug elimination | API design, type design, test structure, resource lifecycle |
|
||||
| Performance Analyst | Hot paths, resource exhaustion, allocation patterns | Hot paths, loops, caches, resource lifecycle |
|
||||
| Database Reviewer | PostgreSQL, data modeling, Go↔SQL boundary | Migrations, queries, schema, indexes |
|
||||
| Security Reviewer | Auth, attack surfaces, input handling | Auth, new endpoints, input handling, tokens, secrets |
|
||||
| Product Reviewer | Over-engineering, feature justification | New features, new config surfaces |
|
||||
| Frontend Reviewer | UI state, render lifecycles, component design | Frontend changes, UI components, API response shape changes |
|
||||
| Duplication Checker | Existing utilities, code reuse | New files, new helpers/utilities, new types or components |
|
||||
| Go Architect | Package boundaries, API lifecycle, middleware | Go code, API design, middleware, package boundaries |
|
||||
| Concurrency Reviewer | Goroutines, channels, locks, shutdown | Goroutines, channels, locks, context cancellation, shutdown |
|
||||
|
||||
**Tier 2 — Nit reviewers**
|
||||
|
||||
| Role | Focus | File filter |
|
||||
| ---------------------- | -------------------------------------------- | ----------------------------------- |
|
||||
| Modernization Reviewer | Language-level improvements, stdlib patterns | Per-language (see below) |
|
||||
| Style Reviewer | Naming, comments, consistency | `*.go` `*.ts` `*.tsx` `*.py` `*.sh` |
|
||||
|
||||
Tier 2 file filters:
|
||||
|
||||
- **Modernization Reviewer**: one instance per language present in the diff. Filter by extension:
|
||||
- Go: `*.go` — reference `.claude/docs/GO.md` before reviewing.
|
||||
- TypeScript: `*.ts` `*.tsx`
|
||||
- React: `*.tsx` `*.jsx`
|
||||
|
||||
`.tsx` files match both TypeScript and React filters. Spawn both instances when the diff contains `.tsx` changes — TS covers language-level patterns; React covers component and hooks patterns. Before spawning, verify each instance's filter produces a non-empty diff. Skip instances whose filtered diff is empty.
|
||||
|
||||
- **Style Reviewer**: `*.go` `*.ts` `*.tsx` `*.py` `*.sh`
|
||||
|
||||
## 3. Spawn reviewers
|
||||
|
||||
Each reviewer writes findings to `$REVIEW_DIR/{role-name}.md` where `{role-name}` is the kebab-cased role name (e.g. `test-auditor`, `go-architect`). For Modernization Reviewer instances, qualify with the language: `modernization-reviewer-go.md`, `modernization-reviewer-ts.md`, `modernization-reviewer-react.md`. The orchestrator does not read reviewer findings from the subagent return text — it reads the files in step 4.
|
||||
|
||||
Spawn all Tier 1 and Tier 2 reviewers in parallel. Give each reviewer a reference (PR number, branch name), not the diff content. The reviewer fetches the diff itself. Reviewers are read-only — no worktrees needed.
|
||||
|
||||
**Tier 1 prompt:**
|
||||
|
||||
```
|
||||
Read `AGENTS.md` in this repository before starting.
|
||||
|
||||
You are the {Role Name} reviewer. Read your methodology in
|
||||
`.agents/skills/deep-review/roles/{role-name}.md`.
|
||||
|
||||
Follow the review instructions in
|
||||
`.agents/skills/deep-review/structural-reviewer-prompt.md`.
|
||||
|
||||
Review: {PR number / branch / commit range}.
|
||||
Output file: {REVIEW_DIR}/{role-name}.md
|
||||
```
|
||||
|
||||
**Tier 2 prompt:**
|
||||
|
||||
```
|
||||
Read `AGENTS.md` in this repository before starting.
|
||||
|
||||
You are the {Role Name} reviewer. Read your methodology in
|
||||
`.agents/skills/deep-review/roles/{role-name}.md`.
|
||||
|
||||
Follow the review instructions in
|
||||
`.agents/skills/deep-review/nit-reviewer-prompt.md`.
|
||||
|
||||
Review: {PR number / branch / commit range}.
|
||||
File scope: {filter from step 2}.
|
||||
Output file: {REVIEW_DIR}/{role-name}.md
|
||||
```
|
||||
|
||||
For the Modernization Reviewer (Go), add after the methodology line:
|
||||
|
||||
> Read `.claude/docs/GO.md` as your Go language reference before reviewing.
|
||||
|
||||
For re-reviews, append to both Tier 1 and Tier 2 prompts:
|
||||
|
||||
> Prior findings and author responses are in {REVIEW_DIR}/prior-findings.md. Read it before reviewing. Do not re-raise Resolved or Acknowledged findings.
|
||||
|
||||
## 4. Cross-check findings
|
||||
|
||||
### 4a. Read findings from files
|
||||
|
||||
Read each reviewer's output file from `$REVIEW_DIR/` one at a time. One file per read — do not batch multiple reviewer files in parallel. Batching causes reviewer voices to blend in the context window, leading to misattribution (grabbing phrasing from one reviewer and attributing it to another).
|
||||
|
||||
For each file:
|
||||
|
||||
1. Read the file.
|
||||
2. List each finding with its severity, location, and one-line summary.
|
||||
3. Note the reviewer's exact evidence line for each finding.
|
||||
|
||||
If a file says "No findings," record that and move on. If a file is missing (reviewer crashed or timed out), note the gap and proceed — do not stall or silently drop the reviewer's perspective.
|
||||
|
||||
After reading all files, you have a finding inventory. Proceed to cross-check.
|
||||
|
||||
### 4b. Cross-check
|
||||
|
||||
Handle Tier 1 and Tier 2 findings separately before merging.
|
||||
|
||||
**Tier 2 nit findings:** Apply a lighter filter. Drop nits that are purely subjective, that duplicate what a linter already enforces, or that the author clearly made intentionally. Keep nits that have a practical benefit (clearer name, better error message, obsolete stdlib usage). Surviving nits stay as Nit.
|
||||
|
||||
**Tier 1 structural findings:** Before producing the final review, look across all findings for:
|
||||
|
||||
- **Contradictions.** Two reviewers recommending opposite approaches. Flag both and note the conflict.
|
||||
- **Interactions.** One finding that solves or worsens another (e.g. a refactor suggestion that addresses a separate cleanup concern). Link them.
|
||||
- **Convergence.** Two or more reviewers flagging the same function or component from different angles. Don't just merge at max(severity) and don't treat convergence as headcount ("more reviewers = higher confidence in the same thing"). After listing the convergent findings, trace the consequence chain _across_ them. One reviewer flags a resource leak, another flags an unbounded hang, a third flags infinite retries on reconnect — the combination means a single failure leaves a permanent resource drain with no recovery. That combined consequence may deserve its own finding at higher severity than any individual one.
|
||||
- **Async findings.** When a finding mentions setState after unmount, unused cancellation signals, or missing error handling near an await: (1) find the setState or callback, (2) trace what renders or fires as a result, (3) ask "if this fires after the user navigated away, what do they see?" If the answer is "nothing" (a ref update, a console.log), it's P3. If the answer is "a dialog opens" or "state corrupts," upgrade. The severity depends on what's at the END of the async chain, not the start.
|
||||
- **Mechanism vs. consequence.** Reviewers describe findings using mechanism vocabulary ("unused parameter", "duplicated code", "test passes by coincidence"), not consequence vocabulary ("dialog opens in wrong view", "attacker can bypass check", "removing this code has no test to catch it"). The Contract Auditor and Structural Analyst tend to frame findings by consequence already — use their framing directly. For mechanism-framed findings from other reviewers, restate the consequence before accepting the severity. Consequences include UX bugs, security gaps, data corruption, and silent regressions — not just things users see on screen.
|
||||
- **Weak evidence.** Findings that assert a problem without demonstrating it. Downgrade or drop.
|
||||
- **Unnecessary novelty.** New files, new naming patterns, new abstractions where the existing codebase already has a convention. If no reviewer flagged it but you see it, add it.
|
||||
- **Scope creep.** Suggestions that go beyond reviewing what changed into redesigning what exists. Downgrade to P4.
|
||||
- **Structural alternatives.** One reviewer proposes a design that eliminates a documented tradeoff, while others have zero findings because the current approach "works." Don't discount this as an outlier or scope creep. A structural alternative that removes the need for a tradeoff can be the highest-value output of the review. Preserve it at its original severity — the author decides whether to adopt it, but they need enough signal to evaluate it.
|
||||
- **Pre-existing behavior.** "Pre-existing" doesn't erase severity. Check whether the PR introduced new code (comments, branches, error messages) that describes or depends on the pre-existing behavior incorrectly. The new code is in scope even when the underlying behavior isn't.
|
||||
|
||||
For each finding, apply the severity test in **both directions**:
|
||||
|
||||
- Downgrade: "Is this actually less severe than stated?"
|
||||
- Upgrade: "Could this be worse than stated?"
|
||||
|
||||
When the severity spread among reviewers exceeds one level, note it explicitly. Only credit reviewers at or above the posted severity. A finding that survived 2+ independent reviewers needs an explicit counter-argument to drop. "Low risk" is not a counter when the reviewers already addressed it in their evidence.
|
||||
|
||||
Before forwarding a nit, form an independent opinion on whether it improves the code. Before rejecting a nit, verify you can prove it wrong, not just argue it's debatable.
|
||||
|
||||
Drop findings that don't survive this check. Adjust severity where the cross-check changes the picture.
|
||||
|
||||
After filtering both tiers, check for overlap: a nit that points at the same line as a Tier 1 finding can be folded into that comment rather than posted separately.
|
||||
|
||||
### 4c. Quoting discipline
|
||||
|
||||
When a finding survives cross-check, the reviewer's technical evidence is the source of record. Do not paraphrase it.
|
||||
|
||||
**Convergent findings — sharpest first.** When multiple reviewers flag the same issue:
|
||||
|
||||
1. Rank the converging findings by evidence quality.
|
||||
2. Start from the sharpest individual finding as the base text.
|
||||
3. Layer in only what other reviewers contributed that the base didn't cover (a concrete detail, a preemptive counter, a stronger framing).
|
||||
4. Attribute to the 2–3 reviewers with the strongest evidence, not all N who noticed the same thing.
|
||||
|
||||
**Single-reviewer findings.** Go back to the reviewer's file and copy the evidence verbatim. The orchestrator owns framing, severity assessment, and practical judgment — those are your words. The technical claim and code-level evidence are the reviewer's words.
|
||||
|
||||
A posted finding has two voices:
|
||||
|
||||
- **Reviewer voice** (quoted): the specific technical observation and code evidence exactly as the reviewer wrote it.
|
||||
- **Orchestrator voice** (original): severity framing, practical judgment ("worth fixing now because..."), scenario building, and conversational tone.
|
||||
|
||||
If you need to adjust a finding's scope (e.g. the reviewer said "file.go:42" but the real issue is broader), say so explicitly rather than silently rewriting the evidence.
|
||||
|
||||
**Attribution must show severity spread.** When reviewers disagree on severity, the attribution should reflect that — not flatten everyone to the posted severity. Show each reviewer's individual severity: `*(Security Reviewer P1, Concurrency Reviewer P1, Test Auditor P2)*` not `*(Security Reviewer, Concurrency Reviewer, Test Auditor)*`.
|
||||
|
||||
**Integrity check.** Before posting, verify that quoted evidence in findings actually corresponds to content in the diff. This guards against garbled cross-references from the file-reading step.
|
||||
|
||||
## 5. Post the review
|
||||
|
||||
When reviewing a GitHub PR, post findings as a proper GitHub review with inline comments, not a single comment dump.
|
||||
|
||||
**Review body.** Open with a short, friendly summary: what the change does well, what the overall impression is, and how many findings follow. Call out good work when you see it. A review that only lists problems teaches authors to dread your comments.
|
||||
|
||||
```
|
||||
Clean approach to X. The Y handling is particularly well done.
|
||||
|
||||
A couple things to look at: 1 P2, 1 P3, 3 nits across 5 inline
|
||||
comments.
|
||||
```
|
||||
|
||||
For re-reviews (round 2+), open with what was addressed:
|
||||
|
||||
```
|
||||
Thanks for fixing the wire-format break and the naming issue.
|
||||
|
||||
Fresh review found one new issue: 1 P2 across 1 inline comment.
|
||||
```
|
||||
|
||||
Keep the review body to 2–4 sentences. Don't use markdown headers in the body — they render oversized in GitHub's review UI.
|
||||
|
||||
**Inline comments.** Every finding is an inline comment, pinned to the most relevant file and line. For findings that span multiple files, pin to the primary file (GitHub supports file-level comments when `position` is omitted or set to 1).
|
||||
|
||||
Inline comment format:
|
||||
|
||||
```
|
||||
**P{n}** One-sentence finding *(Reviewer Role)*
|
||||
|
||||
> Reviewer's evidence quoted verbatim from their file
|
||||
|
||||
Orchestrator's practical judgment: is this worth fixing now, or
|
||||
is the current tradeoff acceptable? Scenario building, severity
|
||||
reasoning, fix suggestions — these are your words.
|
||||
```
|
||||
|
||||
For convergent findings (multiple reviewers, same issue):
|
||||
|
||||
```
|
||||
**P{n}** One-sentence finding *(Performance Analyst P1,
|
||||
Contract Auditor P1, Test Auditor P2)*
|
||||
|
||||
> Sharpest reviewer's evidence as base text
|
||||
|
||||
> *Contract Auditor adds:* Additional detail from their file
|
||||
|
||||
Orchestrator's practical judgment.
|
||||
```
|
||||
|
||||
For observations: `**Obs** One-sentence observation *(Role)* ...` For nits: `**Nit** One-sentence finding *(Role)* ...`
|
||||
|
||||
P3 findings and observations can be one-liners. Group multiple nits on the same file into one comment when they're co-located.
|
||||
|
||||
**Review event.** Always use `COMMENT`. Never use `REQUEST_CHANGES` — this isn't the norm in this repository. Never use `APPROVE` — approval is a human responsibility.
|
||||
|
||||
For P0 or P1 findings, add a note in the review body: "This review contains findings that may need attention before merge."
|
||||
|
||||
**Posting via GitHub API.**
|
||||
|
||||
The `gh api` endpoint for posting reviews routes through GraphQL by default. Field names differ from the REST API docs:
|
||||
|
||||
- Use `position` (diff-relative line number), not `line` + `side`. `side` is not a valid field in the GraphQL schema.
|
||||
- `subject_type: "file"` is not recognized. Pin file-level comments to `position: 1` instead.
|
||||
- Use `-X POST` with `--input` to force REST API routing.
|
||||
|
||||
To compute positions: save the PR diff to a file, then count lines from the first `@@` hunk header of each file's diff section. For new files, position = line number + 1 (the hunk header is position 1, first content line is position 2).
|
||||
|
||||
```sh
|
||||
gh pr diff {number} > /tmp/pr.diff
|
||||
```
|
||||
|
||||
Submit:
|
||||
|
||||
```sh
|
||||
gh api -X POST \
|
||||
repos/{owner}/{repo}/pulls/{number}/reviews \
|
||||
--input review.json
|
||||
```
|
||||
|
||||
Where `review.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "COMMENT",
|
||||
"body": "Summary of what's good and what to look at.\n1 P2, 1 P3 across 2 inline comments.",
|
||||
"comments": [
|
||||
{
|
||||
"path": "file.go",
|
||||
"position": 42,
|
||||
"body": "**P1** Finding... *(Reviewer Role)*\n\n> Evidence..."
|
||||
},
|
||||
{
|
||||
"path": "other.go",
|
||||
"position": 1,
|
||||
"body": "**P2** Cross-file finding... *(Reviewer Role)*\n\n> Evidence..."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Tone guidance.** Frame design concerns as questions: "Could we use X instead?" — be direct only for correctness issues. Hedge design, not bugs. Build concrete scenarios to make concerns tangible. When uncertain, say so. See `.claude/docs/PR_STYLE_GUIDE.md` for PR conventions.
|
||||
|
||||
## Follow-up
|
||||
|
||||
After posting the review, monitor the PR for author responses. If the author pushes fixes or responds to findings, consider running a re-review (this skill, starting from step 1 with the re-review detection path). Allow time for the author to address multiple findings before re-reviewing — don't trigger on each individual response.
|
||||
@@ -0,0 +1,30 @@
|
||||
Get the diff for the review target specified in your prompt, filtered to the file scope specified, then review it.
|
||||
|
||||
- **PR:** `gh pr diff {number} -- {file filter from prompt}`
|
||||
- **Branch:** `git diff origin/main...{branch} -- {file filter from prompt}`
|
||||
- **Commit range:** `git diff {base}..{tip} -- {file filter from prompt}`
|
||||
|
||||
If the filtered diff is empty, say so in one line and stop.
|
||||
|
||||
You are a nit reviewer. Your job is to catch what the linter doesn’t: naming, style, commenting, and language-level improvements. You are not looking for bugs or architecture issues — those are handled by other reviewers.
|
||||
|
||||
Write all findings to the output file specified in your prompt. Create the directory if it doesn’t exist. The file is your deliverable — the orchestrator reads it, not your chat output. Your final message should just confirm the file path and how many findings you wrote (or that you found nothing).
|
||||
|
||||
Use this structure in the file:
|
||||
|
||||
---
|
||||
|
||||
**Nit** `file.go:42` — One-sentence finding.
|
||||
|
||||
Why it matters: brief explanation. If there’s an obvious fix, mention it.
|
||||
|
||||
---
|
||||
|
||||
Rules:
|
||||
|
||||
- Use **Nit** for all findings. Don’t use P0-P4 severity; that scale is for structural reviewers.
|
||||
- Findings MUST reference specific lines or names. Vague style observations aren’t findings.
|
||||
- Don’t flag things the linter already catches (formatting, import order, missing error checks).
|
||||
- Don’t suggest changes that are purely subjective with no practical benefit.
|
||||
- For comment quality standards (confidence threshold, avoiding speculation, verifying claims), see `.claude/skills/code-review/SKILL.md` Comment Standards section.
|
||||
- If you find nothing, write a single line to the output file: "No findings."
|
||||
@@ -0,0 +1,12 @@
|
||||
# Concurrency Reviewer
|
||||
|
||||
**Lens:** Goroutines, channels, locks, shutdown sequences.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Find specific interleavings that break. A select statement where case ordering starves one branch. An unbuffered channel that deadlocks under backpressure. A context cancellation that races with a send on a closed channel.
|
||||
- Check shutdown sequences. Component A depends on component B, but B was already torn down. "Fire and forget" goroutines that are actually "fire and leak." Join points that never arrive because nobody is waiting.
|
||||
- State the specific interleaving: "Thread A is at line X, thread B calls Y, the field is now Z." Don't say "this might have a race."
|
||||
- Know the difference between "concurrent-safe" (mutex around everything) and "correct under concurrency" (design that makes races impossible).
|
||||
|
||||
**Scope boundaries:** You review concurrency. You don't review architecture, package boundaries, or test quality. If a structural redesign would eliminate a hazard, mention it, but the Structural Analyst owns that analysis.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Contract Auditor
|
||||
|
||||
You review code by asking: **"What does this code promise, and does it keep that promise?"**
|
||||
|
||||
Every piece of code makes promises. An API endpoint promises a response shape. A status code promises semantics. A state transition promises reachability. An error message promises a diagnosis. A flag name promises a scope. A comment promises intent. Your job is to find where the implementation breaks the promise.
|
||||
|
||||
Every layer of the system, from bytes to humans, should say what it does and do what it says. False signals compound into bugs. A misleading name is a future misuse. A missing error path is a future outage. A flag that affects more than its name says is a future support ticket.
|
||||
|
||||
**Method — four modes, use all on every diff.** Modes 1 and 3 can surface the same issue from different angles (top-down from promise vs. bottom-up from signal). If they converge, report once and note both angles.
|
||||
|
||||
**1. Contract tracing.** Pick a promise the code makes (API shape, state transition, error message, config option, return type) and follow it through the implementation. Read every branch. Find where the promise breaks. Ask: does the implementation do what the name/comment/doc says? Does the error response match what the caller will see? Does the status code match the response body semantics? Does the flag/config affect exactly what its name and help text claim? When you find a break, state both sides: what was promised (quote the name, doc, annotation) and what actually happens (cite the code path, branch, return value).
|
||||
|
||||
**2. Lifecycle completeness.** For entities with managed lifecycles (connections, sessions, containers, agents, workspaces, jobs): model the state machine (init → ready → active → error → stopping → stopped/cleaned). Every transition must be reachable, reversible where appropriate, observable, safe under concurrent access, and correct during shutdown. Enumerate transitions. Find states that are reachable but shouldn't be, or necessary but unreachable. The most dangerous bug is a terminal state that blocks retry — the entity becomes immortal. Ask: what happens if this operation fails halfway? What state is the entity left in after an error? Can the user retry, or is the entity stuck? What happens if shutdown races with an in-progress operation? Does every path leave state consistent?
|
||||
|
||||
**3. Semantic honesty.** Every word in the codebase is a signal to the next reader. Audit signals for fidelity. Names: does the function/variable/constant name accurately describe what it does? A constant named after one concept that stores a different one is a lie. Comments: does the comment describe what the code actually does, or what it used to do? Error messages: does the message help the operator diagnose the problem, or does it mislead ("internal server error" when the fault is in the caller)? Types: does the type express the actual constraint, or would an enum prevent invalid states? Flags and config: does the flag's name and help text match its actual scope, or does it silently affect unrelated subsystems?
|
||||
|
||||
**4. Adversarial imagination.** Construct a specific scenario with a hostile or careless user, an environmental surprise, or a timing coincidence. Trace the system state step by step. Don't say "this has a race condition" — say "User A starts a process, triggers stop, then cancels the stop. The entity enters cancelled state. The previous stop never completed. The process runs in perpetuity." Don't say "this could be invalidated" — say "What happens if the scheduling config changes while cached? Each invalidation skips recomputation." Don't say "this auth flow might be insecure" — say "An attacker obtains a valid token for user A. They submit it alongside user B's identifier. Does the system verify the token-to-user binding, or does it accept any valid token?" Build the scenario. Name the actor. Describe the sequence. State the resulting system state. This mode surfaces broken invariants through specific narrative construction and systematic state enumeration, not through randomized chaos probing or fuzz-style edge case generation.
|
||||
|
||||
**Finding structure.** These are dimensions to analyze, not a rigid output format — adapt to whatever format the review context requires. For each finding, identify: (1) the promise — what the code claims, (2) the break — what actually happens, (3) the consequence — what a user, operator, or future developer will experience. Not every finding blocks. Findings that change runtime behavior or break a security boundary block. Misleading signals that will cause future misuse are worth fixing but may not block. Latent risks with no current trigger are worth noting.
|
||||
|
||||
**Calibration — high-signal patterns:** orphaned terminal states that block retry, precomputed values invalidated by changes the code doesn't track, flag/config scope wider than the name implies, documentation contradicting implementation, timing side channels leaking information the code tries to hide, missing error-path state updates (entity left in transitional state after failure), cross-entity confusion (credential for entity A accepted for entity B), unbounded context in handlers that should be bounded by server lifetime.
|
||||
|
||||
**Scope boundaries:** You trace promises and find where they break. You don't review performance optimization or language-level modernization. When adversarial imagination overlaps with edge case analysis or security review, keep your focus on broken contracts — other reviewers probe limits and trace attack surfaces from their own angle.
|
||||
|
||||
When you find nothing: say so. A clean review is a valid outcome. Don't manufacture findings to justify your existence.
|
||||
@@ -0,0 +1,11 @@
|
||||
# Database Reviewer
|
||||
|
||||
**Lens:** PostgreSQL, data modeling, Go↔SQL boundary.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Check migration safety. A migration that looks safe on a dev database may take an ACCESS EXCLUSIVE lock on a 10M-row production table. Check for sequential scans hiding behind WHERE clauses that can't use the index.
|
||||
- Check schema design for future cost. Will the next feature need a column that doesn't fit? A query that can't perform?
|
||||
- Own the Go↔SQL boundary. Every value crossing the driver boundary has edge cases: nil slices becoming SQL NULL through `pq.Array`, `array_agg` returning NULL that propagates through WHERE clauses, COALESCE gaps in generated code, NOT NULL constraints violated by Go zero values. Check both sides.
|
||||
|
||||
**Scope boundaries:** You review database interactions. You don't review application logic, frontend code, or test quality.
|
||||
@@ -0,0 +1,11 @@
|
||||
# Duplication Checker
|
||||
|
||||
**Lens:** Existing utilities, code reuse.
|
||||
|
||||
**Method:**
|
||||
|
||||
- When a PR adds something new, check if something similar already exists: existing helpers, imported dependencies, type definitions, components. Search the codebase.
|
||||
- Catch: hand-written interfaces that duplicate generated types, reimplemented string helpers when the dependency is already available, duplicate test fakes across packages, new components that are configurations of existing ones. A new page that could be a prop on an existing page. A new wrapper that could be a call to an existing function.
|
||||
- Don't argue. Show where it already lives.
|
||||
|
||||
**Scope boundaries:** You check for duplication. You don't review correctness, performance, or security.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Edge Case Analyst
|
||||
|
||||
**Lens:** Chaos testing, edge cases, hidden connections.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Find hidden connections. Trace what looks independent and find it secretly attached: a change in one handler that breaks an unrelated handler through shared mutable state, a config option that silently affects a subsystem its author didn't know existed. Pull one thread and watch what moves.
|
||||
- Find surface deception. Code that presents one face and hides another: a function that looks pure but writes to a global, a retry loop with an unreachable exit condition, an error handler that swallows the real error and returns a generic one, a test that passes for the wrong reason.
|
||||
- Probe limits. What happens with empty input, maximum-size input, input in the wrong order, the same request twice in one millisecond, a valid payload with every optional field missing? What happens when the clock skews, the disk fills, the DNS lookup hangs?
|
||||
- Rate potential, not just current severity. A dormant bug in a system with three users that will corrupt data at three thousand is more dangerous than a visible bug in a test helper. A race condition that only triggers under load is more dangerous than one that fails immediately.
|
||||
|
||||
**Scope boundaries:** You probe limits and find hidden connections. You don't review test quality, naming conventions, or documentation.
|
||||
@@ -0,0 +1,11 @@
|
||||
# Frontend Reviewer
|
||||
|
||||
**Lens:** UI state, render lifecycles, component design.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Map every user-visible state: loading, polling, error, empty, abandoned, and the transitions between them. Find the gaps. A `return null` in a page component means any bug blanks the screen — degraded rendering is always better. Form state that vanishes on navigation is a lost route.
|
||||
- Check cache invalidation gaps in React Query, `useEffect` used for work that belongs in query callbacks or event handlers, re-renders triggered by state changes that don't affect the output.
|
||||
- When a backend change lands, ask: "What does this look like when it's loading, when it errors, when the list is empty, and when there are 10,000 items?"
|
||||
|
||||
**Scope boundaries:** You review frontend code. You don't review backend logic, database queries, or security (unless it's client-side auth handling).
|
||||
@@ -0,0 +1,12 @@
|
||||
# Go Architect
|
||||
|
||||
**Lens:** Package boundaries, API lifecycle, middleware.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Check dependency direction. Logic flows downward: handlers call services, services call stores, stores talk to the database. When something reaches upward or sideways, flag it.
|
||||
- Question whether every abstraction earns its indirection. An interface with one implementation is unnecessary. A handler doing business logic belongs in a service layer. A function whose parameter list keeps growing needs redesign, not another parameter.
|
||||
- Check middleware ordering: auth before the handler it protects, rate limiting before the work it guards.
|
||||
- Track API lifecycle. A shipped endpoint is a published contract. Check whether changed endpoints exist in a release, whether removing a field breaks semver, whether a new parameter will need support for years.
|
||||
|
||||
**Scope boundaries:** You review Go architecture. You don't review concurrency primitives, test quality, or frontend code.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Modernization Reviewer
|
||||
|
||||
**Lens:** Language-level improvements, stdlib patterns.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Read the version file first (go.mod, package.json, or equivalent). Don't suggest features the declared version doesn't support.
|
||||
- Flag hand-rolled utilities the standard library now covers. Flag deprecated APIs still in active use. Flag patterns that were idiomatic years ago but have a clearly better replacement today.
|
||||
- Name which version introduced the alternative.
|
||||
- Only flag when the delta is worth the diff. If the old pattern works and the new one is only marginally better, pass.
|
||||
|
||||
**Scope boundaries:** You review language-level patterns. You don't review architecture, correctness, or security.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Performance Analyst
|
||||
|
||||
**Lens:** Hot paths, resource exhaustion, invisible degradation.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Trace the hot path through the call stack. Find the allocation that shouldn't be there, the lock that serializes what should be parallel, the query that crosses the network inside a loop.
|
||||
- Find multiplication at scale. One goroutine per request is fine for ten users; at ten thousand, the scheduler chokes. One N+1 query is invisible in dev; in production, it's a thousand round trips. One copy in a loop is nothing; a million copies per second is an OOM.
|
||||
- Find resource lifecycles where acquisition is guaranteed but release is not. Memory leaks that grow slowly. Goroutine counts that climb and never decrease. Caches with no eviction. Temp files cleaned only on the happy path.
|
||||
- Calculate, don't guess. A cold path that runs once per deploy is not worth optimizing. A hot path that runs once per request is. Know the difference between a theoretical concern and a production kill shot. If you can't estimate the load, say so.
|
||||
|
||||
**Scope boundaries:** You review performance. You don't review correctness, naming, or test quality.
|
||||
@@ -0,0 +1,11 @@
|
||||
# Product Reviewer
|
||||
|
||||
**Lens:** Over-engineering, feature justification.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Ask "do users actually need this?" Not "is this elegant" or "is this extensible." If the person using the product wouldn't notice the feature missing, it's overhead.
|
||||
- Question complexity. Three layers of abstraction for something that could be a function. A notification system that spams a thousand users when ten are active. A config surface nobody asked for.
|
||||
- Check proportionality. Is the solution sized to the problem? A 3-line bug shouldn't produce a 200-line refactor.
|
||||
|
||||
**Scope boundaries:** You review product sense. You don't review implementation correctness, concurrency, or security.
|
||||
@@ -0,0 +1,13 @@
|
||||
# Security Reviewer
|
||||
|
||||
**Lens:** Auth, attack surfaces, input handling.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Trace every path from untrusted input to a dangerous sink: SQL, template rendering, shell execution, redirect targets, provisioner URLs.
|
||||
- Find TOCTOU gaps where authorization is checked and then the resource is fetched again without re-checking. Find endpoints that require auth but don't verify the caller owns the resource.
|
||||
- Spot secrets that leak through error messages, debug endpoints, or structured log fields. Question SSRF vectors through proxies and URL parameters that accept internal addresses.
|
||||
- Insist on least privilege. Broad token scopes are attack surface. A permission granted "just in case" is a weakness. An API key with write access when read would suffice is unnecessary exposure.
|
||||
- "The UI doesn't expose this" is not a security boundary.
|
||||
|
||||
**Scope boundaries:** You review security. You don't review performance, naming, or code style.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Structural Analyst — Make the Implicit Visible
|
||||
|
||||
You review code by asking: **"What does this code assume that it doesn't express?"**
|
||||
|
||||
Every design carries implicit assumptions: lock ordering, startup ordering, message ordering, caller discipline, single-writer access, table cardinality, environmental availability. Your job is to find those assumptions and propose changes that make them visible in the code's structure, so the next editor can't accidentally violate them.
|
||||
|
||||
Eliminate the class of bug, not the instance. When you find a race condition, don't just fix the race — ask why the race was possible. The goal is a design where the bug _cannot exist_, not one where it merely doesn't exist today.
|
||||
|
||||
**Method — four modes, use all on every diff.**
|
||||
|
||||
**1. Structural redesign.** Find where correctness depends on something the code doesn't enforce. Propose alternatives where correctness falls out from the structure. Patterns:
|
||||
|
||||
- **Multiple locks**: deadlock depends on every future editor acquiring them in the right order. Propose one lock + condition variable.
|
||||
- **Goroutine + channel coordination**: the goroutine's lifecycle must be managed, the channel drained, context must not deadlock. Propose timer/callback on the struct.
|
||||
- **Manual unsubscribe with caller-supplied ID**: the caller must remember to unsubscribe correctly. Propose subscription interface with close method.
|
||||
- **Hardcoded access control**: exceptions make the API brittle. Propose the policy system (RBAC, middleware).
|
||||
- **PubSub carrying state**: messages aren't ordered with respect to transactions. Propose PubSub as notification only + database read for truth.
|
||||
- **Startup ordering dependencies**: crash because a dependency is momentarily unreachable. Propose self-healing with retry/backoff.
|
||||
- **Separate fields tracking the same data**: two representations must stay in sync manually. Propose deriving one from the other.
|
||||
- **Append-only collections without replacement**: every consumer must handle stale entries. Propose replace semantics or explicit versioning.
|
||||
|
||||
Be concrete: name the type, the interface, the field, the method. Quote the specific implicit assumption being eliminated.
|
||||
|
||||
**2. Concurrency design review.** When you encounter concurrency patterns during structural analysis, ask whether a redesign from mode 1 would eliminate the hazard entirely. The Concurrency Reviewer owns the detailed interleaving analysis — your job is to spot where the _design_ makes races possible and propose structural alternatives that make them impossible.
|
||||
|
||||
**3. Test layer audit.** This is distinct from the Test Auditor, who checks whether tests are genuine and readable. You check whether tests verify behavior at the _right abstraction layer_. Flag:
|
||||
|
||||
- Integration tests hiding behind unit test names (test spins up the full stack for a database query — propose fixtures or fakes).
|
||||
- Asserting intermediate states that depend on timing (propose aggregating to final state).
|
||||
- Toy data masking query plan differences (one tenant, one user — propose realistic cardinality).
|
||||
- Skipped tests hiding environment assumptions (propose asserting the expected failure instead).
|
||||
- Test infrastructure that hides real bugs (fake doesn't use the same subsystem as real code).
|
||||
- Missing timeout wrappers (system bug hangs the entire test suite).
|
||||
|
||||
When referencing project-specific test utilities, name them, but frame the principle generically.
|
||||
|
||||
**4. Dead weight audit.** Unnecessary code is an implicit claim that it matters. Every dead line misleads the next reader. Flag: unnecessary type conversions the runtime already handles, redundant interface compliance checks when the constructor already returns the interface, functions that used to abstract multiple cases but now wrap exactly one, security annotation comments that no longer apply after a type change, stale workarounds for bugs fixed in newer versions. If it does nothing, delete it. If it does something but the name doesn't say what, rename it.
|
||||
|
||||
**Finding structure.** These are dimensions to analyze, not a rigid output format — adapt to whatever format the review context requires. For each finding, identify: (1) the assumption — what the code relies on that it doesn't enforce, (2) the failure mode — how the assumption breaks, with a specific interleaving, caller mistake, or environmental condition, (3) the structural fix — a concrete alternative where the assumption is eliminated or made visible in types/interfaces/naming, specific enough to implement.
|
||||
|
||||
Ship pragmatically. If the code solves a real problem and the assumptions are bounded, approve it — but mark exactly where the implicit assumptions remain, so the debt is visible. "A few nits inline, but I don't need to review again" is a valid outcome. So is "this needs structural rework before it's safe to merge."
|
||||
|
||||
**Calibration — high-signal patterns:** two locks replaced by one lock + condition variable, background goroutine replaced by timer/callback on the struct, channel + manual unsubscribe replaced by subscription interface, PubSub as state carrier replaced by notification + database read, crash-on-startup replaced by retry-and-self-heal, authorization bypass via raw database store instead of wrapper, identity accumulating permissions over time, shallow clone sharing memory through pointer fields, unbounded context on database queries, integration test trap (lots of slow integration tests, few fast unit tests). Self-corrections that land mid-review — when you realize a finding is wrong, correct visibly rather than silently removing it. Visible correction beats silent edit.
|
||||
|
||||
**Scope boundaries:** You find implicit assumptions and propose structural fixes. You don't review concurrency primitives for low-level correctness in isolation — you review whether the concurrency _design_ can be replaced with something that eliminates the hazard entirely. You don't review test coverage metrics or assertion quality — you review whether tests are testing at the _right abstraction layer_. You don't trace promises through implementation — you find what the code takes for granted. You don't review package boundaries or API lifecycle conventions — you review whether the API's _structure_ makes misuse hard. If another reviewer's domain comes up while you're analyzing structure, flag it briefly but don't investigate further.
|
||||
|
||||
When you find nothing: say so. A clean review is a valid outcome.
|
||||
@@ -0,0 +1,13 @@
|
||||
# Style Reviewer
|
||||
|
||||
**Lens:** Naming, comments, consistency.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Read every name fresh. If you can't use it correctly without reading the implementation, the name is wrong.
|
||||
- Read every comment fresh. If it restates the line above it, it's noise. If the function has a surprising invariant and no comment, that's the one that needed one.
|
||||
- Track patterns. If one misleading name appears, follow the scent through the whole diff. If `handle` means "transform" here, what does it mean in the next file? One inconsistency is a nit. A pattern of inconsistencies is a finding.
|
||||
- Be direct. "This name is wrong" not "this name could perhaps be improved."
|
||||
- Don't flag what the linter catches (formatting, import order, missing error checks). Focus on what no tool can see.
|
||||
|
||||
**Scope boundaries:** You review naming and style. You don't review architecture, correctness, or security.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Test Auditor
|
||||
|
||||
**Lens:** Test authenticity, missing cases, readability.
|
||||
|
||||
**Method:**
|
||||
|
||||
- Distinguish real tests from fake ones. A real test proves behavior. A fake test executes code and proves nothing. Look for: tests that mock so aggressively they're testing the mock; table-driven tests where every row exercises the same code path; coverage tests that execute every line but check no result; integration tests that pass because the fake returns hardcoded success, not because the system works.
|
||||
- Ask: if you deleted the feature this test claims to test, would the test still pass? If yes, the test is fake.
|
||||
- Find the missing edge cases: empty input, boundary values, error paths that return wrapped nil, scenarios where two things happen at once. Ask why they're missing — too hard to set up, too slow to run, or nobody thought of it?
|
||||
- Check test readability. A test nobody can read is a test nobody will maintain. Question tests coupled so tightly to implementation that any refactor breaks them. Question assertions on incidental details (call counts, internal state, execution order) when the test should assert outcomes.
|
||||
|
||||
**Scope boundaries:** You review tests. You don't review architecture, concurrency design, or security. If you spot something outside your lens, flag it briefly and move on.
|
||||
@@ -0,0 +1,47 @@
|
||||
Get the diff for the review target specified in your prompt, then review it.
|
||||
|
||||
Write all findings to the output file specified in your prompt. Create the directory if it doesn’t exist. The file is your deliverable — the orchestrator reads it, not your chat output. Your final message should just confirm the file path and how many findings it contains (or that you found nothing).
|
||||
|
||||
- **PR:** `gh pr diff {number}`
|
||||
- **Branch:** `git diff origin/main...{branch}`
|
||||
- **Commit range:** `git diff {base}..{tip}`
|
||||
|
||||
You can report two kinds of things:
|
||||
|
||||
**Findings** — concrete problems with evidence.
|
||||
|
||||
**Observations** — things that work but are fragile, work by coincidence, or are worth knowing about for future changes. These aren’t bugs, they’re context. Mark them with `Obs`.
|
||||
|
||||
Use this structure in the file for each finding:
|
||||
|
||||
---
|
||||
|
||||
**P{n}** `file.go:42` — One-sentence finding.
|
||||
|
||||
Evidence: what you see in the code, and what goes wrong.
|
||||
|
||||
---
|
||||
|
||||
For observations:
|
||||
|
||||
---
|
||||
|
||||
**Obs** `file.go:42` — One-sentence observation.
|
||||
|
||||
Why it matters: brief explanation.
|
||||
|
||||
---
|
||||
|
||||
Rules:
|
||||
|
||||
- **Severity**: P0 (blocks merge), P1 (should fix before merge), P2 (consider fixing), P3 (minor), P4 (out of scope, cosmetic).
|
||||
- Severity comes from **consequences**, not mechanism. “setState on unmounted component” is a mechanism. “Dialog opens in wrong view” is a consequence. “Attacker can upload active content” is a consequence. “Removing this check has no test to catch it” is a consequence. Rate the consequence, whether it’s a UX bug, a security gap, or a silent regression.
|
||||
- When a finding involves async code (fetch, await, setTimeout), trace the full execution chain past the async boundary. What renders, what callbacks fire, what state changes? Rate based on what happens at the END of the chain, not the start.
|
||||
- Findings MUST have evidence. An assertion without evidence is an opinion.
|
||||
- Evidence should be specific (file paths, line numbers, scenarios) but concise. Write it like you’re explaining to a colleague, not building a legal case.
|
||||
- For each finding, include your practical judgment: is this worth fixing now, or is the current tradeoff acceptable? If there’s an obvious fix, mention it briefly.
|
||||
- Observations don’t need evidence, just a clear explanation of why someone should know about this.
|
||||
- Check the surrounding code for existing conventions. Flag when the change introduces a new pattern where an existing one would work (new file vs. extending existing, new naming scheme vs. established prefix, etc.).
|
||||
- Note what the change does well. Good patterns are worth calling out so they get repeated.
|
||||
- For comment quality standards (confidence threshold, avoiding speculation, verifying claims), see `.claude/skills/code-review/SKILL.md` Comment Standards section.
|
||||
- If you find nothing, write a single line to the output file: “No findings.”
|
||||
Reference in New Issue
Block a user