mirror of
https://github.com/bmad-code-org/BMAD-METHOD.git
synced 2026-08-29 03:44:18 +08:00
feat(bmm): add point-of-use sub-agent contracts to bmad-retro-new
Delegation contracts stated in the steps that use them, not a preamble: aggregate views prefer deterministic scripts and sub-agents return evidence only; review passes run as independent sub-agents with audit-plan-chosen asymmetric lenses and degrade to human-run prompt files, never to inline role-play; routing regrounds findings per write-authority grade before actuation. Capability inventory added to evidence collection. rationale.md records the placement rule and the graded-recheck reasoning.
This commit is contained in:
@@ -19,15 +19,15 @@ description: 'Close out a completed epic: analyze the evidence, verify behavior,
|
||||
|
||||
## Phase 1: Prepare
|
||||
|
||||
1. **Collect Evidence** — Enumerate what actually exists: spec, story files, diff, session logs, git history. Each later step declares what it needs and degrades when something is missing.
|
||||
1. **Collect Evidence** — Enumerate what actually exists: spec, story files, diff, session logs, git history. Inventory capabilities too: sub-agent availability shapes later steps. Each later step declares what it needs and degrades when something is missing.
|
||||
|
||||
2. **Load Audit Plan / Profile Epic** — Read the acceptance criteria declared in the epic spec; if absent, profile the epic from its diff and stories to select which analyses to run.
|
||||
|
||||
## Phase 2: Analyze
|
||||
|
||||
3. **Build Aggregate Views** — Derive whole-epic views that no single diff hunk shows: architecture delta, duplication map, pattern divergence, size trajectories, spec-to-implementation reconciliation.
|
||||
3. **Build Aggregate Views** — Derive whole-epic views that no single diff hunk shows: architecture delta, duplication map, pattern divergence, size trajectories, spec-to-implementation reconciliation. Prefer deterministic scripts; sub-agents return evidence with source refs, never findings. No sub-agents: compute inline and record the narrowed scope.
|
||||
|
||||
4. **Run Review Passes** — Machine review at diff scope, weighted per audit plan: correctness at boundaries between stories, verification gaps, intent conformance.
|
||||
4. **Run Review Passes** — Independent sub-agent reviews at diff scope, with asymmetric lenses chosen by the audit plan: correctness at boundaries between stories, verification gaps, intent conformance. Outputs are hypotheses with source refs and checked scope. No sub-agents: emit prompt files for the human to run in separate sessions, or record the skip — never run "independent" reviews inline.
|
||||
|
||||
5. **Verify Behavior** — Exercise the changed behavior end-to-end as the audit plan directs: run the system, drive the affected flows, record what was observed. Passing tests do not substitute for this step.
|
||||
|
||||
@@ -35,7 +35,7 @@ description: 'Close out a completed epic: analyze the evidence, verify behavior,
|
||||
|
||||
## Phase 3: Decide
|
||||
|
||||
7. **Route Findings** — Assign each finding a disposition (fix now / defer / accept as-is) and a lesson destination by lifetime: repo is permanent, spec files last the project, skill changes go via PR.
|
||||
7. **Route Findings** — Assign each finding a disposition (fix now / defer / accept as-is) and a lesson destination by lifetime: repo is permanent, spec files last the project, skill changes go via PR. Sub-agent findings are testimony, not truth: before actuation, reground per write authority — reopen primary sources for spec and skill targets; repo fixes are rechecked by the remediation session itself.
|
||||
|
||||
8. **Emit Questions** — For each delta a human answer would materially change: a pointed question, a machine default, links to evidence. Also ask one open question for observations the analyses missed.
|
||||
|
||||
|
||||
@@ -57,6 +57,7 @@ Five findings force this shape, and each anchors one section below:
|
||||
- **Design as if no human is involved.** Inputs are evidence; outputs are deltas. Human input enters as *solicited evidence*: pointed questions where an answer would materially change a delta, with evidence links, answerable in any amount or never — plus one open question for what the analyses missed. The skill never asks anyone to "review the epic"; humans review however they please. (The alternative — a curated review experience / narrative dossier — was designed in detail and cut: story-level HITL tooling had already failed ergonomically here (checkpoint-preview), epic scale is strictly harder, and the efficacy evidence for presentation-layer remedies is absent. Baum: "We cannot conclude reliably whether the order of presenting the code change parts influences the efficiency of code review"; [Bouraffa et al. 2025](https://arxiv.org/pdf/2506.10654) shows revealed *demand* — 44.6% of 23K mined reviews deviate from alphabetical order, and deviators cover more of the diff and approve less readily — not efficacy. [Devin Review](https://cognition.com/blog/devin-review) ships narrative diff review per-PR with zero independent efficacy evidence: built ≠ works.)
|
||||
- **The tiers** (stated as the Write Authority section at the top of SKILL.md). Repo: machine may commit, and machine defaults may execute — tests, CI, and revert catch a wrong default. Spec layer: evidence-backed edits apply directly; *uncertain* edits require a human answer, and headless runs hold them as pending proposals — the spec layer has no immune system, and applying an uncertain interpretation "with a question attached" would launder machine judgment into the contract governing all remaining epics (worst around accepted deviations and re-planning, where the point of the question is that the answer could change the write). Skills: pull requests only, human merges, always — widest blast radius (every future epic), slowest feedback (weeks, confounded), no enforcement (a bad edit degrades silently); by the artifact law, the human gate *is* their grafted-on enforcement mechanism.
|
||||
- **The acceptance gate is explicit and asymmetric.** The epic is judged (step 17) against its declared criteria on verification evidence and unresolved findings: accepted / accepted with open items / rejected. A human decision always overrides; the machine may default to accept only when criteria are demonstrably met. An epic failing its criteria with no human decision closes as *not accepted* — the one behavior the skill must be incapable of is silently accepting failure. The verdict sits after the lesson-bearing steps because lessons and re-planning are more valuable on a rejected epic, not less. Step 18 then records the verdict and the full trail — what ran, what was skipped, what remains open. (Design history: an early skeleton conflated cutting the presentation layer with cutting acceptance and closed the epic with no gate at all; that was flagged and fixed. Do not re-conflate them.)
|
||||
- **Sub-agents generate hypotheses, never conclusions — and the recheck is graded by the same authority ladder.** The retro cannot be single-context (it reads more evidence than any epic session), so indexing, derivation, and review delegate to sub-agents. Their outputs are testimony: source refs and checked scope are required fields in the retro file, so a finding without them cannot be routed — structure enforcing what prose could only request. Before actuation, findings are regrounded per write-authority grade: the parent reopens primary sources for spec-layer and skill targets (no enforcement catches an error there); repo-bound fixes get their recheck from the remediation session itself, which touches the actual code and fails on contact with a hallucinated finding. Asymmetric review carries one special rule: independence cannot be faked inline — without sub-agents it degrades to prompt files a human runs in separate sessions, or an explicitly recorded skip, never to the parent role-playing independent reviewers in its own context.
|
||||
- **Why defect-finding is the machine's job at all:** the review literature settled this before agents existed. [Czerwonka et al. 2015](https://www.microsoft.com/en-us/research/publication/code-reviews-do-not-find-bugs-how-the-current-code-review-best-practice-slows-us-down/): ~15% of review comments concern defects, mostly minor; Bacchelli & Bird 2013: review's real products were comprehension transfer and norm enforcement. In the agent world those two functions transform — comprehension reverses direction (no author to teach; the human needs the mental model), and norms stop being conversational (an agent never sees the PR comment again; norms must land in artifacts or gates). This skill is those two surviving functions rebuilt for agents, with defect-finding delegated to machines at the scope where machines are competent.
|
||||
|
||||
## 5. Mechanism must not outrun incidents
|
||||
@@ -68,6 +69,7 @@ Five findings force this shape, and each anchors one section below:
|
||||
- **The accounting caution, with its one valid exception.** Building the full control system before the first transaction is the naive-systematizer move — accounting insights applied to software management. The kept disanalogy: stateless agents, unlike the humans metrics regimes misjudge, have *no tacit layer* — everything a future session knows arrives through artifacts. Writing things down is not the accounting mistake; deriving judgment from counts of what was written is.
|
||||
- **Iteration strategy: backtest, and build disposable.** Epics complete roughly weekly; learning from live runs alone would take a year the field will not sit still for. Instead: (a) run the retro retroactively on *completed* epics, whose futures are known — which deferred items bit, which drift hardened, which defects escaped — validating the analysis half against ground truth, rerunnable many times a day, decoupled from epic cadence; (b) separate what ages fast from what ages slowly — the skill, its prompts, and its routing are disposable (lifetime months, rebuild cost a day; never encode procedure where model judgment will improve past it, and never build machinery whose validation period exceeds its lifetime), while the repo — including its gates and commit-message record — is the only accumulating asset, which is also why it is the primary delta destination.
|
||||
- **Remediation is bounded by construction.** Fix-now findings become story-shaped work executed by *the same dev loop that built the epic* (no second, bespoke code-changing mechanism), one round only, then fast checks and re-driven flows — without a convergence bound the phase can loop on its own output indefinitely.
|
||||
- **Sub-agent instructions live at the point of use, never in a preamble.** This skill accumulates large contexts; a rule stated at the top of the file is out of recall by the time it is needed, and there is no single global policy to state anyway — each delegation has its own fallback (derivation degrades to inline computation; asymmetric review never degrades inline; remediation hands to the dev loop, which owns its own policy). When this skeleton shards into step files, the delegation contract goes in the step that delegates and the consumption contract in the step that consumes — quick-dev is the house pattern (interactive, capability-tolerant, per-step fallbacks, and "disregard sub-agent severity" stated at the consumption point); dev-auto's refuse-without-sub-agents stance is wrong for an interactive skill.
|
||||
- **Resumability is artifact-based and phase-grained — needed by construction, not by elegance.** The workflow stops by design: questions wait for humans, remediation runs for hours, evidence arrives late. Single-shot execution is unrealistic, so a run may stop at any phase boundary and resume there. State lives in one per-epic retro file in the spec layer — frontmatter status (last completed phase, open questions), body accumulating each phase's products — which doubles as step 18's closeout record: the state file and the record are the same artifact, dying with the scaffolding per the artifact law, so no second memory system is invented. Because the file can drift from the world (commits land, questions get answered elsewhere, the spec changes), every run begins by reconciling file against reality, and reality wins: analyses may be recomputed, routing revised on new evidence, writes reconciled before reapplying — never duplicated. Coarse phase-level status, no opaque session state, no workflow engine inside the workflow. This also makes backtesting cleaner: a historical epic's retro file is just a retro file that starts at phase 1.
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user