opensource/BMAD-METHOD
3
0
Fork 0
mirror of https://github.com/bmad-code-org/BMAD-METHOD.git synced 2026-08-28 19:20:41 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix(dev-auto): move deferred findings into the spec (#2640)

Browse Source 
Record deferred review findings only in spec frontmatter and remove the
deferred-work output. Make updates safe for legacy specs and YAML-special
content, strengthen contract coverage, and synchronize the reference docs.
This commit is contained in:
Alex Verkhovsky
2026-07-28 18:42:42 -07:00
committed by GitHub
parent c2530ea53f
commit cfee292715
6 changed files with 210 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/dev-auto.md
+42 -27
View File
@@ -7,6 +7,8 @@ sidebar:
To use BMad in an autonomous development loop, use the `bmad-dev-auto` skill. It is like [Quick Dev](../explanation/quick-dev.md), but designed to keep moving without human interaction. You can use it in an interactive session, but its main purpose is to be used by an orchestrator.
The important architectural boundary is this: `bmad-dev-auto` owns the implementation run and the spec artifact it produces, but it does not own your backlog policy. When review finds something real that is not this story's problem, the skill records that finding in the spec it owns and stops there. Deciding whether to queue it, deduplicate it, escalate it, or ignore it is the orchestrator's responsibility.
## What It Does
`bmad-dev-auto` performs one unattended development-loop iteration:
@@ -41,14 +43,14 @@ Supported intent shapes include:
If the invocation points to an existing spec file with one of the known `status` values in the frontmatter, the workflow resumes from that state:
| Spec status | Entry point |
| --- | --- |
| `draft` | plan |
| `ready-for-dev` | implement |
| `in-progress` | implement |
| `in-review` | review |
| `done` | review again as a fresh follow-up pass |
| `blocked` | halt immediately |
| Spec status | Entry point |
| --------------- | -------------------------------------- |
| `draft` | plan |
| `ready-for-dev` | implement |
| `in-progress` | implement |
| `in-review` | review |
| `done` | review again as a fresh follow-up pass |
| `blocked` | halt immediately |
### Folder+ID Dispatch
@@ -58,11 +60,11 @@ The workflow reads `<spec-folder>/stories.yaml` and looks up the entry whose `id
It then checks `<spec-folder>/stories/<story-id>-*.md` (id-prefix match) to tell a first dispatch from a resume:
| On-disk match | Outcome |
| --- | --- |
| None | First dispatch. Requires `<spec-folder>/SPEC.md` to exist (otherwise halts `blocked` / `no epic spec found`). Loads `SPEC.md` and its companions, then proceeds to planning. |
| Exactly one | Resume: routes on that file's `status` exactly like the Resume Input table above. A `blocked` status here reports blocking condition `story already blocked`, not `blocked spec supplied` — dev-auto discovered the file by id, the caller didn't hand it a blocked spec. A missing or unrecognized `status` halts `blocked` / `unrecognized status in existing story file`. |
| More than one | Halts `blocked` / `ambiguous story file match`. |
| On-disk match | Outcome |
| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| None | First dispatch. Requires `<spec-folder>/SPEC.md` to exist (otherwise halts `blocked` / `no epic spec found`). Loads `SPEC.md` and its companions, then proceeds to planning. |
| Exactly one | Resume: routes on that file's `status` exactly like the Resume Input table above. A `blocked` status here reports blocking condition `story already blocked`, not `blocked spec supplied` — dev-auto discovered the file by id, the caller didn't hand it a blocked spec. A missing or unrecognized `status` halts `blocked` / `unrecognized status in existing story file`. |
| More than one | Halts `blocked` / `ambiguous story file match`. |
A `blocked` story file is permanent: every later dispatch of that id halts with `story already blocked`, even after the cause is fixed. To retry, delete the story file — the id then reads as pending and the next dispatch starts fresh.
@@ -90,14 +92,25 @@ It may also look at:
The spec frontmatter `status` is the main machine-readable state for orchestration:
| Spec Status | Meaning |
| --- | --- |
| `draft` | Spec exists but has not passed ready-for-dev validation |
| `ready-for-dev` | Spec is complete enough to implement |
| `in-progress` | Implementation is underway |
| `in-review` | Review/triage is underway |
| `done` | Workflow completed successfully |
| `blocked` | Workflow cannot safely continue unattended |
| Spec Status | Meaning |
| --------------- | ------------------------------------------------------- |
| `draft` | Spec exists but has not passed ready-for-dev validation |
| `ready-for-dev` | Spec is complete enough to implement |
| `in-progress` | Implementation is underway |
| `in-review` | Review/triage is underway |
| `done` | Workflow completed successfully |
| `blocked` | Workflow cannot safely continue unattended |
### Deferred Findings
`deferred` is where the skill reports real findings that are not this story's problem. Each item contains:
- `summary` — one-sentence description of the deferred issue
- `evidence` — why the finding is real
- `location` — optional file:line or component hint
- `severity` — optional final triage severity (`high`, `medium`, `low`)
This is intentionally not a backlog. It is a machine-readable review output. The orchestrator has to decide what happens next: create a ticket, append to a central queue, correlate duplicates across runs, or do nothing.
### On `ready-for-dev`
@@ -116,6 +129,7 @@ On successful completion, the workflow writes or updates the spec with:
- Residual risks
- `followup_review_recommended` flag. True if LLM decided another review pass seems worthwhile. It's a suggestion, not a must. Simplest way to give it a second review pass is to re-run the skill pointing it at the spec file.
- `baseline_revision` and `final_revision` — the full canonical revisions before implementation and at the reviewed change's endpoint. `git log baseline_revision..final_revision` lists the reviewed change commits. Both are `NO_VCS` without version control.
- `deferred` frontmatter entries for review findings triaged `defer`. Each item records `summary`, `evidence`, and, when known, `location` plus `severity`.
The workflow commits but does not push. If the spec is tracked in the implementation repository, clean HEAD is one spec-finalization commit beyond `final_revision`; otherwise the implementation repository ends at `final_revision`.
@@ -158,6 +172,7 @@ For new work, the workflow creates:
That spec is the contract between planning, implementation, and review. It contains:
- Frontmatter status
- Frontmatter machine state (`followup_review_recommended`, `warnings`, `deferred`, revision markers)
- The immutable `<intent-contract>` block
- Code map
- Tasks and acceptance criteria
@@ -171,11 +186,11 @@ Under folder+id dispatch, the workflow writes to `<spec-folder>/stories/<story-i
When a halt happens before a slug can be derived from the story's title, the write-back falls back to a fixed slug segment instead:
| Situation | Slug segment used |
| --- | --- |
| `stories.yaml` missing/unparseable, or no entry matches the story id | `unresolved` |
| More than one on-disk file already matches `<story-id>-*.md` | `ambiguous` |
| Entry resolved and no on-disk ambiguity | slug derived from `title` (and `description` if needed) |
| Situation | Slug segment used |
| -------------------------------------------------------------------- | ------------------------------------------------------- |
| `stories.yaml` missing/unparseable, or no entry matches the story id | `unresolved` |
| More than one on-disk file already matches `<story-id>-*.md` | `ambiguous` |
| Entry resolved and no on-disk ambiguity | slug derived from `title` (and `description` if needed) |
If the resolved path already exists, the workflow updates its `status` frontmatter and appends result details under `## Auto Run Result`, same as the primary spec artifact. If it doesn't exist, the workflow creates a skeletal story spec: frontmatter status, a heading (the entry's title, or `Story <story_id>` if the entry couldn't be resolved or the on-disk match was ambiguous), and an `## Auto Run Result` section.
@@ -192,7 +207,6 @@ This records the terminal status and blocking condition.
Depending on the route, the workflow may also write:
- `{implementation_artifacts}/epic-<N>-context.md`
- `{implementation_artifacts}/deferred-work.md`
- A patch file preserving the attempted change when the review step halts on `intent gap` (path recorded in the spec's triage log)
## Orchestrator Responsibilities
@@ -203,6 +217,7 @@ An orchestrator integrating `bmad-dev-auto` should:
- Prefer passing a spec path when resuming prior work — or the same spec folder and story id, under folder+id dispatch
- Monitor the produced spec file, story spec artifact, or fallback result file for terminal state
- Read `status`, `blocking condition`, and `followup_review_recommended` rather than inferring success from chat output alone
- Read deferred findings from the spec frontmatter `deferred:` list
- Use `baseline_revision..final_revision` to identify the reviewed change commits; do not assume `final_revision` equals HEAD
- Expect autonomous file changes and local commits
- Handle `blocked` as a routing signal, not just a failure signal
docs/zh-cn/reference/dev-auto.md
+16 -1
View File
@@ -7,6 +7,8 @@ sidebar:
要在自主开发循环里使用 BMad，请用 `bmad-dev-auto` skill。它类似 [Quick Dev](../explanation/quick-dev.md)，但设计为在无人交互的情况下持续推进。你可以在交互式会话里用它，主要用途是被 orchestrator 调用。
这里有一条重要的架构边界：`bmad-dev-auto` 负责 implementation run 及其生成的 spec artifact，但不负责 backlog policy。当 review 发现真实但不属于当前 story 的问题时，skill 会把 finding 记录在自己负责的 spec 中，仅此而已。是排入队列、去重、升级还是忽略，由 orchestrator 决定。
## 它做什么
`bmad-dev-auto` 执行一次无人值守的开发循环迭代：
@@ -99,6 +101,17 @@ spec frontmatter 的 `status` 是 orchestration 的主要 machine-readable 状
| `done` | Workflow 成功完成 |
| `blocked` | Workflow 无法安全 unattended 继续 |
### Deferred Findings
`deferred` 用于记录 skill 发现的真实问题，但这些问题不属于当前 story。每个条目包含：
- `summary` —— deferred issue 的单句描述
- `evidence` —— 证明 finding 真实存在的依据
- `location` —— 可选的 file:line 或 component 提示
- `severity` —— 可选的最终 triage severity（`high`、`medium`、`low`）
它不是 backlog，而是 machine-readable review output。Orchestrator 必须决定下一步：创建 ticket、追加到 central queue、关联多次 run 中的重复项，或不做处理。
### 在 `ready-for-dev` 时
`ready-for-dev` 通常是 workflow 直通 implement 的 resume 状态。当 invocation prompt 指示 planning 后 halt 时，它成为真正的 halt 结果：spec 通过 READY FOR DEVELOPMENT gate 后，workflow 设 status `ready-for-dev` 并停在那里，而不是继续 implement。重新 dispatch 同一 spec（或同一 spec 文件夹和 story id）会经上述路由在 implement resume。
@@ -116,6 +129,7 @@ spec frontmatter 的 `status` 是 orchestration 的主要 machine-readable 状
- Residual risks
- `followup_review_recommended` 标志。若 LLM 认为值得再 review 一轮则为 true。只是建议，非必须。最简单的二次 review 是重新运行 skill 并指向 spec 文件。
- `baseline_revision` 和 `final_revision` —— implementation 前与 reviewed change endpoint 的完整 canonical revision。`git log baseline_revision..final_revision` 列出 reviewed change commits。无版本控制时两者均为 `NO_VCS`。
- triage 为 `defer` 的 review findings 会写入 frontmatter 的 `deferred` 条目。每个条目记录 `summary`、`evidence`，以及已知时的 `location` 和 `severity`。
Workflow 会 commit，但不会 push。若 spec 由 implementation repository track，clean HEAD 会比 `final_revision` 多一个 spec-finalization commit；否则 implementation repository 会停在 `final_revision`。
@@ -158,6 +172,7 @@ workflow 总是尽量留下 durable artifact 描述发生了什么。
该 spec 是 planning、implementation 和 review 之间的 contract，包含：
- Frontmatter status
- Frontmatter machine state（`followup_review_recommended`、`warnings`、`deferred`、revision markers）
- 不可变的 `<intent-contract>` 块
- Code map
- Tasks 和 acceptance criteria
@@ -192,7 +207,6 @@ workflow 在尚无 valid `spec_file` 时 halt（folder+id dispatch 外 ——
视路由，workflow 还可能写入：
- `{implementation_artifacts}/epic-<N>-context.md`
- `{implementation_artifacts}/deferred-work.md`
- review step 因 `intent gap` halt 时保存 attempted change 的 patch 文件（路径记录在 spec triage log）
## Orchestrator 职责
@@ -203,6 +217,7 @@ workflow 在尚无 valid `spec_file` 时 halt（folder+id dispatch 外 ——
- Resume 时优先传 spec 路径 —— 或 folder+id dispatch 下同一 spec 文件夹和 story id
- 监控产出的 spec 文件、story spec artifact 或 fallback result 文件的 terminal state
- 读 `status`、`blocking condition`、`followup_review_recommended`，不要只从 chat 输出推断成功
- 从 spec frontmatter 的 `deferred:` list 读取 deferred findings
- 用 `baseline_revision..final_revision` 识别 reviewed change commits；不要假设 `final_revision` 等于 HEAD
- 预期 autonomous 文件变更和 local commits
- 把 `blocked` 当作 routing signal，而不只是 failure signal
package.json
+3 -2
View File
@@ -40,10 +40,11 @@
"lint:fix": "eslint . --ext .js,.cjs,.mjs,.yaml --fix",
"lint:md": "markdownlint-cli2 \"**/*.md\"",
"prepare": "command -v husky >/dev/null 2>&1 && husky || exit 0",
"quality": "npm run format:check && npm run lint && npm run lint:md && npm run docs:build && npm run test:install && npm run test:urls && npm run test:renderer && npm run validate:refs && npm run validate:skills && npm run docs:validate-sidebar",
"quality": "npm run format:check && npm run lint && npm run lint:md && npm run docs:build && npm run test:install && npm run test:urls && npm run test:renderer && npm run test:dev-auto-contract && npm run validate:refs && npm run validate:skills && npm run docs:validate-sidebar",
"rebundle": "node tools/installer/bundlers/bundle-web.js rebundle",
"test": "npm run test:refs && npm run test:install && npm run test:urls && npm run test:channels && npm run test:renderer && npm run test:skills && npm run lint && npm run lint:md && npm run format:check",
"test": "npm run test:refs && npm run test:install && npm run test:urls && npm run test:channels && npm run test:renderer && npm run test:dev-auto-contract && npm run test:skills && npm run lint && npm run lint:md && npm run format:check",
"test:channels": "node test/test-installer-channels.js",
"test:dev-auto-contract": "node test/test-dev-auto-contract.js",
"test:install": "node test/test-installation-components.js",
"test:refs": "node test/test-file-refs-csv.js",
"test:renderer": "uv run --python 3.11 python -m unittest src/scripts/tests/test_config_utils.py src/scripts/tests/test_resolve_config.py src/scripts/tests/test_resolve_customization.py && node test/test-quick-dev-renderer.js && node test/test-dev-auto-renderer.js",
src/bmm-skills/4-implementation/bmad-dev-auto/spec-template.md
+1
View File
@@ -7,6 +7,7 @@ review_loop_iteration: 0 # incremented by step-04 before each review loopback
followup_review_recommended: false # set by step-04 on status: done — true if the LLM decided another review pass is worthwhile
context: [] # optional: `{project-root}/`-prefixed paths to project-wide standards/docs the implementation agent should load. Keep short — only what isn't already distilled into the spec body.
warnings: [] # optional: machine-readable warnings for orchestration, e.g. oversized, multiple-goals
deferred: [] # append-only machine-readable deferred review findings; each item carries summary/evidence and optional location/severity
---
<!-- Aim for 900–1600 tokens. If larger, add `oversized` to frontmatter `warnings` and continue.
src/bmm-skills/4-implementation/bmad-dev-auto/step-04-review.md
+11 -5
View File
@@ -60,12 +60,18 @@ Execute these review layers in parallel wherever their execution methods allow:
- **intent_gap** — Root cause is inside `<intent-contract>`. Save the attempted change as a patch file in `{{.implementation_artifacts}}` and reference it from the triage-log entry, then revert code changes. Append the triage-log entry for this pass with `addressed_findings: none`, then HALT with status `blocked`, blocking condition `intent gap`, and include the unresolved questions and the saved patch path.
- **bad_spec** — Root cause is outside `<intent-contract>`. Do not modify content inside `<intent-contract>`. Before reverting code: extract KEEP instructions for positive preservation (what worked well and must survive re-derivation). Revert code changes. Read the `## Spec Change Log` in `{spec_file}` and strictly respect all logged constraints when amending the sections outside `<intent-contract>` that contain the root cause. Append a new change-log entry recording: the triggering finding, what was amended, the known-bad state avoided, and the KEEP instructions. Append the triage-log entry for this pass, listing every bad_spec finding that triggered the spec amendment and implementation loopback under `addressed_findings`. Read fully and follow `[[bmad-snapshot:step-03-implement.md]]` to re-derive the code, then this step will run again.
- **patch** — Auto-fix. These are the only findings that survive loopbacks. If the step-03 implementation subagent can be re-engaged with its context intact, send it all patch findings in one synchronous message — for each: the file, what is wrong, and what the fix must do. If it cannot be re-engaged, apply the patches yourself. Then re-run the commands in `{spec_file}`'s `## Verification` section (or perform its manual checks); if verification fails and the failure cannot be fixed, HALT with status `blocked` and blocking condition `patch verification failed`. Append the triage-log entry for this pass, listing every patch fixed in this pass under `addressed_findings`.
- **defer** — Append one new entry to `{{.implementation_artifacts}}/deferred-work.md` using this format. Do not modify existing entries or look for duplicates.
```markdown
- source_spec: `{spec_file}`
summary: <one sentence>
evidence: <why this is real>
- **defer** — Update the single `deferred` list in `{spec_file}` frontmatter. If the field is absent (including on specs created before this field existed), add it once as an empty list. If it is `deferred: []`, replace that empty value when adding the first item; otherwise append to the existing list. Preserve every existing item, do not look for duplicates, and never add a second `deferred:` key. Serialize free-form values as YAML block scalars so characters such as `:`, `#`, quotes, and line breaks remain data. Each item uses this shape:
```yaml
deferred:
- summary: >-
<one sentence>
evidence: |-
<why this is real>
location: >- # optional — file:line or component
src/foo.py:42
severity: medium # optional — high | medium | low
```
After all appends, parse the complete frontmatter as YAML and verify that `deferred` is one list containing every prior item plus the new items with their intended text. Repair serialization errors before continuing.
- **reject** — Drop silently.
## Finalize
test/test-dev-auto-contract.js
+137
View File
@@ -0,0 +1,137 @@
/**
* Regression coverage for bmad-dev-auto's deferred-finding contract.
*
* Ensures the canonical source keeps:
* 1. Machine-readable `deferred` frontmatter on the spec template.
* 2. Review-step instructions that persist deferred findings only in the spec.
* 3. Reference docs that tell orchestrators to read deferred findings from the spec.
*/
'use strict';
const fs = require('node:fs');
const path = require('node:path');
const yaml = require('yaml');
const colors = {
reset: '\u001B[0m',
green: '\u001B[32m',
red: '\u001B[31m',
cyan: '\u001B[36m',
};
let totalTests = 0;
let passedTests = 0;
const failures = [];
function test(name, fn) {
totalTests++;
try {
fn();
passedTests++;
console.log(` ${colors.green}\u2713${colors.reset} ${name}`);
} catch (error) {
console.log(` ${colors.red}\u2717${colors.reset} ${name} ${colors.red}${error.message}${colors.reset}`);
failures.push({ name, message: error.message });
}
}
function assert(condition, message) {
if (!condition) throw new Error(message);
}
function read(relativePath) {
return fs.readFileSync(path.join(__dirname, '..', relativePath), 'utf-8');
}
function parseFrontmatter(content, relativePath) {
assert(content.startsWith('---\n'), `${relativePath} must start with a frontmatter delimiter`);
const end = content.indexOf('\n---\n', 4);
assert(end !== -1, `${relativePath} must close its frontmatter delimiter`);
return yaml.parse(content.slice(4, end));
}
function dedent(content) {
const lines = content.split('\n');
const indents = lines.filter((line) => line.trim()).map((line) => line.match(/^ */)[0].length);
const width = Math.min(...indents);
return lines.map((line) => line.slice(width)).join('\n');
}
console.log(`\n${colors.cyan}bmad-dev-auto deferred contract${colors.reset}\n`);
test('spec template exposes machine-readable deferred frontmatter', () => {
const relativePath = 'src/bmm-skills/4-implementation/bmad-dev-auto/spec-template.md';
const frontmatter = parseFrontmatter(read(relativePath), relativePath);
assert(Array.isArray(frontmatter.deferred), 'spec-template.md frontmatter must declare deferred as a list');
assert(frontmatter.deferred.length === 0, 'spec-template.md deferred list must start empty');
});
test('dev-auto steps preserve their frontmatter boundaries', () => {
const root = 'src/bmm-skills/4-implementation/bmad-dev-auto';
const stepOnePath = `${root}/step-01-clarify-and-route.md`;
const stepOneFrontmatter = parseFrontmatter(read(stepOnePath), stepOnePath);
assert(stepOneFrontmatter.spec_file === '', 'step-01 must define spec_file in frontmatter');
assert(stepOneFrontmatter.spec_folder === '', 'step-01 must define spec_folder in frontmatter');
assert(stepOneFrontmatter.story_id === '', 'step-01 must define story_id in frontmatter');
for (const filename of ['step-02-plan.md', 'step-04-review.md']) {
const relativePath = `${root}/${filename}`;
const content = read(relativePath);
if (content.startsWith('---\n')) parseFrontmatter(content, relativePath);
}
});
test('review step safely records deferred findings only in the spec', () => {
const content = read('src/bmm-skills/4-implementation/bmad-dev-auto/step-04-review.md');
assert(content.includes('If the field is absent'), 'step-04-review.md must initialize deferred for legacy specs');
assert(content.includes('never add a second `deferred:` key'), 'step-04-review.md must forbid duplicate deferred keys');
assert(content.includes('parse the complete frontmatter as YAML'), 'step-04-review.md must validate the updated frontmatter');
assert(!content.includes('deferred_work_file'), 'step-04-review.md must not mention a deferred-work ledger path');
assert(!content.includes('deferred-work.md'), 'step-04-review.md must not mention the deferred-work ledger artifact');
const example = content.match(/```yaml\n([\s\S]*?)\n[ \t]*```/);
assert(example, 'step-04-review.md must include the deferred YAML example');
const specialCharacters = dedent(example[1])
.replace('<one sentence>', 'Parser fails: malformed # input')
.replace('<why this is real>', 'Observed: value # remains data\n Second evidence line');
const parsed = yaml.parse(specialCharacters);
assert(parsed.deferred[0].summary === 'Parser fails: malformed # input', 'summary example must preserve YAML-special characters');
assert(
parsed.deferred[0].evidence === 'Observed: value # remains data\nSecond evidence line',
'evidence example must preserve YAML-special characters and line breaks',
);
});
test('reference docs direct orchestrators to the spec deferred list', () => {
const content = read('docs/reference/dev-auto.md');
assert(
content.includes('Read deferred findings from the spec frontmatter `deferred:` list'),
'docs/reference/dev-auto.md must tell orchestrators where to read deferred findings',
);
assert(!content.includes('deferred-work.md'), 'docs/reference/dev-auto.md must not describe a deferred-work ledger artifact');
});
test('Chinese reference documents the same deferred contract', () => {
const content = read('docs/zh-cn/reference/dev-auto.md');
assert(content.includes('spec frontmatter 的 `deferred:` list'), 'Chinese reference must direct orchestrators to the deferred list');
assert(!content.includes('deferred-work.md'), 'Chinese reference must not describe a deferred-work ledger artifact');
});
console.log(`\n${colors.cyan}${'═'.repeat(55)}${colors.reset}`);
console.log(`${colors.cyan}Test Results:${colors.reset}`);
console.log(` Total: ${totalTests}`);
console.log(` Passed: ${colors.green}${passedTests}${colors.reset}`);
console.log(` Failed: ${passedTests === totalTests ? colors.green : colors.red}${totalTests - passedTests}${colors.reset}`);
console.log(`${colors.cyan}${'═'.repeat(55)}${colors.reset}\n`);
if (failures.length > 0) {
console.log(`${colors.red}FAILED TESTS:${colors.reset}\n`);
for (const failure of failures) {
console.log(`${colors.red}\u2717${colors.reset} ${failure.name}`);
console.log(` ${failure.message}\n`);
}
process.exit(1);
}
console.log(`${colors.green}All tests passed!${colors.reset}\n`);