Emmanuel Atsé
cede485217
* feat(docs): add sidebar order validator
Adds tools/validate-sidebar-order.js to validate sidebar.order values
in YAML frontmatter across English and translated docs.
Checks for duplicate orders, gaps in sequence, and missing order fields.
For translations, also warns on order drift from English counterparts.
Wired into the quality script as docs:validate-sidebar.
* fix(validate-sidebar): tighten language detection and drift guard, add docstrings
* fix(validate-sidebar): replace subdirectory heuristic with locale pattern matching
detectLanguageDirs() previously classified any top-level docs/ directory
containing subdirectories as a translation language. This was too broad —
if an English section ever gained nested subfolders it would be silently
excluded from validation.
Replaced with a BCP 47 locale-code regex (/^[a-z]{2}(?:-[a-zA-Z]{2})?$/)
that matches known patterns (cs, fr, vi-vn, zh-cn) and won't falsely
classify content sections like explanation/ or reference/.
* fix(validate-sidebar): guard drift check against undefined order values
extractSidebarOrder() returns { hasSidebar: false } when no sidebar block
exists, leaving order as undefined rather than null. The drift check only
guarded against null, allowing undefined values to emit noisy warnings
like "Order drift: ... order undefined".
Changed the guard to typeof === 'number' which correctly excludes both
undefined and null without relying on a specific sentinel value.
* chore(validate-sidebar): add JSDoc docstrings to all functions
Adds @param and @returns annotations to extractSidebarOrder,
detectLanguageDirs, getEnglishSections, checkDirectory,
checkTranslationDrift, and relativePath.
* fix(validate-sidebar): add to pre-commit hook
* refactor(validate-sidebar): harden parsing and edge-case handling
Refactor to main() wrapper with pure return-based APIs, single directory
scan, and shared reporting. Harden frontmatter parsing (anchored delimiter,
direct-child-only order extraction, flow mapping support) and validation
(Infinity/zero guard, gap flood cap, multi-segment locales, graceful ENOENT).
* docs: fix sidebar.order duplicates and gaps across all locales
Resolves all validator errors flagged by the new
tools/validate-sidebar-order.js check.
English (docs/{explanation,how-to,reference}/):
- Renumbered to remove duplicates; established reading order
for new explanation pages added since orders were last set.
Translations (cs, fr, vi-vn, zh-cn):
- Mirrored English structural ordering where files exist, then
compacted to 1..N within each directory to eliminate gaps
caused by missing translation files.
Non-blocking drift warnings remain where translation directories
have fewer files than English; these are expected per the
validator's design.
---------
Co-authored-by: Brian Madison <bmadcode@gmail.com>
5.7 KiB
title, description, sidebar
| title | description | sidebar | ||
|---|---|---|---|---|
| Testing Options | Comparing the built-in QA workflow with the Test Architect (TEA) module for test automation. |
|
BMad provides two testing paths: a built-in QA workflow for fast test generation and an installable Test Architect module for enterprise-grade test strategy.
Which Should You Use?
| Factor | Built-in QA | TEA Module |
|---|---|---|
| Best for | Small-medium projects, quick coverage | Large projects, regulated or complex domains |
| Setup | Nothing to install -- included in BMM | Install separately via npx bmad-method install |
| Approach | Generate tests fast, iterate later | Plan first, then generate with traceability |
| Test types | API and E2E tests | API, E2E, ATDD, NFR, and more |
| Strategy | Happy path + critical edge cases | Risk-based prioritization (P0-P3) |
| Workflow count | 1 (Automate) | 9 (design, ATDD, automate, review, trace, and others) |
:::tip[Start with built-in QA] Most projects should start with the built-in QA workflow. If you later need test strategy, quality gates, or requirements traceability, install TEA alongside it. :::
Built-in QA Workflow
The built-in QA workflow (bmad-qa-generate-e2e-tests) is part of the BMM (Agile suite) module, available through the Developer agent. It generates working tests quickly using your project's existing test framework -- no configuration or additional installation required.
Trigger: QA (via the Developer agent) or bmad-qa-generate-e2e-tests
What It Does
The QA workflow (Automate) walks through five steps:
- Detect test framework -- scans
package.jsonand existing test files for your framework (Jest, Vitest, Playwright, Cypress, or any standard runner). If none exists, analyzes the project stack and suggests one. - Identify features -- asks what to test or auto-discovers features in the codebase.
- Generate API tests -- covers status codes, response structure, happy path, and 1-2 error cases.
- Generate E2E tests -- covers user workflows with semantic locators and visible-outcome assertions.
- Run and verify -- executes the generated tests and fixes failures immediately.
The workflow produces a test summary saved to your project's implementation artifacts folder.
Test Patterns
Generated tests follow a "simple and maintainable" philosophy:
- Standard framework APIs only -- no external utilities or custom abstractions
- Semantic locators for UI tests (roles, labels, text rather than CSS selectors)
- Independent tests with no order dependencies
- No hardcoded waits or sleeps
- Clear descriptions that read as feature documentation
:::note[Scope]
The QA workflow generates tests only. For code review and story validation, use the Code Review workflow (CR) instead.
:::
When to Use Built-in QA
- Quick test coverage for a new or existing feature
- Beginner-friendly test automation without advanced setup
- Standard test patterns that any developer can read and maintain
- Small-medium projects where comprehensive test strategy is unnecessary
Test Architect (TEA) Module
TEA is a standalone module that provides an expert agent (Murat) and nine structured workflows for enterprise-grade testing. It goes beyond test generation into test strategy, risk-based planning, quality gates, and requirements traceability.
- Documentation: TEA Module Docs
- Install:
npx bmad-method installand select the TEA module - npm:
bmad-method-test-architecture-enterprise
What TEA Provides
| Workflow | Purpose |
|---|---|
| Test Design | Create a comprehensive test strategy tied to requirements |
| ATDD | Acceptance-test-driven development with stakeholder criteria |
| Automate | Generate tests with advanced patterns and utilities |
| Test Review | Validate test quality and coverage against strategy |
| Traceability | Map tests back to requirements for audit and compliance |
| NFR Assessment | Evaluate non-functional requirements (performance, security) |
| CI Setup | Configure test execution in continuous integration pipelines |
| Framework Scaffolding | Set up test infrastructure and project structure |
| Release Gate | Make data-driven go/no-go release decisions |
TEA also supports P0-P3 risk-based prioritization and optional integrations with Playwright Utils and MCP tooling.
When to Use TEA
- Projects that require requirements traceability or compliance documentation
- Teams that need risk-based test prioritization across many features
- Enterprise environments with formal quality gates before release
- Complex domains where test strategy must be planned before tests are written
- Projects that have outgrown the built-in QA's single-workflow approach
How Testing Fits into Workflows
The QA Automate workflow appears in Phase 4 (Implementation) of the BMad Method workflow map. It is designed to run after a full epic is complete — once all stories in an epic have been implemented and code-reviewed. A typical sequence:
- For each story in the epic: implement with Dev (
DS), then validate with Code Review (CR) - After the epic is complete: generate tests with
QA(via the Developer agent) or TEA's Automate workflow - Run retrospective (
bmad-retrospective) to capture lessons learned
The built-in QA workflow works directly from source code without loading planning documents (PRD, architecture). TEA workflows can integrate with upstream planning artifacts for traceability.
For more on where testing fits in the overall process, see the Workflow Map.