mirror of
https://github.com/coder/coder.git
synced 2026-10-16 14:48:56 +08:00
feat: fail CI when the style guide claims enforcement it doesn't have (#29299)
## Summary PR 1 corrected 9 annotations that claimed enforcement the config doesn't provide. This adds the check that keeps them correct, because the same drift will recur: annotations are prose, `.vale.ini` is config, and nothing tied them together. `scripts/styleclaims` reconciles three sources and fails `make lint`. It also runs in `lint-light`, so the fast pre-commit hook covers docs-only changes: the checker's inputs all live under `docs/.style/`, and leaving it out would let this drift land through the hook that docs changes actually use. It is the only `lint-light` entry that compiles Go, which costs one small build on first run and nothing after. What it reports: 1. `.vale.ini` plus `docs/.style/styles/Coder/`, which is the truth. 2. The per-rule annotations in the style guide subpages. 3. The two coverage tables on the style guide landing page. ### What it reports | Finding | Example it would have caught | |---|---| | A `Coder.*` rule cited as active with no rule file | `Coder.FirstPersonSingular`, `Coder.FirstPersonPlural` | | A third-party rule cited as active while `BasedOnStyles = Coder` | the 8 `Google.*` citations | | A rule file that no section cites as active | a rule landing without the section the per-rule PR pattern requires | | Checks-table severity, scope, or membership drift | a rule re-scoped in `.vale.ini` without updating the table | | Coverage-count drift | the per-section and total rows | ### Also in this PR - One annotation reworded so each planned citation is marked individually (`Google.Gender` and `Google.GenderBias`). A trailing `(both planned)` covering several citations isn't machine-readable, and the checker depends on that precision. - The annotation format documented in `scripts/styleclaims/README.md`, with the per-rule PR pattern in `docs/.style/README.md` pointing at it. - `make lint/prose` prints a coverage pointer after running, so the advisory result doesn't read as a pass. ## Verification - `go test ./scripts/styleclaims/` covers the parser and each finding class, including the mixed `scripts/check_emdash.sh` plus `Coder.EmDash` (planned) annotation and the "rule mentioned but not claimed" case that `Google.Passive` represents. - `go run ./scripts/styleclaims` is clean on the branch. - Regression check: reverting `Coder.FirstPersonSingular` to its pre-PR-1 wording reproduces 3 findings (the false claim plus the 2 count rows). Restoring it returns to clean. - `golangci-lint run scripts/styleclaims/...` reports 0 issues; `gofmt` clean. - `make lint/markdown`, `make lint/emdash` clean. <details> <summary>Design notes</summary> ### Why Go, not bash `scripts/` is mostly bash, but this parses Markdown tables and multi-line annotations, which bash does badly. `scripts/docshtmlcheck` sets the precedent for a Go checker invoked from a lint target, so this follows its layout: `main.go`, `main_test.go`, `README.md`, and a `go run ./scripts/...` target. ### Why not generate the tables Considered and rejected. Generation and checking need the same parse, so once the checker exists the table's correctness is guaranteed either way. Generation would only save a one-row edit in a PR that already edits the guide by doctrine, at the cost of a generated block inside prose contributors are invited to edit. The checker also catches a case generation can't: a rule enabled in `.vale.ini` with no style guide section at all. ### Parsing tradeoff The checker reads prose annotations, so it depends on their format staying regular: start with a recognized keyword, and mark each planned citation individually. That regularity is enforced by the checker itself, which is the point, but it does mean an author who invents a new annotation phrasing gets a failure rather than silent acceptance. If that proves annoying in practice, the fallback is a small YAML manifest of rule status that both the tables and the checker read. I'd rather not add that until the prose form actually hurts. ### Scope validation Scope cells are checked against `.vale.ini` section globs: every glob that sets a rule to `YES` must appear in the cell, and a rule with no `NO` toggle must claim `docs/**`. That's how the `OneSentencePerLine` row stays honest about running on contributor docs only. </details> --- Linear: [DOCS-887](https://linear.app/codercom/issue/DOCS-887/make-clear-that-vale-passing-is-not-style-guide-conformance) 🤖 Generated with Coder Agents on behalf of @nickvigilante.
This commit is contained in:
Nick Vigilante
@@ -738,11 +738,11 @@ endif
|
||||
# GitHub Actions linters are run in a separate CI job (lint-actions) that only
|
||||
# triggers when workflow files change, so we skip them here when CI=true.
|
||||
LINT_ACTIONS_TARGETS := $(if $(CI),,lint/actions/actionlint)
|
||||
lint: lint/shellcheck lint/go lint/ts lint/examples lint/helm lint/site-icons lint/markdown lint/docs-html lint/check-scopes lint/migrations lint/bootstrap lint/architecture lint/emdash lint/agents lint/mise-versions $(LINT_ACTIONS_TARGETS)
|
||||
lint: lint/shellcheck lint/go lint/ts lint/examples lint/helm lint/site-icons lint/markdown lint/docs-html lint/style-claims lint/check-scopes lint/migrations lint/bootstrap lint/architecture lint/emdash lint/agents lint/mise-versions $(LINT_ACTIONS_TARGETS)
|
||||
.PHONY: lint
|
||||
|
||||
# Fast lint subset for lightweight hooks. Some targets use mise-managed tools.
|
||||
lint-light: lint/shellcheck lint/markdown lint/helm lint/bootstrap lint/migrations lint/actions/actionlint lint/typos lint/emdash lint/mise-versions
|
||||
lint-light: lint/shellcheck lint/markdown lint/helm lint/bootstrap lint/migrations lint/actions/actionlint lint/typos lint/emdash lint/style-claims lint/mise-versions
|
||||
.PHONY: lint-light
|
||||
|
||||
lint/site-icons:
|
||||
@@ -791,6 +791,17 @@ lint/docs-html:
|
||||
go run ./scripts/docshtmlcheck
|
||||
.PHONY: lint/docs-html
|
||||
|
||||
# Fails when the style guide claims a prose rule is enforced by tooling that is
|
||||
# not enabled, when an enabled rule has no style guide section, or when the
|
||||
# coverage tables on the style guide landing page drift from the annotations.
|
||||
# Vale checks a subset of the guide and runs advisory, so the guide's own claims
|
||||
# about what is enforced are the only signal an author has; this keeps them
|
||||
# true. See scripts/styleclaims/README.md.
|
||||
lint/style-claims:
|
||||
echo "--- check docs style guide enforcement claims"
|
||||
go run ./scripts/styleclaims
|
||||
.PHONY: lint/style-claims
|
||||
|
||||
lint/architecture:
|
||||
./scripts/check_architecture.sh
|
||||
.PHONY: lint/architecture
|
||||
@@ -875,6 +886,8 @@ docs/.style/.vale-synced: .vale.ini
|
||||
lint/prose: docs/.style/.vale-synced
|
||||
@echo "$(GREEN)==>$(RESET) $(BOLD)lint/prose$(RESET)"
|
||||
mise exec "aqua:errata-ai/vale" -- vale --no-exit docs/
|
||||
@echo "$(GREEN)==>$(RESET) Vale checks a subset of the style guide and never fails this target."
|
||||
@echo " Coverage: docs/.style/style-guide/README.md#what-the-tooling-checks-and-what-it-doesnt"
|
||||
.PHONY: lint/prose
|
||||
|
||||
# pre-commit and pre-push mirror CI checks locally.
|
||||
|
||||
@@ -78,6 +78,9 @@ The PR that adds a rule is the rule's complete unit:
|
||||
The cleanup ships in the same PR as the enable, ordered first.
|
||||
2. **Enable commit**: add the rule to `.vale.ini` at its chosen severity, write a corresponding section under the matching subpage of `docs/.style/style-guide/`, and add the custom rule YAML under `docs/.style/styles/Coder/` if applicable.
|
||||
The rule's `message:` field points at the relevant style-guide subpage anchor.
|
||||
Update the coverage tables on [`style-guide/README.md`](style-guide/README.md) in the same commit.
|
||||
`make lint/style-claims` fails when a rule lands without its section, when the tables disagree with `.vale.ini`, or when an annotation claims enforcement that doesn't exist.
|
||||
Refer to [`scripts/styleclaims/README.md`](../../scripts/styleclaims/README.md) for the annotation format the checker reads, including how to mark a planned rule.
|
||||
|
||||
Severity is a deliberate per-rule choice:
|
||||
|
||||
|
||||
@@ -83,7 +83,7 @@ Avoid `he or she`, `(s)he`, and similar constructions.
|
||||
|
||||
> When a user opens a workspace, he or she connects to the agent over a Tailscale tunnel.
|
||||
|
||||
*Enforced by `Google.Gender` and `Google.GenderBias` (planned; third-party rules aren't loaded by default).*
|
||||
*Enforced by `Google.Gender` (planned) and `Google.GenderBias` (planned); third-party rules aren't loaded by default.*
|
||||
|
||||
## Inclusive-language substitutions
|
||||
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# styleclaims
|
||||
|
||||
`styleclaims` fails CI when the Coder documentation style guide claims a prose rule is enforced by tooling that isn't actually enabled.
|
||||
It runs as `make lint/style-claims` (part of `make lint` and `make lint-light`).
|
||||
|
||||
## Why it exists
|
||||
|
||||
Each rule section in [`docs/.style/style-guide/`](../../docs/.style/style-guide/README.md) ends with an annotation naming what enforces it.
|
||||
Those annotations drifted from the configuration: 8 cited third-party `Google.*` rules that the repo-root `.vale.ini` never loads, and 2 cited `Coder.*` rules that have no YAML under `docs/.style/styles/Coder/`.
|
||||
A reader who trusts an annotation believes a rule is checked when it isn't, which is worse than an unenforced rule the guide is honest about.
|
||||
|
||||
## What it catches
|
||||
|
||||
- **A `Coder.*` rule cited as active with no rule file.** The rule doesn't exist; the citation needs `(planned)`.
|
||||
- **A third-party rule cited as active.** `.vale.ini` sets `BasedOnStyles = Coder`, and third-party rules return one per PR after their corpus is clean, so any `Google.*`, `alex.*`, or `write-good.*` citation is planned until that happens.
|
||||
- **A rule with no style guide section.** A rule file exists but no annotation cites it as active, so a rule landed without the section the per-rule PR pattern requires.
|
||||
- **Checks-table drift.** A severity that disagrees with the rule file's `level:`, a row for a rule that doesn't exist, a missing row for a rule that does, or a scope cell that doesn't read the way `.vale.ini` configures the rule. The scope check compares the rendered text, not the set of paths, so a cell that says "and" where the config says "except" is caught along with a cell that omits or invents a path.
|
||||
- **A rule cited as enforcement while `.vale.ini` leaves it off.** A rule file can exist and still run nowhere.
|
||||
- **Coverage-table drift.** A per-section or total count that disagrees with the annotations.
|
||||
- **A rule heading with no footer.** A `##` or `###` heading that states a rule but carries no annotation, so the coverage table can't count it.
|
||||
|
||||
The checker verifies a `Coder.*` citation against the rule files, so a typo is caught.
|
||||
A third-party citation is verified only against the loaded styles: those rule files are vendored rather than kept in this repository, so the guarantee is weaker on that side.
|
||||
|
||||
## The annotation format it relies on
|
||||
|
||||
An annotation starts with one of `Enforced`, `Documentation-only`, `Adapted`, or `Vale rule`, and runs to the line ending in `*`.
|
||||
|
||||
A citation counts as a claim only when its sentence contains `enforc` or names a `Vale rule`.
|
||||
A sentence that mentions a rule without claiming it runs, such as the note explaining why `Google.Passive` stays out of the package, is ignored.
|
||||
|
||||
Mark each planned citation individually.
|
||||
`(planned)` after the citation, or `Planned Vale rule` before it, both work:
|
||||
|
||||
```md
|
||||
*Enforced by `Coder.BrandNames`.*
|
||||
*Enforced by `Coder.LearnMore` (planned).*
|
||||
*Enforced by `Google.Gender` (planned) and `Google.GenderBias` (planned).*
|
||||
*Enforced by `scripts/check_emdash.sh` (existing CI script) and `Coder.EmDash` (planned).*
|
||||
*Documentation-only. No Vale rule.*
|
||||
```
|
||||
|
||||
A trailing clause such as `(both planned)` that covers several citations at once isn't understood; mark each one.
|
||||
|
||||
A marker that appears before any citation introduces the list that follows, as in "Planned Vale rules" followed by names, and it carries until a later clause re-asserts enforcement.
|
||||
So "Planned `A`, enforced by `B`." reads `A` planned and `B` active.
|
||||
|
||||
## Coverage counts
|
||||
|
||||
Each annotation is classified once for the coverage table, so the table counts rule sections rather than linter rule names.
|
||||
One annotation footer is one rule section: a heading without a footer, such as an example, a navigation heading, or the out-of-scope Color contrast section, isn't a rule of the guide and isn't counted.
|
||||
|
||||
The checker also reports a `##` or `###` heading that states a rule but carries no footer, so a rule can't drop out of the table by never being annotated.
|
||||
A footer belongs to the nearest heading above it, and a heading that has sub-headings is treated as covered by them, so a parent rule whose footer sits after a scoping sub-heading isn't flagged.
|
||||
A heading is also exempt when it is the `Learn more` navigation section, or when it carries a note that opens with `Out of scope` or `Not a rule`, which is how a section that organizes a page without stating a rule declares itself.
|
||||
Headings deeper than `###` are detail inside a section, not rules.
|
||||
A Markdown file that has no footers and no coverage row, such as `editor-setup.md`, isn't a rule section and isn't scanned.
|
||||
|
||||
- **Documentation-only**: the annotation declares the section `Documentation-only`. This wins over everything else, so a section that only cross-references a rule another section owns counts under that other section and no rule is counted twice. The match is case-sensitive; a lowercase "documentation-only" describes part of a section, not the section.
|
||||
- **Tool-checked**: cites as active an enabled Coder rule, a third-party rule whose style `.vale.ini` loads, `markdownlint`, an `MD###` rule, or `scripts/check_emdash.sh`. A Coder rule that exists but is disabled everywhere doesn't count: it checks no file.
|
||||
- **Planned**: cites only planned checkers.
|
||||
|
||||
## Usage
|
||||
|
||||
```sh
|
||||
go run ./scripts/styleclaims
|
||||
```
|
||||
|
||||
It takes no arguments and runs from the repository root.
|
||||
Findings print as `file:line: message`, or as `file: message` when the problem is the file as a whole, such as a rule with no style guide section or a missing coverage row.
|
||||
Any finding exits 1.
|
||||
|
||||
The set of style guide sections comes from the directory, not from a list in the source.
|
||||
A Markdown file is a rule section when it carries at least one enforcement footer, or when the coverage table gives it a row, so a new subpage is validated as soon as either is true.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,924 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestParseValeConfig(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
src := `
|
||||
StylesPath = docs/.style/styles
|
||||
MinAlertLevel = suggestion
|
||||
|
||||
[*.md]
|
||||
BasedOnStyles = Coder
|
||||
|
||||
Coder.OneSentencePerLine = NO
|
||||
|
||||
[docs/.style/*.md]
|
||||
Coder.OneSentencePerLine = YES
|
||||
|
||||
[docs/.style/style-guide/**]
|
||||
BasedOnStyles =
|
||||
`
|
||||
styles, sectionStyles, toggles := parseValeConfig(src)
|
||||
if len(styles) != 1 || styles[0] != "Coder" {
|
||||
t.Fatalf("styles = %v, want [Coder]", styles)
|
||||
}
|
||||
got := toggles["Coder.OneSentencePerLine"]
|
||||
if got["*.md"] != "NO" {
|
||||
t.Errorf("default toggle = %q, want NO", got["*.md"])
|
||||
}
|
||||
if got["docs/.style/*.md"] != "YES" {
|
||||
t.Errorf("scoped toggle = %q, want YES", got["docs/.style/*.md"])
|
||||
}
|
||||
if loaded, ok := sectionStyles["docs/.style/style-guide/**"]; !ok || len(loaded) != 0 {
|
||||
t.Errorf("sectionStyles[style-guide] = %v, want an empty style list", loaded)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewRuleScope(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
toggles := map[string]map[string]string{
|
||||
"Coder.OneSentencePerLine": {"*.md": "NO", "docs/.style/*.md": "YES"},
|
||||
}
|
||||
sectionStyles := map[string][]string{"*.md": {"Coder"}, "docs/.style/style-guide/**": nil}
|
||||
scoped := newRule("OneSentencePerLine", "extends: existence\nlevel: warning\n", sectionStyles, toggles)
|
||||
if scoped.defaultOn {
|
||||
t.Error("rule disabled in the catch-all section should not be on by default")
|
||||
}
|
||||
if len(scoped.enabledIn) != 1 || scoped.enabledIn[0] != "docs/.style/*.md" {
|
||||
t.Errorf("enabledIn = %v, want [docs/.style/*.md]", scoped.enabledIn)
|
||||
}
|
||||
if scoped.severity != "warning" {
|
||||
t.Errorf("severity = %q, want warning", scoped.severity)
|
||||
}
|
||||
|
||||
global := newRule("BrandNames", "level: error\n", sectionStyles, toggles)
|
||||
if !global.defaultOn {
|
||||
t.Error("rule with no toggle should be on by default")
|
||||
}
|
||||
if len(global.disabledIn) != 1 || global.disabledIn[0] != "docs/.style/style-guide/**" {
|
||||
t.Errorf("disabledIn = %v, want [docs/.style/style-guide/**]", global.disabledIn)
|
||||
}
|
||||
}
|
||||
|
||||
func TestActiveCitations(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
text string
|
||||
wantActive []string
|
||||
wantPlanned []string
|
||||
}{
|
||||
{
|
||||
name: "plain claim",
|
||||
text: "*Enforced by `Coder.BrandNames`.*",
|
||||
wantActive: []string{"Coder.BrandNames"},
|
||||
},
|
||||
{
|
||||
name: "planned suffix",
|
||||
text: "*Enforced by `Coder.LearnMore` (planned).*",
|
||||
wantPlanned: []string{"Coder.LearnMore"},
|
||||
},
|
||||
{
|
||||
name: "planned prefix",
|
||||
text: "*Documentation-only. Planned Vale rule `Coder.Idioms`.*",
|
||||
wantPlanned: []string{"Coder.Idioms"},
|
||||
},
|
||||
{
|
||||
name: "mixed active and planned in one sentence",
|
||||
text: "*Enforced by `scripts/check_emdash.sh` (existing CI script) and `Coder.EmDash` (planned).*",
|
||||
wantActive: []string{"scripts/check_emdash.sh"},
|
||||
wantPlanned: []string{"Coder.EmDash"},
|
||||
},
|
||||
{
|
||||
name: "rule named without a claim is ignored",
|
||||
text: "*Documentation-only. No Vale rule. Imprecise rules like `Google.Passive` and `write-good.Passive` fire on every passive construction, so they stay out of the package.*",
|
||||
},
|
||||
{
|
||||
name: "claim after the citation",
|
||||
text: "*Documentation-only. No Vale rule for the general principle; `Coder.SelectClick` enforces the specific case.*",
|
||||
wantActive: []string{"Coder.SelectClick"},
|
||||
},
|
||||
{
|
||||
name: "vale rule form",
|
||||
text: "*Vale rule: `Coder.OneSentencePerLine` (warning).*",
|
||||
wantActive: []string{"Coder.OneSentencePerLine"},
|
||||
},
|
||||
{
|
||||
name: "each citation marked separately",
|
||||
text: "*Enforced by `Google.Gender` (planned) and `Google.GenderBias` (planned); third-party rules aren't loaded by default.*",
|
||||
wantPlanned: []string{"Google.Gender", "Google.GenderBias"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
active, planned := activeCitations(tc.text)
|
||||
assertSame(t, "active", active, tc.wantActive)
|
||||
assertSame(t, "planned", planned, tc.wantPlanned)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassify(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
rules := map[string]bool{"BrandNames": true}
|
||||
cases := []struct {
|
||||
text string
|
||||
want class
|
||||
}{
|
||||
{"*Enforced by `Coder.BrandNames`.*", classTool},
|
||||
{"*Enforced by `markdownlint` rule `MD040` for the missing-language case.*", classTool},
|
||||
{"*Enforced by `Coder.LearnMore` (planned).*", classPlanned},
|
||||
{"*Documentation-only. No Vale rule.*", classDocumentationOnly},
|
||||
{"*Adapted from ASD-STE100 Issue 9, rule 5.2. Documentation-only. No Vale rule.*", classDocumentationOnly},
|
||||
// A rule cited as active that does not exist is not tool coverage.
|
||||
{"*Enforced by `Coder.FirstPersonSingular`.*", classDocumentationOnly},
|
||||
// A documentation-only section that cross-references a rule another
|
||||
// section owns stays documentation-only, so the rule is counted once.
|
||||
{"*Documentation-only. No Vale rule for the general principle; `Coder.BrandNames` enforces one case.*", classDocumentationOnly},
|
||||
// A documentation-only section that names a planned rule is still
|
||||
// documentation-only.
|
||||
{"*Documentation-only. Planned Vale rule `Coder.Idioms`.*", classDocumentationOnly},
|
||||
// A lowercase mention qualifies part of a section, not the section.
|
||||
{"*Enforced by `markdownlint` rule `MD001`. The nested rule is documentation-only.*", classTool},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := classify(tc.text, rules, map[string]bool{"Coder": true}); got != tc.want {
|
||||
t.Errorf("classify(%q) = %v, want %v", tc.text, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
src := strings.Join([]string{
|
||||
"## A rule",
|
||||
"",
|
||||
"Body prose.",
|
||||
"",
|
||||
"*Enforced by `Coder.BrandNames`.*",
|
||||
"",
|
||||
"## Another rule",
|
||||
"",
|
||||
"*Adapted from ASD-STE100 Issue 9, rule 5.2.",
|
||||
"Documentation-only.",
|
||||
"No Vale rule.*",
|
||||
"",
|
||||
}, "\n")
|
||||
|
||||
got := parseAnnotations("page.md", src)
|
||||
if len(got) != 2 {
|
||||
t.Fatalf("parsed %d annotations, want 2", len(got))
|
||||
}
|
||||
if got[0].line != 5 {
|
||||
t.Errorf("first annotation line = %d, want 5", got[0].line)
|
||||
}
|
||||
if !strings.HasSuffix(got[1].text, "No Vale rule.*") {
|
||||
t.Errorf("multi-line annotation not joined: %q", got[1].text)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckCatchesFalseClaims covers the two regressions this checker exists to
|
||||
// prevent: a Coder rule cited as active with no rule file, and a third-party
|
||||
// rule cited as active while .vale.ini loads only the Coder package.
|
||||
func TestCheckCatchesFalseClaims(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
rules := []valeRule{{name: "BrandNames", severity: "error", defaultOn: true}}
|
||||
annotations := map[string][]annotation{
|
||||
"voice-and-tone.md": {
|
||||
{file: "voice-and-tone.md", line: 10, text: "*Enforced by `Coder.FirstPersonSingular`.*"},
|
||||
{file: "voice-and-tone.md", line: 20, text: "*Enforced by `Google.OxfordComma`.*"},
|
||||
{file: "voice-and-tone.md", line: 30, text: "*Enforced by `Coder.BrandNames`.*"},
|
||||
},
|
||||
}
|
||||
landing := checksTable("`Coder.BrandNames`", "`error`", "`docs/**`") + coverageTable(map[string][4]int{
|
||||
"voice-and-tone.md": {3, 1, 0, 2},
|
||||
})
|
||||
|
||||
findings := checkClaims(testPages, []string{"Coder"}, rules, annotations, landing)
|
||||
var msgs []string
|
||||
for _, f := range findings {
|
||||
msgs = append(msgs, f.msg)
|
||||
}
|
||||
joined := strings.Join(msgs, "\n")
|
||||
|
||||
if !strings.Contains(joined, "Coder.FirstPersonSingular") {
|
||||
t.Errorf("missing finding for a rule with no rule file:\n%s", joined)
|
||||
}
|
||||
if !strings.Contains(joined, "Google.OxfordComma") {
|
||||
t.Errorf("missing finding for an unloaded third-party style:\n%s", joined)
|
||||
}
|
||||
for _, m := range msgs {
|
||||
if strings.Contains(m, "Coder.BrandNames") {
|
||||
t.Errorf("unexpected finding for a rule that is enabled: %s", m)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckCatchesTableDrift(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
rules := []valeRule{{name: "BrandNames", severity: "error", defaultOn: true}}
|
||||
annotations := map[string][]annotation{
|
||||
"voice-and-tone.md": {{file: "voice-and-tone.md", line: 10, text: "*Enforced by `Coder.BrandNames`.*"}},
|
||||
}
|
||||
|
||||
t.Run("severity mismatch", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
landing := checksTable("`Coder.BrandNames`", "`warning`", "`docs/**`") +
|
||||
coverageTable(map[string][4]int{"voice-and-tone.md": {1, 1, 0, 0}})
|
||||
if !containsMsg(checkClaims(testPages, []string{"Coder"}, rules, annotations, landing), "severity") {
|
||||
t.Error("severity drift not reported")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("count mismatch", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
landing := checksTable("`Coder.BrandNames`", "`error`", "`docs/**`") +
|
||||
coverageTable(map[string][4]int{"voice-and-tone.md": {9, 9, 9, 9}})
|
||||
if !containsMsg(checkClaims(testPages, []string{"Coder"}, rules, annotations, landing), "annotations give") {
|
||||
t.Error("coverage drift not reported")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rule missing from the checks table", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
landing := coverageTable(map[string][4]int{"voice-and-tone.md": {1, 1, 0, 0}})
|
||||
if !containsMsg(checkClaims(testPages, []string{"Coder"}, rules, annotations, landing), "missing from the checks table") {
|
||||
t.Error("unlisted rule not reported")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rule with no style guide section", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
extra := []valeRule{rules[0], {name: "Orphan", severity: "warning", defaultOn: true}}
|
||||
landing := checksTable("`Coder.BrandNames`", "`error`", "`docs/**`") +
|
||||
checksTable("`Coder.Orphan`", "`warning`", "`docs/**`") +
|
||||
coverageTable(map[string][4]int{"voice-and-tone.md": {1, 1, 0, 0}})
|
||||
if !containsMsg(checkClaims(testPages, []string{"Coder"}, extra, annotations, landing), "no style guide section cites it") {
|
||||
t.Error("undocumented rule not reported")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// testPages is the section set the table helpers build rows for. Production
|
||||
// discovers this set from disk; the tests pin it so the fixtures stay small.
|
||||
var testPages = []string{"voice-and-tone.md"}
|
||||
|
||||
func checksTable(name, severity, scope string) string {
|
||||
return "| " + name + " | Vale | " + severity + " | " + scope + " | What it catches |\n"
|
||||
}
|
||||
|
||||
func coverageTable(counts map[string][4]int) string {
|
||||
var rows []string
|
||||
var total [4]int
|
||||
for _, page := range testPages {
|
||||
c := counts[page]
|
||||
for i := range total {
|
||||
total[i] += c[i]
|
||||
}
|
||||
rows = append(rows, "| [Section](./"+page+") | "+
|
||||
itoa(c[0])+" | "+itoa(c[1])+" | "+itoa(c[2])+" | "+itoa(c[3])+" |")
|
||||
}
|
||||
rows = append(rows, "| **Total** | "+itoa(total[0])+" | "+itoa(total[1])+" | "+itoa(total[2])+" | "+itoa(total[3])+" |")
|
||||
return strings.Join(rows, "\n") + "\n"
|
||||
}
|
||||
|
||||
func itoa(n int) string { return strconv.Itoa(n) }
|
||||
|
||||
func containsMsg(findings []finding, substr string) bool {
|
||||
return slices.ContainsFunc(findings, func(f finding) bool {
|
||||
return strings.Contains(f.msg, substr)
|
||||
})
|
||||
}
|
||||
|
||||
func assertSame(t *testing.T, label string, got, want []string) {
|
||||
t.Helper()
|
||||
if !slices.Equal(got, want) {
|
||||
t.Errorf("%s = %v, want %v", label, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckFooterCoverage(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
src := `# Page
|
||||
|
||||
## Grouping heading
|
||||
|
||||
### Footered sub-rule
|
||||
|
||||
*Documentation-only.
|
||||
No Vale rule.*
|
||||
|
||||
## Rule without a footer
|
||||
|
||||
Prose that states a rule.
|
||||
|
||||
## Example inside a fence
|
||||
|
||||
` + "```md\n## Not a real heading\n```" + `
|
||||
|
||||
*Documentation-only.
|
||||
No Vale rule.*
|
||||
|
||||
## Color contrast
|
||||
|
||||
*Out of scope for this guide.
|
||||
Tracked by the docs site theme.*
|
||||
|
||||
## Personas
|
||||
|
||||
#### Perry the Platform Engineer
|
||||
|
||||
Reference detail, not a rule.
|
||||
|
||||
## Learn more
|
||||
|
||||
- [Voice and tone](./voice-and-tone.md)
|
||||
`
|
||||
|
||||
findings := checkFooterCoverage("page.md", src)
|
||||
if len(findings) != 1 {
|
||||
t.Fatalf("checkFooterCoverage() = %v, want 1 finding", findings)
|
||||
}
|
||||
if !strings.Contains(findings[0].msg, "Rule without a footer") {
|
||||
t.Errorf("finding = %q, want the unfootered rule heading", findings[0].msg)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckScopeCell(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
global := valeRule{name: "BrandNames", severity: "error", defaultOn: true, disabledIn: []string{"docs/.style/style-guide/**"}}
|
||||
scoped := valeRule{
|
||||
name: "OneSentencePerLine",
|
||||
severity: "warning",
|
||||
enabledIn: []string{"docs/.style/*.md", "docs/.style/styles/Coder/*.md"},
|
||||
}
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
rule valeRule
|
||||
cell string
|
||||
want bool
|
||||
}{
|
||||
{
|
||||
name: "global rule names the disable as an exception",
|
||||
rule: global,
|
||||
cell: "`docs/**` except `docs/.style/style-guide/**`",
|
||||
},
|
||||
{
|
||||
name: "global rule omits the disable",
|
||||
rule: global,
|
||||
cell: "`docs/**`",
|
||||
want: true,
|
||||
},
|
||||
{
|
||||
name: "global rule inverts the polarity",
|
||||
rule: global,
|
||||
cell: "`docs/**` and `docs/.style/style-guide/**`",
|
||||
want: true,
|
||||
},
|
||||
{
|
||||
name: "scoped rule names both enabling globs",
|
||||
rule: scoped,
|
||||
cell: "`docs/.style/*.md` and `docs/.style/styles/Coder/*.md` only",
|
||||
},
|
||||
{
|
||||
name: "scoped rule over-claims the whole docs tree",
|
||||
rule: scoped,
|
||||
cell: "`docs/**`",
|
||||
want: true,
|
||||
},
|
||||
{
|
||||
name: "scoped rule omits an enabling glob",
|
||||
rule: scoped,
|
||||
cell: "`docs/.style/*.md` only",
|
||||
want: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
findings := checkScopeCell("README.md", 1, tc.cell, tc.rule)
|
||||
if got := len(findings) > 0; got != tc.want {
|
||||
t.Errorf("%s: findings = %v, want a finding: %v", tc.name, findings, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadRulesSkipsDemoRules(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := t.TempDir()
|
||||
for name, body := range map[string]string{
|
||||
"BrandNames.yml": "extends: substitution\nlevel: error\n",
|
||||
"DemoError.yml": "extends: existence\nlevel: error\n",
|
||||
"DemoWarning.yml": "extends: existence\nlevel: warning\n",
|
||||
"notes.txt": "not a rule",
|
||||
} {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
rules, err := loadRules(dir, map[string][]string{"*.md": {"Coder"}}, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(rules) != 1 {
|
||||
t.Fatalf("loadRules() = %v, want only the non-demo rule", rules)
|
||||
}
|
||||
if rules[0].name != "BrandNames" || rules[0].severity != "error" {
|
||||
t.Errorf("rule = %+v, want BrandNames at error", rules[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotationsIgnoresFencedExamples(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
src := "## A rule\n\n" +
|
||||
"Write the footer like this:\n\n" +
|
||||
"```md\n*Enforced by `Coder.RuleThatOnlyExistsInThisExample`.*\n```\n\n" +
|
||||
"*Documentation-only.\nNo Vale rule.*\n"
|
||||
|
||||
got := parseAnnotations("page.md", src)
|
||||
if len(got) != 1 {
|
||||
t.Fatalf("parseAnnotations() = %v, want only the real footer", got)
|
||||
}
|
||||
if strings.Contains(got[0].text, "RuleThatOnlyExistsInThisExample") {
|
||||
t.Errorf("annotation = %q, want the footer outside the fence", got[0].text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSubpagesDiscoversAnnotatedPages(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := t.TempDir()
|
||||
for name, body := range map[string]string{
|
||||
"README.md": "## Coverage\n\n*Documentation-only.\nNo Vale rule.*\n",
|
||||
"word-choice.md": "## A rule\n\n*Enforced by `Coder.BrandNames`.*\n",
|
||||
"editor-setup.md": "## How to configure your editor\n\nNo rules here.\n",
|
||||
} {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
pages, annotations, sources, err := subpages(dir, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(annotations["word-choice.md"]) != 1 {
|
||||
t.Errorf("annotations = %v, want the page's one footer", annotations["word-choice.md"])
|
||||
}
|
||||
if len(pages) != 1 || pages[0] != "word-choice.md" {
|
||||
t.Fatalf("subpages() = %v, want only the annotated page", pages)
|
||||
}
|
||||
if !strings.Contains(sources["word-choice.md"], "Coder.BrandNames") {
|
||||
t.Errorf("sources missing the page body: %q", sources["word-choice.md"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckCoverageTableReportsMalformedRow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
landing := "| [Section](voice-and-tone.md) | 1 | 1 | 0 | 0 |\n| **Total** | 1 | 1 | 0 | 0 |\n"
|
||||
findings := checkCoverageTable(testPages, landing, map[string][4]int{"voice-and-tone.md": {1, 1, 0, 0}})
|
||||
if !containsMsg(findings, "no link to a style guide section") {
|
||||
t.Errorf("findings = %v, want a malformed-row finding", findings)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckCoverageTableReportsUnknownSection(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
landing := "| [Section](./not-a-section.md) | 1 | 1 | 0 | 0 |\n" +
|
||||
coverageTable(map[string][4]int{"voice-and-tone.md": {1, 1, 0, 0}})
|
||||
findings := checkCoverageTable(testPages, landing, map[string][4]int{"voice-and-tone.md": {1, 1, 0, 0}})
|
||||
if !containsMsg(findings, "not a style guide section") {
|
||||
t.Errorf("findings = %v, want an unknown-section finding", findings)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckChecksTableReportsUnknownRule(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
landing := checksTable("`Coder.NotARule`", "`error`", "`docs/**`")
|
||||
if !containsMsg(checkChecksTable(landing, nil), "no rule file under") {
|
||||
t.Error("a table row for a nonexistent rule was not reported")
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckClaimsCountsPlannedAnnotations drives a planned annotation through
|
||||
// the reconciler, so the planned column of the coverage table is exercised end
|
||||
// to end rather than only in the classifier.
|
||||
func TestCheckClaimsCountsPlannedAnnotations(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
rules := []valeRule{{name: "BrandNames", severity: "error", defaultOn: true}}
|
||||
annotations := map[string][]annotation{
|
||||
"voice-and-tone.md": {
|
||||
{file: "voice-and-tone.md", line: 10, text: "*Enforced by `Coder.BrandNames`.*"},
|
||||
{file: "voice-and-tone.md", line: 20, text: "*Enforced by `Coder.LearnMore` (planned).*"},
|
||||
{file: "voice-and-tone.md", line: 30, text: "*Documentation-only. No Vale rule.*"},
|
||||
},
|
||||
}
|
||||
landing := checksTable("`Coder.BrandNames`", "`error`", "`docs/**`") +
|
||||
coverageTable(map[string][4]int{"voice-and-tone.md": {3, 1, 1, 1}})
|
||||
|
||||
if findings := checkClaims(testPages, []string{"Coder"}, rules, annotations, landing); len(findings) != 0 {
|
||||
t.Fatalf("checkClaims() = %v, want no findings for a table that matches", findings)
|
||||
}
|
||||
|
||||
drifted := checksTable("`Coder.BrandNames`", "`error`", "`docs/**`") +
|
||||
coverageTable(map[string][4]int{"voice-and-tone.md": {3, 1, 0, 2}})
|
||||
if !containsMsg(checkClaims(testPages, []string{"Coder"}, rules, annotations, drifted), "annotations give") {
|
||||
t.Error("a planned annotation counted as documentation-only was not reported")
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunReconcilesTheRepository exercises the whole wiring, from .vale.ini and
|
||||
// the rule files through page discovery to the landing page tables, against the
|
||||
// real style guide. It is the test that fails if the repository and the
|
||||
// checker disagree.
|
||||
//
|
||||
//nolint:paralleltest // t.Chdir cannot be used in a parallel test.
|
||||
func TestRunReconcilesTheRepository(t *testing.T) {
|
||||
t.Chdir("../..")
|
||||
|
||||
findings, err := run()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, f := range findings {
|
||||
t.Errorf("%s:%d: %s", f.file, f.line, f.msg)
|
||||
}
|
||||
}
|
||||
|
||||
// TestScopeText pins every shape the rule model can take, including the ones
|
||||
// no rule in .vale.ini has today, so a future config change meets a test rather
|
||||
// than a surprise.
|
||||
func TestScopeText(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
rule valeRule
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "on everywhere",
|
||||
rule: valeRule{defaultOn: true},
|
||||
want: "`docs/**`",
|
||||
},
|
||||
{
|
||||
name: "on everywhere except one section",
|
||||
rule: valeRule{defaultOn: true, disabledIn: []string{"docs/.style/style-guide/**"}},
|
||||
want: "`docs/**` except `docs/.style/style-guide/**`",
|
||||
},
|
||||
{
|
||||
name: "off by default, on in two sections",
|
||||
rule: valeRule{enabledIn: []string{"docs/a/*.md", "docs/b/*.md"}},
|
||||
want: "`docs/a/*.md` and `docs/b/*.md` only",
|
||||
},
|
||||
{
|
||||
name: "a single-directory glob is not reached by a subdirectory disable",
|
||||
rule: valeRule{enabledIn: []string{"docs/.style/*.md"}, disabledIn: []string{"docs/.style/style-guide/**"}},
|
||||
want: "`docs/.style/*.md` only",
|
||||
},
|
||||
{
|
||||
name: "a recursive glob is reached by a disable inside it",
|
||||
rule: valeRule{enabledIn: []string{"docs/admin/**"}, disabledIn: []string{"docs/admin/generated/**"}},
|
||||
want: "`docs/admin/**` except `docs/admin/generated/**`, and nowhere else",
|
||||
},
|
||||
{
|
||||
name: "off everywhere",
|
||||
rule: valeRule{},
|
||||
want: "nowhere",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
if got := tc.rule.scopeText(); got != tc.want {
|
||||
t.Errorf("%s: scopeText() = %q, want %q", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckClaimsRejectsDisabledRule covers the enablement gate: a rule file can
|
||||
// exist and still run nowhere, and a section must not cite it as enforcement.
|
||||
func TestCheckClaimsRejectsDisabledRule(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
rules := []valeRule{{name: "BrandNames", severity: "error"}}
|
||||
annotations := map[string][]annotation{
|
||||
"voice-and-tone.md": {{file: "voice-and-tone.md", line: 10, text: "*Enforced by `Coder.BrandNames`.*"}},
|
||||
}
|
||||
landing := coverageTable(map[string][4]int{"voice-and-tone.md": {1, 0, 0, 1}})
|
||||
|
||||
findings := checkClaims(testPages, []string{"Coder"}, rules, annotations, landing)
|
||||
if !containsMsg(findings, "leaves it off everywhere") {
|
||||
t.Errorf("findings = %v, want a finding for a rule that runs nowhere", findings)
|
||||
}
|
||||
// A rule that runs nowhere must not also be demanded in the tables, or the
|
||||
// author is left with no way to satisfy the checker.
|
||||
if containsMsg(findings, "missing from the checks table") {
|
||||
t.Errorf("findings = %v, want no table demand for a disabled rule", findings)
|
||||
}
|
||||
if containsMsg(findings, "no style guide section cites it") {
|
||||
t.Errorf("findings = %v, want no citation demand for a disabled rule", findings)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnfencedTracksTheOpeningDelimiter(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
src := "```md\n~~~\n*Enforced by `Coder.InsideTheFence`.*\n```\n\n*Documentation-only.\nNo Vale rule.*\n"
|
||||
got := parseAnnotations("page.md", src)
|
||||
if len(got) != 1 || strings.Contains(got[0].text, "InsideTheFence") {
|
||||
t.Fatalf("parseAnnotations() = %v, want only the footer outside the fence", got)
|
||||
}
|
||||
|
||||
tilde := "~~~md\n*Enforced by `Coder.InsideTheFence`.*\n~~~\n"
|
||||
if found := parseAnnotations("page.md", tilde); len(found) != 0 {
|
||||
t.Errorf("parseAnnotations() = %v, want nothing from a tilde-fenced example", found)
|
||||
}
|
||||
}
|
||||
|
||||
func TestActiveCitationsSharedPlannedPrefix(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
active, planned := activeCitations("*Planned Vale rules `Coder.One` and `Coder.Two`.*")
|
||||
if len(active) != 0 {
|
||||
t.Errorf("active = %v, want none: the prefix marks both citations planned", active)
|
||||
}
|
||||
assertSame(t, "planned", planned, []string{"Coder.One", "Coder.Two"})
|
||||
|
||||
active, planned = activeCitations("*Enforced by `Coder.One` (planned) and `Coder.Two`.*")
|
||||
assertSame(t, "active", active, []string{"Coder.Two"})
|
||||
assertSame(t, "planned", planned, []string{"Coder.One"})
|
||||
}
|
||||
|
||||
func TestUnfencedRespectsFenceRunLength(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// A 4-backtick block teaching nested fences: the inner 3-backtick lines are
|
||||
// content, and the block closes only on a run at least as long as its
|
||||
// opener.
|
||||
src := "````md\n```sh\n*Enforced by `Coder.InsideTheOuterFence`.*\n```\n````\n\n*Documentation-only.\nNo Vale rule.*\n"
|
||||
got := parseAnnotations("page.md", src)
|
||||
if len(got) != 1 || strings.Contains(got[0].text, "InsideTheOuterFence") {
|
||||
t.Fatalf("parseAnnotations() = %v, want only the footer outside the fence", got)
|
||||
}
|
||||
|
||||
// A fence line carrying an info string opens a block; it never closes one.
|
||||
if found := parseAnnotations("page.md", "```\n```md\n*Enforced by `Coder.StillFenced`.*\n```\n"); len(found) != 0 {
|
||||
t.Errorf("parseAnnotations() = %v, want nothing: the info-string line cannot close the block", found)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckChecksTableRejectsInactiveRow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
landing := checksTable("`Coder.BrandNames`", "`error`", "`docs/**`")
|
||||
rules := []valeRule{{name: "BrandNames", severity: "error"}}
|
||||
findings := checkChecksTable(landing, rules)
|
||||
if !containsMsg(findings, "leaves it off everywhere") {
|
||||
t.Errorf("findings = %v, want a finding for a row that claims a disabled rule runs", findings)
|
||||
}
|
||||
if containsMsg(findings, "severity") || containsMsg(findings, "scope") {
|
||||
t.Errorf("findings = %v, want no severity or scope comparison against a disabled rule", findings)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyUsesEnablementAndLoadedStyles(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
enabled := map[string]bool{"BrandNames": true}
|
||||
loaded := map[string]bool{"Coder": true, "Google": true}
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
text string
|
||||
want class
|
||||
}{
|
||||
{
|
||||
name: "an enabled Coder rule is tool coverage",
|
||||
text: "*Enforced by `Coder.BrandNames`.*",
|
||||
want: classTool,
|
||||
},
|
||||
{
|
||||
name: "a rule file that runs nowhere is not tool coverage",
|
||||
text: "*Enforced by `Coder.SwitchedOff`.*",
|
||||
want: classDocumentationOnly,
|
||||
},
|
||||
{
|
||||
name: "a loaded third-party rule is tool coverage",
|
||||
text: "*Enforced by `Google.OxfordComma`.*",
|
||||
want: classTool,
|
||||
},
|
||||
{
|
||||
name: "an unloaded third-party rule is not",
|
||||
text: "*Enforced by `Microsoft.Foo`.*",
|
||||
want: classDocumentationOnly,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
if got := classify(tc.text, enabled, loaded); got != tc.want {
|
||||
t.Errorf("%s: classify() = %v, want %v", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestActiveCitationsBindsMarkersToClauses(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
text string
|
||||
wantActive []string
|
||||
wantPlanned []string
|
||||
}{
|
||||
{
|
||||
name: "a marker after the second citation leaves the first active",
|
||||
text: "*Enforced by `Coder.One` and `Coder.Two` (planned).*",
|
||||
wantActive: []string{"Coder.One"},
|
||||
wantPlanned: []string{"Coder.Two"},
|
||||
},
|
||||
{
|
||||
name: "a marker after the first citation leaves the second active",
|
||||
text: "*Enforced by `Coder.One` (planned) and `Coder.Two`.*",
|
||||
wantActive: []string{"Coder.Two"},
|
||||
wantPlanned: []string{"Coder.One"},
|
||||
},
|
||||
{
|
||||
name: "an opening marker covers every citation",
|
||||
text: "*Planned Vale rules `Coder.One` and `Coder.Two`.*",
|
||||
wantPlanned: []string{"Coder.One", "Coder.Two"},
|
||||
},
|
||||
{
|
||||
name: "a marker in a trailing clause stays there",
|
||||
text: "*Enforced by `scripts/check_emdash.sh`, with `Coder.EmDash` (planned) to follow.*",
|
||||
wantActive: []string{"scripts/check_emdash.sh"},
|
||||
wantPlanned: []string{"Coder.EmDash"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
active, planned := activeCitations(tc.text)
|
||||
assertSame(t, tc.name+" active", active, tc.wantActive)
|
||||
assertSame(t, tc.name+" planned", planned, tc.wantPlanned)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnderPathMatchesOnSegments(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cases := []struct {
|
||||
glob, prefix string
|
||||
want bool
|
||||
}{
|
||||
{"docs/foo/bar/**", "docs/foo/", true},
|
||||
{"docs/foobar/**", "docs/foo", false},
|
||||
{"docs/foo", "docs/foo", true},
|
||||
{"docs/anything/**", "", true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := underPath(tc.glob, tc.prefix); got != tc.want {
|
||||
t.Errorf("underPath(%q, %q) = %v, want %v", tc.glob, tc.prefix, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestActiveCitationsMixedClauseOrderings(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
text string
|
||||
wantActive []string
|
||||
wantPlanned []string
|
||||
}{
|
||||
{
|
||||
name: "an introducer in a citation-less clause carries",
|
||||
text: "*Planned Vale rules, `Coder.One` and `Coder.Two`.*",
|
||||
wantPlanned: []string{"Coder.One", "Coder.Two"},
|
||||
},
|
||||
{
|
||||
name: "a later clause re-asserting enforcement stops the carry",
|
||||
text: "*Planned `Coder.One`, enforced by `Coder.Two`.*",
|
||||
wantActive: []string{"Coder.Two"},
|
||||
wantPlanned: []string{"Coder.One"},
|
||||
},
|
||||
{
|
||||
name: "a separator inside a parenthetical does not split the clause",
|
||||
text: "*Enforced by `Coder.One` (planned, ships at warning) and `Coder.Two`.*",
|
||||
wantActive: []string{"Coder.Two"},
|
||||
wantPlanned: []string{"Coder.One"},
|
||||
},
|
||||
{
|
||||
name: "a separator inside a code span does not split the clause",
|
||||
text: "*Enforced by `scripts/check_emdash.sh` and `Coder.One`.*",
|
||||
wantActive: []string{"scripts/check_emdash.sh", "Coder.One"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
active, planned := activeCitations(tc.text)
|
||||
assertSame(t, tc.name+" active", active, tc.wantActive)
|
||||
assertSame(t, tc.name+" planned", planned, tc.wantPlanned)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseFenceIgnoresIndentedCode(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
if _, ok := parseFence(" ```"); ok {
|
||||
t.Error("a line indented four spaces is an indented code block, not a fence")
|
||||
}
|
||||
if f, ok := parseFence(" ```"); !ok || f.length != 3 {
|
||||
t.Errorf("parseFence(three spaces) = %+v, %v, want a fence", f, ok)
|
||||
}
|
||||
src := " ```\n*Enforced by `Coder.NotFenced`.*\n"
|
||||
if got := parseAnnotations("page.md", src); len(got) != 1 {
|
||||
t.Errorf("parseAnnotations() = %v, want the footer: the indented line opens no fence", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCarryStopsOnValeRuleWording(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// The sentence gate and the carry stop must speak the same language: a
|
||||
// clause that reasserts enforcement as "Vale rule `X`" stops a carried
|
||||
// marker, exactly as "enforced by" does.
|
||||
active, planned := activeCitations("*Planned Vale rules, `Coder.One`, Vale rule `Coder.Two` flags the plural.*")
|
||||
assertSame(t, "active", active, []string{"Coder.Two"})
|
||||
assertSame(t, "planned", planned, []string{"Coder.One"})
|
||||
}
|
||||
|
||||
func TestSplitClausesGuards(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// A separator inside a code span or a parenthetical must not end a clause.
|
||||
got := splitClauses("Enforced by `scripts/check_emdash.sh, and more` (planned, still) and `Coder.One`")
|
||||
if len(got) != 2 {
|
||||
t.Fatalf("splitClauses() = %q, want two clauses", got)
|
||||
}
|
||||
if !strings.Contains(got[0], "planned, still") {
|
||||
t.Errorf("first clause = %q, want the parenthetical intact", got[0])
|
||||
}
|
||||
if !strings.Contains(got[0], "check_emdash.sh, and more") {
|
||||
t.Errorf("first clause = %q, want the code span intact", got[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseFenceRejectsTabIndentation(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
if _, ok := parseFence("\t```"); ok {
|
||||
t.Error("a tab-indented line is an indented code block, not a fence")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckCoverageAnchor(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
recipe := "\t@echo \" Coverage: docs/.style/style-guide/README.md#what-the-tooling-checks-and-what-it-doesnt\"\n"
|
||||
|
||||
if findings := checkCoverageAnchor(recipe, "## What the tooling checks, and what it doesn't\n"); len(findings) != 0 {
|
||||
t.Errorf("checkCoverageAnchor() = %v, want no findings when the heading matches the recipe", findings)
|
||||
}
|
||||
if findings := checkCoverageAnchor(recipe, "## Something else\n"); len(findings) != 1 {
|
||||
t.Errorf("checkCoverageAnchor() = %v, want a finding when the heading is gone", findings)
|
||||
}
|
||||
// Renaming the heading and the recipe together keeps the pointer honest.
|
||||
renamed := "\t@echo \"docs/.style/style-guide/README.md#what-runs-today\"\n"
|
||||
if findings := checkCoverageAnchor(renamed, "## What runs today\n"); len(findings) != 0 {
|
||||
t.Errorf("checkCoverageAnchor() = %v, want no findings when both moved", findings)
|
||||
}
|
||||
if findings := checkCoverageAnchor("no pointer here\n", "## What the tooling checks, and what it doesn't\n"); len(findings) != 1 {
|
||||
t.Errorf("checkCoverageAnchor() = %v, want a finding when the recipe lost the pointer", findings)
|
||||
}
|
||||
// A marker with nothing after it names no anchor, so it is not a pointer.
|
||||
bare := "\t@echo \"docs/.style/style-guide/README.md#\"\n"
|
||||
if findings := checkCoverageAnchor(bare, "## What the tooling checks, and what it doesn't\n"); len(findings) != 1 {
|
||||
t.Errorf("checkCoverageAnchor() = %v, want a finding when the recipe names no anchor", findings)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user