Skip to content

feat: add doctest tool and Bats harness for Markdown code blocks#84

Open
trevor-vaughan wants to merge 1 commit into
mainfrom
feat/testable-documentaiton
Open

feat: add doctest tool and Bats harness for Markdown code blocks#84
trevor-vaughan wants to merge 1 commit into
mainfrom
feat/testable-documentaiton

Conversation

@trevor-vaughan

Copy link
Copy Markdown
Member

Go tool (cmd/doctest/) uses goldmark to parse fenced code blocks
annotated with {test="..."} from Markdown files. Bats test harness
runs extracted snippets. Motivated by 11 open issues where Getting
Started page commands were broken or untested.

  • Add cmd/doctest/ with extract and coverage subcommands
  • Add Bats test harness via npm (bats-core, bats-support, bats-assert)
  • Add Makefile targets: test-docs-extract, test-docs, test-docs-coverage
  • Wire doc tests into CI (informational, non-blocking until blocks are annotated)
  • Add spec and plan in specs/015-testable-documentation/
  • Update CONTRIBUTING.md and README.md; create AGENTS.md

Details:

Go tool (cmd/doctest/):

  • goldmark AST parser extracts annotated fenced code blocks to disk
  • coverage subcommand exits non-zero when untested executable blocks exist
  • Table-driven unit tests cover parsing, extraction, slug derivation, and coverage

Bats harness (tests/docs/):

  • Installed as npm devDependencies (project already uses npm for Hugo)
  • Helper scripts wrap snippet execution; skeleton test for getting-started guide

Build and CI:

  • DOCTEST_DIR variable decouples extract output path from test input path
  • Go tests run before doc tests so coverage failures don't block unit tests
  • Doc test step uses || true until remaining blocks are annotated
  • .gitignore: .test-output/, doctest binary, cmd/sync-content/sync-content

Assisted-by: Claude Opus 4.6
Signed-off-by: Trevor Vaughan tvaughan@redhat.com

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

✅ CRAP Load Analysis: PASS (no baseline)

No baseline file found at .gaze/baseline.json. Showing current scores without regression detection.

How to Enable Regression Detection

Generate and commit a baseline file to track CRAP score changes over time:

# 1. Install gaze
go install github.com/unbound-force/gaze/cmd/gaze@latest

# 2. Run tests and generate baseline
go test -coverprofile=coverage.out ./...
mkdir -p .gaze
gaze crap --format=json --coverprofile=coverage.out ./... > .gaze/baseline.json

# 3. Commit the baseline
git add .gaze/baseline.json
git commit -m "chore: add CRAP baseline for regression detection"

For more information:

Summary

Metric Value
Functions analysed 12
Avg complexity 8.3
Avg line coverage 74.8%
Avg CRAP score 19
CRAPload (>= 15) 3
Avg contract coverage 0%
Avg GazeCRAP score 0
GazeCRAPload (>= 15) 0

View full analysis logs

@trevor-vaughan trevor-vaughan force-pushed the feat/testable-documentaiton branch from 7422178 to 6c3596b Compare July 6, 2026 19:52
@trevor-vaughan trevor-vaughan marked this pull request as ready for review July 6, 2026 20:27
@trevor-vaughan trevor-vaughan requested a review from a team as a code owner July 6, 2026 20:27
Go tool (cmd/doctest/) uses goldmark to parse fenced code blocks
annotated with {test="..."} from Markdown files. Bats test harness
runs extracted snippets. Motivated by 11 open issues where Getting
Started page commands were broken or untested.

- Add cmd/doctest/ with extract and coverage subcommands
- Add Bats test harness via npm (bats-core, bats-support, bats-assert)
- Add Makefile targets: test-docs-extract, test-docs, test-docs-coverage
- Wire doc tests into CI (informational, non-blocking until blocks are annotated)
- Add spec and plan in specs/015-testable-documentation/
- Update CONTRIBUTING.md and README.md; create AGENTS.md

Details:

Go tool (cmd/doctest/):
- goldmark AST parser extracts annotated fenced code blocks to disk
- coverage subcommand exits non-zero when untested executable blocks exist
- Table-driven unit tests cover parsing, extraction, slug derivation, and coverage
- Ignores gitignored files during validation

Bats harness (tests/docs/):
- Installed as npm devDependencies (project already uses npm for Hugo)
- Helper scripts wrap snippet execution; skeleton test for getting-started guide

Build and CI:
- DOCTEST_DIR variable decouples extract output path from test input path
- Go tests run before doc tests so coverage failures don't block unit tests
- Doc test step uses || true until remaining blocks are annotated
- .gitignore: .test-output/, doctest binary, cmd/sync-content/sync-content

Assisted-by: Claude Opus 4.6
Signed-off-by: Trevor Vaughan <tvaughan@redhat.com>
@trevor-vaughan trevor-vaughan force-pushed the feat/testable-documentaiton branch from 6c3596b to 9a3ecd9 Compare July 7, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant