feat: add doctest tool and Bats harness for Markdown code blocks#84
Open
trevor-vaughan wants to merge 1 commit into
Open
feat: add doctest tool and Bats harness for Markdown code blocks#84trevor-vaughan wants to merge 1 commit into
trevor-vaughan wants to merge 1 commit into
Conversation
✅ CRAP Load Analysis: PASS (no baseline)No baseline file found at How to Enable Regression DetectionGenerate and commit a baseline file to track CRAP score changes over time: # 1. Install gaze
go install github.com/unbound-force/gaze/cmd/gaze@latest
# 2. Run tests and generate baseline
go test -coverprofile=coverage.out ./...
mkdir -p .gaze
gaze crap --format=json --coverprofile=coverage.out ./... > .gaze/baseline.json
# 3. Commit the baseline
git add .gaze/baseline.json
git commit -m "chore: add CRAP baseline for regression detection"For more information: Summary
|
7422178 to
6c3596b
Compare
Go tool (cmd/doctest/) uses goldmark to parse fenced code blocks
annotated with {test="..."} from Markdown files. Bats test harness
runs extracted snippets. Motivated by 11 open issues where Getting
Started page commands were broken or untested.
- Add cmd/doctest/ with extract and coverage subcommands
- Add Bats test harness via npm (bats-core, bats-support, bats-assert)
- Add Makefile targets: test-docs-extract, test-docs, test-docs-coverage
- Wire doc tests into CI (informational, non-blocking until blocks are annotated)
- Add spec and plan in specs/015-testable-documentation/
- Update CONTRIBUTING.md and README.md; create AGENTS.md
Details:
Go tool (cmd/doctest/):
- goldmark AST parser extracts annotated fenced code blocks to disk
- coverage subcommand exits non-zero when untested executable blocks exist
- Table-driven unit tests cover parsing, extraction, slug derivation, and coverage
- Ignores gitignored files during validation
Bats harness (tests/docs/):
- Installed as npm devDependencies (project already uses npm for Hugo)
- Helper scripts wrap snippet execution; skeleton test for getting-started guide
Build and CI:
- DOCTEST_DIR variable decouples extract output path from test input path
- Go tests run before doc tests so coverage failures don't block unit tests
- Doc test step uses || true until remaining blocks are annotated
- .gitignore: .test-output/, doctest binary, cmd/sync-content/sync-content
Assisted-by: Claude Opus 4.6
Signed-off-by: Trevor Vaughan <tvaughan@redhat.com>
6c3596b to
9a3ecd9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Go tool (cmd/doctest/) uses goldmark to parse fenced code blocks
annotated with {test="..."} from Markdown files. Bats test harness
runs extracted snippets. Motivated by 11 open issues where Getting
Started page commands were broken or untested.
Details:
Go tool (cmd/doctest/):
Bats harness (tests/docs/):
Build and CI:
Assisted-by: Claude Opus 4.6
Signed-off-by: Trevor Vaughan tvaughan@redhat.com