tests: property-based tests for the parsers and id derivations - #48
Merged
Conversation
Scorecard's fuzzing check scored zero: nothing exercised the parsers beyond hand-picked examples. Six Hypothesis properties now hold for any input: a rule id is short, lowercase, hyphenated and stable under re-slugging; extracting rules from any file never raises and never repeats an id; input hashes are deterministic and order-sensitive; prompt.md frontmatter round-trips its mapping and body; the run history table names every run and pair; rendered Markdown links point at pages while other links stay. hypothesis joins the dev extra, the hashed CI requirements and the dependency allowlist. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gwmBVUHSgohCLkNVqAR77 Signed-off-by: Claude <noreply@anthropic.com>
matplotlib 3.11 requires Python >= 3.11, and this package still declares requires-python >= 3.10 with a 3.10 job in CI. Dependabot's bump in #47 was compiled on a newer interpreter, so its lockfile dropped the pins that only 3.10 needs (typing-extensions, tomli, exceptiongroup) and the hashed install failed on the first job; it also rewrote the floor in ci.in to >= 3.11.1. Pin the ceiling in ci.in with the reason beside it, and tell Dependabot to skip feature bumps of matplotlib under /requirements so the same PR does not reopen every week. Patch releases still flow. The regenerated ci.txt is byte-identical to the current one, which confirms the lock was already the 3.10-compatible resolution. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gwmBVUHSgohCLkNVqAR77 Signed-off-by: Claude <noreply@anthropic.com>
open-coder-ai-org
marked this pull request as ready for review
September 7, 2026 15:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Scorecard's fuzzing check scored zero: nothing exercised the parsers beyond hand-picked examples. Six Hypothesis properties now hold for any input, 200 or 100 examples each:
input_hashis deterministic and order-sensitiveprompt.mdfrontmatter mapping and body read back as written.mdfiles become.htmllinks while other links stayTwo strategies are scoped on purpose, to plain YAML scalars and to bodies without a bare carriage return, so the properties test the frontmatter split and not YAML's scalar grammar or
read_text's newline normalisation.hypothesisjoins the dev extra, the hash-pinned CI requirements and the chock dependency allowlist. Scorecard recognises Hypothesis for its fuzzing check.Note: this branch conflicts with #43's
requirements/ci.txtonly if merged first; merge #43, #45, #46 then this, or tell me and I will rebase.Definition of done
python -m ruff check .andpython -m ruff format --check .cleanpython -m pytest -qgreen: 400 passed on 3.11; 397 passed and 3 skipped on 3.10 (the version-pinned figure compare and two that need optional packages not installed there)chock checkclean;.agents/policies/unchanged; the regeneratedrequirements/ci.txtinstalls in a fresh venv with--require-hashesClaims
re-derivableunless it actually recomputes from the subjectplus its recorded configuration and carries an
inputHash. A row that could not bemeasured says
NotAvailable,Error, orNotApplicableand why. Tests only.🤖 Generated with Claude Code
https://claude.ai/code/session_014gwmBVUHSgohCLkNVqAR77
Generated by Claude Code