feat(traceability-audit): schema-capability precheck + the rivet mechanisms that shipped - #155
Conversation
…anisms that shipped Mined from ten June field reports (#84, #88-#95, #98) that were filed and never actioned. Two verified defects in this skill: 1. It never checks whether the loaded schema CAN model verification. Two repos independently found — only by running `rivet schema list`/`stats` rather than reading YAML — that no test/verification artifact type exists at all: `dev` is requirement + design-decision + feature, so `verifies` records provenance and gates nothing, and the status lifecycle caps at `implemented` (zero artifacts verified/accepted project-wide). One repo reported requirement-coverage 0/45 = 0.0% with its verification-method fields silently ignored. An audit run against such a schema is itself vacuous: it runs, reports green, gates nothing. New Step 0 makes that a mechanical precheck and a finding, not a clean report. 2. It prescribes only `rivet validate && rivet check && rivet coverage` and never mentions the mechanisms rivet has since shipped. Verified: `rivet verify` exists (main.rs -> cmd_verify), `rivet coverage` carries combined V-closure and a --tests mode, the requirement-verification rule exists — and a grep for any of them across the entire plugin returned ZERO hits. The tool learned; the skills did not. Also adds "warnings are not a pass" — one repo shipped on FAIL (0 errors, 87 warnings, 33 broken cross-refs) where the warnings were the unmapped verification. Both additions are mechanical checks rather than prose, per #94's constraint: "don't add prose-only skills, they get skipped under pressure." traceability-audit 0.1.0 -> 0.2.0; plugin 0.16.2 -> 0.17.0. Refs #88, #89, #90, #91, #92, #93, #94, #95 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Automated review for PR #155pulseengine/pulseengine.eu: Verdict: 💬 Comment Summary: Improper schema validation and audit checks in traceability-audit skill. Findings: 0 mechanical (rivet) · 2 from local AI model. Findings (2):
Generated by a local AI model and post-validated against a strict JSON contract. Each finding includes the verbatim line being criticised — verify by reading the file at the cited location. Reviewed at |
1 participant
feat(traceability-audit): schema-capability precheck + the rivet mechanisms that shipped
Mined from ten June field reports (#84, #88-#95, #98) that were filed and never
actioned. Two verified defects in this skill:
It never checks whether the loaded schema CAN model verification. Two repos
independently found — only by running
rivet schema list/statsrather thanreading YAML — that no test/verification artifact type exists at all:
devisrequirement + design-decision + feature, so
verifiesrecords provenance andgates nothing, and the status lifecycle caps at
implemented(zero artifactsverified/accepted project-wide). One repo reported requirement-coverage
0/45 = 0.0% with its verification-method fields silently ignored. An audit run
against such a schema is itself vacuous: it runs, reports green, gates nothing.
New Step 0 makes that a mechanical precheck and a finding, not a clean report.
It prescribes only
rivet validate && rivet check && rivet coverageand nevermentions the mechanisms rivet has since shipped. Verified:
rivet verifyexists (main.rs -> cmd_verify),
rivet coveragecarries combined V-closure anda --tests mode, the requirement-verification rule exists — and a grep for any
of them across the entire plugin returned ZERO hits. The tool learned; the
skills did not.
Also adds "warnings are not a pass" — one repo shipped on FAIL (0 errors, 87
warnings, 33 broken cross-refs) where the warnings were the unmapped verification.
Both additions are mechanical checks rather than prose, per #94's constraint:
"don't add prose-only skills, they get skipped under pressure."
traceability-audit 0.1.0 -> 0.2.0; plugin 0.16.2 -> 0.17.0.
Refs #88, #89, #90, #91, #92, #93, #94, #95
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com