From 86e2e623dbc382a37aeda1a4937c002f2826daf0 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Wed, 26 Aug 2026 11:39:43 +0300 Subject: [PATCH 1/8] docs(cf13): define quality gate contract --- .../spec.md | 274 ++++++++++++++++++ 1 file changed, 274 insertions(+) create mode 100644 specs/014-cf-13-baselines-suppression-quality-gates/spec.md diff --git a/specs/014-cf-13-baselines-suppression-quality-gates/spec.md b/specs/014-cf-13-baselines-suppression-quality-gates/spec.md new file mode 100644 index 00000000..7559907d --- /dev/null +++ b/specs/014-cf-13-baselines-suppression-quality-gates/spec.md @@ -0,0 +1,274 @@ +# CF-13 — Baselines, Suppressions, and Quality Gates + +Status: PLANNING_CANDIDATE + +## Purpose + +CF-13 makes commandF adoptable in repositories that already contain known compatibility debt without weakening CF-04 compatibility semantics or CF-05 policy semantics. + +The slice adds a deterministic **new-change-first** quality gate. It composes the complete CF-05 `CheckReport`, an optional previously accepted CF-05 baseline, and optional exact finding suppressions. Existing baseline findings and explicitly suppressed findings remain visible evidence but do not block the CF-13 gate. New unsuppressed findings are evaluated with the existing CF-05 direction and severity threshold. + +CF-13 does not redefine `BREAKING`, `RISKY`, or `ADDITIVE`, and it does not remove findings from evidence. + +## User-visible command + +```text +commandf gate \ + --before-lock --before-cache \ + --after-lock --after-cache \ + [--direction both|producer|consumer] \ + [--fail-on breaking|risky|none] \ + [--baseline ] \ + [--suppressions ] \ + [--format json] \ + [--output ] +``` + +Defaults: + +- `--direction both`; +- `--fail-on breaking`; +- no baseline; +- no suppressions; +- JSON output; +- stdout when `--output` is omitted. + +The command performs no package acquisition or network lookup. It reuses the exact existing two-state package loader, CF-04 classifier, and CF-05 policy evaluator for the current candidate. + +## Exit contract + +`commandf gate` follows the CI-stable check/gate contract: + +- `0` — evaluation completed and the CF-13 quality gate passed; +- `1` — usage, input, baseline, suppression, classification, serialization, or output failure; +- `2` — evaluation completed successfully, output was emitted, and the quality gate failed. + +Exit `2` is reserved for a completed quality-gate policy failure. Parse failures for `gate` MUST normalize to exit `1` just as `check` parse failures do. + +## Current-report authority + +The current candidate is evaluated through the existing CF-05 library API. The resulting valid `CheckReport` is embedded unchanged in the CF-13 report. + +CF-13 MUST NOT: + +- reimplement structural diff; +- reimplement CF-04 compatibility classification; +- reinterpret CF-05 direction or `fail_on` semantics; +- mutate or filter the embedded current CF-05 evidence. + +## Baseline contract + +`--baseline` accepts a bounded UTF-8 JSON file containing a valid CF-05 `CheckReport` schema v1. + +A baseline is admissible only when: + +- the report passes existing CF-05 validation; +- its `package_name` exactly equals the current package name; +- its CF-04 ruleset exactly equals the current report ruleset; +- every finding produces a unique CF-13 fingerprint. + +The baseline policy (`direction` / `fail_on`) is not CF-13 authority. The baseline contributes only its validated compatibility findings as previously accepted evidence. + +A baseline may represent different before/after package versions from the current candidate. That is the expected adoption use case. + +If no baseline is supplied, no current finding is treated as pre-existing. + +## Exact finding fingerprint + +CF-13 owns a versioned deterministic finding fingerprint used only for baseline/suppression identity. It is not a replacement for CF-04 `rule_id`. + +V1 fingerprint input includes: + +- fingerprint schema/version; +- CF-04 ruleset; +- `rule_id`; +- compatibility severity; +- compatibility direction; +- structural source change kind; +- exact `ResourceKey` kind/value; +- before filename; +- after filename; +- element view; +- element id; +- field; +- before value; +- after value. + +The human-readable message is deliberately excluded: a wording-only message edit MUST NOT invalidate an otherwise identical accepted finding. + +The input is serialized through a fixed commandF-owned canonical structure and hashed with SHA-256. The public representation is: + +```text +sha256:<64 lowercase hex characters> +``` + +A severity, direction, rule, resource, source-kind, or evidence-value change therefore creates a different fingerprint and is treated as new unless separately accepted/suppressed. + +Duplicate fingerprints in either the current or baseline report are rejected as ambiguous rather than silently collapsed. + +## Suppression contract + +`--suppressions` accepts bounded UTF-8 JSON with this V1 logical shape: + +```json +{ + "schema": 1, + "suppressions": [ + { + "finding_fingerprint": "sha256:<64 lowercase hex characters>", + "rationale": "Approved interoperability exception", + "reference": "optional external tracking reference" + } + ] +} +``` + +Rules: + +- suppression fingerprints MUST use the exact V1 syntax; +- `rationale` MUST be non-empty after trimming and is retained in output; +- `reference` is optional evidence text only and carries no authority; +- duplicate suppression fingerprints fail closed; +- there are no glob, rule-wide, severity-wide, resource-wide, or wildcard suppressions in V1; +- there is no clock-based expiry evaluation in V1, avoiding hidden wall-clock nondeterminism; +- an unmatched/stale suppression is retained as `unused` evidence but does not itself fail the gate; +- a misspelled or stale suppression cannot hide a finding because only exact fingerprint equality suppresses it. + +Suppressions do not alter the embedded CF-05 report. + +## Finding disposition + +Every current finding receives exactly one CF-13 disposition: + +1. `suppressed` — an exact suppression entry matches the fingerprint; +2. `baseline` — otherwise, an exact baseline finding matches the fingerprint; +3. `new` — otherwise. + +Suppression precedence over baseline is intentional so explicit waiver evidence remains visible when both inputs contain the finding. + +Disposition is independent of CF-05 direction selection. All current findings remain classified for auditability. + +## Quality-gate decision + +The CF-13 decision uses the current CF-05 policy and only **new, unsuppressed** findings. + +Direction selection happens exactly as in CF-05. Threshold semantics remain exactly: + +- `breaking` blocks selected `BREAKING` findings; +- `risky` blocks selected `BREAKING` or `RISKY` findings; +- `none` blocks no finding; +- `ADDITIVE` does not block under `breaking` or `risky`. + +A selected finding with disposition `baseline` or `suppressed` never contributes to `blocking_findings` in CF-13 V1, but remains present in the report. + +## CF-13 JSON report + +The versioned report contains at least: + +- schema version; +- current CF-05 policy; +- gate decision counts; +- the complete unmodified current CF-05 `CheckReport`; +- deterministic baseline evidence when supplied; +- deterministic suppression-file evidence when supplied; +- per-current-finding fingerprint and disposition; +- matched suppression rationale/reference when applicable; +- unused suppression fingerprints; +- enough evidence to distinguish new, baseline, and suppressed findings without repository paths or external lookups. + +Decision counts include: + +- total current findings; +- selected findings; +- new findings; +- baseline findings; +- suppressed findings; +- new selected breaking/risky/additive findings; +- blocking findings; +- unused suppressions. + +The report MUST NOT delete findings merely because they are baseline or suppressed. + +## Determinism + +For identical pinned package inputs, policy, canonical baseline content, and suppression content: + +- fingerprints are byte-stable; +- dispositions are stable; +- JSON output is byte-identical; +- no timestamps, host paths, random ids, run ids, environment fields, or wall-clock decisions are emitted. + +Baseline and suppression evidence digests are computed from commandF canonical parsed content rather than original whitespace, so semantically identical JSON formatting does not change the gate result. + +## Output semantics + +`--output` reuses the existing CF-05 same-directory atomic publication contract. Output is fully written before exit `2` is returned. stdout remains quiet when an output path is supplied. + +## Fail-closed behavior + +CF-13 fails closed on: + +- invalid current CF-05 authority; +- unsupported baseline CF-05 schema/ruleset; +- baseline package-name mismatch; +- duplicate current or baseline fingerprints; +- malformed or oversized baseline/suppression files; +- unsupported suppression schema; +- malformed fingerprint syntax; +- empty suppression rationale; +- duplicate suppression fingerprints; +- serialization/output publication failure; +- any underlying CF-03/CF-04/CF-05 operational failure. + +Unknown future disposition/schema values are not coerced. + +## Security and trust boundary + +- no network access is added; +- no repository source discovery is added; +- no PHI or instance data is required; +- no arbitrary code/predicate execution is allowed in suppressions; +- suppression text is evidence, not executable policy; +- diagnostic output remains bounded/sanitized by existing CLI behavior; +- input size limits are explicit and tested. + +## Acceptance + +CF-13 is complete only when all of the following are proven on the exact final implementation head: + +1. `commandf gate --help` exposes two-state inputs, direction, threshold, optional baseline/suppressions, JSON format, and optional output. +2. With no baseline/suppressions, a current selected BREAKING finding is `new`, blocks under default policy, emits JSON, and exits `2`. +3. An exact matching valid baseline finding is `baseline`, remains in evidence, does not block, and allows exit `0` when no other blockers exist. +4. A changed severity/direction/rule/evidence field does not match the baseline fingerprint and remains new. +5. An exact suppression changes only disposition/gate decision, retains the full CF-05 finding plus rationale/reference evidence, and does not remove it. +6. A stale or misspelled suppression is reported unused and cannot hide a current finding. +7. Duplicate finding fingerprints and duplicate suppression fingerprints fail closed. +8. Baseline package mismatch, unsupported schema/ruleset, malformed suppression schema, invalid fingerprint, and empty rationale exit `1`. +9. Direction and `fail_on` semantics match CF-05 exactly, including `none`. +10. Reordered/whitespace-different equivalent suppression and baseline JSON canonicalize deterministically where semantic ordering is irrelevant. +11. Repeated identical evaluation produces byte-identical report bytes. +12. `--output` atomically replaces an existing file and emits complete output before policy-failure exit `2`. +13. Existing `commandf check` JSON/SARIF bytes and exit semantics remain unchanged. +14. Full workspace format, Clippy, tests, security regressions, and configured real-FHIR smoke remain green. +15. A dedicated CF-13 deterministic proof demonstrates baseline-match, suppression-match, new-finding block, repeated-byte equality, and a clean repository. +16. Independent review findings are dispositioned; reviewer unavailability/rate limits are recorded without invented PASS. +17. Convergence records exact final head/tree, workflow run/job/artifact identities, deterministic proof digest, coverage limits, and explicit deferrals. + +## Explicit deferrals / non-goals + +CF-13 V1 does not add: + +- wildcard, regex, rule-wide, severity-wide, or resource-wide suppressions; +- time-based suppression expiry or current-time policy; +- remote/shared baseline registries; +- GitHub API lookups or issue validation; +- organizational CEL/Rego/CUE policy languages; +- repository-level multi-package quality profiles; +- impact/CF-12 reachability as compatibility severity; +- SARIF filtering that removes accepted findings; +- automatic suppression generation; +- AI/model/agent suppression authority; +- changes to CF-04 classification semantics or CF-05 report schema; +- changes to CF-06 production oracle identity or the frozen CF-10 corpus. + +Broader institutional policy/profile systems may build on this exact deterministic gate in a later slice. \ No newline at end of file From deab7d9f1000115c353fb9e6612193e3ccd4ba5d Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Wed, 26 Aug 2026 11:40:36 +0300 Subject: [PATCH 2/8] docs(cf13): plan deterministic quality gate --- .../plan.md | 355 ++++++++++++++++++ 1 file changed, 355 insertions(+) create mode 100644 specs/014-cf-13-baselines-suppression-quality-gates/plan.md diff --git a/specs/014-cf-13-baselines-suppression-quality-gates/plan.md b/specs/014-cf-13-baselines-suppression-quality-gates/plan.md new file mode 100644 index 00000000..1ea42c97 --- /dev/null +++ b/specs/014-cf-13-baselines-suppression-quality-gates/plan.md @@ -0,0 +1,355 @@ +# CF-13 Plan — Deterministic Baselines, Suppressions, and Quality Gates + +Status: PLANNING_CANDIDATE + +## Entry condition + +CF-13 is authorized by Master Architecture V2 as `baselines/suppression/quality gates` and depends on CF-05. Canonical CF-05 behavior is already present on main. CF-13 does not depend on the externally blocked CF-06/CF-10 production-oracle path. + +The implementation starts only after this planning package passes consistency/review/CI and is merged to canonical main. + +## Design summary + +CF-13 adds a new `commandf gate` vertical slice rather than changing CF-05 `commandf check` behavior in place. + +Reasons: + +1. CF-05 JSON/SARIF and exit behavior are already shipped evidence contracts and should remain byte/behavior stable. +2. New-change-first adoption is a separate policy layer over a valid CF-05 current report, not a reinterpretation of compatibility semantics. +3. A separate report schema can preserve baseline/suppression provenance without version-bumping CF-05. +4. `gate` can reuse existing two-state loading/classification/check APIs and existing atomic output plumbing without duplicating semantic engines. + +## Proposed implementation surface + +Expected library modules in `commandf-pkg`: + +- `gate_model.rs` — public V1 schema, suppression schema, dispositions, decisions, evidence; +- `gate.rs` — validation, fingerprinting, canonicalization, baseline/suppression matching, gate evaluation; +- `gate_error.rs` — bounded typed failures; +- `lib.rs` exports. + +Expected CLI surface: + +- `crates/commandf-cli/src/main.rs` — `gate` parser/execution wiring only; +- focused CLI regressions under `crates/commandf-cli/tests/`. + +Expected proof surface: + +- `.github/workflows/cf13-quality-gate-proof.yml` with complete path filters for the CF-13 implementation/test/spec surface; +- retained deterministic evidence artifact containing a report digest and fixture/input identity. + +No new dependency is expected. `sha2`, `serde`, `serde_json`, and existing atomic-output infrastructure are already direct dependencies/available plumbing. + +## Library architecture + +### 1. Build the current CF-05 report through existing authority + +The CLI follows the current `check` path: + +1. load exact before/after lock/cache state using the existing bounded verified loader; +2. build the existing structural diff; +3. classify through CF-04; +4. call `evaluate_compatibility_policy` with the requested CF-05 policy; +5. pass the valid current `CheckReport` into CF-13 evaluation. + +No CF-13 function classifies compatibility itself. + +### 2. Validate baseline authority + +When a baseline is provided: + +- bound the input size before allocation; +- parse as `CheckReport`; +- call existing `validate_check_report`; +- require exact current/baseline package-name equality; +- require exact current/baseline CF-04 ruleset equality; +- compute V1 fingerprints for all baseline findings; +- reject duplicate baseline fingerprints. + +Baseline direction/fail-on metadata is retained by the baseline artifact but ignored for matching. Only validated compatibility findings constitute baseline evidence. + +### 3. Canonical fingerprint key + +Add a private/publicly testable commandF-owned key struct with a fixed field order and explicit fingerprint schema version. + +Fields are exactly those frozen in `spec.md`: ruleset, rule/severity/direction/source-kind/resource, optional filenames/view/element/field, and before/after values. The message is omitted. + +Serialize the key deterministically with `serde_json::to_vec`, hash with `sha2::Sha256`, and format lowercase as `sha256:`. + +`serde_json::Value` object-key determinism must not be assumed accidentally. Before/after JSON values MUST be recursively canonicalized into deterministic object-key order before fingerprint serialization. Arrays preserve their semantic order. + +Tests must prove object-key permutations yield the same fingerprint while semantically meaningful array order changes remain distinguishable. + +### 4. Suppression model + +Public V1 suppression structures: + +```text +GateSuppressions { + schema: 1, + suppressions: Vec +} + +GateSuppression { + finding_fingerprint: String, + rationale: String, + reference: Option +} +``` + +Validation: + +- exact schema 1; +- bounded entry count and bounded individual string lengths; +- exact `sha256:` + 64 lowercase hexadecimal syntax; +- trimmed rationale non-empty; +- duplicate fingerprint rejected. + +The implementation should choose explicit conservative V1 bounds and expose them as named constants so boundary tests can prove acceptance/rejection. No regex crate is required. + +Canonical suppression evidence is produced by sorting entries by fingerprint for digest/output metadata. User input order has no policy meaning. + +### 5. Baseline canonical evidence digest + +Canonicalize the parsed baseline report using its existing `to_json_bytes()` after validation. Because the embedded compatibility findings already have deterministic ordering, compute SHA-256 over those canonical bytes. + +Record a baseline evidence object containing at least: + +- canonical SHA-256; +- package name; +- ruleset; +- finding count. + +Do not retain the local input path in machine-readable output. + +### 6. Suppression canonical evidence digest + +After validation and deterministic sorting, serialize the normalized suppression object through a fixed canonical structure and hash it. Record: + +- canonical SHA-256; +- entry count. + +Do not retain local paths. + +### 7. Current finding uniqueness + +Compute each current finding fingerprint in CF-04 order. Reject duplicate fingerprints as ambiguous. + +This is intentionally stricter than silently deduplicating compatibility evidence. If CF-04 ever produces byte-identical semantic findings twice, the quality gate refuses to guess whether one accepted baseline/suppression identity should cover one or both occurrences. + +### 8. Disposition algorithm + +Build maps keyed by fingerprint only after uniqueness validation. + +For each current finding in original deterministic CF-04 order: + +1. if suppression exists -> `suppressed` and attach its rationale/reference; +2. else if baseline contains fingerprint -> `baseline`; +3. else -> `new`. + +Track suppression fingerprints that never matched a current finding as deterministic sorted `unused_suppressions`. + +### 9. Decision algorithm + +Reuse CF-05 policy semantics rather than duplicating independent rule meaning. The CF-13 evaluator may share/expose a crate-private helper from `check.rs` if needed, but any refactor must preserve CF-05 behavior and tests exactly. + +For counts/decision: + +- all current findings remain in evidence/disposition output; +- `selected` is based on current CF-05 direction; +- only selected findings with disposition `new` can block; +- the threshold is current CF-05 `fail_on`; +- baseline/suppressed findings contribute to their disposition counts but not blockers; +- `fail_on=none` yields zero blockers. + +Prefer reusing the current `CheckReport.decision` for total/selected severity counts where semantically identical, but compute new/baseline/suppressed/blocking counts explicitly and test against CF-05 policy behavior. + +### 10. Report model + +Expected V1 structures: + +```text +QualityGateReport +QualityGateDecision +QualityGateFinding +QualityGateDisposition +QualityGateBaselineEvidence +QualityGateSuppressionEvidence +GateSuppressions +GateSuppression +``` + +`QualityGateReport` embeds the complete current `CheckReport` unchanged and includes the current CF-05 policy explicitly or via that embedded report. Per-finding gate evidence carries fingerprint/disposition and matched suppression metadata. + +Output order: + +- current findings: original CF-04 order; +- unused suppressions: lexicographic fingerprint order; +- suppression normalization/digest: lexicographic fingerprint order. + +### 11. Validation API + +Provide a `validate_quality_gate_report` function that recomputes the gate from embedded current evidence and normalized baseline/suppression evidence where enough source content is present, or otherwise validates all internal invariants deterministically. + +If the report does not embed the full baseline report/suppression source, validation MUST still verify: + +- current CheckReport validity; +- unique current fingerprints; +- per-finding fingerprints match recomputation; +- disposition/count consistency; +- suppression evidence syntax and unique matched identities; +- decision consistency with current policy/dispositions. + +Do not claim the digest alone proves unseen external content. + +## CLI architecture + +Add `Command::Gate` with the exact spec arguments. + +CLI steps: + +1. parse package/policy arguments; +2. build current CF-05 report using existing functions; +3. bounded-read/parse optional baseline; +4. bounded-read/parse optional suppression file; +5. evaluate CF-13 gate; +6. serialize JSON; +7. publish via existing atomic `write_check_output` helper; +8. return 0/2 from the CF-13 decision. + +The process-boundary parse normalization currently special-cases `check`. Extend it narrowly so both `check` and `gate` usage failures return 1 while help remains 0. Do not change other commands' Clap behavior. + +V1 JSON only. SARIF remains the complete CF-05 artifact and is not filtered by baseline/suppression state. + +## Security / trust boundary + +- no network access; +- no arbitrary expressions/scripts; +- bounded baseline and suppression input bytes; +- bounded suppression entry count and string lengths; +- no path values serialized into reports; +- existing sanitized runtime diagnostics retained; +- no PHI/instance fixtures; +- no external tracker lookup for suppression references; +- no current-time evaluation. + +A suppression is explicit local policy evidence, not proof that a finding is safe. + +## Compatibility / migration impact + +- CF-04 model/ruleset unchanged; +- CF-05 `CheckReport` schema unchanged; +- existing `check` CLI arguments/output/exit behavior unchanged; +- existing SARIF/GitHub annotation flows unchanged; +- CF-06 identity and CF-10 corpus unchanged; +- no lockfile schema change; +- no new crate/dependency expected. + +## Test plan + +### Library positive cases + +- current BREAKING finding -> new/blocking; +- exact baseline -> baseline/non-blocking; +- exact suppression -> suppressed/non-blocking with rationale/reference; +- baseline + suppression same fingerprint -> suppression precedence; +- `breaking`, `risky`, `none` parity with CF-05; +- producer/consumer/both parity; +- unused suppression retained; +- empty baseline/no suppressions; +- baseline with different package versions but same package name. + +### Library negative/counterexample cases + +- severity change invalidates baseline match; +- direction/rule/source-kind/resource/evidence changes invalidate match; +- message-only change preserves fingerprint; +- malformed/uppercase/short fingerprint rejected; +- empty rationale rejected; +- duplicate suppressions rejected; +- duplicate current/baseline fingerprints rejected; +- invalid baseline check schema/decision/ruleset rejected; +- baseline package mismatch rejected; +- unsupported suppression schema rejected; +- input/string/count bounds rejected. + +### Determinism + +- repeated report bytes equal; +- baseline whitespace/key-order variants canonicalize to same baseline digest where parsed semantics are identical; +- suppression entry order/key-order variants canonicalize to same suppression digest; +- fingerprint JSON object-key permutations equal; +- output remains stable under deterministic fixture repetition. + +### CLI + +- `gate --help` contract; +- no-baseline BREAKING exit 2 with complete JSON; +- matching baseline exit 0; +- matching suppression exit 0; +- stale suppression still exit 2 when current blocker is new; +- malformed baseline/suppression exit 1; +- gate parse failure exit 1; +- output atomic replace on pass/fail; +- existing `check` behavior regressions unchanged. + +### Repository regression + +- `cargo fmt --all -- --check`; +- `cargo clippy --workspace --all-targets --all-features -- -D warnings`; +- `cargo test --workspace --all-features`; +- configured security regressions; +- configured real-FHIR smoke; +- all path-applicable existing proof workflows. + +## Dedicated proof workflow + +Add `cf13-quality-gate-proof` only in the implementation stack after the library/CLI is functional. + +The proof must use pinned runner/toolchain/action identities consistent with repository policy and must: + +1. create deterministic local synthetic package states without network acquisition; +2. produce a baseline/current scenario with one accepted old finding and one genuinely new finding; +3. prove the baseline finding does not block; +4. prove the new finding blocks under the selected policy; +5. add an exact suppression and prove only that fingerprint becomes suppressed; +6. execute the same final gate twice and compare bytes; +7. emit a SHA-256 identity artifact; +8. assert the repository remains clean. + +Proof output must not contain timestamps/random ids/host paths. + +## Delivery stacks + +### Planning stack + +This Spec Kit package only: + +- `spec.md`; +- `plan.md`; +- `tasks.md`; +- `consistency.md`. + +No production code in planning PR. + +### Stack A — library + +Implement fingerprint, baseline/suppression models/validation, gate evaluator, deterministic report, and library contract tests. No CLI yet. + +### Stack B — CLI + proof + +Add `commandf gate`, bounded input handling, atomic output, exit semantics, integration tests, and dedicated proof workflow. + +### Stack C — convergence + +Documentation-only final convergence record if exact implementation/run/artifact/reviewer identities need canonical capture. + +## Review plan + +- CodeRabbit when available; +- Qodo when connected/available; +- other repository-installed reviewers are informational unless governance makes them authoritative; +- every substantive finding is fixed or rejected with contract-grounded reasoning; +- rate limits/unavailability are recorded, never converted into PASS. + +Every head mutation invalidates prior exact-head qualification and requires re-reading all path-applicable gates. \ No newline at end of file From 753cb0452615127276a80080d5b4a911b6187c28 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Wed, 26 Aug 2026 11:41:03 +0300 Subject: [PATCH 3/8] docs(cf13): define dependency-ordered tasks --- .../tasks.md | 162 ++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 specs/014-cf-13-baselines-suppression-quality-gates/tasks.md diff --git a/specs/014-cf-13-baselines-suppression-quality-gates/tasks.md b/specs/014-cf-13-baselines-suppression-quality-gates/tasks.md new file mode 100644 index 00000000..cf2771bb --- /dev/null +++ b/specs/014-cf-13-baselines-suppression-quality-gates/tasks.md @@ -0,0 +1,162 @@ +# CF-13 Tasks — Baselines, Suppressions, and Quality Gates + +Status: PLANNING_CANDIDATE + +Tasks are dependency ordered. A task is complete only with executable evidence on the exact candidate state. + +## Planning and contract freeze + +- [x] T001 — Confirm CF-13 roadmap authority and dependency eligibility. + - Master Architecture V2 defines CF-13 as `baselines/suppression/quality gates` depending on CF-05. + - CF-13 does not depend on the externally blocked CF-06/CF-10 production-oracle path. + +- [x] T002 — Freeze the V1 user-visible command and exit contract. + - `commandf gate ` with exact before/after lock/cache inputs; + - optional CF-05 baseline and exact suppression file; + - existing CF-05 direction/fail-on semantics; + - JSON-only V1; + - exit 0 pass / 1 operational-or-input / 2 completed gate failure. + +- [x] T003 — Freeze baseline, fingerprint, suppression, and evidence semantics. + - exact unique SHA-256 finding fingerprints; + - valid same-package/same-ruleset CF-05 baseline; + - exact non-wildcard suppressions with mandatory rationale; + - `suppressed > baseline > new` disposition precedence; + - full current CF-05 evidence preserved. + +- [ ] T004 — Complete planning consistency, exact-head CI, and independent planning review; merge planning before implementation. + +## Stack A — deterministic quality-gate library + +- [ ] T010 — Add CF-13 V1 public models. + - quality-gate report/decision/finding/disposition; + - baseline/suppression evidence; + - suppression input schema v1; + - deterministic JSON serialization. + +- [ ] T011 — Implement deterministic finding fingerprint V1. + - fixed semantic key fields from `spec.md`; + - recursive JSON object-key canonicalization; + - SHA-256 `sha256:` output; + - message excluded; + - positive and counterexample fingerprint tests. + +- [ ] T012 — Validate and normalize CF-05 baselines. + - existing `validate_check_report` authority; + - exact package/ruleset compatibility; + - duplicate fingerprint rejection; + - canonical baseline digest/evidence. + +- [ ] T013 — Validate and normalize suppression files. + - schema, syntax, bounds, non-empty rationale; + - duplicate rejection; + - deterministic order and canonical digest; + - unmatched suppression retention. + +- [ ] T014 — Implement deterministic finding disposition. + - suppression precedence; + - baseline matching; + - new finding classification; + - original current finding order preserved. + +- [ ] T015 — Implement quality-gate decision by composing CF-05 policy semantics. + - direction parity; + - breaking/risky/none parity; + - only selected `new` findings can block; + - baseline/suppressed evidence retained but non-blocking. + +- [ ] T016 — Implement CF-13 report validation/invariant checks. + - current CF-05 report validity; + - fingerprint/disposition/count consistency; + - unique identities; + - fail closed on unknown/inconsistent state. + +- [ ] T017 — Prove library determinism and bounds. + - repeated bytes; + - semantic JSON key-order invariance; + - suppression order invariance; + - duplicate/invalid/bounded-input failures; + - no silent evidence deletion. + +- [ ] T018 — Prove CF-05 behavior remains unchanged by Stack A refactors. + - existing check model/evaluator/SARIF tests unchanged and green; + - any shared helper extraction covered by exact parity tests. + +## Stack B — shipped `commandf gate` + +- [ ] T020 — Add the `commandf gate` CLI surface. + - exact two-state package inputs; + - direction/fail-on; + - optional baseline/suppressions; + - JSON/output arguments. + +- [ ] T021 — Add bounded baseline/suppression file loading and fail-closed CLI errors. + - explicit byte limits; + - bounded diagnostic behavior; + - no network acquisition added. + +- [ ] T022 — Preserve atomic output and gate exit semantics. + - complete output before exit 2; + - atomic replacement on pass/fail; + - gate parse failures normalized to 1; + - non-check/non-gate Clap behavior unchanged. + +- [ ] T023 — Add end-to-end CLI fixtures/regressions. + - new blocker; + - baseline pass; + - suppression pass; + - stale suppression cannot hide blocker; + - malformed/mismatched inputs exit 1; + - deterministic repeated bytes. + +- [ ] T024 — Add dedicated `cf13-quality-gate-proof` workflow. + - pinned toolchain/actions; + - complete CF-13 path filters; + - baseline + new + suppression proof; + - repeated byte equality; + - clean repository; + - retained digest artifact. + +- [ ] T025 — Record exact CF-13 deterministic proof identity. + - exact head/tree; + - run/job/artifact ids; + - artifact digest; + - `CF13_GATE_SHA256`. + +- [ ] T026 — Prove existing user-visible command behavior remains unchanged. + - `commandf check` JSON/SARIF/exit semantics; + - CF-01 through CF-12 applicable regressions; + - no CF-06 identity, CF-10 corpus, dependency, or lock-schema mutation. + +## Regression, review, and convergence + +- [ ] T040 — Run mandatory workspace gates on the exact final implementation head. + - format; + - Clippy with `-D warnings`; + - full workspace tests. + +- [ ] T041 — Run every path-applicable repository workflow and configured real-FHIR/security regression. + - do not invent workflows that path filters do not trigger. + +- [ ] T042 — Independent implementation review. + - CodeRabbit when available; + - Qodo when connected/available; + - disposition every substantive finding; + - record rate limits/unavailability without invented PASS. + +- [ ] T043 — Run CF-13 convergence. + - record exact final heads/trees/runs/jobs/artifacts/digests; + - record `CF13_GATE_SHA256`; + - verify unresolved substantive findings = 0; + - record explicit V1 coverage limits/deferrals; + - create docs-only closeout stack if needed. + +## Hard sequencing rules + +1. T004 must be complete and the planning PR merged before T010 implementation starts. +2. T010 precedes T011-T016; T011 precedes baseline/suppression matching; T012/T013 precede T014; T014 precedes T015; T015 precedes T016/T017. +3. Stack A must merge before Stack B starts unless the canonical planning merge explicitly authorizes a stacked implementation base. +4. `commandf check` remains CF-05 authority and must not be silently changed to new-change-first semantics. +5. Baseline/suppression matching is exact; no wildcard or inferred waiver authority is allowed. +6. No time/network/model authority is introduced. +7. No CF-06 production pin, frozen CF-10 corpus, or lock-schema mutation is authorized by CF-13. \ No newline at end of file From bbe57c2a3d684f6418bd299b148e126a80054d54 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Wed, 26 Aug 2026 11:41:38 +0300 Subject: [PATCH 4/8] docs(cf13): close planning consistency --- .../consistency.md | 215 ++++++++++++++++++ 1 file changed, 215 insertions(+) create mode 100644 specs/014-cf-13-baselines-suppression-quality-gates/consistency.md diff --git a/specs/014-cf-13-baselines-suppression-quality-gates/consistency.md b/specs/014-cf-13-baselines-suppression-quality-gates/consistency.md new file mode 100644 index 00000000..bede6457 --- /dev/null +++ b/specs/014-cf-13-baselines-suppression-quality-gates/consistency.md @@ -0,0 +1,215 @@ +# CF-13 Planning Consistency — Baselines, Suppressions, and Quality Gates + +Status: CONSISTENCY_CANDIDATE + +This analysis checks `spec.md`, `plan.md`, `tasks.md`, commandF constitution/engineering rules, Master Architecture V2, and the canonical CF-05 contract before any CF-13 production implementation begins. + +## Authority alignment + +### Roadmap + +Master Architecture V2 defines: + +```text +CF-13 | baselines/suppression/quality gates | CF-05 +``` + +The planned slice stays within that identity. It does not absorb CF-12 impact, CF-14 source profiling, CF-15 recipes, or CF-16 mapping IR. + +### Constitution + +The plan satisfies the constitutional requirements: + +- **vertical capability** — a shipped `commandf gate` command is the user-visible result; +- **determinism** — exact canonical fingerprints, canonical baseline/suppression digests, deterministic report bytes, no time/random/env fields; +- **fail closed** — malformed/ambiguous baseline or suppression state is rejected; +- **evidence explicit** — current CF-05 evidence remains complete; baseline/suppression provenance and rationale remain visible; +- **precision over noise** — adoption debt can be baseline/suppressed without changing CF-04 severity truth; +- **product/research separation** — no AI or speculative semantic policy becomes authority. + +### AGENTS.md + +The plan does not silently discard/coerce source information, does not invent compatibility meaning, keeps ruleset/package evidence explicit, adds no unconsumed crate, and includes positive/counterexample/determinism/failure tests. + +## CF-05 composition check + +No conflict was found with canonical CF-05: + +- CF-05 owns current compatibility policy (`direction`, `fail_on`) and its 0/1/2 check exit contract. +- CF-13 calls the existing CF-05 evaluator for current evidence and preserves the complete `CheckReport`. +- CF-13 does not filter or rewrite CF-05 JSON/SARIF. +- CF-13's gate decision is a new adoption-layer decision over finding disposition, not a replacement compatibility classification. +- Existing `commandf check` remains unchanged for users not invoking `gate`. + +This separation avoids a conditional CF-05 schema/output change and therefore preserves existing SARIF, source-map, GitHub annotation, and downstream report consumers. + +## Baseline semantics check + +A baseline is defined as previously accepted **finding evidence**, not as a compatibility oracle and not as a claim that historical findings are safe. + +The following ambiguities are explicitly closed: + +- baseline policy metadata is not used to decide the current gate; +- only exact package-name and ruleset-compatible valid CF-05 reports are admissible; +- before/after package versions may differ, because otherwise a baseline could not represent historical accepted debt; +- exact semantic fingerprint equality is required; +- duplicate semantic fingerprints fail closed rather than being treated as a set implicitly. + +No contradiction with CF-05 evidence completeness was found. + +## Fingerprint semantics check + +The fingerprint fields include all CF-04 finding semantics/evidence except human-readable message wording. + +Consequences are intentional: + +- severity escalation becomes new; +- direction change becomes new; +- rule change becomes new; +- resource/source-kind/field/value change becomes new; +- message-only rewording remains the same finding identity; +- filename evidence remains identity-bearing because source-artifact movement can be semantically material for an exact finding record. + +Recursive JSON object-key canonicalization is required so map-key insertion order cannot create different fingerprints. Array order remains preserved. + +No cryptographic collision recovery mechanism is needed in V1 beyond exact SHA-256 identity; duplicate output fingerprints are treated as ambiguous and rejected. + +## Suppression semantics check + +Suppressions are exact, explicit waivers keyed to one fingerprint and carrying mandatory rationale. + +The plan deliberately excludes: + +- wildcards; +- rule-wide/severity-wide/resource-wide selectors; +- executable predicates; +- remote issue/tracker authority; +- clock-based expiry. + +This avoids broad accidental evidence hiding and nondeterministic current-time decisions. + +Unmatched suppressions are surfaced as unused rather than causing failure. This is consistent with fail-closed safety because an unmatched suppression cannot affect any current finding; a typo leaves the real finding new/blocking. + +Suppression precedence over baseline is consistent: explicit waiver provenance remains visible even if the same finding was also historically baselined. + +## Gate decision check + +No new compatibility threshold semantics are introduced. + +For a current finding to block CF-13 it must be: + +1. selected by the existing CF-05 direction policy; +2. disposed as `new`; +3. blocking under the existing CF-05 `fail_on` severity threshold. + +Baseline/suppressed findings remain evidence but are excluded from blockers by the purpose of the slice. + +`fail_on=none` continues to mean no policy blockers. + +## CLI / exit consistency + +`commandf gate` mirrors the CF-05 CI distinction: + +- 0 = completed/pass; +- 1 = parse/input/operational failure; +- 2 = completed/fail. + +The existing process boundary already normalizes `check` usage errors to 1. The planned narrow extension to `gate` does not alter other commands. + +Atomic output is reused rather than rebuilt. + +## Security / trust-boundary consistency + +The plan adds no: + +- network lookup; +- package acquisition; +- PHI/instance processing; +- arbitrary code execution; +- current-time authority; +- model/agent authority; +- external tracker verification; +- source path fabrication. + +Baseline/suppression files are bounded local inputs and diagnostics stay under the existing sanitized CLI boundary. + +## Dependency / schema consistency + +- no new crate is planned; +- existing `sha2`, `serde`, and `serde_json` are sufficient; +- lock schema remains unchanged; +- CF-04 ruleset/schema remains unchanged; +- CF-05 report schema remains unchanged; +- CF-06 production oracle identity remains unchanged; +- frozen CF-10 corpus remains unchanged. + +## Task-order consistency + +Task dependencies are coherent: + +- planning closes before implementation; +- models/fingerprint precede matching; +- baseline/suppression validation precede disposition; +- disposition precedes decision/report validation; +- library precedes shipped CLI/proof; +- exact-head gates/review precede convergence. + +No circular task dependency was found. + +## Open questions resolved in planning + +### Should CF-13 mutate `commandf check`? + +No. A separate `commandf gate` preserves CF-05 byte/schema/consumer stability and makes the adoption-layer policy explicit. + +### Should baseline matching use CF-05 policy/decision? + +No. A valid baseline contributes compatibility findings only. Current policy controls the current gate. + +### Should message text be fingerprinted? + +No. Rule/evidence identity is authoritative; prose-only wording churn should not force baseline migration. + +### Should suppressions support wildcard selectors? + +No in V1. Exact fingerprints minimize accidental over-suppression and are auditable. + +### Should suppressions expire by date? + +No in V1. Current-time evaluation would violate identical-input byte determinism unless time were made an explicit pinned input. Expiry/profile policy is deferred. + +### Should stale suppressions fail the gate? + +No. They are retained as unused evidence. They have no authority over current findings and therefore cannot create a false pass. + +### Should SARIF omit baseline/suppressed findings? + +No. CF-05 SARIF remains complete and unchanged. CF-13 V1 is JSON-only. + +## Explicit remaining deferrals + +Planning intentionally defers: + +- repository/org-wide quality profile files; +- wildcard/rule/resource suppressions; +- explicit pinned-time expiry inputs; +- shared/remote baseline stores; +- SARIF disposition overlays; +- GitHub issue/reference verification; +- multi-package aggregate gates; +- CEL/Rego/CUE policy engines; +- impact-informed severity; +- auto-generated waivers; +- AI-based waiver authority. + +These are not accepted CF-13 V1 requirements. + +## Consistency decision + +```text +SPEC_PLAN_TASKS_CONSISTENT=YES +KNOWN_AUTHORITY_CONTRADICTIONS=0 +IMPLEMENTATION_AUTHORIZED_BEFORE_PLANNING_MERGE=NO +``` + +T004 remains open until this planning exact head receives all path-applicable CI and independent review truth and the planning PR is merged. Only canonical planning may authorize Stack A. \ No newline at end of file From c66aafd5bebe3736cc2bc8789e539ef64c6f3f99 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Wed, 26 Aug 2026 12:21:34 +0300 Subject: [PATCH 5/8] docs(cf13): harden fingerprint and baseline evidence contracts --- .../spec.md | 122 ++++++++++++++---- 1 file changed, 99 insertions(+), 23 deletions(-) diff --git a/specs/014-cf-13-baselines-suppression-quality-gates/spec.md b/specs/014-cf-13-baselines-suppression-quality-gates/spec.md index 7559907d..68c057f8 100644 --- a/specs/014-cf-13-baselines-suppression-quality-gates/spec.md +++ b/specs/014-cf-13-baselines-suppression-quality-gates/spec.md @@ -40,7 +40,7 @@ The command performs no package acquisition or network lookup. It reuses the exa `commandf gate` follows the CI-stable check/gate contract: - `0` — evaluation completed and the CF-13 quality gate passed; -- `1` — usage, input, baseline, suppression, classification, serialization, or output failure; +- `1` — usage, input, baseline, suppression, classification, serialization, validation, or output failure; - `2` — evaluation completed successfully, output was emitted, and the quality gate failed. Exit `2` is reserved for a completed quality-gate policy failure. Parse failures for `gate` MUST normalize to exit `1` just as `check` parse failures do. @@ -56,6 +56,8 @@ CF-13 MUST NOT: - reinterpret CF-05 direction or `fail_on` semantics; - mutate or filter the embedded current CF-05 evidence. +The embedded current `CheckReport` retains exact current package name, before/after package versions, archive SHA-256 identities, ruleset, findings, and policy decision. CF-13 must not replace those identities with local file paths. + ## Baseline contract `--baseline` accepts a bounded UTF-8 JSON file containing a valid CF-05 `CheckReport` schema v1. @@ -65,7 +67,7 @@ A baseline is admissible only when: - the report passes existing CF-05 validation; - its `package_name` exactly equals the current package name; - its CF-04 ruleset exactly equals the current report ruleset; -- every finding produces a unique CF-13 fingerprint. +- every finding produces a unique CF-13 V1 fingerprint. The baseline policy (`direction` / `fail_on`) is not CF-13 authority. The baseline contributes only its validated compatibility findings as previously accepted evidence. @@ -73,6 +75,8 @@ A baseline may represent different before/after package versions from the curren If no baseline is supplied, no current finding is treated as pre-existing. +A persisted CF-13 report that claims any finding disposition `baseline` MUST retain sufficient baseline membership evidence to revalidate that claim without re-reading an unseen external file. V1 therefore retains the validated baseline before/after package identities and the complete sorted set of baseline finding fingerprints in `QualityGateBaselineEvidence`, in addition to the canonical baseline digest. A digest plus count alone is not authoritative membership evidence. + ## Exact finding fingerprint CF-13 owns a versioned deterministic finding fingerprint used only for baseline/suppression identity. It is not a replacement for CF-04 `rule_id`. @@ -96,15 +100,22 @@ V1 fingerprint input includes: The human-readable message is deliberately excluded: a wording-only message edit MUST NOT invalidate an otherwise identical accepted finding. -The input is serialized through a fixed commandF-owned canonical structure and hashed with SHA-256. The public representation is: +The input is serialized through a fixed commandF-owned canonical structure. Any nested JSON object in `before` or `after` is recursively canonicalized by lexicographically sorting object keys while preserving array order. Those canonical bytes are hashed with SHA-256. -```text -sha256:<64 lowercase hex characters> +Every persisted fingerprint identity uses the explicit versioned object: + +```json +{ + "schema": 1, + "digest": "sha256:<64 lowercase hex characters>" +} ``` +The schema is part of the persisted identity, not merely an implicit hash-preimage detail. Consumers MUST reject unsupported fingerprint schema values; they MUST NOT compare a V1 digest against an identity from another fingerprint schema. + A severity, direction, rule, resource, source-kind, or evidence-value change therefore creates a different fingerprint and is treated as new unless separately accepted/suppressed. -Duplicate fingerprints in either the current or baseline report are rejected as ambiguous rather than silently collapsed. +Duplicate V1 fingerprints in either the current or baseline report are rejected as ambiguous rather than silently collapsed. ## Suppression contract @@ -115,7 +126,10 @@ Duplicate fingerprints in either the current or baseline report are rejected as "schema": 1, "suppressions": [ { - "finding_fingerprint": "sha256:<64 lowercase hex characters>", + "finding_fingerprint": { + "schema": 1, + "digest": "sha256:<64 lowercase hex characters>" + }, "rationale": "Approved interoperability exception", "reference": "optional external tracking reference" } @@ -125,14 +139,16 @@ Duplicate fingerprints in either the current or baseline report are rejected as Rules: -- suppression fingerprints MUST use the exact V1 syntax; +- every persisted suppression fingerprint MUST carry the explicit supported fingerprint schema; +- fingerprint digests MUST use exact `sha256:` plus 64 lowercase hexadecimal characters; +- unsupported fingerprint schemas fail closed even if the digest text is otherwise valid; - `rationale` MUST be non-empty after trimming and is retained in output; - `reference` is optional evidence text only and carries no authority; - duplicate suppression fingerprints fail closed; - there are no glob, rule-wide, severity-wide, resource-wide, or wildcard suppressions in V1; - there is no clock-based expiry evaluation in V1, avoiding hidden wall-clock nondeterminism; - an unmatched/stale suppression is retained as `unused` evidence but does not itself fail the gate; -- a misspelled or stale suppression cannot hide a finding because only exact fingerprint equality suppresses it. +- a misspelled or stale suppression cannot hide a finding because only exact supported-version fingerprint equality suppresses it. Suppressions do not alter the embedded CF-05 report. @@ -140,8 +156,8 @@ Suppressions do not alter the embedded CF-05 report. Every current finding receives exactly one CF-13 disposition: -1. `suppressed` — an exact suppression entry matches the fingerprint; -2. `baseline` — otherwise, an exact baseline finding matches the fingerprint; +1. `suppressed` — an exact same-version suppression fingerprint matches; +2. `baseline` — otherwise, an exact same-version baseline fingerprint matches; 3. `new` — otherwise. Suppression precedence over baseline is intentional so explicit waiver evidence remains visible when both inputs contain the finding. @@ -171,10 +187,31 @@ The versioned report contains at least: - the complete unmodified current CF-05 `CheckReport`; - deterministic baseline evidence when supplied; - deterministic suppression-file evidence when supplied; -- per-current-finding fingerprint and disposition; +- per-current-finding explicit-version fingerprint and disposition; - matched suppression rationale/reference when applicable; -- unused suppression fingerprints; -- enough evidence to distinguish new, baseline, and suppressed findings without repository paths or external lookups. +- unused explicit-version suppression fingerprints; +- enough evidence to distinguish and revalidate new, baseline, and suppressed findings without repository paths or external lookups. + +`QualityGateBaselineEvidence` V1 contains at least: + +- baseline canonical SHA-256; +- fingerprint schema `1`; +- package name; +- CF-04 ruleset; +- exact baseline before package version and archive SHA-256; +- exact baseline after package version and archive SHA-256; +- finding count; +- the complete lexicographically sorted unique set of baseline V1 fingerprint identities. + +`QualityGateSuppressionEvidence` V1 contains at least: + +- suppression canonical SHA-256; +- suppression schema; +- fingerprint schema `1`; +- entry count; +- normalized suppression entries or equivalent complete membership evidence sufficient to validate every `suppressed` disposition and every `unused` identity. + +Local baseline/suppression/lock/cache paths MUST NOT be serialized as authority. Decision counts include: @@ -189,6 +226,24 @@ Decision counts include: The report MUST NOT delete findings merely because they are baseline or suppressed. +## Canonical evidence digests + +Baseline and suppression evidence digests are semantic-content digests, not original-file-byte digests. + +For the baseline: + +1. bound, parse, and validate the CF-05 `CheckReport`; +2. serialize the validated typed report to a JSON value; +3. recursively sort **every JSON object key at every depth**, including nested `before`/`after` evidence values, while preserving array order; +4. serialize that normalized value with one fixed commandF-owned JSON encoding; +5. hash those bytes with SHA-256. + +Therefore semantically identical baseline reports differing only in whitespace or JSON object-key insertion order produce the same canonical baseline digest. Array reordering remains identity-bearing. + +Suppression evidence is normalized by validating every entry, sorting entries by explicit-version fingerprint identity, recursively canonicalizing JSON objects, and hashing the fixed canonical serialization. + +The canonical digest never substitutes for membership evidence needed to validate a persisted disposition. + ## Determinism For identical pinned package inputs, policy, canonical baseline content, and suppression content: @@ -198,7 +253,7 @@ For identical pinned package inputs, policy, canonical baseline content, and sup - JSON output is byte-identical; - no timestamps, host paths, random ids, run ids, environment fields, or wall-clock decisions are emitted. -Baseline and suppression evidence digests are computed from commandF canonical parsed content rather than original whitespace, so semantically identical JSON formatting does not change the gate result. +Baseline and suppression evidence digests are computed from commandF canonical parsed content rather than original whitespace or object insertion order. ## Output semantics @@ -214,13 +269,17 @@ CF-13 fails closed on: - duplicate current or baseline fingerprints; - malformed or oversized baseline/suppression files; - unsupported suppression schema; -- malformed fingerprint syntax; +- unsupported fingerprint schema; +- malformed fingerprint digest syntax; - empty suppression rationale; - duplicate suppression fingerprints; +- persisted `baseline` disposition without retained matching baseline membership evidence; +- persisted `suppressed` disposition without retained matching suppression membership evidence; +- persisted fingerprint/count/decision mismatch; - serialization/output publication failure; - any underlying CF-03/CF-04/CF-05 operational failure. -Unknown future disposition/schema values are not coerced. +Unknown future disposition/schema/fingerprint-version values are not coerced. ## Security and trust boundary @@ -232,6 +291,22 @@ Unknown future disposition/schema values are not coerced. - diagnostic output remains bounded/sanitized by existing CLI behavior; - input size limits are explicit and tested. +## Provenance and retained proof evidence + +Runtime CF-13 evidence retains exact package identity through the embedded current `CheckReport` and baseline evidence: package name, exact versions, ruleset, and archive SHA-256 values. The product report does not serialize host-local lock/cache paths. + +The dedicated repository proof artifact additionally MUST bind the execution to immutable repository and input provenance without granting those records runtime policy authority. It records at least: + +- exact commandF head SHA and tree SHA; +- repository-relative paths and blob/content SHA identities for the governing CF-13 `spec.md`, `plan.md`, `tasks.md`, constitution, AGENTS.md, and relevant CF-05 implementation authority inspected for the proof; +- pinned Rust/toolchain and GitHub Action identities used by the workflow; +- dependency lockfile identity/digest; +- exact synthetic before/after package names, versions, archive SHA-256 identities, and fixture/source-input SHA-256 values; +- canonical baseline and suppression evidence digests; +- final `CF13_GATE_SHA256`. + +No mutable branch name, local host path, timestamp, or floating dependency reference is sufficient as retained proof identity. + ## Acceptance CF-13 is complete only when all of the following are proven on the exact final implementation head: @@ -243,16 +318,17 @@ CF-13 is complete only when all of the following are proven on the exact final i 5. An exact suppression changes only disposition/gate decision, retains the full CF-05 finding plus rationale/reference evidence, and does not remove it. 6. A stale or misspelled suppression is reported unused and cannot hide a current finding. 7. Duplicate finding fingerprints and duplicate suppression fingerprints fail closed. -8. Baseline package mismatch, unsupported schema/ruleset, malformed suppression schema, invalid fingerprint, and empty rationale exit `1`. +8. Baseline package mismatch, unsupported schema/ruleset, malformed suppression schema, unsupported fingerprint schema, invalid fingerprint digest, and empty rationale exit `1`. 9. Direction and `fail_on` semantics match CF-05 exactly, including `none`. -10. Reordered/whitespace-different equivalent suppression and baseline JSON canonicalize deterministically where semantic ordering is irrelevant. +10. Reordered/whitespace-different equivalent suppression and baseline JSON canonicalize deterministically, including nested object-key permutations; semantically meaningful array-order changes remain distinguishable. 11. Repeated identical evaluation produces byte-identical report bytes. 12. `--output` atomically replaces an existing file and emits complete output before policy-failure exit `2`. 13. Existing `commandf check` JSON/SARIF bytes and exit semantics remain unchanged. -14. Full workspace format, Clippy, tests, security regressions, and configured real-FHIR smoke remain green. -15. A dedicated CF-13 deterministic proof demonstrates baseline-match, suppression-match, new-finding block, repeated-byte equality, and a clean repository. -16. Independent review findings are dispositioned; reviewer unavailability/rate limits are recorded without invented PASS. -17. Convergence records exact final head/tree, workflow run/job/artifact identities, deterministic proof digest, coverage limits, and explicit deferrals. +14. Persisted report validation accepts a legitimate report and rejects forged baseline/suppressed dispositions, altered fingerprints, count mismatches, decision mismatches, unsupported fingerprint/disposition/schema values, and insufficient membership evidence deterministically. +15. Full workspace format, Clippy, tests, security regressions, and configured real-FHIR smoke remain green. +16. A dedicated CF-13 deterministic proof demonstrates baseline-match, suppression-match, new-finding block, repeated-byte equality, immutable repository/input provenance, and a clean repository. +17. Independent review findings are dispositioned; reviewer unavailability/rate limits are recorded without invented PASS. +18. Convergence records exact final head/tree, workflow run/job/artifact identities, deterministic proof digest, coverage limits, and explicit deferrals. ## Explicit deferrals / non-goals From 614ba20e17c3088f4c69ee0630fb7417987faa83 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Wed, 26 Aug 2026 12:22:27 +0300 Subject: [PATCH 6/8] docs(cf13): make retained gate evidence revalidatable --- .../plan.md | 209 +++++++++++++----- 1 file changed, 150 insertions(+), 59 deletions(-) diff --git a/specs/014-cf-13-baselines-suppression-quality-gates/plan.md b/specs/014-cf-13-baselines-suppression-quality-gates/plan.md index 1ea42c97..1fda1ddc 100644 --- a/specs/014-cf-13-baselines-suppression-quality-gates/plan.md +++ b/specs/014-cf-13-baselines-suppression-quality-gates/plan.md @@ -17,14 +17,14 @@ Reasons: 1. CF-05 JSON/SARIF and exit behavior are already shipped evidence contracts and should remain byte/behavior stable. 2. New-change-first adoption is a separate policy layer over a valid CF-05 current report, not a reinterpretation of compatibility semantics. 3. A separate report schema can preserve baseline/suppression provenance without version-bumping CF-05. -4. `gate` can reuse existing two-state loading/classification/check APIs and existing atomic output plumbing without duplicating semantic engines. +4. `gate` can reuse existing two-state loading/classification/check APIs and existing atomic-output plumbing without duplicating semantic engines. ## Proposed implementation surface Expected library modules in `commandf-pkg`: -- `gate_model.rs` — public V1 schema, suppression schema, dispositions, decisions, evidence; -- `gate.rs` — validation, fingerprinting, canonicalization, baseline/suppression matching, gate evaluation; +- `gate_model.rs` — public V1 schema, explicit-version fingerprint identity, suppression schema, dispositions, decisions, evidence; +- `gate.rs` — validation, recursive canonicalization, fingerprinting, baseline/suppression matching, gate evaluation; - `gate_error.rs` — bounded typed failures; - `lib.rs` exports. @@ -36,7 +36,7 @@ Expected CLI surface: Expected proof surface: - `.github/workflows/cf13-quality-gate-proof.yml` with complete path filters for the CF-13 implementation/test/spec surface; -- retained deterministic evidence artifact containing a report digest and fixture/input identity. +- retained deterministic evidence artifact containing report/input/provenance identities and `CF13_GATE_SHA256`. No new dependency is expected. `sha2`, `serde`, `serde_json`, and existing atomic-output infrastructure are already direct dependencies/available plumbing. @@ -54,6 +54,8 @@ The CLI follows the current `check` path: No CF-13 function classifies compatibility itself. +The embedded current `CheckReport` remains the current package-evidence authority and already retains package name, exact before/after versions, archive SHA-256 values, ruleset, findings, policy, and decision. CF-13 must not substitute host-local paths for those identities. + ### 2. Validate baseline authority When a baseline is provided: @@ -64,34 +66,52 @@ When a baseline is provided: - require exact current/baseline package-name equality; - require exact current/baseline CF-04 ruleset equality; - compute V1 fingerprints for all baseline findings; -- reject duplicate baseline fingerprints. +- reject duplicate baseline fingerprints; +- retain baseline before/after `PackageEvidence` and the complete sorted unique baseline fingerprint membership set in CF-13 output. Baseline direction/fail-on metadata is retained by the baseline artifact but ignored for matching. Only validated compatibility findings constitute baseline evidence. -### 3. Canonical fingerprint key +### 3. Explicit-version canonical fingerprint identity + +Add a commandF-owned persisted fingerprint identity: + +```text +FindingFingerprint { + schema: 1, + digest: "sha256:<64 lowercase hex>" +} +``` -Add a private/publicly testable commandF-owned key struct with a fixed field order and explicit fingerprint schema version. +The V1 hash preimage uses a fixed field order and includes the fingerprint schema plus exactly the semantic fields frozen in `spec.md`: ruleset, rule/severity/direction/source-kind/resource, optional filenames/view/element/field, and before/after values. The message is omitted. -Fields are exactly those frozen in `spec.md`: ruleset, rule/severity/direction/source-kind/resource, optional filenames/view/element/field, and before/after values. The message is omitted. +Before serialization, recursively canonicalize every JSON object in `before`/`after` by lexicographically sorting keys at every depth. Arrays preserve order. Serialize the fixed key structure deterministically, hash with `sha2::Sha256`, and format the digest lowercase. -Serialize the key deterministically with `serde_json::to_vec`, hash with `sha2::Sha256`, and format lowercase as `sha256:`. +Every persisted baseline member, suppression selector, current-finding identity, and unused-suppression identity carries `schema: 1` explicitly. Validation rejects unsupported fingerprint schemas before digest comparison. Cross-version fingerprints are never compared as equal merely because their digest strings match. -`serde_json::Value` object-key determinism must not be assumed accidentally. Before/after JSON values MUST be recursively canonicalized into deterministic object-key order before fingerprint serialization. Arrays preserve their semantic order. +Tests must prove: -Tests must prove object-key permutations yield the same fingerprint while semantically meaningful array order changes remain distinguishable. +- nested object-key permutations yield the same V1 fingerprint; +- semantically meaningful array-order changes yield a different fingerprint; +- unsupported fingerprint versions fail closed; +- message-only changes preserve identity while all frozen semantic/evidence fields remain identity-bearing. ### 4. Suppression model Public V1 suppression structures: ```text +FindingFingerprint { + schema: 1, + digest: "sha256:<64 lowercase hex>" +} + GateSuppressions { schema: 1, suppressions: Vec } GateSuppression { - finding_fingerprint: String, + finding_fingerprint: FindingFingerprint, rationale: String, reference: Option } @@ -99,52 +119,74 @@ GateSuppression { Validation: -- exact schema 1; +- exact suppression schema 1; +- exact fingerprint schema 1; - bounded entry count and bounded individual string lengths; -- exact `sha256:` + 64 lowercase hexadecimal syntax; +- exact `sha256:` + 64 lowercase hexadecimal digest syntax; - trimmed rationale non-empty; -- duplicate fingerprint rejected. +- duplicate same-version fingerprint rejected. The implementation should choose explicit conservative V1 bounds and expose them as named constants so boundary tests can prove acceptance/rejection. No regex crate is required. -Canonical suppression evidence is produced by sorting entries by fingerprint for digest/output metadata. User input order has no policy meaning. +Canonical suppression evidence is produced by sorting entries by `(fingerprint.schema, fingerprint.digest)` for digest/output metadata. User input order has no policy meaning. + +### 5. Baseline canonical evidence and membership + +Do **not** hash `CheckReport::to_json_bytes()` directly as the canonical baseline identity. Nested `serde_json::Value` object insertion order is not an accepted source of evidence identity. + +Define one `canonical_json_bytes` helper for CF-13 semantic evidence: -### 5. Baseline canonical evidence digest +1. serialize the already validated typed value to `serde_json::Value`; +2. recursively rebuild every JSON object with lexicographically sorted keys at every depth; +3. preserve array order exactly; +4. serialize with one fixed compact or otherwise fixed commandF-owned JSON encoding; +5. append no environment-dependent data. -Canonicalize the parsed baseline report using its existing `to_json_bytes()` after validation. Because the embedded compatibility findings already have deterministic ordering, compute SHA-256 over those canonical bytes. +Compute the baseline SHA-256 over those canonical bytes. -Record a baseline evidence object containing at least: +`QualityGateBaselineEvidence` V1 contains at least: - canonical SHA-256; +- fingerprint schema 1; - package name; - ruleset; -- finding count. +- exact before `PackageEvidence` (`version`, `archive_sha256`); +- exact after `PackageEvidence` (`version`, `archive_sha256`); +- finding count; +- complete lexicographically sorted unique `Vec` membership. -Do not retain the local input path in machine-readable output. +This membership is authoritative for validating a persisted `baseline` disposition. The digest is evidence binding, not a substitute for unseen membership. -### 6. Suppression canonical evidence digest +Do not retain the local baseline input path in machine-readable output. -After validation and deterministic sorting, serialize the normalized suppression object through a fixed canonical structure and hash it. Record: +### 6. Suppression canonical evidence and membership + +After validation and deterministic sorting, recursively canonicalize and serialize the normalized suppression object using the same fixed canonical JSON rules, then hash it. + +`QualityGateSuppressionEvidence` V1 contains at least: - canonical SHA-256; -- entry count. +- suppression schema 1; +- fingerprint schema 1; +- entry count; +- normalized complete suppression entries, or an equivalent complete membership structure that retains each exact fingerprint plus rationale/reference and is sufficient to validate every `suppressed` disposition and `unused` fingerprint. -Do not retain local paths. +Do not retain local suppression paths. ### 7. Current finding uniqueness -Compute each current finding fingerprint in CF-04 order. Reject duplicate fingerprints as ambiguous. +Compute each current finding V1 fingerprint in CF-04 order. Reject duplicate same-version fingerprints as ambiguous. -This is intentionally stricter than silently deduplicating compatibility evidence. If CF-04 ever produces byte-identical semantic findings twice, the quality gate refuses to guess whether one accepted baseline/suppression identity should cover one or both occurrences. +This is intentionally stricter than silently deduplicating compatibility evidence. If CF-04 ever produces semantically identical findings twice, the quality gate refuses to guess whether one accepted baseline/suppression identity should cover one or both occurrences. ### 8. Disposition algorithm -Build maps keyed by fingerprint only after uniqueness validation. +Build maps keyed by the complete supported-version fingerprint identity only after uniqueness validation. For each current finding in original deterministic CF-04 order: -1. if suppression exists -> `suppressed` and attach its rationale/reference; -2. else if baseline contains fingerprint -> `baseline`; +1. if a same-version suppression exists -> `suppressed` and attach its rationale/reference; +2. else if retained baseline membership contains the same-version fingerprint -> `baseline`; 3. else -> `new`. Track suppression fingerprints that never matched a current finding as deterministic sorted `unused_suppressions`. @@ -169,6 +211,7 @@ Prefer reusing the current `CheckReport.decision` for total/selected severity co Expected V1 structures: ```text +FindingFingerprint QualityGateReport QualityGateDecision QualityGateFinding @@ -179,28 +222,34 @@ GateSuppressions GateSuppression ``` -`QualityGateReport` embeds the complete current `CheckReport` unchanged and includes the current CF-05 policy explicitly or via that embedded report. Per-finding gate evidence carries fingerprint/disposition and matched suppression metadata. +`QualityGateReport` embeds the complete current `CheckReport` unchanged and includes the current CF-05 policy explicitly or via that embedded report. Per-finding gate evidence carries explicit-version fingerprint/disposition and matched suppression metadata. Output order: - current findings: original CF-04 order; -- unused suppressions: lexicographic fingerprint order; -- suppression normalization/digest: lexicographic fingerprint order. +- baseline membership: lexicographic `(schema, digest)` order; +- unused suppressions: lexicographic `(schema, digest)` order; +- normalized suppression evidence: lexicographic `(schema, digest)` order. ### 11. Validation API -Provide a `validate_quality_gate_report` function that recomputes the gate from embedded current evidence and normalized baseline/suppression evidence where enough source content is present, or otherwise validates all internal invariants deterministically. +Provide `validate_quality_gate_report` as a true persisted-evidence validator rather than a partial invariant checker. -If the report does not embed the full baseline report/suppression source, validation MUST still verify: +V1 report evidence is deliberately sufficient to revalidate all disposition authority without an external baseline/suppression file. Validation MUST: -- current CheckReport validity; -- unique current fingerprints; -- per-finding fingerprints match recomputation; -- disposition/count consistency; -- suppression evidence syntax and unique matched identities; -- decision consistency with current policy/dispositions. +- validate the embedded current `CheckReport`; +- validate supported report, suppression, and fingerprint schemas; +- recompute every current V1 fingerprint from the embedded current finding; +- verify unique current identities; +- verify baseline evidence package/ruleset/before/after identity syntax, canonical digest syntax, count, uniqueness, ordering, and membership set; +- verify suppression evidence canonical digest syntax, count, uniqueness, ordering, rationale/reference bounds, and complete membership; +- require every `baseline` disposition to have an exact member in retained baseline evidence; +- require every `suppressed` disposition and attached suppression metadata to match retained suppression evidence; +- recompute `unused_suppressions` from retained suppression membership and current identities; +- recompute all disposition and decision counts from embedded current evidence plus retained memberships; +- reject altered fingerprints, forged dispositions, count mismatches, decision mismatches, unknown values, and insufficient membership evidence. -Do not claim the digest alone proves unseen external content. +The validator does not claim that a digest proves unseen content; it validates the content that the report actually retains and its self-binding invariants. ## CLI architecture @@ -227,7 +276,7 @@ V1 JSON only. SARIF remains the complete CF-05 artifact and is not filtered by b - no arbitrary expressions/scripts; - bounded baseline and suppression input bytes; - bounded suppression entry count and string lengths; -- no path values serialized into reports; +- no host-local path values serialized into reports; - existing sanitized runtime diagnostics retained; - no PHI/instance fixtures; - no external tracker lookup for suppression references; @@ -235,6 +284,34 @@ V1 JSON only. SARIF remains the complete CF-05 artifact and is not filtered by b A suppression is explicit local policy evidence, not proof that a finding is safe. +## Immutable authority and provenance binding + +Runtime product evidence and repository proof evidence have different scopes and must both be explicit. + +### Runtime product evidence + +The CF-13 report retains: + +- current package name and exact before/after package versions/archive SHA-256 values through the embedded CF-05 report; +- baseline package name, ruleset, exact before/after versions/archive SHA-256 values, canonical baseline digest, and full baseline membership; +- canonical suppression digest and complete normalized suppression membership; +- no host-local lock/cache/baseline/suppression paths. + +### Repository proof evidence + +The dedicated proof artifact must bind its result to immutable development/runtime authorities by recording: + +- commandF exact head SHA and tree SHA; +- repository-relative path plus blob/content SHA for `spec.md`, `plan.md`, `tasks.md`, `.specify/memory/constitution.md`, `AGENTS.md`, and the relevant CF-05/CF-04 source files consumed as implementation authority; +- pinned Rust toolchain version and immutable GitHub Action refs used by the proof workflow; +- `Cargo.lock` repository path plus blob/content SHA and a SHA-256 digest of its exact proof-head bytes; +- exact synthetic source fixture paths relative to the repository/workflow staging root plus SHA-256 content digests; +- exact before/after package name/version/archive SHA-256 identities produced from those fixtures; +- baseline/suppression canonical evidence digests; +- final report digest `CF13_GATE_SHA256`. + +Repository-relative paths identify which governed inputs were inspected; immutable SHAs/digests establish identity. Mutable branch names, local absolute paths, timestamps, and floating dependency/action references are insufficient proof identity. + ## Compatibility / migration impact - CF-04 model/ruleset unchanged; @@ -257,28 +334,38 @@ A suppression is explicit local policy evidence, not proof that a finding is saf - producer/consumer/both parity; - unused suppression retained; - empty baseline/no suppressions; -- baseline with different package versions but same package name. +- baseline with different package versions but same package name; +- legitimate serialized gate report validates successfully. ### Library negative/counterexample cases - severity change invalidates baseline match; - direction/rule/source-kind/resource/evidence changes invalidate match; - message-only change preserves fingerprint; -- malformed/uppercase/short fingerprint rejected; +- malformed/uppercase/short digest rejected; +- unsupported fingerprint schema rejected before matching; - empty rationale rejected; - duplicate suppressions rejected; - duplicate current/baseline fingerprints rejected; - invalid baseline check schema/decision/ruleset rejected; - baseline package mismatch rejected; - unsupported suppression schema rejected; -- input/string/count bounds rejected. +- input/string/count bounds rejected; +- forged `baseline` disposition absent from retained membership rejected; +- forged `suppressed` disposition or altered rationale/reference rejected; +- altered persisted current fingerprint rejected; +- baseline/suppression membership count mismatch rejected; +- gate decision/count mismatch rejected; +- unknown disposition/report/fingerprint schema values rejected. ### Determinism - repeated report bytes equal; -- baseline whitespace/key-order variants canonicalize to same baseline digest where parsed semantics are identical; -- suppression entry order/key-order variants canonicalize to same suppression digest; -- fingerprint JSON object-key permutations equal; +- baseline whitespace/top-level/nested-object-key variants canonicalize to the same baseline digest when parsed semantics are identical; +- array order remains identity-bearing in baseline/fingerprint evidence; +- suppression entry order/key-order variants canonicalize to the same suppression digest; +- fingerprint nested JSON object-key permutations equal; +- validation returns the same accepted/rejected result and bounded error classification for repeated identical persisted inputs; - output remains stable under deterministic fixture repetition. ### CLI @@ -289,6 +376,7 @@ A suppression is explicit local policy evidence, not proof that a finding is saf - matching suppression exit 0; - stale suppression still exit 2 when current blocker is new; - malformed baseline/suppression exit 1; +- unsupported fingerprint schema exit 1; - gate parse failure exit 1; - output atomic replace on pass/fail; - existing `check` behavior regressions unchanged. @@ -309,15 +397,18 @@ Add `cf13-quality-gate-proof` only in the implementation stack after the library The proof must use pinned runner/toolchain/action identities consistent with repository policy and must: 1. create deterministic local synthetic package states without network acquisition; -2. produce a baseline/current scenario with one accepted old finding and one genuinely new finding; -3. prove the baseline finding does not block; -4. prove the new finding blocks under the selected policy; -5. add an exact suppression and prove only that fingerprint becomes suppressed; -6. execute the same final gate twice and compare bytes; -7. emit a SHA-256 identity artifact; -8. assert the repository remains clean. - -Proof output must not contain timestamps/random ids/host paths. +2. record exact fixture/source-input SHA-256 values and resulting package name/version/archive SHA-256 identities; +3. produce a baseline/current scenario with one accepted old finding and one genuinely new finding; +4. prove the baseline finding does not block and its disposition validates against retained membership; +5. prove the new finding blocks under the selected policy; +6. add an exact suppression and prove only that explicit-version fingerprint becomes suppressed; +7. validate the persisted final report, including retained baseline/suppression membership; +8. execute the same final gate twice and compare bytes; +9. record exact head/tree, governed repository paths with immutable blob/content identities, toolchain/action refs, and dependency lock identity; +10. emit a retained deterministic evidence artifact containing those identities plus `CF13_GATE_SHA256`; +11. assert the repository remains clean. + +Proof output must not contain timestamps/random ids/host absolute paths. ## Delivery stacks @@ -334,7 +425,7 @@ No production code in planning PR. ### Stack A — library -Implement fingerprint, baseline/suppression models/validation, gate evaluator, deterministic report, and library contract tests. No CLI yet. +Implement explicit-version fingerprint identity, canonicalization, baseline/suppression models/validation, membership-bearing evidence, gate evaluator, deterministic/revalidatable report, and library contract/tamper tests. No CLI yet. ### Stack B — CLI + proof From 4f036480c2f98966332f9ad879c0b3b7d388837e Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Wed, 26 Aug 2026 12:22:53 +0300 Subject: [PATCH 7/8] docs(cf13): require persisted-evidence tamper proofs --- .../tasks.md | 83 +++++++++++++------ 1 file changed, 59 insertions(+), 24 deletions(-) diff --git a/specs/014-cf-13-baselines-suppression-quality-gates/tasks.md b/specs/014-cf-13-baselines-suppression-quality-gates/tasks.md index cf2771bb..3e64d03d 100644 --- a/specs/014-cf-13-baselines-suppression-quality-gates/tasks.md +++ b/specs/014-cf-13-baselines-suppression-quality-gates/tasks.md @@ -18,44 +18,54 @@ Tasks are dependency ordered. A task is complete only with executable evidence o - exit 0 pass / 1 operational-or-input / 2 completed gate failure. - [x] T003 — Freeze baseline, fingerprint, suppression, and evidence semantics. - - exact unique SHA-256 finding fingerprints; + - exact unique SHA-256 finding fingerprints with explicit persisted fingerprint schema `1`; - valid same-package/same-ruleset CF-05 baseline; - - exact non-wildcard suppressions with mandatory rationale; + - recursive canonical JSON object-key normalization for semantic digests/fingerprints while preserving array order; + - baseline evidence retains exact before/after package identities plus complete sorted fingerprint membership; + - suppression evidence retains complete normalized membership with mandatory rationale; + - exact non-wildcard suppressions only; - `suppressed > baseline > new` disposition precedence; - - full current CF-05 evidence preserved. + - full current CF-05 evidence preserved; + - canonical digests bind retained evidence but never substitute for membership needed to revalidate dispositions. - [ ] T004 — Complete planning consistency, exact-head CI, and independent planning review; merge planning before implementation. ## Stack A — deterministic quality-gate library - [ ] T010 — Add CF-13 V1 public models. + - explicit-version `FindingFingerprint { schema, digest }`; - quality-gate report/decision/finding/disposition; - - baseline/suppression evidence; + - membership-bearing baseline/suppression evidence; - suppression input schema v1; - deterministic JSON serialization. - [ ] T011 — Implement deterministic finding fingerprint V1. - fixed semantic key fields from `spec.md`; - recursive JSON object-key canonicalization; - - SHA-256 `sha256:` output; + - SHA-256 `sha256:` digest inside explicit schema `1` identity; - message excluded; - - positive and counterexample fingerprint tests. + - positive and counterexample fingerprint tests; + - unsupported/cross-version fingerprint identities rejected before matching. - [ ] T012 — Validate and normalize CF-05 baselines. - existing `validate_check_report` authority; - exact package/ruleset compatibility; - duplicate fingerprint rejection; - - canonical baseline digest/evidence. + - recursively canonical baseline digest; + - retain exact baseline before/after `PackageEvidence`; + - retain complete lexicographically sorted unique baseline fingerprint membership. - [ ] T013 — Validate and normalize suppression files. - - schema, syntax, bounds, non-empty rationale; + - suppression schema and fingerprint schema validation; + - digest syntax, bounds, non-empty rationale; - duplicate rejection; - - deterministic order and canonical digest; + - deterministic order and recursively canonical digest; + - retain complete normalized suppression membership; - unmatched suppression retention. - [ ] T014 — Implement deterministic finding disposition. - - suppression precedence; - - baseline matching; + - same-version suppression precedence; + - baseline membership matching; - new finding classification; - original current finding order preserved. @@ -65,17 +75,32 @@ Tasks are dependency ordered. A task is complete only with executable evidence o - only selected `new` findings can block; - baseline/suppressed evidence retained but non-blocking. -- [ ] T016 — Implement CF-13 report validation/invariant checks. +- [ ] T016 — Implement CF-13 persisted-report validation/invariant checks. - current CF-05 report validity; - - fingerprint/disposition/count consistency; + - supported report/suppression/fingerprint schema validation; + - recomputed current fingerprints; + - baseline membership/count/package/ruleset/before/after identity validation; + - suppression membership/count/rationale/reference validation; + - every `baseline` disposition must match retained baseline membership; + - every `suppressed` disposition must match retained suppression membership and metadata; + - recomputed unused suppressions, disposition counts, and decision; - unique identities; - - fail closed on unknown/inconsistent state. + - fail closed on unknown/inconsistent/insufficient evidence. -- [ ] T017 — Prove library determinism and bounds. - - repeated bytes; - - semantic JSON key-order invariance; +- [ ] T017 — Prove library determinism, tamper resistance, and bounds. + - legitimate serialized report validates successfully; + - repeated report/validation results are deterministic; + - semantic top-level and nested JSON object-key invariance; + - meaningful array-order differences remain identity-bearing; - suppression order invariance; - duplicate/invalid/bounded-input failures; + - unsupported fingerprint schema rejected; + - forged `baseline` disposition without retained membership rejected; + - forged `suppressed` disposition or altered waiver metadata rejected; + - altered current fingerprint rejected; + - baseline/suppression membership count mismatch rejected; + - decision/count mismatch rejected; + - unknown disposition/report/fingerprint schema values rejected; - no silent evidence deletion. - [ ] T018 — Prove CF-05 behavior remains unchanged by Stack A refactors. @@ -93,6 +118,7 @@ Tasks are dependency ordered. A task is complete only with executable evidence o - [ ] T021 — Add bounded baseline/suppression file loading and fail-closed CLI errors. - explicit byte limits; - bounded diagnostic behavior; + - unsupported fingerprint versions fail closed; - no network acquisition added. - [ ] T022 — Preserve atomic output and gate exit semantics. @@ -106,22 +132,30 @@ Tasks are dependency ordered. A task is complete only with executable evidence o - baseline pass; - suppression pass; - stale suppression cannot hide blocker; - - malformed/mismatched inputs exit 1; + - malformed/mismatched/version-incompatible inputs exit 1; - deterministic repeated bytes. - [ ] T024 — Add dedicated `cf13-quality-gate-proof` workflow. - pinned toolchain/actions; - complete CF-13 path filters; - baseline + new + suppression proof; + - persisted-report validation; - repeated byte equality; - clean repository; - - retained digest artifact. + - retained deterministic evidence artifact. -- [ ] T025 — Record exact CF-13 deterministic proof identity. +- [ ] T025 — Record exact CF-13 deterministic proof identity and immutable provenance. - exact head/tree; - run/job/artifact ids; - artifact digest; - - `CF13_GATE_SHA256`. + - `CF13_GATE_SHA256`; + - repository-relative paths plus immutable blob/content identities for CF-13 spec/plan/tasks, constitution, AGENTS.md, and relevant CF-04/CF-05 implementation authorities; + - pinned Rust/toolchain and GitHub Action identities; + - `Cargo.lock` path/blob identity plus exact-byte SHA-256; + - exact synthetic source/fixture relative paths and content SHA-256 values; + - exact before/after package name/version/archive SHA-256 identities; + - baseline and suppression canonical evidence digests; + - no host-local path/timestamp/floating-ref substituted for immutable identity. - [ ] T026 — Prove existing user-visible command behavior remains unchanged. - `commandf check` JSON/SARIF/exit semantics; @@ -157,6 +191,7 @@ Tasks are dependency ordered. A task is complete only with executable evidence o 2. T010 precedes T011-T016; T011 precedes baseline/suppression matching; T012/T013 precede T014; T014 precedes T015; T015 precedes T016/T017. 3. Stack A must merge before Stack B starts unless the canonical planning merge explicitly authorizes a stacked implementation base. 4. `commandf check` remains CF-05 authority and must not be silently changed to new-change-first semantics. -5. Baseline/suppression matching is exact; no wildcard or inferred waiver authority is allowed. -6. No time/network/model authority is introduced. -7. No CF-06 production pin, frozen CF-10 corpus, or lock-schema mutation is authorized by CF-13. \ No newline at end of file +5. Baseline/suppression matching is exact and fingerprint-version-aware; no wildcard or inferred waiver authority is allowed. +6. A persisted baseline/suppressed disposition is not authoritative unless the report retains membership evidence sufficient to revalidate it. +7. No time/network/model authority is introduced. +8. No CF-06 production pin, frozen CF-10 corpus, or lock-schema mutation is authorized by CF-13. \ No newline at end of file From 33a0536d745d67ac6a094ce891293efa7e2204b9 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Wed, 26 Aug 2026 12:23:32 +0300 Subject: [PATCH 8/8] docs(cf13): reconcile review findings with repository authority --- .../consistency.md | 104 ++++++++++++++++-- 1 file changed, 94 insertions(+), 10 deletions(-) diff --git a/specs/014-cf-13-baselines-suppression-quality-gates/consistency.md b/specs/014-cf-13-baselines-suppression-quality-gates/consistency.md index bede6457..b068359d 100644 --- a/specs/014-cf-13-baselines-suppression-quality-gates/consistency.md +++ b/specs/014-cf-13-baselines-suppression-quality-gates/consistency.md @@ -21,15 +21,15 @@ The planned slice stays within that identity. It does not absorb CF-12 impact, C The plan satisfies the constitutional requirements: - **vertical capability** — a shipped `commandf gate` command is the user-visible result; -- **determinism** — exact canonical fingerprints, canonical baseline/suppression digests, deterministic report bytes, no time/random/env fields; -- **fail closed** — malformed/ambiguous baseline or suppression state is rejected; -- **evidence explicit** — current CF-05 evidence remains complete; baseline/suppression provenance and rationale remain visible; +- **determinism** — explicit-version canonical fingerprints, recursively canonical baseline/suppression digests, deterministic report bytes, no time/random/env fields; +- **fail closed** — malformed, version-incompatible, ambiguous, tampered, or insufficient baseline/suppression evidence is rejected; +- **evidence explicit** — current CF-05 evidence remains complete; baseline/suppression membership, package identities, provenance, and rationale remain visible; - **precision over noise** — adoption debt can be baseline/suppressed without changing CF-04 severity truth; - **product/research separation** — no AI or speculative semantic policy becomes authority. ### AGENTS.md -The plan does not silently discard/coerce source information, does not invent compatibility meaning, keeps ruleset/package evidence explicit, adds no unconsumed crate, and includes positive/counterexample/determinism/failure tests. +The plan does not silently discard/coerce source information, does not invent compatibility meaning, keeps ruleset/package/version/content-digest evidence explicit, adds no unconsumed crate, and includes rationale plus positive/counterexample/determinism/tamper/failure tests for public rules. ## CF-05 composition check @@ -37,6 +37,7 @@ No conflict was found with canonical CF-05: - CF-05 owns current compatibility policy (`direction`, `fail_on`) and its 0/1/2 check exit contract. - CF-13 calls the existing CF-05 evaluator for current evidence and preserves the complete `CheckReport`. +- The embedded current report already retains current package name, exact before/after versions and archive SHA-256 identities, ruleset, findings, and decision. - CF-13 does not filter or rewrite CF-05 JSON/SARIF. - CF-13's gate decision is a new adoption-layer decision over finding disposition, not a replacement compatibility classification. - Existing `commandf check` remains unchanged for users not invoking `gate`. @@ -52,11 +53,28 @@ The following ambiguities are explicitly closed: - baseline policy metadata is not used to decide the current gate; - only exact package-name and ruleset-compatible valid CF-05 reports are admissible; - before/after package versions may differ, because otherwise a baseline could not represent historical accepted debt; -- exact semantic fingerprint equality is required; -- duplicate semantic fingerprints fail closed rather than being treated as a set implicitly. +- exact same-fingerprint-schema semantic identity is required; +- duplicate semantic fingerprints fail closed rather than being treated as a set implicitly; +- baseline evidence retains exact before/after package versions/archive SHA-256 identities; +- baseline evidence retains the complete sorted unique fingerprint membership set, so a persisted `baseline` disposition can be revalidated without trusting an unseen external file; +- the canonical baseline digest binds normalized baseline semantics but is not treated as proof of membership by itself. No contradiction with CF-05 evidence completeness was found. +## Canonical evidence digest check + +The initial planning draft incorrectly proposed hashing `CheckReport::to_json_bytes()` directly for baseline identity. Review correctly identified that nested `serde_json::Value` objects can preserve insertion order, so semantically equivalent nested object-key order could change bytes. + +The corrected contract defines one CF-13 semantic canonicalization rule: + +1. serialize the validated typed value to JSON value form; +2. recursively sort object keys at every depth; +3. preserve array order; +4. serialize with a fixed commandF-owned JSON encoding; +5. hash those bytes. + +The same recursive canonicalization discipline applies to normalized suppression evidence and the JSON-valued components of finding fingerprints. Therefore whitespace and object insertion order are non-authoritative while meaningful array order remains identity-bearing. + ## Fingerprint semantics check The fingerprint fields include all CF-04 finding semantics/evidence except human-readable message wording. @@ -70,13 +88,21 @@ Consequences are intentional: - message-only rewording remains the same finding identity; - filename evidence remains identity-bearing because source-artifact movement can be semantically material for an exact finding record. +The persisted identity is explicitly versioned as: + +```text +FindingFingerprint { schema: 1, digest: sha256:<64 lowercase hex> } +``` + +Versioning only an internal preimage would be insufficient because persisted baseline/suppression/current identities could otherwise be compared across incompatible future algorithms. The corrected contract requires schema validation before matching and an explicit cross-version rejection test. + Recursive JSON object-key canonicalization is required so map-key insertion order cannot create different fingerprints. Array order remains preserved. No cryptographic collision recovery mechanism is needed in V1 beyond exact SHA-256 identity; duplicate output fingerprints are treated as ambiguous and rejected. ## Suppression semantics check -Suppressions are exact, explicit waivers keyed to one fingerprint and carrying mandatory rationale. +Suppressions are exact, explicit waivers keyed to one explicit-version fingerprint and carrying mandatory rationale. The plan deliberately excludes: @@ -92,7 +118,9 @@ Unmatched suppressions are surfaced as unused rather than causing failure. This Suppression precedence over baseline is consistent: explicit waiver provenance remains visible even if the same finding was also historically baselined. -## Gate decision check +The retained suppression evidence is complete enough to validate every persisted `suppressed` disposition, rationale/reference, and unused identity. A suppression digest/count alone is not treated as membership authority. + +## Gate decision and persisted-report validation check No new compatibility threshold semantics are introduced. @@ -106,12 +134,24 @@ Baseline/suppressed findings remain evidence but are excluded from blockers by t `fail_on=none` continues to mean no policy blockers. +The corrected report-validation contract does not accept disposition/count consistency alone. It requires enough retained baseline/suppression membership to recompute authority and rejects: + +- forged baseline dispositions; +- forged suppressed dispositions or waiver metadata; +- altered current fingerprints; +- baseline/suppression membership/count mismatches; +- decision mismatches; +- unsupported fingerprint/report/disposition values; +- insufficient retained membership evidence. + +A legitimate serialized report is a required positive test, and each tampering class is a required deterministic negative/counterexample test. + ## CLI / exit consistency `commandf gate` mirrors the CF-05 CI distinction: - 0 = completed/pass; -- 1 = parse/input/operational failure; +- 1 = parse/input/validation/operational failure; - 2 = completed/fail. The existing process boundary already normalizes `check` usage errors to 1. The planned narrow extension to `gate` does not alter other commands. @@ -133,6 +173,29 @@ The plan adds no: Baseline/suppression files are bounded local inputs and diagnostics stay under the existing sanitized CLI boundary. +## Immutable provenance check + +Repository review correctly required stronger retained provenance for reproducible proof. The corrected design distinguishes product/runtime evidence from repository proof evidence rather than serializing host-local paths into the product contract. + +Runtime CF-13 evidence retains: + +- current exact package identities through the embedded CF-05 report; +- baseline package name/ruleset and exact before/after versions/archive SHA-256 identities; +- baseline/suppression canonical digests and complete membership evidence. + +The dedicated deterministic proof artifact additionally records: + +- exact commandF head/tree; +- repository-relative paths and immutable blob/content identities for CF-13 spec/plan/tasks, constitution, AGENTS.md, and relevant CF-04/CF-05 implementation authority; +- pinned Rust/toolchain and GitHub Action identities; +- `Cargo.lock` repository path/blob identity and exact-byte SHA-256; +- exact synthetic fixture/source relative paths and content SHA-256 values; +- exact before/after package name/version/archive SHA-256 identities; +- baseline/suppression canonical evidence digests; +- final `CF13_GATE_SHA256`. + +Mutable branch names, timestamps, host-local absolute paths, or floating dependency/action refs cannot substitute for these identities. + ## Dependency / schema consistency - no new crate is planned; @@ -140,6 +203,7 @@ Baseline/suppression files are bounded local inputs and diagnostics stay under t - lock schema remains unchanged; - CF-04 ruleset/schema remains unchanged; - CF-05 report schema remains unchanged; +- CF-13 introduces only its own report/suppression/fingerprint schema V1; - CF-06 production oracle identity remains unchanged; - frozen CF-10 corpus remains unchanged. @@ -151,11 +215,26 @@ Task dependencies are coherent: - models/fingerprint precede matching; - baseline/suppression validation precede disposition; - disposition precedes decision/report validation; +- persisted-report validation precedes proof authority; - library precedes shipped CLI/proof; - exact-head gates/review precede convergence. No circular task dependency was found. +## Review-derived corrections + +The first independent review of planning head `bbe57c2a3d684f6418bd299b148e126a80054d54` produced five substantive threads. They are treated as valid planning defects and corrected in this head series: + +1. **Baseline canonical digest** — replace direct `CheckReport::to_json_bytes()` hashing with recursive semantic JSON object-key canonicalization. +2. **Immutable provenance** — require explicit package/source/governance/dependency identities in retained proof while keeping host-local paths out of runtime product evidence. +3. **Baseline membership authority** — persist the complete baseline fingerprint membership set and validate every `baseline` disposition against it. +4. **Fingerprint schema visibility** — persist fingerprint schema `1` with every fingerprint identity and reject cross-version identities. +5. **Persisted-report tamper tests** — require positive legitimate-report validation plus negative forged disposition/fingerprint/count/decision/unknown-value cases. + +These corrections strengthen determinism and fail-closed evidence without widening CF-13 into remote policy, CF-12 impact, or CF-06/CF-10 authority. + +Because the PR head changed, all prior exact-head CI and reviewer status is invalidated and must be re-run before T004 can close. + ## Open questions resolved in planning ### Should CF-13 mutate `commandf check`? @@ -186,6 +265,10 @@ No. They are retained as unused evidence. They have no authority over current fi No. CF-05 SARIF remains complete and unchanged. CF-13 V1 is JSON-only. +### Is a baseline digest/count sufficient to validate a persisted baseline disposition? + +No. V1 retains complete baseline fingerprint membership so validation can recompute the disposition without trusting unseen external content. + ## Explicit remaining deferrals Planning intentionally defers: @@ -209,7 +292,8 @@ These are not accepted CF-13 V1 requirements. ```text SPEC_PLAN_TASKS_CONSISTENT=YES KNOWN_AUTHORITY_CONTRADICTIONS=0 +REVIEW_DEFECTS_INCORPORATED=5 IMPLEMENTATION_AUTHORIZED_BEFORE_PLANNING_MERGE=NO ``` -T004 remains open until this planning exact head receives all path-applicable CI and independent review truth and the planning PR is merged. Only canonical planning may authorize Stack A. \ No newline at end of file +T004 remains open until the **new exact planning head** receives all path-applicable CI and independent review truth with zero unresolved substantive findings and the planning PR is merged. Only canonical planning may authorize Stack A. \ No newline at end of file