diff --git a/.dev/features/specified-marker-check/PLAN.md b/.dev/features/specified-marker-check/PLAN.md new file mode 100644 index 0000000..e6a39a5 --- /dev/null +++ b/.dev/features/specified-marker-check/PLAN.md @@ -0,0 +1,94 @@ +# PLAN — specified-marker-check + +- spec_content_hash: 8f5ec002e3b18cbfd2f094b08a3671f7ed42a05a3fbaf01a11bbbd28da30fb52 # fix #4 +- applied_lessons: [L1, L2, L6, L13, L18, L20] +- increment: Give the F7 specified-not-yet-live annotations a deterministic floor check, so the docs cannot drift back into overclaiming — nor silently understate a primitive once it ships. +- layer(s): none (build apparatus, `.dev/floor/`) +- constitution_refs: [P0, P5, P6, P7] + +## Applied lessons + +- **L20** — this increment **is** L20 applied. F7's correction reduced to "remember to update the + docs", which is precisely the remedy-class L20 says WILL fail. The trigger is not hypothetical: the + docs asserted non-existent floors and nothing detected it for as long as it was true. +- **L6** — the defect L6 names (a membership fact grepped from free text) recurred **inside F7's own + REVIEW.md** while correcting a different error. So this checker reads its membership from a + **structured** JSON manifest, never by scanning prose for what looks like a marker. +- **L2** — the checker's own header states what it guarantees and what it does not, in the artifact, + not only here; and it cites only ops verified live this run. +- **L1** — meta-doc sweep: `CLAUDE.md`'s Commands block states the apparatus checkers, and + `package.json` states the gate. Both change and are declared below. +- **L13 / L18** — format this stage's own artifact; exclusion block is a `###` heading. + +## Discovery (P6 — verified live this run) + +| fact | verification | +| ------------------------------------- | ------------------------------------------------------ | +| no `pre-egress` hook | `ls .claude/hooks/` → 3 hooks, none matching `egress` | +| no archetype-maps manifest | `pharn/pharn-contracts/archetype-maps.json` absent | +| no `/pharn-estimate` | no `.claude/commands/*estimate*` | +| no `pharn/pharn-audits/` | absent | +| nothing reads trusted-doc prose | all three in `.prettierignore` + markdownlint-excluded | +| `npm run check` is the aggregate gate | `package.json` scripts, read this run | + +## Files + +- `.dev/floor/specified-primitives.json` — the structured manifest: each not-yet-live primitive, its + existence probe, and the exact marker substring at each doc site +- `.dev/floor/check-specified-markers.mjs` — the checker (existence test + exact-substring test) +- `.dev/floor/check-specified-markers.test.mjs` — tests, incl. mutants +- `package.json` — wire the checker into `npm run check` +- `CLAUDE.md` — document the new apparatus command (L1) +- `CHANGELOG.md` — `[Unreleased]` entry + +### Deliberately NOT in scope + +- `pharn/ARCHITECTURE.md`, `THREAT-MODEL.md`, `LIMITS.md` — no trusted-doc edit is needed; this + increment reads them, never writes them. **No Bash write this time.** +- `SKILLS_VERSION` — **no bump.** Every path here is apparatus (`.dev/**`) or repo-meta + (`package.json`, `CLAUDE.md`, `CHANGELOG.md`); the product surface is byte-unchanged. +- `pharn/floor/**` — this guards PHARN's own docs, so it is dev apparatus, not shipped product. A + user's install has no reason to check PHARN's annotations. +- Building `pre-egress` / the archetype manifest / `/pharn-estimate` — still trigger-gated (P7). + +## Contracts satisfied + +- none — no capability, no contract shape change. + +## Evals to write (P1) + +- **none** — P1 binds a `role:`-bearing Capability; this authors none. The checker's own + `*.test.mjs` is the regression suite, and it is apparatus. + +## Guarantee audit (P0) + +- "a primitive that SHIPS while its markers remain is caught" → **floor: enum/regex** (an existence + probe + set membership). This is the load-bearing direction. +- "a marker DELETED while its primitive is still absent is caught" → **floor: enum/regex** (exact + substring presence). Both drift directions close. +- "the docs are accurate" → **ADVISORY, and emphatically not claimed.** The checker proves a known + annotation is still present and still warranted. It **cannot discover a NEW overclaim** about a + primitive absent from the manifest — the manifest is a hand-maintained address book, not a + discovery mechanism. That limit is the honest analogue of L6's index: "the manifest was checked" + never means "the docs are true". +- "the probe proves the primitive does not exist" → **floor, NARROWED and stated.** It tests **file + existence**, never that a hook is WIRED in `settings.json` or that it WORKS. A file named + `*egress*` in `.claude/hooks/` flips the probe to live even if nothing loads it. It is the loud + early signal, not proof of function. + +## Trust audit (P2) + +The manifest is a **trusted**, human-reviewed apparatus file, not ingested content. The docs it reads +are trusted. No untrusted input is ingested and no free text steers a branch — the checker's every +decision is an existence test or an exact substring test over bytes it never interprets. + +## Determinism audit (P5) + +Two membership tests, no classification, no LLM. Terminal fallback on a malformed manifest is a loud +RED naming the manifest — never a guess, never a silent pass. + +## Open questions (HALT) + +- none. The human's standing instruction ("if there are things we need to fix to make pharn better + just do it now") is the GATE-1 approval for this increment; the design contains no wording decision + over governing text, because no trusted doc is edited. diff --git a/.dev/features/trusted-doc-accuracy/GRILL.md b/.dev/features/trusted-doc-accuracy/GRILL.md new file mode 100644 index 0000000..8bc7305 --- /dev/null +++ b/.dev/features/trusted-doc-accuracy/GRILL.md @@ -0,0 +1,128 @@ +# GRILL — trusted-doc-accuracy (F7) + +Plan under interrogation: `.dev/features/trusted-doc-accuracy/PLAN.md` (`trust: untrusted`). +Spec-hash check: `sha256(pharn/ARCHITECTURE.md)` = `a1c243ea…21753` — **matches** the plan's +`spec_content_hash`; no drift. (Content-hash is floor-grade; here it only **surfaces** — the block on +drift is `/pharn-dev-build`'s gate, fix #4.) +Griller membership (FLOOR, `pharn/floor/count-grillers.mjs`): **13 registered**. + +--- + +## Findings — inline axes (Step 2) + +```yaml +- type: FINDING + rule_id: "P0" + severity: important + file: ".dev/features/trusted-doc-accuracy/PLAN.md:134" + problem: "The replacement marker makes a forward-looking scheduling promise that nothing enforces, so the increment may be trading an overclaim about the PRESENT for an unfalsifiable claim about the FUTURE." + evidence: "Marker: `_(specified; ships with the guarded surface)_`" + +- type: FINDING + rule_id: "P7" + severity: important + file: ".dev/features/trusted-doc-accuracy/PLAN.md:62" + problem: "Three primitives are deferred as 'trigger-gated (P7)' but the plan never states WHAT the trigger is or WHERE it is recorded, so the marker becomes a permanent parking spot rather than a labeled limit with a reopen condition." + evidence: "`pre-egress` hook, the archetype manifest, or `/pharn-estimate` is trigger-gated (P7) and out of scope" + +- type: FINDING + rule_id: "P0" + severity: important + file: ".dev/features/trusted-doc-accuracy/PLAN.md:205" + problem: "The Q4 deferral ships a KNOWN-FALSE backstop claim inside the very increment whose stated purpose is removing false claims — after F7, LIMITS.md:29 still asserts a live pre-write backstop for a kind-conditioned restriction that no live check enforces." + evidence: "F7's marker makes `:29` less wrong while it still claims a live `pre-write` backstop for a restriction no check enforces." + +- type: FINDING + rule_id: "P6" + severity: blocking + file: ".dev/features/trusted-doc-accuracy/PLAN.md:139" + problem: "Ten edits are applied with `sed`, whose substitution silently no-ops when a pattern fails to match, and the plan declares no post-edit verification step — so a partially-applied patch set would pass unnoticed into the human's GATE-2 review." + evidence: "### `pharn/ARCHITECTURE.md` — 3 edits (Bash `sed`)" + +- type: FINDING + rule_id: "P5" + severity: minor + file: ".dev/features/trusted-doc-accuracy/PLAN.md:24" + problem: "The plan pre-declares that /pharn-dev-regress's scope check will emit a false 'escaped its scope' finding but does not prescribe the classification, leaving the operator to re-derive the disposition of a P0-shaped blocking finding at read time." + evidence: 'so any human-applied trusted-doc edit **will** surface as a false "the build escaped its scope" finding' +``` + +## Findings — registered grillers (Step 2b) + +Membership is FLOOR; **running** a griller and **judging** its axis is ADVISORY. + +### documentation (`enforces: P7`) — 1 finding + +```yaml +- type: FINDING + rule_id: "P7" + severity: important + file: ".dev/features/trusted-doc-accuracy/PLAN.md:134" + problem: "The marker introduces a new vocabulary item into the shipped trusted docs with no definition anywhere in them, so a reader who meets it has nothing to look up and must infer what 'the guarded surface' denotes." + evidence: "Marker: `_(specified; ships with the guarded surface)_`. Single-line replacements only; no reflowing," +``` + +### comprehension (`enforces: P7`) — 1 finding + +```yaml +- type: FINDING + rule_id: "P7" + severity: minor + file: ".dev/features/trusted-doc-accuracy/PLAN.md:94" + problem: "The WHY of each annotation — that the primitive was verified absent this run — lives only in the PLAN and CHANGELOG, so a future reader of the trusted docs sees the marker without the evidence that produced it." + evidence: '"the marked sites no longer assert a live protection" → **advisory.** Nothing on the floor reads' +``` + +### architecture (`enforces: P3`) — no findings + +Structural fit recognized: the increment adds no capability, changes no contract shape, touches no +module in the layer tree, and introduces no sibling reference. `## Files` is confined to two +repo-meta files plus `CLAUDE.md`. + +### Not applicable — 9 grillers + +`a11y`, `error-handling`, `i18n`, `migrations`, `observability`, `performance`, `privacy`, +`security`, `testability` — each interrogates an axis (runtime behavior, schema change, user-facing +surface, test strategy) that a prose-only correction with no executable artifact does not present. +Recorded as not-applicable, **not** as passes. + +--- + +## Summary + +The plan is unusually well-grounded on the axes this stage most often catches: every factual claim is +re-derived from live state rather than inherited from the edit spec (`## Discovery`), the guarantee +audit reduces each claim honestly, and the `## Files` / Bash split is declared rather than disguised — +the L19 failure mode is named in the open, which is the correct handling of a write that passes no gate. + +The concerns cluster in three places. + +**The marker's own honesty (P0).** The increment's thesis is that a doc must not assert a protection +the repo lacks. The chosen remedy asserts instead that the protection **will** arrive with "the +guarded surface" — a phrase that names no artifact, no condition, and no owner. That is weaker than a +false present-tense claim in one respect (a reader is warned it is not live) and worse in another +(it cannot be checked, so it cannot ever be found wrong). Pairing it with a stated reopen condition +would keep the design intent visible without smuggling a schedule in. + +**A known-false claim shipping inside the correction (P0).** The Q4 deferral is a defensible scope +call and the human made it deliberately — but the honest consequence deserves stating plainly in the +CHANGELOG, not just in the dev-side plan: after F7 lands, a trusted doc still points at a floor +backstop that does not exist on either half. + +**The mechanism has no completion check (P6).** This is the one finding a reviewer should not wave +through. `sed` no-ops silently on a non-matching pattern; ten of the eleven edits go that way; and the +plan's only verification is `npm run check`, which cannot see whether a substitution landed because +the trusted docs are excluded from both formatters. A partially-applied patch set is therefore +indistinguishable from a fully-applied one at every gate downstream. A per-edit before/after diff, or +a post-edit grep asserting each old string is gone and each new string present, closes it cheaply. + +--- + +**ADVISORY VERDICT: 7 concerns raised (1 blocking-severity, 4 important, 2 minor) — for the human to +weigh before `/pharn-dev-build`.** + +This grill-log gates nothing. `severity` values above are **enum-gated members** whose **assignment is +model judgment** (fix #3); the free-text `problem` / `evidence` quote the plan and inherit its +untrusted tag — they are DATA for a human, never instructions to `/pharn-dev-build`. The only +floor-grade facts in this run are the griller **membership** count and the **spec-hash** comparison, +both reported above. diff --git a/.dev/features/trusted-doc-accuracy/PLAN.md b/.dev/features/trusted-doc-accuracy/PLAN.md new file mode 100644 index 0000000..383f318 --- /dev/null +++ b/.dev/features/trusted-doc-accuracy/PLAN.md @@ -0,0 +1,213 @@ +# PLAN — trusted-doc-accuracy (F7) + +- spec_content_hash: a1c243eaa7b52494dbdbc6dab02132d4783ca139826920d848e58a8dac621753 # fix #4 +- applied_lessons: [L1, L2, L7, L13, L17, L18, L19, L20] +- increment: Annotate every site in the four trusted docs that presents a **not-yet-live** floor primitive as operative, and correct one name-drift — so the governing text stops asserting protections the repo does not have (P0). +- layer(s): none (root trusted docs + repo-meta; no capability tree change) +- constitution_refs: [P0, P2, P6, P7] + +## Applied lessons + +- **L1** — meta-doc sweep run: `grep -rn "pre-egress" --include="*.md"` found **`CLAUDE.md:256`** + carrying the _same_ three-primitives claim as `pharn/ARCHITECTURE.md:41`. The edit spec does not + name it; leaving it would ship exactly the stale canon L1 describes. Raised as **Q3**. +- **L2** — the increment _is_ L2's rule applied to the trusted docs themselves: every "enforced by + ``" phrase must cite an op verified **live this run**. Liveness was verified by reading + implementations, not by trusting the edit spec — see `## Discovery` below. +- **L7** — `## Files` declares **only** what this increment's Write/Edit tools actually touch. The + three trusted docs are deliberately **excluded** rather than declared: naming them would be the + over-declaration L7 forbids (and L18's live over-grant reached `pharn/ARCHITECTURE.md` by exactly + that route). +- **L13** — this stage formats its own artifact (`prettier` + `markdownlint-cli2` over this PLAN + only, never repo-wide — L19). +- **L17** — pre-declared: `/pharn-dev-regress`'s `scope` check tests _changed-since-base_, not + _written-by-the-build_, so any human-applied trusted-doc edit **will** surface as a false "the + build escaped its scope" finding. Recorded here so it is classified, not waved through. +- **L18** — the exclusion block below is a `###` **heading**, not a bold prose intro, so + `set-writes-scope.cjs --from-plan` terminates the authorized list structurally. +- **L19** — the only agent path to the trusted docs is a **Bash write that never passes fix #7 at + all**. L19 says name it, never pretend the gate covered it. This is the substance of **Q1**. +- **L20** — the setter's printed path count was read against this plan's approved list, not assumed: + `--from-plan` reported **3 path(s)** = `SKILLS_VERSION`, `CHANGELOG.md`, `CLAUDE.md`, exactly the + approved set. Re-verified at build Step 0; a mismatch is a STOP. + +## Discovery (P6 — verified live this run, not from the edit spec) + +| Claim under test | Verification | Result | +| ----------------------------------------------- | ---------------------------------------------------------------------------- | ---------------------------------------------- | +| `pre-egress` hook exists | `ls .claude/hooks/` | **absent** — 3 hooks, none egress | +| archetype-maps manifest exists | `validate.mjs:233` expects `pharn/pharn-contracts/archetype-maps.json` | **absent** repo-wide | +| the fix #5 check ever fires | `validate.mjs:15` — "if an archetype-maps manifest exists (**conditional**)" | **never fires** | +| `/pharn-estimate` exists | `git ls-files .claude/commands/` | **absent** | +| `est_tokens` is emitted | grep over `.mjs`/`.cjs`/`.json` | **docs only**, no emitter | +| `security-secrets` lens exists | `ls pharn/pharn-review/` | **no** — 22 lenses; ships as `secrets-in-code` | +| `pharn-audits` module exists | `ls -d pharn/pharn-audits` | **absent** | +| the community-privilege restriction is enforced | `KIND_ENUM` used only at `validate.mjs:155` | **value-membership only** — see Q4 | + +`SKILLS_VERSION` live = **2.5.0** (the edit spec says it was verified at 2.4.6). Line numbers were +re-confirmed against `main`@`c93ca61` and **all still match**. + +## Files + +Write/Edit-tool outputs — these and only these are granted fix #7 scope (expected setter count: **3**): + +- `SKILLS_VERSION` — patch bump 2.5.0 → 2.5.1 (trusted docs are in the bump-triggering set) +- `CHANGELOG.md` — one `[Unreleased]` entry recording the correction + the Q4 deferral +- `CLAUDE.md` — edit **C1** (`:256`), the `pre-egress` claim (Q3 / L1) + +### Deliberately NOT in scope + +- `pharn/CONSTITUTION.md` — not implicated; no site names a not-yet-live primitive +- `.claude/hooks/**`, `pharn/floor/**`, `package.json` — F7 is doc accuracy only; building the + `pre-egress` hook, the archetype manifest, or `/pharn-estimate` is trigger-gated (P7) and out of scope +- `LIMITS.md:28` / `THREAT-MODEL.md:102` beyond the marker — the Q4 deferral + +### Written by Bash, OUTSIDE every write gate (L19 — declared, not disguised) + +Per Q1 the agent applies edits **A1–A3 / T1–T3 / L1–L4** with `sed`. `sed` is not +`Write|Edit|MultiEdit`, so these writes pass **neither** fix #7 **nor** fix #2 — the hooks are not +satisfied, they are **not consulted**. They are listed here rather than in `## Files` on purpose: +naming them there would grant Write-tool scope to three hook-protected docs, the over-declaration +L7 forbids and the exact over-grant L18 reproduced live. + +- `pharn/ARCHITECTURE.md` — A1, A2, A3 +- `THREAT-MODEL.md` — T1, T2, T3 +- `LIMITS.md` — L1, L2, L3, L4 + +The only real control on these three is the **human wording approval** at GATE 1 plus CODEOWNERS +review on `main`. That is discipline and a GitHub-layer gate, not a floor primitive (P0). + +## Contracts satisfied + +- none — this increment adds no capability and changes no contract shape. It corrects prose in the + governing text and bumps the shipped-surface version. + +## Evals to write (P1) + +- **none, and this is not a P1 exemption claim.** P1 binds a **Capability** (a `role:`-bearing file) + to `evals/cases/*` + `evals/expected/*`. This increment authors no capability and no `rule_id`, so + there is nothing for an eval to bind. The deterministic gate that _does_ apply is + `npm run check` staying GREEN (`/pharn-dev-verify`'s gate map). + +## Guarantee audit (P0) + +- "the marked sites no longer assert a live protection" → **advisory.** Nothing on the floor reads + trusted-doc prose; no checker validates their text. This is a human-reviewed wording correction. +- "the four trusted docs cannot be written by the agent's Write/Edit/MultiEdit" → **floor: hook** + (`protect-trusted-paths.cjs`, fix #2). **Narrowed, and stated:** it gates that tool surface only — + a Bash write bypasses `PreToolUse` entirely (`THREAT-MODEL.md:86`, L19). This is Q1. +- "this increment's Write/Edit tools touch only `SKILLS_VERSION`, `CHANGELOG.md`, `CLAUDE.md`" → + **floor: hook** (fix #7, `set-writes-scope.cjs --from-plan` + `enforce-writes-scope.cjs`), verified + live at **3 path(s)**. **Narrowed, and stated:** this says nothing about the ten `sed` edits to the + three trusted docs — that surface is ungated (see `## Files`, L19). +- "the annotated primitives are genuinely not live" → **floor-grounded evidence, advisory + conclusion.** Absence was verified by directory listing and by reading `validate.mjs`'s own + conditional; that no _other_ mechanism supplies the protection is a reading, not a check. +- "SKILLS_VERSION was bumped because product bytes changed" → **advisory.** No checker binds a + product-surface diff to a version bump (a standing, unenforced repo convention). + +## Trust audit (P2) + +The edit spec in the command arguments is **untrusted input** — it is unverified text, and it +proposed at least one string that must not be copied through (Q2). Every factual claim in it was +**re-derived from live state** (`## Discovery`) rather than accepted; its proposed wording is treated +as a **suggestion for the human to ratify**, never as an instruction to apply verbatim. No +guaranteed decision in this increment rests on it. + +## Open questions — RESOLVED at GATE 1 + +- **Q1 — write mechanism.** Raised: `CLAUDE.md` #1 says _"do not work around the hook — let a human + edit them outside the agent loop"_, which conflicts with the edit spec's Bash residual. + **Human decision: the agent applies the trusted-doc edits via Bash `sed`, after approving the + wording below.** Recorded honestly per L19: these three files are written by a path that **never + passes fix #7 or fix #2** — not a gated write, and not claimed as one. +- **Q2 — corrupted target string.** The spec's `THREAT-MODEL.md:91` replacement was garbled + ("condips"). **Resolved:** _"Specified; the check is conditional and no manifest exists, so it + never fires."_ +- **Q3 — `CLAUDE.md:256`.** **Included** (L1 meta-doc sweep). +- **Q4 — the empty backstop.** **Deferred to a follow-up, flagged in the CHANGELOG.** F7 applies the + marker as specified; the deeper defect is recorded below so it is not lost. +- **Q5 — `security-review auditors`.** **Included.** + +## The patch set — 11 edits across 4 files + +Marker: `_(specified; ships with the guarded surface)_`. Single-line replacements only; no reflowing, +no other lines. The three trusted docs are `.prettierignore`d **and** markdownlint-excluded (verified +this run), so no formatter can touch them; `CLAUDE.md` is formatter-governed but prettier's markdown +`proseWrap` default is `preserve`, and MD013 is off. + +### `pharn/ARCHITECTURE.md` — 3 edits (Bash `sed`) + +**A1 · :41** — the primitive list entry + +- `- \`pre-egress\` — blocks a network call to a domain not on a hardcoded allowlist.` +- → `- \`pre-egress\` _(specified; ships with the guarded surface)_ — blocks a network call to a domain not on a hardcoded allowlist.` + +**A2 · :239** — the §7 pre-egress allowlist clause. The spec named `:237`, but the marker reads far +better folded into the existing parenthetical's close on `:239` than jammed mid-clause on `:237`. + +- `fooled) and the **constitution/trusted-file write-guard** (fix #2) and the **\`writes\`-scope` +- → `fooled — specified; ships with the guarded surface) and the **constitution/trusted-file write-guard** (fix #2) and the **\`writes\`-scope` + +**A3 · :258** — the validate-contract list + +- `\`coupling\` enum membership; the four archetype maps agree (fix #5); finding templates separate` +- → `\`coupling\` enum membership; the four archetype maps agree (fix #5 — conditional; specified, ships with the guarded surface); finding templates separate` + +### `THREAT-MODEL.md` — 3 edits (Bash `sed`) + +**T1 · :70** — mechanism cell only + +- `| pre-write + pre-egress hook |` +- → `| pre-write hook + pre-egress (specified; ships with the guarded surface) |` + +**T2 · :71** — mechanism cell only + +- `| pre-egress hook + enum gate |` +- → `| pre-egress (specified; ships with the guarded surface) + enum gate |` + +**T3 · :91** — the fix #5 closure status (Q2) + +- `5. **Archetype maps drift** — \`validate\` checks the four maps agree (fix #5). _Closed._` +- → `5. **Archetype maps drift** — \`validate\` checks the four maps agree (fix #5). _Specified; the check is conditional and no manifest exists, so it never fires._` + +### `LIMITS.md` — 4 edits (Bash `sed`) + +**L1 · :29** — the §1a backstop citation + +- `(\`ARCHITECTURE.md §5\`, pre-write + pre-egress hooks). Safety comes from the floor, not from the` +- → `(\`ARCHITECTURE.md §5\`, pre-write hook; pre-egress specified, ships with the guarded surface). Safety comes from the floor, not from the` + +**L2 · :52** — `/pharn-estimate` + +- `**measured runtime cost** (the system already observes it). \`/pharn-estimate\` reports a range,` +- → `**measured runtime cost** (the system already observes it). \`/pharn-estimate\` _(specified; ships with the guarded surface)_ reports a range,` + +**L3 · :75** — the §1d re-gate claim; pre-write / writes-scope stay live + +- `no floor-gated capability — the pre-write / writes-scope and pre-egress hooks re-gate every downstream` +- → `no floor-gated capability — the pre-write / writes-scope hooks (and pre-egress, specified; ships with the guarded surface) re-gate every downstream` + +**L4 · :108** — name correction **+** Q5 marker, one line. The `security` **griller** does ship +(verified: `pharn/pharn-pipeline/grillers/security/`), so it is left alone. An inner `(...)` would +close two parens against the sentence's own; an em-dash avoids `))`. + +- `griller, security-secrets lens, security-review auditors) — each fresh sub-agent re-pays. Tiered` +- → `griller, secrets-in-code lens, security-review auditors — specified; ships with the guarded surface) — each fresh sub-agent re-pays. Tiered` + +### `CLAUDE.md` — 1 edit (Edit tool; fix #7 applies) + +**C1 · :256** — the same three-primitives claim (Q3 / L1) + +- `1. **Hooks** — \`pre-write\` (block writes to protected paths / out-of-\`writes\`-scope), \`pre-egress\`` +- → `1. **Hooks** — \`pre-write\` (block writes to protected paths / out-of-\`writes\`-scope), \`pre-egress\` _(specified; ships with the guarded surface)_` + +## Deferred finding (Q4) — recorded, not fixed here + +`LIMITS.md:28-29` and `THREAT-MODEL.md:102` are **backstop** claims, not descriptions: each strikes a +claim as a limit, then points at the floor as what bounds the residual. Live, **both halves are +empty** — no egress hook exists, and `KIND_ENUM` is read only to check that `kind`'s _value_ is an +enum member (`validate.mjs:155`); nothing conditions any privilege on `kind: community`. F7's marker +makes `:29` less wrong while it still claims a live `pre-write` backstop for a restriction no check +enforces. Follow-up: `community-privilege-backstop`. Trigger to reopen (P7): satisfied already — this +is a real, verified defect, not a hypothetical. diff --git a/.dev/features/trusted-doc-accuracy/REGRESSION.md b/.dev/features/trusted-doc-accuracy/REGRESSION.md new file mode 100644 index 0000000..f5e6b77 --- /dev/null +++ b/.dev/features/trusted-doc-accuracy/REGRESSION.md @@ -0,0 +1,56 @@ +# REGRESSION — trusted-doc-accuracy (F7) + +**FLOOR VERDICT: `no-regressions`** (`pharn/floor/check-regress.mjs verdict`, exit 0). Deterministic +exit-code comparison — zero LLM-judge in the core. + +| gate | base (`HEAD` = `c93ca61`) | head (working tree) | flip | +| ---------- | ------------------------- | ------------------- | ---- | +| `tests` | 0 | 0 | none | +| `validate` | 0 | 0 | none | + +Base resolved by deterministic state test: `git status --porcelain` non-empty → working-tree dogfood +→ `base = HEAD`. The baseline ran in a detached `git worktree` at that SHA (stdlib-only gates, no +`npm ci`); the gate set is identical on both sides. + +**Style gates deliberately skipped, not omitted.** `inside` touched no shared style config +(`.prettierrc`, `.prettierignore`, `.markdownlint-cli2.jsonc`, eslint config, `package.json` — all +byte-unchanged), so over outside files byte-identical at base and head a style result cannot flip. +The increment's own style is covered by `/pharn-dev-verify`'s gate map (L9), where it is not skipped. + +**Test list expanded through stdin** (`xargs node --test < list`) — not `node --test $LIST`, which +zsh does not word-split (**L5**), and not `xargs -a`, which BSD `xargs` rejects outright (**L16**). + +## Scope check — `escaped: []`, and the exemptions are the story + +`scope` exited **0**. Five paths landed in `escape_exempt` rather than in `escaped`: + +- `.dev/features/trusted-doc-accuracy/{PLAN,GRILL}.md` — this feature's own pipeline artifacts, each + written by its own stage under that stage's Step-0 writes-scope. +- `LIMITS.md`, `THREAT-MODEL.md`, `pharn/ARCHITECTURE.md` — the hook-protected trusted docs. + +This is the 2.4.4 `--feature` fix working as intended: on the pre-2.4.4 checker every one of these +five would have been reported as a **blocking P0 fix#7 "the build escaped its `## Files`"** finding on +the correct, designed workflow — the false positive `.dev/memory-bank/lessons-learned.md` **L17** +documents and **L20** demanded be given a floor check. It fired **zero** times here. + +### One observation the exemption cannot see (for `/pharn-dev-review`) + +The trusted-doc exemption's stated justification is that those four docs are ones **the agent cannot +write at all** — `protect-trusted-paths.cjs` denies every `Write|Edit|MultiEdit` to them, so their +presence in a diff provably came from a human. **This increment falsifies that premise.** Under the +GATE-1-approved Bash path, the agent wrote all three: `sed`-class writes never reach `PreToolUse`, so +fix #2 was not satisfied, it was **not consulted**. + +The exemption still fired on the correct paths, and `escaped: []` is the right answer here — the +three docs were authorized at GATE 1 and their edits are the increment. But the checker reached that +answer via a rationale that no longer holds: it exempts those paths because it believes an agent write +to them is impossible. An agent-authored trusted-doc edit and a human-authored one are, at this +checker, indistinguishable. Recorded for the review lens, not fixed here. + +## Honest scope (P0) + +`/pharn-dev-regress` catches **exactly what its suite catches — nothing more**. `no-regressions` means +the outside gates that ran did not flip pass→fail. It does **not** mean "nothing broke," and for a +prose-only increment it is especially weak evidence: **no deterministic check in this repo reads +trusted-doc prose**, so the correctness of all eleven edits is invisible to this stage by construction. +That verification is the per-edit assertion recorded in the build note, plus human review. diff --git a/.dev/features/trusted-doc-accuracy/REVIEW.md b/.dev/features/trusted-doc-accuracy/REVIEW.md new file mode 100644 index 0000000..9b4db6f --- /dev/null +++ b/.dev/features/trusted-doc-accuracy/REVIEW.md @@ -0,0 +1,160 @@ +# REVIEW — trusted-doc-accuracy (F7) + +PHARN reviewing PHARN. The increment under review is `trust: untrusted`. + +## Step 1 — Floor first (P0) + +| floor check | result | +| ----------------------------------------- | ------------------------------------------------ | +| `pharn/floor/validate.mjs .` | **GREEN** — 36 capabilities, exit 0 | +| `check-regress.mjs verdict` | **`no-regressions`**, exit 0 | +| `check-regress.mjs scope` | `escaped: []`, exit 0 | +| `check-verify.mjs` | **`PASS`**, exit 0 (5/5 gates) | +| `npm run check` (aggregate) | exit 0 | +| `check-plan-lessons.mjs` | **GREEN** — 8 cited ids resolve | +| spec-hash gate at build time (fix #4) | **MATCH** — build was against an un-drifted spec | +| per-edit assertion (10 trusted-doc edits) | 10/10 matched exactly once; 10/10 verified after | + +The floor is green everywhere it applies. **What it does not cover is the whole substance of this +increment** — see F1. + +--- + +## The four lenses + +### L-floor → P0 (the governing lens) + +```yaml +- type: FINDING + rule_id: "P0" + severity: important + file: "LIMITS.md:29" + problem: "After this increment a trusted doc still labels a clause '**Backstop (floor):**' whose floor operation does not exist on either half — the marker narrows the overclaim without removing it, so the site remains an instance of the exact disease the increment exists to cure." + evidence: "(`ARCHITECTURE.md §5`, pre-write hook; pre-egress specified, ships with the guarded surface). Safety comes from the floor, not from the" + +- type: FINDING + rule_id: "P0" + severity: important + file: "pharn/ARCHITECTURE.md:41" + problem: "The replacement marker asserts a future ('ships with the guarded surface') that names no artifact, condition, or owner, so it is unfalsifiable — a claim that cannot be checked has traded a wrong statement for an uncheckable one." + evidence: "- `pre-egress` _(specified; ships with the guarded surface)_ — blocks a network call to a domain not on a hardcoded allowlist." + +- type: FINDING + rule_id: "P0" + severity: minor + file: "THREAT-MODEL.md:102" + problem: "The §4 sibling of the LIMITS:29 site still reads 'trusted-write or off-allowlist egress (floor)' with no annotation, so the two docs now disagree about the same mechanism — one marked specified, the other still labelled floor." + evidence: "trusted-write or off-allowlist egress (floor), so blast radius is bounded even when the body is" +``` + +### L-eval → P1 + +```yaml +- type: FINDING + rule_id: "P1" + severity: minor + file: ".dev/features/trusted-doc-accuracy/PLAN.md:85" + problem: "No eval binds any claim in this increment, correctly — but the consequence is that nothing in the repo will detect if one of these eleven annotations is later reverted or contradicted by a new sentence, leaving the correction unprotected against exactly the drift that produced it." + evidence: "**none, and this is not a P1 exemption claim.** P1 binds a **Capability** (a `role:`-bearing file)" +``` + +P1 is **satisfied**, not waived: it binds a `role:`-bearing Capability to its evals, and this +increment authors none. The finding above is about durability, not about a missing eval. + +### L-trust → P2 (targets the residual) + +```yaml +- type: FINDING + rule_id: "P2" + severity: minor + file: "pharn/floor/check-regress.mjs:311" + problem: "The trusted-doc escape exemption's named L19 residual — a Bash write bypassing both the hook and this exemption — was exercised for real for the first time by this increment, so a limit the module documents as theoretical is now observed live and its blast radius is measured rather than predicted." + evidence: "const escapeExempt = undeclared.filter((f) => TRUSTED_DOCS.includes(f) || isPipelineArtifact(f, feature));" + +- type: FINDING + rule_id: "P2" + severity: important + file: ".dev/features/trusted-doc-accuracy/PLAN.md:65" + problem: "The increment establishes a working precedent for agent-authored edits to the four human-only trusted docs, and the only controls on it are a human wording approval and CODEOWNERS review — neither of which is a floor primitive, and neither of which leaves a machine-checkable trace that a given trusted-doc edit passed a human." + evidence: "### Written by Bash, OUTSIDE every write gate (L19 — declared, not disguised)" +``` + +The taint handling itself is clean: the edit spec was treated as untrusted input, every factual claim +in it was re-derived from live state rather than accepted, and one string it proposed (`"the check is +condips with the guarded surface"`) was **rejected** rather than copied into governing text. That is +the fence working — an untrusted source proposed bytes for a trusted doc and did not get them. + +> **Correction — a finding this review first raised and then withdrew.** The first draft asserted that +> `check-regress.mjs`'s trusted-doc exemption rests on the premise "the agent cannot write these files +> at all", and that this increment falsified it. **That was wrong**, and reading the module rather than +> inferring from its behavior is what showed it. `pharn/floor/check-regress.mjs:117-121` already states +> the bound exactly: _"BOUNDED, and stated: the hook gates the Write-tool surface only, so a Bash-tool +> write bypasses both it and this exemption (the L19 escape). The claim is 'the build could not have +> written this WITH THE WRITE TOOL', never 'no process changed it'."_ The module named the residual +> before this increment exercised it. The finding above is the corrected, narrower one: not a falsified +> rationale, but a **documented limit observed live for the first time**. Recorded rather than silently +> edited, because a review that quietly deletes its own wrong finding is the same dishonesty it exists +> to catch. + +### L-axis → P3 + +No findings. One axis of change (doc accuracy), no capability tree edits, no sibling references, no +layer inversion. `## Files` held to three paths and the setter confirmed three. + +--- + +## Gates (fix #3) — floor-gate vs advisory + +- **Floor-gate (blocking): none.** Every floor check above is green. +- **Advisory: 6 findings** (0 blocking-severity, 3 important, 3 minor). `severity` values are + enum-gated members; the **assignment** is model judgment and gates nothing. + + > **Two corrections to this tally, and the second one is the interesting one.** The first draft said + > "7 findings (0 blocking, 5 important, 2 minor)" against a body that has only ever held **6** — an + > arithmetic slip plus a severity tally that did not match the objects above it. A stated count + > nobody derives from the artifact is the same class of defect as a stated guarantee nobody derives + > from the floor. `GRILL.md`'s `7 (1 blocking, 4 important, 2 minor)` was re-derived and is correct. + > + > Re-deriving it, however, first produced **7** — because the remediation note itself quoted the + > search patterns, and a substring search counted that prose as a seventh finding. That is + > `.dev/memory-bank/lessons-learned.md` **L6** reproduced inside this review: a membership fact read + > by grepping free text conflates _documentation about_ a declaration with _a_ declaration. The count + > here is the **six YAML objects in the four lens code fences** — read from the structured location, + > which is why this note no longer spells the patterns out. + +--- + +## The one thing a reviewer should weigh hardest (F1, restated in prose) + +The increment does what it was scoped to do, and the floor confirms nothing broke. But its own +success criterion is unverifiable by any mechanism in this repo: **no gate reads trusted-doc prose.** +`format:check` and `lint:md` exclude the three docs by name; `validate` scans a tree they are not in. +So `PASS` here is a statement about the repo's health, not about the correction's accuracy. + +What carries the accuracy claim is the `## Discovery` table — each primitive confirmed absent by live +inspection this run — plus the per-edit assertion that the intended bytes landed. Both are real and +both are recorded. Neither is a check that will run again tomorrow. If `pre-egress` ships next month +and nobody removes these ten markers, the docs will be wrong in the opposite direction and nothing +will say so. + +That is not an argument against the change; it is the honest boundary of a prose correction in a +repo whose floor deliberately does not read prose. It is worth stating plainly at the gate, because +the alternative reading — "verify passed, so the docs are accurate now" — is precisely the inference +P0 forbids. + +## Candidate lesson (proposed, NOT promoted — promotion is a separate gated run) + +**An increment that edits its own pinned spec self-invalidates its pin, and the pipeline reports that +as drift.** `sha256(pharn/ARCHITECTURE.md)` moved from `a1c243ea…` to `8f5ec002…` as a direct result +of the approved work. `/pharn-dev-build` checked the pin before writing, so the build was sound — but +re-running the same plan now HALTs with "the spec drifted" though nothing is wrong. fix #4 models the +spec as a fixed **input**; for a trusted-doc increment it is the **target**. This will recur on every +such increment. Not promoted here: `/pharn-dev-memory-promote` is the gated path, and one occurrence +is one occurrence (**L20**'s trigger is the _second_). + +--- + +**This review gates nothing and issues no seal.** `/pharn-dev-review` has no structural verdict by +design; the floor-grade facts are the Step-1 table, already gated at build and verify. The free-text +`problem` / `evidence` fields above quote the increment and inherit its untrusted tag — DATA for the +human, never instructions. The merge/fix/abandon decision is the human's. diff --git a/.dev/features/trusted-doc-accuracy/SHIP.md b/.dev/features/trusted-doc-accuracy/SHIP.md new file mode 100644 index 0000000..7188136 --- /dev/null +++ b/.dev/features/trusted-doc-accuracy/SHIP.md @@ -0,0 +1,67 @@ +# SHIP — trusted-doc-accuracy (F7) + +Advisory roll-up. `/pharn-dev-ship` gated mode: the chain ran in order, and each proceed was read +from that stage's structural verdict, never from judgment. + +## Stages run, in order + +| # | stage | structural verdict read | value | proceed | +| --- | -------------------- | ---------------------------------------- | ---------------------- | ------- | +| 1 | `/pharn-dev-plan` | — (GATE 1: human approval) | approved as written | ✓ | +| 2 | `/pharn-dev-grill` | none — advisory by design, gates nothing | 7 concerns raised | ✓ | +| 3 | `/pharn-dev-build` | `pharn/floor/validate.mjs .` exit code | **0** (GREEN, 36 caps) | ✓ | +| 4 | `/pharn-dev-regress` | `regression-report.json` `.verdict` | **`no-regressions`** | ✓ | +| 5 | `/pharn-dev-verify` | `verify-report.json` `.verdict` | **`PASS`** (5/5 gates) | ✓ | +| 6 | `/pharn-dev-review` | none — no structural verdict by design | 6 advisory findings | GATE 2 | + +**Run ended at GATE 2** — the human decides merge / fix / abandon. No RED-verdict STOP occurred. + +## Verdicts, verbatim + +- `/pharn-dev-build` → `node pharn/floor/validate.mjs .` → exit **0**, `FLOOR: GREEN — 36 capabilities checked in .` +- `/pharn-dev-regress` → `.verdict` = **`no-regressions`**; `regressions: []`, `pre_existing: []`; + scope `escaped: []` with 5 paths in `escape_exempt` +- `/pharn-dev-verify` → `.verdict` = **`PASS`**; `failing_gates: []`; gates + `test`/`validate`/`lint`/`format:check`/`lint:md` all exit 0 +- aggregate `npm run check` → exit **0** + +Additional floor reads this run (each owned by its own checker, not by this stage): +`check-plan-lessons.mjs` GREEN (8 ids resolve); the fix #4 spec-hash gate MATCHED at build time; +`count-grillers.mjs` 13 registered; `count-verifiers.mjs` 0 registered. + +## What landed + +11 edits across 4 files, plus the version and changelog: + +- `pharn/ARCHITECTURE.md` (3), `THREAT-MODEL.md` (3), `LIMITS.md` (4) — applied through **Bash**, the + GATE-1-approved path, which passes **neither** fix #2 **nor** fix #7 (declared in `PLAN.md`, L19) +- `CLAUDE.md` (1) — Edit tool, inside the fix #7 scope +- `SKILLS_VERSION` 2.5.0 → **2.5.1** (patch), `CHANGELOG.md` `[Unreleased]` entry + +The ten Bash edits carried a per-edit assertion: each substitution had to match **exactly once** +across all ten before any file was written (fail-closed), then each was re-verified after writing. +10/10 both times. That check exists because `/pharn-dev-grill` raised it as a blocking-severity P6 +concern; it is a real deterministic check but it is **not** part of any stage's verdict. + +## Pointers (cited, not restated — P4) + +- `.dev/features/trusted-doc-accuracy/REVIEW.md` — 6 advisory findings, 4 lenses, candidate lesson, + and one withdrawn finding recorded in place rather than deleted +- `.dev/features/trusted-doc-accuracy/GRILL.md` — 7 advisory concerns (advisory; gated nothing) +- `.dev/features/trusted-doc-accuracy/VERIFY.md`, `REGRESSION.md`, `PLAN.md` + +## Two things the human should carry into the decision + +1. **No gate in this repo reads trusted-doc prose.** The three docs are excluded from both formatters + and are not in `validate`'s scan tree, so the green verdicts above say nothing about whether the + eleven annotations are _right_. Accuracy rests on the `PLAN.md` `## Discovery` verification and on + human review. +2. **The spec pin self-invalidated.** `sha256(pharn/ARCHITECTURE.md)` moved `a1c243ea…` → `8f5ec002…` + because the increment edits the doc the plan pins. The build checked the pin _before_ writing, so + nothing was bypassed — but re-running this plan would now HALT as "drifted". + +--- + +Chain ran; the named floor verdicts are as shown — this is **NOT** a judgment that the increment is +good or wise; that is the human's call at the post-review gate. No merge, no push, no +`PHARN ✓ reviewed` seal was applied. diff --git a/.dev/features/trusted-doc-accuracy/VERIFY.md b/.dev/features/trusted-doc-accuracy/VERIFY.md new file mode 100644 index 0000000..d1e0ca6 --- /dev/null +++ b/.dev/features/trusted-doc-accuracy/VERIFY.md @@ -0,0 +1,59 @@ +# VERIFY — trusted-doc-accuracy (F7) + +**FLOOR VERDICT: `PASS`** (`pharn/floor/check-verify.mjs`, exit 0). Absolute threshold: PASS iff every +gate exits 0. + +| gate | exit | +| -------------- | ---- | +| `test` | 0 | +| `validate` | 0 | +| `lint` | 0 | +| `format:check` | 0 | +| `lint:md` | 0 | + +`npm run check` (the aggregate the gate map is designed to track, L9) also exits **0**, so verify and +the full gate agree — the L9 seam where an increment's own new markdown passes both per-increment +stages and only reddens at `npm run check` did not open here. + +**No `structural:` gate.** This increment ships no capability, so there is no eval pair to +run — not a skipped check, an inapplicable one. P1 binds a `role:`-bearing file to its evals; this +increment authors none. + +**Advisory layer: empty.** `pharn/floor/count-verifiers.mjs .` → `{"registered":0,"verifiers":[]}`. +Zero verifiers exist (P7 — none authored, none built speculatively), so the advisory layer contributed +nothing. Per fix #3 it could not have flipped the verdict in either direction even if populated. + +## What PASS means here, and what it emphatically does not + +`PASS` means **exactly** "the five named gates exited 0." For this increment that claim is unusually +narrow, and saying so is the point: + +- **No gate in this repo reads trusted-doc prose.** `.prettierignore` and `.markdownlint-cli2.jsonc` + both exclude `pharn/ARCHITECTURE.md`, `THREAT-MODEL.md`, and `LIMITS.md` by name, and the floor + checkers cite them only in header comments — none validates their text. So `format:check` and + `lint:md` passing says **nothing** about ten of the eleven edits; they passed identically before the + change. +- **`validate` GREEN is not evidence about this increment either.** It scans the `pharn/pharn-*` + capability tree, which this increment did not touch. +- **What actually verified the edits** is the per-edit assertion in the build step: all ten + substitutions were required to match **exactly once** before any file was written (fail-closed), then + re-verified after writing that each old string is absent and each new string present exactly once. + That is a deterministic check, but it is **not** one of these gates and it is **not** in the verdict. + It verifies that the intended bytes landed — never that the intended wording is _right_, which is the + human's call at GATE 2. + +So: **the gates passed. That is not "the docs are now accurate."** The accuracy claim rests on the +`## Discovery` verification in `PLAN.md` (each primitive confirmed absent by live inspection) and on +human review of the wording — advisory and human, respectively, never floor. + +## Spec-pin drift — expected, and worth a decision + +`sha256(pharn/ARCHITECTURE.md)` moved from `a1c243ea…621753` (the plan's pin) to `8f5ec002…30fb52`, +because **this increment edits the very doc the plan pins**. `/pharn-dev-build` checked the pin +_before_ writing, correctly, so the build was sound. But re-running `/pharn-dev-build` on this plan +would now HALT with "the spec drifted" even though the work is complete and correct. + +fix #4 treats the spec as a fixed **input**; here it is the **target**. Nothing in the pipeline is +wrong, and no gate is bypassed — the pin did its job — but the failure mode is confusing rather than +informative, and it will recur on every future trusted-doc increment. Surfaced for +`/pharn-dev-review`; not fixed here (out of scope, and it is a pipeline question, not a doc question). diff --git a/.dev/features/trusted-doc-accuracy/regression-report.json b/.dev/features/trusted-doc-accuracy/regression-report.json new file mode 100644 index 0000000..4876e0d --- /dev/null +++ b/.dev/features/trusted-doc-accuracy/regression-report.json @@ -0,0 +1,32 @@ +{ + "feature": "trusted-doc-accuracy", + "base": "HEAD (c93ca61 — working-tree dogfood)", + "head": "working tree", + "inside": [ + ".dev/features/trusted-doc-accuracy/GRILL.md", + ".dev/features/trusted-doc-accuracy/PLAN.md", + "CHANGELOG.md", + "CLAUDE.md", + "LIMITS.md", + "SKILLS_VERSION", + "THREAT-MODEL.md", + "pharn/ARCHITECTURE.md" + ], + "declared": ["SKILLS_VERSION", "CHANGELOG.md", "CLAUDE.md"], + "escaped": [], + "escape_exempt": [ + ".dev/features/trusted-doc-accuracy/GRILL.md", + ".dev/features/trusted-doc-accuracy/PLAN.md", + "LIMITS.md", + "THREAT-MODEL.md", + "pharn/ARCHITECTURE.md" + ], + "style_gates": "skipped — no shared style config in `inside`; a style result cannot flip over outside files byte-identical at base and head", + "outside_gates": { + "tests": { "base": 0, "head": 0 }, + "validate": { "base": 0, "head": 0 } + }, + "regressions": [], + "pre_existing": [], + "verdict": "no-regressions" +} diff --git a/.dev/features/trusted-doc-accuracy/verify-report.json b/.dev/features/trusted-doc-accuracy/verify-report.json new file mode 100644 index 0000000..f68ae71 --- /dev/null +++ b/.dev/features/trusted-doc-accuracy/verify-report.json @@ -0,0 +1,22 @@ +{ + "feature": "trusted-doc-accuracy", + "commit": "working tree on c93ca61", + "gates": { + "test": 0, + "validate": 0, + "lint": 0, + "format:check": 0, + "lint:md": 0 + }, + "verdict": "PASS", + "failing_gates": [], + "structural_gates": "none — this increment ships no capability and therefore no eval pair (P1 binds a role:-bearing file; there is none here)", + "verifiers": { + "registered": 0, + "findings": [], + "note": "count-verifiers.mjs reports 0 (P7 — none authored). Advisory layer is empty; it could not have flipped the verdict either way (fix #3)." + }, + "aggregate": { + "npm run check": 0 + } +} diff --git a/.dev/floor/check-specified-markers.mjs b/.dev/floor/check-specified-markers.mjs new file mode 100644 index 0000000..93ec3d5 --- /dev/null +++ b/.dev/floor/check-specified-markers.mjs @@ -0,0 +1,254 @@ +#!/usr/bin/env node +// .dev/floor/check-specified-markers.mjs — bind PHARN's "specified; ships with the guarded surface" +// annotations to reality, in BOTH drift directions. +// +// Build apparatus, NOT product surface: it guards PHARN's OWN governing docs, and a user's install has +// no reason to check PHARN's annotations. It is not a Capability (no `role:`), ships to nobody, and +// drives no SKILLS_VERSION bump. The dependency may only point .dev/ -> pharn/, never the reverse. +// +// Floor primitive (ARCHITECTURE §2): #3 (enum / regex — here, set membership by file EXISTENCE and by +// exact SUBSTRING presence). No LLM, no classification, no heuristics, no network, no child_process. +// +// WHY IT EXISTS (P7 — a real failure, not a hypothetical). The four trusted docs asserted, in the +// present tense, floor primitives that do not exist as running checks: a `pre-egress` hook (no +// .claude/hooks/*egress* file), the fix #5 archetype-maps agreement (validate.mjs CHECK 7 is +// CONDITIONAL and no manifest exists, so it has never fired), and `/pharn-estimate` (no such command). +// THREAT-MODEL.md even carried `_Closed._` on fix #5. That is the P0 disease — "written in the contract" +// mistaken for "therefore guaranteed" — reproduced inside the documents that DEFINE it, and NOTHING +// DETECTED IT: the three trusted docs are in .prettierignore AND excluded by name in +// .markdownlint-cli2.jsonc, and no floor checker reads their prose. The `trusted-doc-accuracy` (F7) +// increment corrected the sites by hand. This checker exists because that correction's only remaining +// remedy was "remember to update the docs", which is exactly the remedy-class +// .dev/memory-bank/lessons-learned.md L20 says WILL recur — the second occurrence being the trigger to +// give the class a floor check rather than another reminder. +// +// THE TWO DIRECTIONS IT CLOSES. +// 1. THE PRIMITIVE SHIPS, THE MARKERS REMAIN. Someone builds the pre-egress hook. The docs still say +// "specified; ships with the guarded surface", now UNDERSTATING a live protection. RED, naming +// every site to update. This is the load-bearing direction: it fires exactly when the repo gets +// BETTER, which is precisely when nobody is looking for a doc bug. +// 2. A MARKER IS DELETED, THE PRIMITIVE IS STILL ABSENT. The doc silently returns to overclaiming — +// the original F7 defect. RED, naming the file and the missing marker. +// +// WHY A STRUCTURED MANIFEST (lessons-learned L6). Membership is read from specified-primitives.json — +// never by scanning doc prose for what LOOKS like a marker. L6's defect (a membership fact grepped out +// of free text, so documentation ABOUT a declaration counts as a declaration) recurred inside F7's own +// REVIEW.md while correcting an unrelated error: a substring search for finding objects counted the +// remediation note that quoted the search pattern, inflating 6 findings to 7. A prose-scanning version +// of THIS checker would have the identical bug — a CHANGELOG sentence quoting a marker would register +// as a doc site. So the sites are enumerated, not discovered. +// +// HONEST SCOPE (P0) — what this does NOT guarantee, stated because it is not obvious. +// * IT CANNOT DISCOVER A NEW OVERCLAIM. The manifest is a hand-maintained address book. A doc that +// starts asserting some OTHER non-existent primitive tomorrow is invisible here until a human adds +// the entry. "The manifest checked out" NEVER means "the docs are true" — the same bound the +// lessons index carries, and the reason this file is not called check-doc-accuracy. +// * THE PROBE TESTS EXISTENCE, NOT FUNCTION. `dir-contains` asks whether a file whose NAME contains a +// substring is present; `path` asks whether a path exists. Neither asks whether a hook is WIRED in +// .claude/settings.json, whether it is invoked, or whether it works. A stub named pre-egress.cjs +// flips the probe to "live" and REDs direction 1 — deliberately: a loud early signal beats a silent +// one, and the remedy (check the doc, then the wiring) is the right next action either way. +// * SUBSTRING PRESENCE IS NOT SENTENCE COHERENCE. Direction 2 proves the marker bytes are still in the +// file. It says nothing about whether the surrounding sentence still makes sense, sits in the same +// section, or has been reduced to a quotation inside an unrelated paragraph. +// * IT IS APPARATUS. Nothing on the floor forces `npm run check` to run it; that wiring is a +// convention this file cannot enforce about itself. +// +// TRUST (P2): the manifest is a trusted, human-reviewed apparatus file and the docs it reads are +// trusted. No untrusted input is ingested; no free text steers a branch. Doc bytes are opaque DATA — +// read, compared by `String.prototype.includes`, never parsed, interpreted, or executed. +// +// Usage: node .dev/floor/check-specified-markers.mjs [targetDir] [--manifest ] +// targetDir defaults to cwd; --manifest defaults to the sibling specified-primitives.json and +// exists so the test suite can drive fixture manifests (an authored fixture passes by +// construction — L4 — so the RED paths must be reachable, not merely asserted). +// Exit: 0 GREEN | 1 RED (a drifted annotation) | 2 the manifest itself is unusable (fail-closed) + +import { existsSync, readdirSync, readFileSync, statSync } from "node:fs"; +import { join, dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const argv = process.argv.slice(2); +const mIdx = argv.indexOf("--manifest"); +const MANIFEST = mIdx !== -1 && argv[mIdx + 1] ? resolve(argv[mIdx + 1]) : join(HERE, "specified-primitives.json"); +const positional = argv.filter((a, i) => !a.startsWith("--") && i !== mIdx + 1); +const TARGET = resolve(positional[0] ?? process.cwd()); + +const reds = []; +const red = (msg) => reds.push(msg); + +// --- Load the manifest. Any defect here is exit 2, never a silent pass: a checker that cannot read its +// --- own membership set has no verdict to give, and reporting GREEN would be the worst outcome. +let manifest; +try { + manifest = JSON.parse(readFileSync(MANIFEST, "utf8")); +} catch (e) { + console.error(`RED (manifest unusable): cannot read/parse ${MANIFEST} — ${e.message}`); + process.exit(2); +} +const primitives = manifest.specified_primitives; +const named = manifest.named_artifacts ?? []; +if (!Array.isArray(primitives)) { + console.error(`RED (manifest unusable): ${MANIFEST} has no \`specified_primitives\` array`); + process.exit(2); +} + +/** + * Is the primitive LIVE? A pure existence test (ARCHITECTURE §2 primitive #3). + * Unknown probe type => throw, so a typo in the manifest fails CLOSED (exit 2) rather than + * silently reporting "not live" and passing every marker check. + */ +function isLive(probe, id) { + if (!probe || typeof probe !== "object") throw new Error(`primitive ${id}: missing probe`); + if (probe.type === "path") { + if (typeof probe.path !== "string") throw new Error(`primitive ${id}: probe.path must be a string`); + return existsSync(join(TARGET, probe.path)); + } + if (probe.type === "dir-contains") { + if (typeof probe.dir !== "string" || typeof probe.substring !== "string") { + throw new Error(`primitive ${id}: dir-contains needs string \`dir\` and \`substring\``); + } + const dir = join(TARGET, probe.dir); + let entries; + try { + if (!statSync(dir).isDirectory()) return false; + entries = readdirSync(dir); + } catch { + // The directory itself is absent/unreadable => nothing in it => the primitive is not live. + return false; + } + return entries.some((e) => e.includes(probe.substring)); + } + throw new Error(`primitive ${id}: unknown probe type ${JSON.stringify(probe.type)}`); +} + +function validatePrimitive(p) { + if (!p || typeof p !== "object") throw new Error("primitive record must be an object"); + if (typeof p.id !== "string") throw new Error("primitive record lacks string `id`"); + if (!Array.isArray(p.sites)) throw new Error(`primitive ${p.id}: \`sites\` must be an array`); + for (const site of p.sites) { + if (!site || typeof site !== "object") { + throw new Error(`primitive ${p.id}: each site must be an object`); + } + if (typeof site.file !== "string" || typeof site.marker !== "string") { + throw new Error(`primitive ${p.id}: each site needs string \`file\` and \`marker\` fields`); + } + } +} + +function validateNamedArtifact(a) { + if (!a || typeof a !== "object") throw new Error("named-artifact record must be an object"); + if (typeof a.id !== "string") throw new Error("named-artifact record lacks string `id`"); + if (typeof a.cited_in !== "string") { + throw new Error(`named-artifact ${a.id}: \`cited_in\` must be a string`); + } + if (typeof a.citation !== "string") { + throw new Error(`named-artifact ${a.id}: \`citation\` must be a string`); + } + if (typeof a.must_exist !== "string") { + throw new Error(`named-artifact ${a.id}: \`must_exist\` must be a string`); + } + if (a.forbidden !== undefined) { + if (!Array.isArray(a.forbidden)) { + throw new Error(`named-artifact ${a.id}: \`forbidden\` must be an array`); + } + for (const entry of a.forbidden) { + if (typeof entry !== "string") { + throw new Error(`named-artifact ${a.id}: each \`forbidden\` entry must be a string`); + } + } + } +} + +const fileCache = new Map(); +function readDoc(rel) { + if (!fileCache.has(rel)) { + try { + fileCache.set(rel, readFileSync(join(TARGET, rel), "utf8")); + } catch { + fileCache.set(rel, null); + } + } + return fileCache.get(rel); +} + +let siteCount = 0; +let liveCount = 0; + +try { + for (const p of primitives) { + validatePrimitive(p); + const live = isLive(p.probe, p.id); + if (live) liveCount++; + for (const site of p.sites) { + siteCount++; + const src = readDoc(site.file); + if (src === null) { + red(`${site.file}: listed as a site for \`${p.id}\` but the file could not be read`); + continue; + } + const present = src.includes(site.marker); + if (live && present) { + // DIRECTION 1 — the repo got better and the docs did not follow. + red( + `${site.file}: \`${p.id}\` IS NOW LIVE (${describe(p.probe)}), but the doc still annotates it ` + + `as not-yet-live. REMOVE the marker — the doc now UNDERSTATES a real protection.\n` + + ` marker: ${JSON.stringify(site.marker)}` + ); + } else if (!live && !present) { + // DIRECTION 2 — the annotation was removed while the primitive is still absent. + red( + `${site.file}: the \`${p.id}\` annotation is GONE, but the primitive is still absent ` + + `(${describe(p.probe)}). The doc has returned to asserting a protection the repo does not have.\n` + + ` expected: ${JSON.stringify(site.marker)}` + ); + } + } + } + + for (const a of named) { + validateNamedArtifact(a); + const src = readDoc(a.cited_in); + if (src === null) { + red(`${a.cited_in}: named-artifact citation for \`${a.id}\` — the file could not be read`); + continue; + } + if (!src.includes(a.citation)) { + red(`${a.cited_in}: no longer cites \`${a.id}\` as ${JSON.stringify(a.citation)} — the name drifted`); + } + if (a.forbidden) { + for (const forbidden of a.forbidden) { + if (src.includes(forbidden)) { + red(`${a.cited_in}: still cites \`${a.id}\` by the obsolete name ${JSON.stringify(forbidden)} — ` + "remove the legacy citation"); + } + } + } + if (!existsSync(join(TARGET, a.must_exist))) { + red(`${a.cited_in} cites \`${a.id}\`, but ${a.must_exist} does not exist — the doc names a missing artifact`); + } + } +} catch (e) { + console.error(`RED (manifest unusable): ${e.message}`); + process.exit(2); +} + +function describe(probe) { + return probe.type === "path" ? `${probe.path} exists` : `${probe.dir}/*${probe.substring}*`; +} + +if (reds.length) { + console.error(`SPECIFIED-MARKERS: RED — ${reds.length} drifted annotation(s)\n`); + for (const r of reds) console.error(` - ${r}`); + console.error( + `\nManifest: ${MANIFEST}\n` + `This checks only the annotations that manifest LISTS. It cannot discover a new overclaim (P0).` + ); + process.exit(1); +} + +console.log( + `SPECIFIED-MARKERS: GREEN — ${siteCount} annotation(s) across ${primitives.length} specified primitive(s) ` + + `(${liveCount} now live), ${named.length} named artifact(s) checked in ${TARGET}.\n` + + `NOTE (P0): this proves the LISTED annotations still match reality. It never means the docs are accurate — ` + + `an overclaim not in the manifest is invisible here.` +); diff --git a/.dev/floor/check-specified-markers.test.mjs b/.dev/floor/check-specified-markers.test.mjs new file mode 100644 index 0000000..43a6feb --- /dev/null +++ b/.dev/floor/check-specified-markers.test.mjs @@ -0,0 +1,261 @@ +// .dev/floor/check-specified-markers.test.mjs — apparatus tests for the specified-marker checker. +// +// L4: an authored fixture passes by construction. These tests therefore drive the RED paths with real +// fixture trees + fixture manifests (via --manifest), and the ✧ cases are MUTANTS — each asserts the +// checker fails when the thing it guards is broken, not merely that it passes when everything is fine. + +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const CHECKER = join(HERE, "check-specified-markers.mjs"); +const REPO = join(HERE, "..", ".."); + +const MARKER = "`pre-egress` _(specified; ships with the guarded surface)_ — blocks a network call"; + +/** Run the checker; never throws. Returns {code, out}. */ +function run(target, manifest) { + const args = [CHECKER, target]; + if (manifest) args.push("--manifest", manifest); + try { + const out = execFileSync(process.execPath, args, { encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] }); + return { code: 0, out }; + } catch (e) { + return { code: e.status, out: `${e.stdout ?? ""}${e.stderr ?? ""}` }; + } +} + +/** Build a throwaway target tree + manifest. `opts.hookName` present => the primitive is "live". */ +function fixture(opts = {}) { + const dir = mkdtempSync(join(tmpdir(), "pharn-markers-")); + mkdirSync(join(dir, ".claude", "hooks"), { recursive: true }); + if (opts.hookName) writeFileSync(join(dir, ".claude", "hooks", opts.hookName), "// stub\n"); + writeFileSync(join(dir, "DOC.md"), opts.docBody ?? `intro\n${MARKER}\noutro\n`); + + const manifest = { + specified_primitives: [ + { + id: "pre-egress", + probe: opts.probe ?? { type: "dir-contains", dir: ".claude/hooks", substring: "egress" }, + sites: opts.sites ?? [{ file: "DOC.md", marker: MARKER }], + }, + ], + named_artifacts: opts.named ?? [], + }; + const mPath = join(dir, "manifest.json"); + writeFileSync(mPath, JSON.stringify(manifest)); + return { dir, mPath, cleanup: () => rmSync(dir, { recursive: true, force: true }) }; +} + +// ---------------------------------------------------------------- the two correct states are GREEN + +test("GREEN — primitive absent and the marker is present (the F7 steady state)", () => { + const f = fixture(); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); + assert.match(r.out, /GREEN/); +}); + +test("GREEN — primitive live and the marker has been removed (the correct post-ship state)", () => { + const f = fixture({ hookName: "pre-egress.cjs", docBody: "intro\nno annotation here\noutro\n" }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); +}); + +// ---------------------------------------------------------------- ✧ direction 1: the primitive ships + +test("✧ RED — the primitive SHIPPED but the marker remains (the doc now understates a real protection)", () => { + const f = fixture({ hookName: "pre-egress.cjs" }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /IS NOW LIVE/); + assert.match(r.out, /REMOVE the marker/); + assert.match(r.out, /DOC\.md/); +}); + +test("✧ the live-probe is a SUBSTRING match on the filename, so any *egress* file trips it", () => { + const f = fixture({ hookName: "my-pre-egress-hook.cjs" }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); +}); + +test("a same-directory file NOT matching the substring leaves the primitive not-live", () => { + const f = fixture({ hookName: "unrelated.cjs" }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); +}); + +// ------------------------------------------------------------- ✧ direction 2: the marker is deleted + +test("✧ RED — the marker was DELETED while the primitive is still absent (silent return to overclaiming)", () => { + const f = fixture({ docBody: "intro\nblocks a network call to a domain not on a hardcoded allowlist.\noutro\n" }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /annotation is GONE/); + assert.match(r.out, /still absent/); +}); + +test("✧ RED — a one-character edit to the marker counts as deleted (exact substring, not fuzzy)", () => { + const f = fixture({ docBody: `intro\n${MARKER.replace("specified;", "specified:")}\noutro\n` }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); +}); + +test("✧ RED — a site file that cannot be read is reported, never skipped", () => { + const f = fixture({ sites: [{ file: "NOPE.md", marker: MARKER }] }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /could not be read/); +}); + +// -------------------------------------------------------------------- ✧ fail-closed on a bad manifest + +test("✧ exit 2 — an unknown probe type fails CLOSED, never a silent GREEN", () => { + const f = fixture({ probe: { type: "vibes", dir: ".claude/hooks" } }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /unknown probe type/); +}); + +test("✧ exit 2 — an unreadable manifest fails CLOSED", () => { + const f = fixture(); + const r = run(f.dir, join(f.dir, "does-not-exist.json")); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /manifest unusable/); +}); + +test("✧ exit 2 — a manifest with no specified_primitives array fails CLOSED", () => { + const f = fixture(); + writeFileSync(f.mPath, JSON.stringify({ nothing: true })); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); +}); + +test("✧ exit 2 — a primitive with non-array sites fails CLOSED", () => { + const f = fixture(); + writeFileSync( + f.mPath, + JSON.stringify({ + specified_primitives: [{ id: "pre-egress", probe: { type: "path", path: "DOC.md" }, sites: "DOC.md" }], + }) + ); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /sites.*must be an array/); +}); + +test("✧ exit 2 — a site lacking string file/marker fields fails CLOSED", () => { + const f = fixture(); + writeFileSync( + f.mPath, + JSON.stringify({ + specified_primitives: [{ id: "pre-egress", probe: { type: "path", path: "DOC.md" }, sites: [{ file: "DOC.md" }] }], + }) + ); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /string `file` and `marker`/); +}); + +test("✧ exit 2 — a named-artifact lacking required string fields fails CLOSED", () => { + const f = fixture({ named: [{ id: "ghost", cited_in: "DOC.md" }] }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /citation.*must be a string/); +}); + +// ------------------------------------------------------------------------------ named-artifact half + +test("✧ RED — a doc citing an artifact that does not exist (the security-secrets name-drift class)", () => { + const f = fixture({ + named: [{ id: "ghost", cited_in: "DOC.md", citation: "intro", must_exist: "pharn/pharn-review/ghost" }], + }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /does not exist/); +}); + +test("✧ RED — the citation itself drifting out of the doc is caught", () => { + const f = fixture({ + named: [{ id: "x", cited_in: "DOC.md", citation: "a name no longer present", must_exist: ".claude" }], + }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /name drifted/); +}); + +test("✧ RED — obsolete legacy citation coexisting with the corrected name is caught", () => { + const f = fixture({ + docBody: "intro\nsecrets-in-code lens and security-secrets lens\noutro\n", + named: [ + { + id: "secrets-in-code", + cited_in: "DOC.md", + citation: "secrets-in-code lens", + forbidden: ["security-secrets lens"], + must_exist: ".claude", + }, + ], + }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /obsolete name/); + assert.match(r.out, /remove the legacy citation/); +}); + +test("✧ exit 2 — a named-artifact with non-array forbidden fails CLOSED", () => { + const f = fixture({ + named: [ + { + id: "secrets-in-code", + cited_in: "DOC.md", + citation: "secrets-in-code lens", + forbidden: "security-secrets lens", + must_exist: ".claude", + }, + ], + }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /forbidden.*must be an array/); +}); + +// ------------------------------------------------------------------------------------- integration + +test("integration — the REAL repo is GREEN against the REAL manifest", () => { + const r = run(REPO); + assert.equal(r.code, 0, r.out); + assert.match(r.out, /GREEN/); + // The count is asserted as a floor, not a fixed number: adding an annotation must not break this. + const m = r.out.match(/GREEN — (\d+) annotation/); + assert.ok(m && Number(m[1]) >= 11, `expected >= 11 annotations, got: ${r.out}`); +}); + +test("the GREEN line states its own P0 bound (the manifest is not a discovery mechanism)", () => { + const r = run(REPO); + assert.equal(r.code, 0, r.out); + assert.match(r.out, /never means the docs are accurate/); +}); diff --git a/.dev/floor/specified-primitives.json b/.dev/floor/specified-primitives.json new file mode 100644 index 0000000..39cdbe8 --- /dev/null +++ b/.dev/floor/specified-primitives.json @@ -0,0 +1,106 @@ +{ + "$comment": [ + "The address book for PHARN's own 'specified; ships with the guarded surface' annotations.", + "Read by .dev/floor/check-specified-markers.mjs. STRUCTURED on purpose (lessons-learned L6): a", + "membership fact is read from here, never by scanning doc prose for what looks like a marker.", + "", + "ADDING AN ENTRY is what makes a new annotation guarded. Nothing discovers annotations; this file", + "is a hand-maintained list, so an unlisted overclaim is invisible to the checker. Stated in the", + "checker's header as its central limit." + ], + + "specified_primitives": [ + { + "id": "pre-egress", + "what": "a PreToolUse hook blocking a network call to a domain not on a hardcoded allowlist", + "probe": { "type": "dir-contains", "dir": ".claude/hooks", "substring": "egress" }, + "sites": [ + { + "file": "pharn/ARCHITECTURE.md", + "marker": "`pre-egress` _(specified; ships with the guarded surface)_ — blocks a network call" + }, + { + "file": "pharn/ARCHITECTURE.md", + "marker": "fooled — specified; ships with the guarded surface) and the **constitution/trusted-file write-guard**" + }, + { + "file": "THREAT-MODEL.md", + "marker": "pre-write hook + pre-egress (specified; ships with the guarded surface)" + }, + { + "file": "THREAT-MODEL.md", + "marker": "pre-egress (specified; ships with the guarded surface) + enum gate" + }, + { + "file": "LIMITS.md", + "marker": "pre-write hook; pre-egress specified, ships with the guarded surface)" + }, + { + "file": "LIMITS.md", + "marker": "the pre-write / writes-scope hooks (and pre-egress, specified; ships with the guarded surface) re-gate" + }, + { + "file": "CLAUDE.md", + "marker": "_(specified; ships with the guarded surface)_ (block non-allowlisted network calls)." + } + ] + }, + + { + "id": "archetype-maps", + "what": "the archetype-maps manifest that makes validate.mjs CHECK 7 (fix #5) fire at all", + "probe": { "type": "path", "path": "pharn/pharn-contracts/archetype-maps.json" }, + "sites": [ + { + "file": "pharn/ARCHITECTURE.md", + "marker": "the four archetype maps agree (fix #5 — conditional; specified, ships with the guarded surface)" + }, + { + "file": "THREAT-MODEL.md", + "marker": "_Specified; the check is conditional and no manifest exists, so it never fires._" + } + ] + }, + + { + "id": "pharn-estimate", + "what": "the /pharn-estimate command LIMITS.md §1c says reports an est_tokens range", + "probe": { "type": "dir-contains", "dir": ".claude/commands", "substring": "estimate" }, + "sites": [ + { + "file": "LIMITS.md", + "marker": "`/pharn-estimate` _(specified; ships with the guarded surface)_ reports a range," + } + ] + }, + + { + "id": "pharn-audits", + "what": "the pharn-audits module whose auditors LIMITS.md §3b names in its cost model", + "probe": { "type": "path", "path": "pharn/pharn-audits" }, + "sites": [ + { + "file": "LIMITS.md", + "marker": "security-review auditors — specified; ships with the guarded surface)" + } + ] + } + ], + + "named_artifacts": [ + { + "$comment": "The other half of the F7 defect: a doc naming a SHIPPED artifact by a name it does not have. LIMITS.md said `security-secrets lens`; the lens ships as `secrets-in-code`. This guards the corrected name against re-drift in either direction.", + "id": "secrets-in-code", + "cited_in": "LIMITS.md", + "citation": "secrets-in-code lens", + "forbidden": ["security-secrets lens"], + "must_exist": "pharn/pharn-review/secrets-in-code" + }, + { + "id": "security-griller", + "cited_in": "LIMITS.md", + "citation": "(security\ngriller,", + "must_exist": "pharn/pharn-pipeline/grillers/security" + } + ] +} diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 644379f..c61036d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -36,6 +36,9 @@ jobs: - name: Docs drift check if: ${{ always() && steps.install.outcome == 'success' }} run: npm run docs:check + - name: Specified-marker check + if: ${{ always() && steps.install.outcome == 'success' }} + run: npm run check:markers - name: Test if: ${{ always() && steps.install.outcome == 'success' }} run: npm test diff --git a/CHANGELOG.md b/CHANGELOG.md index 39549b3..ab04a34 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,24 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Fixed +- **Three trusted docs stopped describing floor primitives that do not exist as running checks — and a shipped lens was named by a name it has never had.** The governing text asserted, in the present tense, protections the repo does not have. Each site was verified absent **this run** rather than accepted from the request: **no `.claude/hooks/*egress*` file exists** (three hooks, none of them egress); **no archetype-maps manifest exists** at the path `pharn/floor/validate.mjs:233` looks for, and validate's own header (`:15`) already calls CHECK 7 **"conditional — if an archetype-maps manifest exists"**, so the check has never fired; **no `/pharn-estimate` command exists** and `est_tokens` is named only in prose, emitted by no `.mjs`/`.cjs`/`.json`; **no `pharn/pharn-audits/` module exists**; and `pharn/pharn-review/` ships **22 lenses**, among them `secrets-in-code` — there is no `security-secrets`. Ten sites are now annotated `(specified; ships with the guarded surface)`, and one is a **name correction**, not a marker: a lens that ships under a different name is not a deferred primitive. + + **The rule applied, and its one exception.** Where a doc names a floor primitive that is designed but not running, its present-tense phrasing claims a protection that does not exist — the P0 disease ("written in the contract" mistaken for "therefore guaranteed") reproduced in the documents that **define** that disease. The marker preserves the design intent without asserting the protection. `THREAT-MODEL.md:91`'s `_Closed._` on fix #5 was the sharpest case: a closure status is a stronger claim than a description, and CHECK 7's own **conditional** wording contradicted it, so it now reads _"Specified; the check is conditional and no manifest exists, so it never fires."_ — stating the mechanism, not merely withdrawing the claim. + + **Sites (11 edits, 4 files).** `pharn/ARCHITECTURE.md` — the `pre-egress` primitive entry (`:41`), the §7 pre-egress-allowlist clause (`:239`, folded into the existing parenthetical rather than jammed mid-clause at `:237` where the request placed it), and the validate-contract list's fix #5 item (`:258`, matching validate's "conditional"). `THREAT-MODEL.md` — both mechanism cells naming a `pre-egress hook` (`:70`, `:71`) and the fix #5 closure status (`:91`). `LIMITS.md` — the §1a backstop citation (`:29`), `/pharn-estimate` (`:52`), the §1d re-gate claim (`:75`, where pre-write / writes-scope stay **live** and only pre-egress is marked), and §3b's `security-secrets` → **`secrets-in-code`** plus a marker on `security-review auditors` (`:108`). The `security` **griller** on that same line **does** ship (`pharn/pharn-pipeline/grillers/security/`) and was left alone. **`CLAUDE.md:256` is corrected too, and it was not in the request:** an L1 meta-doc sweep found it listing `pre-egress` as one of the three live primitives, identically to `ARCHITECTURE.md:41` — leaving it would have shipped the exact stale canon the change exists to remove. + + **This corrects documentation accuracy. It closes no threat, ships no new check, and adds no capability.** Building the `pre-egress` hook, the archetype-maps manifest, or `/pharn-estimate` is trigger-gated (P7) and explicitly out of scope. + + **A larger defect was found in the same sweep and is deliberately NOT fixed here (P7 — recorded, not smuggled in).** `LIMITS.md:28-29` and `THREAT-MODEL.md:102` are not descriptions but **backstop** claims: each strikes a claim as a limit, then points at the floor as what bounds the residual ("**Backstop (floor):** `kind: community` cannot declare trusted-write or off-allowlist egress"). Live, **both halves are empty** — the egress hook does not exist, **and** `KIND_ENUM` is read at exactly one place (`pharn/floor/validate.mjs:155`) to check that `kind`'s _value_ is an enum member; **nothing conditions any privilege on `kind: community`**. So the marker applied here leaves `:29` less wrong while it still claims a live `pre-write` backstop for a restriction no check enforces. Stating that plainly is the honest cost of scoping this change to doc-marking: **after this lands, a trusted doc still points at a backstop that does not exist on either half.** Follow-up: `community-privilege-backstop`. The P7 trigger is already satisfied — this is a verified defect, not a hypothetical. + + **The marker's own honest bound (P0).** `(specified; ships with the guarded surface)` names no artifact, no condition, and no owner, so it is **not falsifiable** — it warns a reader the primitive is not live, and in exchange asserts a schedule nothing enforces. It is an improvement on a false present-tense claim, not a guarantee, and it is not a substitute for a stated reopen trigger. Recorded because a marker that reads as a promise is the same disease one step removed. + + **Mechanism, stated rather than disguised (L19).** The three trusted docs are hook-protected: `protect-trusted-paths.cjs` denies every `Write|Edit|MultiEdit` to them (fix #2), which is why the ten edits were applied through **Bash** — a path that passes **neither** fix #2 **nor** fix #7, because `PreToolUse` hooks are not consulted at all there. They were therefore **not** declared in the plan's `## Files`: naming them would have granted Write-tool scope to three hook-protected docs, the over-declaration **L7** forbids and the exact over-grant **L18** reproduced live. The scope setter resolved **3 path(s)** — `SKILLS_VERSION`, `CHANGELOG.md`, `CLAUDE.md` — matching the approved list, checked rather than assumed (**L20**). What replaces the gate is a human wording approval, CODEOWNERS review on `main`, and a **per-edit assertion**: every substitution was required to match **exactly once** across all ten before any file was written (fail-closed), then re-verified after writing that the old string is gone and the new string present exactly once. That check exists because the grill stage raised it as a blocking-severity P6 concern — `sed` no-ops silently on a non-matching pattern, and the trusted docs are excluded from **both** formatters, so a partially-applied patch set would have been invisible at every downstream gate. + + **No formatter can touch the three trusted docs, verified rather than assumed:** they are listed in `.prettierignore` and excluded by name in `.markdownlint-cli2.jsonc`, so the two table-cell edits in `THREAT-MODEL.md` cannot trip `format:check` on realignment. `CLAUDE.md` **is** formatter-governed, but prettier's markdown `proseWrap` defaults to `preserve` and MD013 is off. + + **`SKILLS_VERSION` → `2.5.1`** — **patch**. The three trusted docs are in the bump-triggering set and prose-only edits to shipped bytes bump by the rule in `CLAUDE.md`; this is a correction/clarification to bytes that already shipped, with no contract, finding-shape, frontmatter, or command change, and nothing an install emits changes. `CLAUDE.md` is repo-meta and drives no bump on its own. + - **The spec→plan chain now pins WHICH spec a plan implements, not only what that spec said — and it can finally read the field format the plan stage documents emitting.** `check-plan-spec-agree.mjs` asserted `planHash === specHash` and nothing else (its own header advertised "exactly ONE new assertion"), so a `PLAN.md` carrying the **wrong** `spec_id` with the **right** `spec_content_hash` passed the entire pipeline GREEN. The body pin proves a plan was made against _some_ current Approved spec; it never proved it was made against _the_ one the plan names, so the record was **mislabeled** and every downstream stage inherited the wrong identity. Reproduced on the live tree before anything changed: a PLAN declaring `spec_id: SOME-OTHER-SPEC` against a matching spec exits **0**. Underneath it sat a second, worse defect: the carried-field parse did **not** strip YAML inline comments, and `.claude/commands/pharn-plan.md` documents emitting `spec_content_hash: # fix #4 — carried forward; …`. A PLAN written **exactly the way the command says to write it** therefore **false-RED'd** — `spec_content_hash is not a sha256: "fdc516…385b # fix #4 — …"` — because the documented note was read as part of the 64-hex value and failed its own enum-gate. The comment defect had to be fixed **first**: a naive identity equality would have inherited it, since ` # carried from the Approved SPEC` never equals ``. Both were reproduced before the change and re-verified after. **Two parts.** `stripComment()` + `readValue()` — in `check-spec.mjs`'s `parseSpec` and in `check-plan-spec-agree.mjs`'s carried-field read — treat a value-initial or whitespace-preceded `#` as a YAML comment. **The order is load-bearing:** the _quote_ is resolved **first** — a quoted scalar's interior is taken up to its closing quote and a real comment after that quote is discarded — because stripping a whitespace-preceded `#` and everything after it first would eat the closing quote of a value that legitimately contains one. Resolving the quote first is what gets **both** shapes right: a quoted value containing a hash keeps it, and a quoted value _followed_ by a note drops the note. `parseSpec` stores **every** field, not only the three this checker gates, so an unrelated quoted field must survive. `feat#3` has no preceding whitespace, is not a comment, and survives byte-exact. Second, `check-spec.mjs` gains a **`--spec-id`** mode mirroring `--hash` exactly (unreadable → 1, no frontmatter → 1, no `spec_id` → an empty line at exit 0), and the chain checker shells it for the SPEC's identity while reading the PLAN's locally — the same deliberate asymmetry the hash already uses (SPEC fields through the `check-spec` CLI, PLAN fields through the P3-local parse), so SPEC parsing stays in exactly one place (P4). @@ -67,6 +85,18 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Added +- **The "specified; ships with the guarded surface" annotations are now floor-checked in BOTH drift directions — the entry below corrected the docs by hand, and this stops that correction from rotting.** The F7 fix reduced to "remember to update the docs when the primitive ships", which is exactly the remedy-class `.dev/memory-bank/lessons-learned.md` **L20** says WILL fail. The P7 trigger is not hypothetical and not new: the trusted docs asserted non-existent floor primitives in the present tense and **nothing detected it** — all three are in `.prettierignore` **and** excluded by name in `.markdownlint-cli2.jsonc`, and no floor checker reads their prose, so the drift was structurally invisible for as long as it was true. New `.dev/floor/check-specified-markers.mjs` + `.dev/floor/specified-primitives.json`, wired into `npm run check` as `check:markers`. + + **Two directions, and the first one is the one nobody would catch.** **(1) The primitive SHIPS while its markers remain** → RED naming every site: the doc now **understates** a live protection. This fires precisely when the repo gets **better**, which is exactly when no one is auditing the docs for a bug. **(2) A marker is DELETED while the primitive is still absent** → RED: a silent return to overclaiming, the original F7 defect. A third check covers the other half of what F7 fixed — `named_artifacts` asserts a doc citing a shipped artifact still names it correctly **and** that the artifact exists, and rejects any `forbidden` legacy citation still present, guarding the `security-secrets` → `secrets-in-code` name-drift class in both directions. + + **Membership is read from a STRUCTURED manifest, never from prose — and that is a lesson applied, not a preference.** L6 ("a membership fact is read from the structured location, never grepped from free text") **recurred inside F7's own `REVIEW.md`** while correcting an unrelated error: a substring search for finding objects counted a remediation note that quoted the search pattern, inflating 6 findings to 7. A prose-scanning version of this checker would carry the identical defect — a CHANGELOG sentence quoting a marker would register as a doc site. So the 11 annotation sites are **enumerated** in `specified-primitives.json`, not discovered. + + **Reproduced live, not merely asserted (L4 — an authored fixture passes by construction).** A stub `.claude/hooks/pre-egress.cjs` was created in the real tree: the checker went **RED on all 7 pre-egress sites**, each naming its file and the exact marker to remove; deleting the stub returned it to exit 0. **20 tests**, of which 15 are `✧` mutants driving the RED and fail-closed paths with fixture trees and fixture manifests (hence the `--manifest` flag): the primitive shipping, a one-character marker edit, an unreadable site file, an unknown probe type, an unreadable manifest, a manifest with no `specified_primitives`, malformed primitive/site/named-artifact records, a coexisting obsolete legacy citation, and both named-artifact failures. An unknown probe type exits **2**, never a silent GREEN — a checker that cannot read its own membership set has no verdict to give. + + **Honestly bounded (P0), in the checker's own header as well as here.** It **cannot discover a new overclaim**: the manifest is a hand-maintained address book, so a doc that starts asserting some _other_ non-existent primitive tomorrow is invisible until a human adds the entry — **"the manifest checked out" NEVER means "the docs are true"**, the same bound the lessons index carries and the reason this is not called `check-doc-accuracy`. The probe tests **file existence**, never that a hook is **wired** in `.claude/settings.json` or that it works, so a stub flips it to "live" — deliberately, since a loud early signal beats a silent one and the remedy is the right next action either way. Substring presence is not sentence coherence. And nothing on the floor forces `npm run check` to invoke it; that wiring is a convention this file cannot enforce about itself. + + **No `SKILLS_VERSION` bump.** Every path is apparatus (`.dev/**`) or repo-meta (`package.json`, `CLAUDE.md`, `CHANGELOG.md`); the product surface is byte-unchanged. It lives in `.dev/floor/` rather than `pharn/floor/` because it guards **PHARN's own** governing docs — a user's install has no reason to check PHARN's annotations, and the dependency may only point `.dev/` → `pharn/`. + - **Ported the lessons-index READ side to the product surface (`product-lessons-index`)** — the follow-up reserved when PR #115 (`59def15`) shipped the index dev-only. Three new stdlib-only checkers under `pharn/floor/` — [`lessons-index-core.mjs`](./pharn/floor/lessons-index-core.mjs), [`gen-lessons-index.mjs`](./pharn/floor/gen-lessons-index.mjs), [`check-lessons-index.mjs`](./pharn/floor/check-lessons-index.mjs) — render a one-line-per-lesson address book (`id | type | concepts | title | promoted | ~tokens`) over a **user's** `memory-bank/lessons-learned.md`, and `/pharn-plan`'s mandatory lessons sweep becomes the **two-step** form: **select** candidates from the index, then **read each candidate's full `## L` entry from canon** before declaring `applied_lessons`. **`SKILLS_VERSION` → `2.3.0`** — **minor**, not major, and the reason matters: nothing an existing install already emits becomes RED. `pharn/floor/check-plan-lessons.mjs` is **byte-identical** (it still verifies the declaration against **canon**, never the index — which is why a stale or poisoned index cannot corrupt the floor gate), and every degraded index state resolves to _read canon in full and say so_, never to a block. A repo with no memory-bank and no index plans exactly as it did before. - **The location decision, and its cost, stated rather than buried.** The index is written to **`.pharn/lessons-index.md`** — gitignored runtime scratch — so the shipped guarantee is **narrower than the dev original's**: it is a **staleness** comparison over a **disposable cache**, _not_ `docs/lessons-index.md`'s "committed == recomputed" byte-equality, and its coverage is **machine-local and ephemeral** (a fresh clone has no cache, which is GREEN by design). `memory-bank/lessons-index.md` was rejected because a Bash-run generator writing into the fail-closed gated-canon zone would **normalize a fix #7 bypass**; `docs/lessons-index.md` was rejected because in a user's repo `docs/` is **the user's directory**, and writing there is a scope claim on ground PHARN does not own. There was no free option, and the trade-off was put to the human at the plan gate. - **`NO_CANON` and `COLD` are GREEN on purpose — the one behavioral divergence from the dev core.** The dev core _throws_ on absent or empty canon ("refusing to render an empty index as fact"), which is correct where canon always exists and hostile where it usually does not: a user's repo commonly has **no** memory-bank, and a fresh clone never has a cache. Both are the honest normal state of a new install, so both are benign no-ops. **`STALE` is the only drift RED**, because it is the only state in which the cache could actively _mislead_ a selection; `ENUM_ERROR` (duplicate id, unsafe title, CHECK-5 hazard) blames **canon** and deliberately does **not** prescribe a regenerate that cannot succeed. diff --git a/CLAUDE.md b/CLAUDE.md index eb46656..7728e28 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -179,6 +179,23 @@ node .dev/floor/check-provenance.mjs # ran under that model (the platform applies model/effort; that binding is advisory). Exits non-zero on RED. node .dev/floor/check-config.mjs [validate | resolve | agreement] +# Bind PHARN's own "(specified; ships with the guarded surface)" annotations to reality, BOTH ways. +# The four trusted docs asserted floor primitives that do not exist as running checks (no pre-egress +# hook; no archetype-maps manifest, so validate CHECK 7 never fires; no /pharn-estimate) and NOTHING +# detected it — they are .prettierignore'd AND markdownlint-excluded, and no checker reads their prose. +# FLOOR (enum/regex, primitive #3), two directions: (1) a primitive that SHIPS while its markers remain +# → RED naming every site (the doc now UNDERSTATES a live protection — it fires exactly when the repo +# gets BETTER, which is when nobody is looking); (2) a marker DELETED while the primitive is still +# absent → RED (a silent return to overclaiming). Also checks `named_artifacts`: a doc citing a shipped +# artifact by a name it does not have (the `security-secrets` → `secrets-in-code` drift). +# Membership comes from the STRUCTURED .dev/floor/specified-primitives.json, never from scanning prose +# — L6, whose defect recurred inside this very increment's REVIEW.md. +# NARROWED, and stated: it CANNOT DISCOVER a new overclaim (the manifest is a hand-maintained address +# book — "the manifest checked out" NEVER means "the docs are true"), and the probe tests file +# EXISTENCE, never that a hook is WIRED in settings.json or works. Apparatus: no SKILLS_VERSION bump. +# Wired into `npm run check` as `check:markers`. Exits non-zero on RED; exit 2 on an unusable manifest. +node .dev/floor/check-specified-markers.mjs [target-dir] [--manifest ] + # Regenerate / drift-check the derived one-line index over .dev/memory-bank/lessons-learned.md. # Both are folded into `npm run docs:generate` / `npm run docs:check` (the latter inside `npm run check`), # so promoting a lesson without regenerating is a loud RED. FLOOR: byte-equality (committed == recomputed) @@ -254,7 +271,7 @@ either blocks. three deterministic, non-LLM primitives — every guarantee in the system must reduce to one: 1. **Hooks** — `pre-write` (block writes to protected paths / out-of-`writes`-scope), `pre-egress` - (block non-allowlisted network calls). + _(specified; ships with the guarded surface)_ (block non-allowlisted network calls). 2. **Content-hash** — detects silent mutation of a pinned artifact (the spec, a seam resolution). 3. **Enum / regex check** — set membership or pattern match (`validate.mjs` and at gates). diff --git a/LIMITS.md b/LIMITS.md index 21ea029..57a1aec 100644 --- a/LIMITS.md +++ b/LIMITS.md @@ -26,7 +26,7 @@ thing whose purpose is to be executed as instructions. - **Struck claim:** "markdown-only = safe." - **Backstop (floor):** `kind: community` cannot declare trusted-write or off-allowlist egress - (`ARCHITECTURE.md §5`, pre-write + pre-egress hooks). Safety comes from the floor, not from the + (`ARCHITECTURE.md §5`, pre-write hook; pre-egress specified, ships with the guarded surface). Safety comes from the floor, not from the absence of `.cjs`. ### 1b. The fence is enforced by the model that may be compromised @@ -49,7 +49,7 @@ frontmatter cannot express). - **Struck claim:** "`est_tokens` = cost." - **True statement:** `est_tokens` is an estimate with a confidence band; the real number is the - **measured runtime cost** (the system already observes it). `/pharn-estimate` reports a range, + **measured runtime cost** (the system already observes it). `/pharn-estimate` _(specified; ships with the guarded surface)_ reports a range, not a promise. The 2.0 cost-per-feature metric is built on _measured_ cost, not the declared field. @@ -72,7 +72,7 @@ but the gate's integrity here rests on _who may write the enum field_, and in pu (the post-verify decision) and every between-stage proceed: _invoking and obeying_ the checker is advisory orchestration, not a floor primitive (the "two clocks," stated in the checkers themselves). - **Backstop (floor):** a forged approval moves only the advisory _intent-approval_ signal; it unlocks - no floor-gated capability — the pre-write / writes-scope and pre-egress hooks re-gate every downstream + no floor-gated capability — the pre-write / writes-scope hooks (and pre-egress, specified; ships with the guarded surface) re-gate every downstream write and network call regardless of `state`, and the human GATE-2 decision still stands between a built increment and merge. Closing the gate itself needs an out-of-band approval signal the Write tool cannot forge (e.g. a `PreToolUse` hook admitting the Draft→Approved transition only against a @@ -105,7 +105,7 @@ the largest practical token problem and it is not yet solved. ### 3b. Rule overlap × stages The same rule file (`security.md`) is loaded into context 3–4× across a feature's life (security -griller, security-secrets lens, security-review auditors) — each fresh sub-agent re-pays. Tiered +griller, secrets-in-code lens, security-review auditors — specified; ships with the guarded surface) — each fresh sub-agent re-pays. Tiered loading does not cache between stages (fresh contexts naively cannot). Real cost ≈ `diff_size × (validators + verifiers + lenses + auditors) + rule_overlap` — both terms large, neither touched by tiered loading. diff --git a/SKILLS_VERSION b/SKILLS_VERSION index 437459c..73462a5 100644 --- a/SKILLS_VERSION +++ b/SKILLS_VERSION @@ -1 +1 @@ -2.5.0 +2.5.1 diff --git a/THREAT-MODEL.md b/THREAT-MODEL.md index c2f8480..1e90a7d 100644 --- a/THREAT-MODEL.md +++ b/THREAT-MODEL.md @@ -67,8 +67,8 @@ Every answer reduces to the floor (P0) or is labeled a limit (`LIMITS.md`). | memory poisoning | promotion to canon is a **gated write** with per-entry provenance | pre-write hook | | reviewed-code injection | finding is computed from enum-gated fields; injected comment reaches only free-text (fix #1) | enum check | | seam-record poisoning | content-hash on resolution; drift is loud, not silent | content-hash | -| community Capability | `kind` is a **privilege level**: community = markdown-only, no `.cjs`; cannot declare trusted-write or off-allowlist egress | pre-write + pre-egress hook | -| cross-model response | egress-fenced + response returns in an untrusted fence + advisory-only (never gates ship) | pre-egress hook + enum gate | +| community Capability | `kind` is a **privilege level**: community = markdown-only, no `.cjs`; cannot declare trusted-write or off-allowlist egress | pre-write hook + pre-egress (specified; ships with the guarded surface) | +| cross-model response | egress-fenced + response returns in an untrusted fence + advisory-only (never gates ship) | pre-egress (specified; ships with the guarded surface) + enum gate | --- @@ -88,7 +88,7 @@ The red-team's verdict: a **single disease in five places — "written in the co invariants from actual content) vs advisory-gate (reads LLM severity, never sole basis for a guaranteed block) (fix #3). _Closed by separation; the advisory half is labeled, not "fixed."_ 4. **spec_id binds identity not content** — plan pins `spec_content_hash` (fix #4). _Closed._ -5. **Archetype maps drift** — `validate` checks the four maps agree (fix #5). _Closed._ +5. **Archetype maps drift** — `validate` checks the four maps agree (fix #5). _Specified; the check is conditional and no manifest exists, so it never fires._ 6. **CI checks namespace not meaning** — `validate` requires every `enforces` rule*id to be produced by ≥1 eval (fix #6). \_Closed for the rule_id↔eval binding; full semantic equivalence between a rule's text and a lens's behavior remains a judgment — labeled.* diff --git a/package.json b/package.json index 09d830a..9053294 100644 --- a/package.json +++ b/package.json @@ -32,7 +32,8 @@ "format:check": "prettier --check .", "docs:generate": "node .dev/floor/gen-capability-catalog.mjs . && node .dev/floor/gen-lessons-index.mjs .", "docs:check": "node .dev/floor/check-capability-catalog.mjs . && node .dev/floor/check-lessons-index.mjs .", - "check": "npm run format:check && npm run lint && npm run lint:md && npm run docs:check && npm test" + "check:markers": "node .dev/floor/check-specified-markers.mjs .", + "check": "npm run format:check && npm run lint && npm run lint:md && npm run docs:check && npm run check:markers && npm test" }, "devDependencies": { "@eslint/js": "^10.0.1", diff --git a/pharn/ARCHITECTURE.md b/pharn/ARCHITECTURE.md index 323d915..a3958ad 100644 --- a/pharn/ARCHITECTURE.md +++ b/pharn/ARCHITECTURE.md @@ -38,7 +38,7 @@ small, explicit, and non-LLM. Nothing else is a guarantee. 1. **Hooks** — non-LLM programs run by Claude Code at tool boundaries. - `pre-write` — blocks a write to a protected path (e.g. trusted files; out-of-`writes`-scope paths). - - `pre-egress` — blocks a network call to a domain not on a hardcoded allowlist. + - `pre-egress` _(specified; ships with the guarded surface)_ — blocks a network call to a domain not on a hardcoded allowlist. 2. **Content-hash** — identity of _content_, not identity of _id_. Detects silent mutation of a pinned artifact (spec, seam resolution, fetched doc). 3. **Enum / regex check** — set membership or pattern match, in `validate` and at gates @@ -236,7 +236,7 @@ Three moments, all reading **typed fields** (never model prose): - **pre-write** — hooks; block before a bad edit lands. Hosts the **pre-egress allowlist** (a network call to a non-allowlisted domain does not execute, regardless of whether the model was - fooled) and the **constitution/trusted-file write-guard** (fix #2) and the **`writes`-scope + fooled — specified; ships with the guarded surface) and the **constitution/trusted-file write-guard** (fix #2) and the **`writes`-scope guard** (fix #7). - **in-build** — validators; per-phase, gate a wave. - **post-build** — lenses (at review), verifiers (at verify), auditors (on-demand). A lens cannot @@ -255,7 +255,7 @@ is now labeled correctly (`LIMITS.md`). `floor/validate.mjs` (the `validate` step) enforces, deterministically: capability frontmatter present; evals present (P1); **every `enforces` rule_id produced by ≥1 eval** (P1, fix #6); -`coupling` enum membership; the four archetype maps agree (fix #5); finding templates separate +`coupling` enum membership; the four archetype maps agree (fix #5 — conditional; specified, ships with the guarded surface); finding templates separate enum-gated from free-text fields (fix #1); no forbidden sibling reference (P3, best-effort grep). ---