Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions .dev/features/features-readme-spec-live/GRILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# GRILL — features-readme-spec-live

**Plan:** `.dev/features/features-readme-spec-live/PLAN.md` · **Spec-hash check (content-hash primitive,
surfaced not blocking):** `sha256(pharn/ARCHITECTURE.md)` live =
`8f5ec002e3b18cbfd2f094b08a3671f7ed42a05a3fbaf01a11bbbd28da30fb52` == the plan's `spec_content_hash` →
**no drift**. The block on drift is `/pharn-dev-build`'s floor-gate (fix #4), not this stage.
**Registered grillers (membership, FLOOR — `pharn/floor/count-grillers.mjs`):** 13.

> The PLAN is `trust: untrusted` here. Every `problem` / `evidence` below quotes it as DATA.

## Findings

### Axis: discovery-first / doc-vs-live-state (P6)

```yaml
- type: FINDING
rule_id: "P6"
severity: important
file: ".dev/features/features-readme-spec-live/PLAN.md:103"
problem: "The plan still carries three questions under `## Open questions (HALT)` in unresolved form, but all three were answered at the plan-approval gate — and `/pharn-dev-build` refuses a plan whose open questions are unresolved, so the build stage will read a stale blocker that live state has already cleared."
evidence: "PLAN.md:103 `## Open questions (HALT)` followed by items 1-3 ('Proposed: `as the user runs each stage`. Alternative: leave line 8-9 untouched', 'Proposed: include it, carrying no version bump', 'Confirm it should be a follow-up rather than folded in') — against `.claude/commands/pharn-dev-build.md:42`: 'Read `PLAN.md`. If it has unresolved `## Open questions (HALT)` → **HALT**; it is not approved.'"
Comment thread
PrzemekGalarowicz marked this conversation as resolved.
```

```yaml
- type: FINDING
rule_id: "P6"
severity: minor
file: ".dev/features/features-readme-spec-live/PLAN.md:75"
problem: "The guarantee audit asserts a mechanism about the floor scanner's behavior that no read of that scanner this run supports — it is inferred from a promoted lesson plus a green baseline, which is evidence the check does not fire today but not a reading of why."
evidence: "PLAN.md:75 — 'so no CHECK fires on its content (L10's product-surface asymmetry is therefore inert here)'. The plan cites no line of `pharn/floor/validate.mjs` for this, and the surrounding `## Trust audit (P2)` elsewhere holds itself to live verification ('it was still treated as a claim to verify rather than a fact to copy (P6)')."
```

### Axis: guarantee-audit completeness (P0)

```yaml
- type: FINDING
rule_id: "P0"
severity: minor
file: ".dev/features/features-readme-spec-live/PLAN.md:68"
problem: "A floor reduction is claimed without the narrowing the same plan states elsewhere: the writes-scope hook gates only the Write/Edit/MultiEdit surface, so the claim covers less than the unqualified wording implies."
evidence: 'PLAN.md:68 — ''"the edit lands only in the two declared files" → **floor: hook** (fix #7 `set-writes-scope.cjs` + `enforce-writes-scope.cjs`, composed with fix #2''s denylist).'' The same document''s applied-lessons block already names the bound at PLAN.md:31 (''a repo-wide `prettier --write .` whose Bash writes escape the fix #7 gate entirely''), but the audit entry does not carry it.'
```

### Axes that produced no finding

- **Eval coverage (P1) + the structural/semantic split (`eval-format.md`, cited not restated).** Not
engaged, and the plan says so correctly: P1 binds a Capability, and `features/README.md` carries no
`---`-fenced frontmatter, hence no `role:` and no `enforces` rule id owing a fixture. Read
structurally from the file, not from the plan's self-claim (L6). Nothing is laundered into a judge
because there is no eval to route.
- **Trust propagation (P2).** The increment ingests no untrusted artifact, and the plan's trust audit
goes further than required — it records two request premises that failed live verification
(`SKILLS_VERSION` 2.4.6 vs 2.5.1; the `pharn/floor/README.md` half already fixed) rather than
inheriting them. That is the correct direction for this axis.
- **One axis of change (P3).** Two files, one change-reason: the README correction and the changelog
line that records it co-change by the repo's own "all notable changes are documented" contract. No
sibling reference — the increment touches no module and routes nothing leaf→leaf.
- **Determinism (P5).** Every branch named is a membership test (a grep for `node floor/…`; the
setter's parse of `## Files`; exit codes). The one irreducible judgment — whether line 8–9 reads as
stale — terminated in a question to the human, not a guess.
- **Honest scope (P7).** The increment adds no capability, rule, or enforcer, so the speculative-addition
hazard does not arise. Its exclusion block is a `###` heading (L18) and names four deferrals with
reasons, including one same-class defect it deliberately declines to fold in.
- **Documentation griller (`pharn/pharn-pipeline/grillers/documentation/documentation.md`, P7).** Layer 1
presence: the "needs it" trigger does not fire — the increment adds no public API, no config key, and
no non-obvious behavior; it **is** a documentation correction, and its `## Files` states the exact
prose changing. No absence finding. Layer 2 adequacy is not reached.
- **Remaining registered grillers.** Applied inline over a plan that builds no code: a11y,
architecture, comprehension, coupling, error-handling, i18n, migrations, observability, performance,
privacy, security, testability. None fired — there is no interface, no data flow, no schema, no
failure path, and no user-visible string beyond the corrected sentences themselves.

## Summary

The plan's substance holds: the defect is real and was reproduced live, the fix is one axis, and the
versioning call (no `SKILLS_VERSION` bump) follows the repo's own repo-meta rule rather than the
request's assertion. Two of the three findings are wording-level honesty gaps in the guarantee audit —
worth closing because this repo's whole thesis is that an unqualified floor claim is the disease, and
the plan states the missing qualifier elsewhere in its own text.

The first finding is the one with teeth, and it is procedural rather than substantive: the plan's open
questions were all answered at the approval gate, but the document still reads as though they are open,
and the build stage's documented refusal rule keys on exactly that section. Left as-is it produces a
halt that live state does not justify — the mirror image of the very defect this increment fixes, a
document asserting a state the repo has moved past.

ADVISORY VERDICT: 3 concerns raised (0 blocking-severity, 1 important, 2 minor) — for the human to
weigh before `/pharn-dev-build`. Nothing here blocks; the grill stage gates nothing.
139 changes: 139 additions & 0 deletions .dev/features/features-readme-spec-live/PLAN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# PLAN — features/README.md stops calling the shipped product pipeline unbuilt

- spec_content_hash: 8f5ec002e3b18cbfd2f094b08a3671f7ed42a05a3fbaf01a11bbbd28da30fb52 # fix #4
- applied_lessons: [L1, L11, L13, L18, L19, L20]
- increment: Reword the two sentences in `features/README.md` that describe the product pipeline as
still under construction, so the guide matches the live repo where all seven stages ship.
- layer(s): none — repo-meta. No capability, no `role:` frontmatter, no node in the layer tree
(`pharn/ARCHITECTURE.md §4`).
- constitution_refs: [P0, P6, P7]

## Applied lessons

- **L1** — ran the meta-doc sweep this increment invalidates: grepped the tree for every doc asserting a
product stage is unbuilt. It returned the two in-scope lines plus three classes deliberately left
alone (see `### Deliberately NOT in scope`), and settled the `CHANGELOG.md` question — L1's own
provenance names a missing `[Unreleased]` entry as half of the defect that promoted it, so the entry
is in `## Files` rather than left optional.
- **L11** — ran the whole-repo `npm run check` at BASELINE before scoping, so a pre-existing style red
could not later be blamed on this increment. It is green (1302 tests, all gates pass), so any red at
verify is this increment's.
- **L13** — this stage formats its own artifact: `prettier --write` + `markdownlint-cli2 --fix` scoped
to this `PLAN.md` alone, before the halt.
- **L18** — the exclusion block below is a `###` HEADING (`### Deliberately NOT in scope`), never a bold
prose intro, so `set-writes-scope.cjs --from-plan` terminates the authorized list structurally rather
than on a prose cue it may not recognize.
- **L19** — the L13 formatters are pinned to this one artifact path, never `npm run format` (a
repo-wide `prettier --write .` whose Bash writes escape the fix #7 gate entirely).
- **L20** — read the scope-setter's printed count as a checkable number, not decoration: Step 0 printed
`1 path(s)`, matching the one `PLAN.md` this stage writes. The same read is owed at `/pharn-dev-build`
Step 0, where the expected count is the number of `## Files` entries below.

## Files

- `features/README.md` — reword lines 18–20 (drop "(a later increment)" / "Until then") and lines 8–9
("as those stages are built"), so neither asserts the product pipeline is unbuilt — layer none
- `CHANGELOG.md` — one `## [Unreleased]` entry recording the correction, carrying **no** version bump
(L1) — layer none

### Deliberately NOT in scope

- `SKILLS_VERSION` — **must stay at 2.5.1.** `features/README.md` is a README, and CLAUDE.md's
discipline exempts repo-meta from the bump; the concrete bump-triggering set does not list it either.
- `pharn/floor/README.md` — verified live this run: it already reads `node pharn/floor/validate.mjs`
(line 34) and `node pharn/floor/check-structural.mjs` (line 55). There is no `node floor/…` path left,
so the second half of the original finding is already fixed (PR #126) and this increment must not
touch it.
- `pharn/pharn-contracts/finding-shape.md` — line 81 says the `check-structural` wiring over emitted
output "is increment **3c, not yet built**", which the sweep found stale (see `## Open questions`).
Same defect class, but it is **product surface**, so correcting it bumps `SKILLS_VERSION` — a
different axis and a separate increment (P3, P7).
- `.claude/commands/pharn-ship.md` — line 350's "`--loop` … is a **separate follow-up increment**" while
`/pharn-loop` ships the capability under a different name. Judgment call, product surface, separate.
- `.dev/features/**` — every other sweep hit is a frozen audit artifact that was true when written.
Rewriting history is not a staleness fix.

## Contracts satisfied

- none. This increment adds no capability, cites no `pharn-contracts` schema, and emits no finding.

## Evals to write (P1)

- none, and P1 is not engaged: P1 binds a **Capability** (a file whose frontmatter carries `role:`).
`features/README.md` has no frontmatter and declares no `enforces` rule id, so there is no rule id
owing an eval fixture.

## Guarantee audit (P0)

- "the edit lands only in the two declared files" → **floor: hook** (fix #7 `set-writes-scope.cjs` +
`enforce-writes-scope.cjs`, composed with fix #2's denylist) — **narrowed, and stated:** the hook
gates the `Write|Edit|MultiEdit` surface only, so a tool this increment runs through Bash (the L13
formatters) writes outside it unchecked. That is why those are pinned to one artifact path each
(L19), and the bound belongs on the claim, not only in the lessons block.
- "the repo stays style-clean and the floor stays GREEN with this in it" → **floor: enum-regex** — the
exit codes of `npm run check` (`format:check`, `lint:md`, `lint`, `docs:check`, `node --test`) and
`pharn/floor/validate.mjs`.
- "the reworded sentences are TRUE" → **advisory.** Nothing on the floor reads this file's prose.
Read live this run, not inferred: `validate.mjs:120` admits a file to the capability set only when
parsed frontmatter carries `role:`, and `validate.mjs:130` arms CHECK 5 only when the text matches
**both** `/rule_id:/` and `/problem:/`. `features/README.md` has no frontmatter and neither marker,
and the reword adds neither — so the file is walked (root `features/` is not in `EXCLUDE_SEGMENTS`,
L10) yet trips no content check. The truth of the sentences rests on the live reads recorded in
`## Open questions` — P6 evidence, not a floor reduction.
- "no `SKILLS_VERSION` bump is owed" → **advisory.** The bump-triggering set is a CLAUDE.md discipline;
no checker computes it from a diff. `check-specified-markers.mjs` binds trusted-doc marker claims, not
README bump eligibility.
- "the CHANGELOG entry carries no version" → **advisory** (the same discipline; nothing gates it).

## Trust audit (P2)

- The increment ingests no untrusted artifact: it reads repo-owned files and edits one of them; it
fetches nothing and emits no finding, so no taint propagates to any output.
- The **request text** driving this increment is human-authored intent, not fetched content — but it
was still treated as a claim to verify rather than a fact to copy (P6). Two of its premises did not
survive: it states the repo is at `SKILLS_VERSION 2.4.6` (live: **2.5.1**), and it offers the
`pharn/floor/README.md` half as needing confirmation (live: already fixed). Neither changes the fix;
both are recorded so no downstream stage inherits an unverified premise.

## Determinism audit (P5)

- "is `pharn/floor/README.md` in scope?" → membership test, not judgment: grep for a `node floor/…`
invocation. Zero matches live → excluded.
- "which paths may the build write?" → parsed deterministically by `set-writes-scope.cjs --from-plan`
from `## Files` above; no model chooses it.
- "is the increment green?" → exit codes only (`npm run check`, `validate.mjs`).
- The one irreducible judgment — whether line 8–9's "as those stages are built" reads as stale — does
**not** end in a guess: it is carried to the human as an open question below (the terminal fallback).

## Open questions — RESOLVED at the approval gate

**None open.** All three were put to the human as a selectable form at the plan-approval halt and
answered before any file was written; the plan was then approved as written. Recorded here so no
downstream stage reads a cleared blocker as live (`/pharn-dev-build` refuses a plan with unresolved
open questions).

1. **Line 8–9 — "as those stages are built".** My read was **stale, and the same axis**: "built" is
what happens to PHARN, not what a user does to a stage, so the phrase dates the sentence to when the
pipeline was unbuilt — the same premise as line 18, written at the same time. Live check: all seven
spine stages ship as `.claude/commands/pharn-{spec,plan,grill,build,regress,verify,ship}.md`.
**RESOLVED — reword to `as the user runs each stage`.**
2. **`CHANGELOG.md` entry.** L1 argued for it; the request called it optional.
**RESOLVED — include it, carrying no version bump.**
3. **Sibling staleness found by the L1 sweep.** `pharn/pharn-contracts/finding-shape.md:81` calls the
`check-structural`-over-emitted-output wiring "increment 3c, **not yet built**", but
`/pharn-dev-eval` (`:125`) and `/pharn-verify` (`:210`) both invoke `check-structural.mjs` over an
emitted `findings.json` today, and canon L4 records two live runs of it. Same defect class on the
**product surface** (so: a bump). **RESOLVED — defer to a separate follow-up increment**, recorded
in `### Deliberately NOT in scope` above.

## Grill findings folded back in (advisory — `GRILL.md` gates nothing)

`/pharn-dev-grill` raised three concerns against this plan; all three were closed here **before** build,
which is why the text above differs from the approved version in exactly these three places:

- **P6, important** — the open-questions section read as unresolved after the gate had cleared it.
Closed by the section immediately above.
- **P6, minor** — the floor-scanner claim was inferred rather than read. Closed by reading
`pharn/floor/validate.mjs:120` / `:130` live and citing both lines in the guarantee audit.
- **P0, minor** — a floor reduction stated without the Bash-escape narrowing the same plan names
elsewhere. Closed by moving the bound onto the claim itself.
66 changes: 66 additions & 0 deletions .dev/features/features-readme-spec-live/REGRESSION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# REGRESSION — features-readme-spec-live

**Base:** `2b4fec89cbd68e03544b9bad4254360ee029f040` (working-tree dogfood — `git status --porcelain`
was non-empty, so the deterministic state test resolved `base = HEAD`, P5; not a model's choice).

## Partition (computed by `pharn/floor/check-regress.mjs scope`, exit 0)

**Inside (the changed scope), 4 paths:**

- `features/README.md`, `CHANGELOG.md` — the two the plan's `## Files` declared and
`/pharn-dev-build` was pinned to.
- `.dev/features/features-readme-spec-live/PLAN.md`, `.../GRILL.md` — this feature's own pipeline
artifacts, each written by its own stage under that stage's own Step-0 writes-scope.

**No scope breach.** `scope` exited 0: nothing changed outside the declared writes. The two artifacts
above are listed in the helper's returned `escape_exempt`, read here rather than assumed — that
exemption is the floor check L20 demanded for the false-escape class L17 documents (cited, not
restated — P4). A stray file in the feature dir, another feature's artifact, or any real source path
would still have been an escape.

**Outside:** 61 test files + 1 committed eval pair
(`trust-fence/evals/expected/expected-injection-comment.json` ↔ `.dev/features/trust-fence/findings.json`).

## Gate set and the style-gate skip

The gate set was decided **once** and applied identically at base and head (a mismatch is
`inconclusive`, never a silent pass): `tests`, `validate`, `structural:expected-injection-comment`.

The style gates (`lint` / `format:check` / `lint:md`) were **deterministically skipped**: `inside`
touches no shared style config (`eslint.config.mjs`, `.prettierrc.json`, `.prettierignore`,
`.markdownlint-cli2.jsonc`), and over outside files that are byte-identical at base and head a style
result cannot flip. The skip also spares the baseline worktree an `npm ci` — the core gates are
stdlib-only. Absent from **both** maps, so the sets still match.

The `tests` list was expanded through `xargs` reading **stdin**, never `xargs -a` (GNU-only, which
BSD/macOS `xargs` rejects outright) and never an unquoted `$LIST` (zsh does not word-split it). Both
forms fabricate an equal-at-both-sides red that reads as `pre_existing` while masking a real
tests-gate regression — L5's failure mode and L16's correction to L5's own remedy.

## Per-gate exit codes

| gate | base | head | classification |
| --------------------------------------- | ---- | ---- | -------------- |
| `tests` (61 outside files) | 0 | 0 | stable |
| `validate` (whole-repo) | 0 | 0 | stable |
| `structural:expected-injection-comment` | 0 | 0 | stable |

`regressions[]`: **empty.** `pre_existing[]`: **empty.**

## Verdict (FLOOR — the helper's, not this stage's)

**REGRESSIONS: none — no deterministically-detectable breakage outside the feature.**
`check-regress.mjs verdict` exit **0**, `"verdict": "no-regressions"`, recorded verbatim in
`regression-report.json`.

**The honest residual (P0/P7):** this catches **exactly what its suite catches — nothing more.** A
regression no deterministic check covers is invisible to it. The claim is "deterministically-detectable
breakage outside the feature is caught," **not** "nothing broke" — and emphatically not that the
increment is good. That the verdict is floor-grade is true of the **comparison** only; choosing the
base, partitioning inside/outside, and running the suite are this stage's advisory work.

For this increment the residual is unusually wide and worth stating plainly: the change is two
sentences of English prose in a README, and **no deterministic check in the suite reads that prose**.
A green comparison here means the repo still builds and passes exactly as it did — it says nothing
about whether the reworded sentences are true. That claim rests on the live reads recorded in the
plan, and it is advisory.
Loading