From 734a8dfdfe62e9ebe1d699edf04e1e65668e302d Mon Sep 17 00:00:00 2001 From: David Asamonye Date: Fri, 28 Aug 2026 12:38:27 +0000 Subject: [PATCH 1/2] feat(kit): v2.3 deterministic gates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the third layer. Rules and skills are instructions to a model that can misread them under load; a gate is a fact that exits non-zero. scripts/verify-harness.sh checks discipline rather than code: ledger integrity, every PENDING_REVIEW has a contract, skills and rules will actually load, every [GROUNDED] path in the contract exists, the diff is contained by the impact map, and RED was recorded and failed on an assertion. Contract-dependent checks skip visibly when no contract is reachable; --strict turns skips into failures. The contract form asked for "Expected RED output" — a prediction, written before anything ran, never compared to reality. That becomes recorded RED, BASELINE and GREEN blocks. A test that never failed proves nothing: it may be vacuous or already passing, and one that failed on an import error proves the test file is wrong rather than the behaviour absent. Slices without an ERD are covered by the same gate, and it tightens rather than relaxes: if you cannot write a test that fails today you do not have a falsifiable criterion, and the contract is not approvable. CI re-runs both the gate and the contract's own VERIFY block, so an agent's claim that verification passed stops mattering. One script, invoked from CI and optionally from an editor hook, never reimplemented per surface. Contracts become tracked. The review process grades a PR against its contract and CI cannot check a file it cannot see; the same invisibility hid contracts from Finder and the editor sidebar. Session state, drafts and PR bodies stay local. This also anchors the legacy .harness/ ignore rule, which was unanchored and matched skills/.harness at any depth. Adds Appendices R and S and bumps the kit to 2.3.0. Contract: skills/.harness/contracts/KIT-GATES-001.md --- .github/workflows/harness.yml | 38 ++ .gitignore | 13 +- BOOTSTRAP.md | 338 ++++++++++++++++- FEATURES.json | 8 + README.md | 16 +- scripts/build-bootstrap.sh | 29 +- scripts/verify-harness.sh | 211 +++++++++++ skills/.harness/contracts/KIT-AUDIT-001.md | 339 +++++++++++++++++ skills/.harness/contracts/KIT-ERD-001.md | 340 +++++++++++++++++ skills/.harness/contracts/KIT-GATES-001.md | 345 ++++++++++++++++++ skills/.harness/contracts/KIT-QUALITY-001.md | 283 ++++++++++++++ skills/.harness/contracts/KIT-RULES-001.md | 311 ++++++++++++++++ skills/.harness/contracts/KIT-SKILLS-001.md | 323 ++++++++++++++++ skills/.harness/contracts/KIT-V13-001.md | 264 ++++++++++++++ .../templates/CODE_QUALITY.md | 3 + .../templates/AGENT_PROCESS.md | 14 + skills/harness-review/templates/REVIEW.md | 5 + .../templates/SPRINT_CONTRACT.md | 32 +- 18 files changed, 2888 insertions(+), 24 deletions(-) create mode 100644 .github/workflows/harness.yml create mode 100755 scripts/verify-harness.sh create mode 100644 skills/.harness/contracts/KIT-AUDIT-001.md create mode 100644 skills/.harness/contracts/KIT-ERD-001.md create mode 100644 skills/.harness/contracts/KIT-GATES-001.md create mode 100644 skills/.harness/contracts/KIT-QUALITY-001.md create mode 100644 skills/.harness/contracts/KIT-RULES-001.md create mode 100644 skills/.harness/contracts/KIT-SKILLS-001.md create mode 100644 skills/.harness/contracts/KIT-V13-001.md diff --git a/.github/workflows/harness.yml b/.github/workflows/harness.yml new file mode 100644 index 0000000..cc566f4 --- /dev/null +++ b/.github/workflows/harness.yml @@ -0,0 +1,38 @@ +name: harness + +# The unforgeable layer. Everything else in the kit is a instruction to a model +# that can misread it; this re-runs the checks itself, so an agent's claim that +# VERIFY passed stops mattering. +# +# It invokes scripts/verify-harness.sh — it never reimplements a check. + +on: + pull_request: + workflow_dispatch: + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # scope containment needs the merge base + + - name: Harness discipline + run: | + test -x scripts/verify-harness.sh || chmod +x scripts/verify-harness.sh + ./scripts/verify-harness.sh --base "origin/${{ github.base_ref }}" + + - name: Contract VERIFY block + run: | + # Re-run the contract's own verify commands. A local green is not evidence. + state=skills/.harness/STATE.md + [ -f "$state" ] || state=.harness/STATE.md + contract=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$state" 2>/dev/null | head -1) + if [ -z "$contract" ] || [ ! -f "$contract" ]; then + echo "no contract reachable — skipping contract VERIFY"; exit 0 + fi + echo "running VERIFY from $contract" + awk '/^## VERIFY/{f=1} f&&/^```(bash|sh)$/{c=1;next} c&&/^```$/{c=0} c' "$contract" > /tmp/verify.sh + if [ ! -s /tmp/verify.sh ]; then echo "contract has no VERIFY block"; exit 1; fi + sh -e /tmp/verify.sh diff --git a/.gitignore b/.gitignore index 211534e..322a5d9 100644 --- a/.gitignore +++ b/.gitignore @@ -13,11 +13,16 @@ Thumbs.db .env .env.* -# Ephemeral review packets — never committed (legacy path) -.harness/ +# Ephemeral review packets — never committed (legacy root path only). +# Anchored: unanchored '.harness/' also matched skills/.harness at any depth. +/.harness/ -# Harness working directory — contracts, local templates, session state -skills/.harness/ +# Harness working directory. Contracts are TRACKED: the review process grades a PR +# against its contract, and CI cannot check a file it cannot see. Session state, +# drafts and PR bodies stay local. +skills/.harness/* +!skills/.harness/contracts +!skills/.harness/contracts/** # Project instances (this repo ships TEMPLATES, not filled-in copies) # Uncomment if you keep a live filled AGENTS.md / FEATURES.json elsewhere diff --git a/BOOTSTRAP.md b/BOOTSTRAP.md index fdc9951..d8920c2 100644 --- a/BOOTSTRAP.md +++ b/BOOTSTRAP.md @@ -1,13 +1,28 @@ # BOOTSTRAP.md — Harness Bootstrap -> **DEPRECATED — kit 2.2.0 adopts via git submodule; see README.md. This file is retained for +> **DEPRECATED — kit 2.3.0 adopts via git submodule; see README.md. This file is retained for > repos already bootstrapped from it and will be removed in KIT-SKILLS-002.** > -> **Kit version 2.2.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, +> **Kit version 2.3.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, > Rust, Go, Solidity build, and Solidity auditing. Copy this file into **any** repo. Prompt your LLM: > *"Read BOOTSTRAP.md and complete Step 0 — Bootstrap harness files."* -## v2.2 changelog +## v2.3 changelog + +- **`scripts/verify-harness.sh`** — the third layer. Rules and skills are instructions to a model that + can misread them; this is a fact that exits non-zero. Nine checks: ledger integrity, every + `PENDING_REVIEW` has a contract, skills and rules will load, every `[GROUNDED]` path **exists**, + the diff is **contained by the impact map**, RED was recorded and failed on an **assertion**. +- **Prediction becomes recording** — the contract form's "Expected RED output" is replaced by recorded + **RED / BASELINE / GREEN** blocks. A test that never failed proves nothing. +- **No ERD, same gate** — without a design document each success criterion must be expressible as a + failing test before work starts, or the contract is not approvable. +- **CI re-runs VERIFY** (`.github/workflows/harness.yml`) — the only layer an agent cannot fake. One + script, invoked from CI and optionally from an editor hook; never reimplemented. +- **Contracts are tracked** — the review process grades a PR against its contract, and CI cannot check + a file it cannot see. Session state, drafts and PR bodies stay local. + +## v2.2 changelog (retained) - **`rules/` directory** — six always-on process-law rules as `.mdc`, linked into the editor's rules directory and assembled (frontmatter stripped) into a marked block in `AGENTS.md`, so agents without @@ -119,7 +134,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco | 1 | kit skills (symlinked by `setup-harness-kit.sh`) | Appendices B–P | N/A — canonical in the submodule | | 2 | `skills/.harness/contracts/` | — | Create empty directory | | 3 | `skills/.harness/STATE.md` | Appendix F | Fill `` + date | -| 4 | `skills/.harness/VERSION` | — | `version=2.2.0` + `bootstrapped=` | +| 4 | `skills/.harness/VERSION` | — | `version=2.3.0` + `bootstrapped=` | | 5 | `README.md` | Appendix G | **No** — overwrite with slim pointer (after brownfield recovery if needed) | | 6 | `AGENTS.md` | Appendix B | **Yes** if already filled — scaffold only; never overwrite harvested/verified content | | 7 | `FEATURES.json` | Appendix C | **Yes** if seeded — scaffold only | @@ -137,7 +152,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco **Done when:** -- `skills/.harness/` tree exists with VERSION `2.2.0` +- `skills/.harness/` tree exists with VERSION `2.3.0` - `python3 -m json.tool FEATURES.json` passes (if scaffolded) - `.gitignore` contains `skills/.harness/` - No `HARNESS.md` in repo @@ -457,6 +472,20 @@ is **N/A** for `ops-docs`, `code-audit` and `product-erd`, and for `solidity-aud ship code of their own. Credential findings belong to `SECURITY_CHECKLIST.md`, not this gate — raise them there and do not double-report. +Then run the gate: + +```sh +./scripts/verify-harness.sh # or the path this repo declares +``` + +It checks the discipline rather than the code: ledger integrity, that every `PENDING_REVIEW` has a +contract, that skills and rules will actually load, that every `[GROUNDED]` path in the contract +**exists**, that the diff is **contained by the impact map**, and that RED was recorded and failed on +an assertion. **The executor may set `PENDING_REVIEW` only when this exits 0.** + +Rules and skills are instructions to a model that can misread them under load. This is a fact — and +**CI runs it again**, so a claim that it passed locally is not evidence. + If verify fails: stay `FAIL`, fix or revise the contract. #### 9. REVIEW → PENDING_REVIEW → HUMAN PASS @@ -841,10 +870,34 @@ Pick **one**: - **Test files to create or extend:** - `` — -- **Expected RED output:** -- **Verify commands (GREEN):** - - `` - - `` +> **A test that never failed proves nothing.** It may be vacuous, or already passing. Record the real +> run — command, exit code, and the failure message — not a prediction of it. A RED that failed on an +> import or path error proves the test file is wrong, not that the behaviour is absent. +> +> **No ERD? Same gate.** Without a design document the success criteria below are the whole story, and +> each must be expressible as a **failing test before work starts**. If you cannot write a test that +> fails today, you do not have a falsifiable criterion and this contract is not approvable. + +### RED (recorded before EXECUTE) + +- **Command:** `` +- **Exit code:** `` +- **Failure reason:** `` +- **Captured:** `` + +**N/A** only when this sprint changes no behaviour (`ops-docs`, `code-audit`, `product-erd`). + +### BASELINE (recorded before EXECUTE — behaviour changes only) + +- **Command:** `` +- **Result:** `` + +### GREEN (recorded after EXECUTE) + +- **Command:** `` → **exit 0** +- **Command:** `` → **``** + +Green-after means nothing without green-before. A count that moved is a regression until explained. ### Vertical slice (when non-trivial) @@ -1096,6 +1149,11 @@ Before deep code review, locate and read the relevant contract: code-quality gate (`CODE_QUALITY.md`: lint command recorded, violations = 0 or documented), solidity-audit PoCs. Mark each **Pass / Fail / N/A**. 8. **VERIFY commands** — run or confirm the contract's verify block passed; note any skipped or failing commands. +9. **Gate artifacts** — confirm the contract carries a recorded **RED** (real command, non-zero exit, + assertion-shaped failure), a **BASELINE**, and a **GREEN** whose counts match the baseline. A + predicted RED, or one that failed on an import error, is a finding. `N/A` is only valid when the + sprint changed no behaviour. +10. **Gate ran** — confirm `verify-harness.sh` exited 0 **in CI**, not only locally. If no contract exists (hotfix, drive-by), state that explicitly and review on production-readiness only. Retroactive contract may be required before PASS. @@ -1608,6 +1666,9 @@ explicitly approves that scope. > `CODE_AUDIT_CONTRACT.md` asks "where should we invest refactoring effort?" on a cadence, over a > declared subsystem. Do not substitute one for the other. +> Process discipline — ledger, contract, scope containment, RED artifact — is checked by +> `verify-harness.sh`, not here. This gate judges the code; that one judges the contract. + ## 0. Sensor - [ ] Repo's declared lint / static-analysis command from `AGENTS.md` was **run** on the diff @@ -2421,6 +2482,265 @@ Kit upgrades are a submodule pointer bump and a setup re-run. Never edit these f --- +## Appendix R — verify-harness.sh (the gate) + +#!/bin/sh +# verify-harness.sh — deterministic checks on harness discipline. +# +# Rules and skills are instructions to a model that can misread them under load. +# This is a fact. Run it from a repo root; it exits non-zero on any violation. +# +# ./verify-harness.sh # skip contract checks when no contract is reachable +# ./verify-harness.sh --strict # a skip becomes a failure +# ./verify-harness.sh --contract # check a specific contract +# ./verify-harness.sh --base # diff base for scope containment (default: origin/HEAD) +# +# CI and any editor hook invoke THIS script. Never reimplement a check elsewhere. +set -u + +STRICT=0; CONTRACT=""; BASE="" +while [ $# -gt 0 ]; do + case "$1" in + --strict) STRICT=1 ;; + --contract) shift; CONTRACT="${1:-}" ;; + --base) shift; BASE="${1:-}" ;; + -h|--help) sed -n '2,14p' "$0"; exit 0 ;; + *) echo "unknown argument: $1" >&2; exit 2 ;; + esac + shift +done + +fails=0; skips=0 +pass() { printf ' PASS %s\n' "$1"; } +fail() { printf ' FAIL %s\n' "$1"; fails=$((fails + 1)); } +skip() { + if [ "$STRICT" -eq 1 ]; then fail "$1 (skipped, and --strict is on)" + else printf ' SKIP %s\n' "$1"; skips=$((skips + 1)); fi +} + +CONTRACTS_DIR=skills/.harness/contracts +[ -d "$CONTRACTS_DIR" ] || CONTRACTS_DIR=.harness/contracts +STATE=skills/.harness/STATE.md +[ -f "$STATE" ] || STATE=.harness/STATE.md + +echo "harness verify" + +# --- 1. ledger integrity ------------------------------------------------------ +if [ -f FEATURES.json ]; then + if python3 -m json.tool FEATURES.json >/dev/null 2>&1; then + bad=$(python3 - <<'PY' +import json +d = json.load(open("FEATURES.json")) +legend = set(d.get("legend", {}).get("status") or ["FAIL", "PENDING_REVIEW", "PASS"]) +out = [] +for f in d.get("features", []): + fid = f.get("id", "") + if f.get("status") not in legend: + out.append("%s: status %r not in legend" % (fid, f.get("status"))) + v = (f.get("verify") or "").strip() + if not v: + out.append("%s: empty verify" % fid) + elif not any(c in v for c in "&|;$/") and " " not in v: + out.append("%s: verify does not look like a command" % fid) +print("\n".join(out)) +PY +) + if [ -z "$bad" ]; then pass "FEATURES.json: parses, statuses legal, verify fields runnable" + else printf '%s\n' "$bad" | while IFS= read -r l; do [ -n "$l" ] && printf ' %s\n' "$l"; done + fail "FEATURES.json: invalid entries (above)"; fi + else + fail "FEATURES.json: does not parse" + fi +else + skip "FEATURES.json: not present" +fi + +# --- 2. every PENDING_REVIEW has a contract ---------------------------------- +if [ -f FEATURES.json ] && [ -d "$CONTRACTS_DIR" ]; then + missing=$(python3 - "$CONTRACTS_DIR" <<'PY' +import json, os, sys +d = json.load(open("FEATURES.json")) +out = [f["id"] for f in d.get("features", []) + if f.get("status") == "PENDING_REVIEW" + and not os.path.exists(os.path.join(sys.argv[1], f["id"] + ".md"))] +print(" ".join(out)) +PY +) + if [ -z "$missing" ]; then pass "every PENDING_REVIEW feature has a contract file" + else fail "PENDING_REVIEW without a contract: $missing"; fi +else + skip "PENDING_REVIEW/contract cross-check" +fi + +# --- 3. STATE points at a real contract -------------------------------------- +if [ -f "$STATE" ]; then + ref=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$STATE" | head -1) + if [ -z "$ref" ]; then skip "STATE.md names no contract" + elif [ -f "$ref" ]; then pass "STATE.md current contract exists ($ref)" + else fail "STATE.md points at a missing contract: $ref"; fi +else + skip "STATE.md not present" +fi + +# --- 4/5. skills and rules load ---------------------------------------------- +if [ -d skills ]; then + bad="" + for s in skills/*/SKILL.md; do + [ -f "$s" ] || continue + head -1 "$s" | grep -q '^---$' && grep -q '^name:' "$s" && grep -q '^description:' "$s" || bad="$bad $s" + done + if [ -z "$bad" ]; then pass "skills: frontmatter valid (retrievable)" + else fail "skills with bad frontmatter:$bad"; fi +else + skip "no skills directory" +fi + +if [ -d rules ]; then + bad="" + for r in rules/*.mdc; do + [ -f "$r" ] || continue + head -1 "$r" | grep -q '^---$' && grep -q '^description:' "$r" && grep -q '^alwaysApply:' "$r" || bad="$bad $r" + done + if [ -z "$bad" ]; then pass "rules: frontmatter valid (will load)" + else fail "rules with bad frontmatter:$bad"; fi +else + skip "no rules directory" +fi + +# --- resolve the active contract --------------------------------------------- +contract_file="" +if [ -n "$CONTRACT" ]; then + contract_file="$CONTRACTS_DIR/$CONTRACT.md" + [ -f "$contract_file" ] || contract_file="$CONTRACT" +elif [ -f "$STATE" ]; then + ref=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$STATE" | head -1) + [ -n "$ref" ] && [ -f "$ref" ] && contract_file="$ref" +fi + +# --- 6. grounded paths exist (catches invented files) ------------------------ +if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then + # Only backticked tokens that look like paths: they contain a slash, or carry a + # known file extension. Branch names and commit SHAs are also backticked on + # [GROUNDED] lines and are not paths. + missing="" + for p in $(grep -o '`[^`]*`[^|]*\[GROUNDED\]' "$contract_file" | grep -o '`[^`]*`' | tr -d '`' | sort -u); do + case "$p" in + *" "*|"") continue ;; + */*) : ;; + *.md|*.sh|*.json|*.yml|*.yaml|*.mdc|*.ts|*.js|*.go|*.rs|*.sol|*.py) : ;; + *) continue ;; + esac + [ -e "$p" ] || missing="$missing $p" + done + if [ -z "$missing" ]; then pass "contract: every [GROUNDED] path exists" + else fail "contract cites paths that do not exist:$missing"; fi +else + skip "contract [GROUNDED] paths (no contract)" +fi + +# --- 7. diff is contained by the impact map ---------------------------------- +if [ -n "$contract_file" ] && [ -f "$contract_file" ] && git rev-parse --git-dir >/dev/null 2>&1; then + base="$BASE" + [ -n "$base" ] || base=$(git rev-parse --abbrev-ref origin/HEAD 2>/dev/null || echo origin/master) + if git rev-parse --verify -q "$base" >/dev/null; then + undeclared="" + for f in $(git diff --name-only "$base"...HEAD 2>/dev/null); do + grep -qF "$f" "$contract_file" || undeclared="$undeclared $f" + done + if [ -z "$undeclared" ]; then pass "diff is contained by the contract impact map" + else fail "changed but not declared in the contract:$undeclared"; fi + else + skip "scope containment (base $base unresolvable)" + fi +else + skip "scope containment (no contract)" +fi + +# --- 8. RED was recorded, and failed for the right reason -------------------- +if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then + if grep -q 'RED (recorded before EXECUTE)' "$contract_file"; then + if grep -qi 'N/A' "$(printf %s "$contract_file")" && grep -A4 'RED (recorded before EXECUTE)' "$contract_file" | grep -qi 'N/A'; then + pass "RED: declared N/A (no behaviour change)" + elif grep -A6 'RED (recorded before EXECUTE)' "$contract_file" | grep -qiE 'assert|expect|to (be|equal)|assertion'; then + pass "RED: recorded and failed on an assertion" + else + fail "RED: recorded but no assertion-shaped failure — a test that never failed proves nothing" + fi + else + skip "RED artifact (contract predates the recording format)" + fi +else + skip "RED artifact (no contract)" +fi + +# --- 9. branch discipline ----------------------------------------------------- +if git rev-parse --git-dir >/dev/null 2>&1; then + cur=$(git branch --show-current 2>/dev/null || echo "") + def=$(git rev-parse --abbrev-ref origin/HEAD 2>/dev/null | sed 's|^origin/||') + [ -n "$def" ] || def=master + if [ -z "$cur" ]; then skip "branch discipline (detached HEAD)" + elif [ "$cur" != "$def" ]; then pass "on a feature branch ($cur)" + else + if [ -z "$(git status --porcelain 2>/dev/null)" ]; then pass "on $def and clean" + else fail "uncommitted implementation on the default branch ($def)"; fi + fi +else + skip "branch discipline (not a git repo)" +fi + +echo +if [ "$fails" -gt 0 ]; then + echo "harness verify: $fails failure(s), $skips skipped" + exit 1 +fi +echo "harness verify: green ($skips skipped)" +exit 0 + +--- + +## Appendix S — CI workflow (re-runs the gate and the contract VERIFY block) + +name: harness + +# The unforgeable layer. Everything else in the kit is a instruction to a model +# that can misread it; this re-runs the checks itself, so an agent's claim that +# VERIFY passed stops mattering. +# +# It invokes scripts/verify-harness.sh — it never reimplements a check. + +on: + pull_request: + workflow_dispatch: + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # scope containment needs the merge base + + - name: Harness discipline + run: | + test -x scripts/verify-harness.sh || chmod +x scripts/verify-harness.sh + ./scripts/verify-harness.sh --base "origin/${{ github.base_ref }}" + + - name: Contract VERIFY block + run: | + # Re-run the contract's own verify commands. A local green is not evidence. + state=skills/.harness/STATE.md + [ -f "$state" ] || state=.harness/STATE.md + contract=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$state" 2>/dev/null | head -1) + if [ -z "$contract" ] || [ ! -f "$contract" ]; then + echo "no contract reachable — skipping contract VERIFY"; exit 0 + fi + echo "running VERIFY from $contract" + awk '/^## VERIFY/{f=1} f&&/^```(bash|sh)$/{c=1;next} c&&/^```$/{c=0} c' "$contract" > /tmp/verify.sh + if [ ! -s /tmp/verify.sh ]; then echo "contract has no VERIFY block"; exit 1; fi + sh -e /tmp/verify.sh + +--- + ## Kit repo note (maintainers) *Maintainers: edit `templates/*` and this file's Step 0 / paths sections, then run diff --git a/FEATURES.json b/FEATURES.json index 6d267ef..1aee429 100644 --- a/FEATURES.json +++ b/FEATURES.json @@ -13,6 +13,14 @@ "verify": "runnable test command or command sequence — not prose" }, "features": [ + { + "id": "KIT-GATES-001", + "name": "Deterministic gates: RED/BASELINE/GREEN artifacts, verify-harness.sh, CI re-running VERIFY", + "priority": 1, + "verify": "test -x scripts/verify-harness.sh && ./scripts/verify-harness.sh && test -f .github/workflows/harness.yml && ! grep -q 'Expected RED output' skills/sprint-contract/templates/SPRINT_CONTRACT.md && grep -q '2.3.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "PENDING_REVIEW", + "notes": "Contract: skills/.harness/contracts/KIT-GATES-001.md; profile: ops-docs; EXECUTE+VERIFY green 2026-08-28 on feat/kit-gates-001-deterministic-gates; contracts now tracked; awaiting human PASS" + }, { "id": "KIT-RULES-001", "name": "rules/ directory as .mdc with AGENTS.md assembly, plus ERD routing before contract drafting", diff --git a/README.md b/README.md index 9f3b3ae..e256458 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ A reusable, **tool-agnostic** framework for AI-assisted engineering — contract-first, TDD when behaviour changes, multi-language (TypeScript, Rust, Go, Solidity build + audit). -**Kit v2.2** — the process ships as **skills**, adopted as a **git submodule**. Skills stay canonical +**Kit v2.3** — the process ships as **skills**, adopted as a **git submodule**. Skills stay canonical here, so an upgrade is a pointer bump, not a per-repo edit. ## Adopt @@ -24,6 +24,20 @@ git submodule update --remote harness-kit ./harness-kit/scripts/setup-harness-kit.sh ``` +## Gates + +```sh +./scripts/verify-harness.sh # exits non-zero on any violation +./scripts/verify-harness.sh --strict # skips become failures +``` + +Rules and skills are instructions to a model that can misread them under load. A gate is a fact. +It checks ledger integrity, that every `PENDING_REVIEW` has a contract, that skills and rules will +load, that every `[GROUNDED]` path exists, that the diff is contained by the impact map, and that RED +was recorded and failed on an assertion. **CI runs it again** — a local green is not evidence. + +> Gate it if a command can enforce it, skill it if it is procedural, rule it only if it holds everywhere. + ## Rules Six always-on constraints in `rules/*.mdc`, linked into the editor's rules directory and assembled into diff --git a/scripts/build-bootstrap.sh b/scripts/build-bootstrap.sh index ff63e8f..e70f127 100755 --- a/scripts/build-bootstrap.sh +++ b/scripts/build-bootstrap.sh @@ -1,5 +1,5 @@ #!/bin/sh -# DEPRECATED as of kit 2.2.0 — adoption moved to git submodule + scripts/setup-harness-kit.sh. +# DEPRECATED as of kit 2.3.0 — adoption moved to git submodule + scripts/setup-harness-kit.sh. # Retained so repos bootstrapped from BOOTSTRAP.md keep working. Removal: KIT-SKILLS-002. # Generates BOOTSTRAP.md from the forms now bundled inside skills//templates/. set -eu @@ -17,14 +17,29 @@ append() { cat <<'HEADER' # BOOTSTRAP.md — Harness Bootstrap -> **DEPRECATED — kit 2.2.0 adopts via git submodule; see README.md. This file is retained for +> **DEPRECATED — kit 2.3.0 adopts via git submodule; see README.md. This file is retained for > repos already bootstrapped from it and will be removed in KIT-SKILLS-002.** > -> **Kit version 2.2.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, +> **Kit version 2.3.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, > Rust, Go, Solidity build, and Solidity auditing. Copy this file into **any** repo. Prompt your LLM: > *"Read BOOTSTRAP.md and complete Step 0 — Bootstrap harness files."* -## v2.2 changelog +## v2.3 changelog + +- **`scripts/verify-harness.sh`** — the third layer. Rules and skills are instructions to a model that + can misread them; this is a fact that exits non-zero. Nine checks: ledger integrity, every + `PENDING_REVIEW` has a contract, skills and rules will load, every `[GROUNDED]` path **exists**, + the diff is **contained by the impact map**, RED was recorded and failed on an **assertion**. +- **Prediction becomes recording** — the contract form's "Expected RED output" is replaced by recorded + **RED / BASELINE / GREEN** blocks. A test that never failed proves nothing. +- **No ERD, same gate** — without a design document each success criterion must be expressible as a + failing test before work starts, or the contract is not approvable. +- **CI re-runs VERIFY** (`.github/workflows/harness.yml`) — the only layer an agent cannot fake. One + script, invoked from CI and optionally from an editor hook; never reimplemented. +- **Contracts are tracked** — the review process grades a PR against its contract, and CI cannot check + a file it cannot see. Session state, drafts and PR bodies stay local. + +## v2.2 changelog (retained) - **`rules/` directory** — six always-on process-law rules as `.mdc`, linked into the editor's rules directory and assembled (frontmatter stripped) into a marked block in `AGENTS.md`, so agents without @@ -136,7 +151,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco | 1 | kit skills (symlinked by `setup-harness-kit.sh`) | Appendices B–P | N/A — canonical in the submodule | | 2 | `skills/.harness/contracts/` | — | Create empty directory | | 3 | `skills/.harness/STATE.md` | Appendix F | Fill `` + date | -| 4 | `skills/.harness/VERSION` | — | `version=2.2.0` + `bootstrapped=` | +| 4 | `skills/.harness/VERSION` | — | `version=2.3.0` + `bootstrapped=` | | 5 | `README.md` | Appendix G | **No** — overwrite with slim pointer (after brownfield recovery if needed) | | 6 | `AGENTS.md` | Appendix B | **Yes** if already filled — scaffold only; never overwrite harvested/verified content | | 7 | `FEATURES.json` | Appendix C | **Yes** if seeded — scaffold only | @@ -154,7 +169,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco **Done when:** -- `skills/.harness/` tree exists with VERSION `2.2.0` +- `skills/.harness/` tree exists with VERSION `2.3.0` - `python3 -m json.tool FEATURES.json` passes (if scaffolded) - `.gitignore` contains `skills/.harness/` - No `HARNESS.md` in repo @@ -257,6 +272,8 @@ HEADER append "Appendix O — ERD.md template" skills/erd-authoring/templates/ERD.md append "Appendix P — ARCHITECTURE.md template" skills/erd-authoring/templates/ARCHITECTURE.md append "Appendix Q — rules/README.md (rule budget + ownership tiers)" rules/README.md + append "Appendix R — verify-harness.sh (the gate)" scripts/verify-harness.sh + append "Appendix S — CI workflow (re-runs the gate and the contract VERIFY block)" .github/workflows/harness.yml cat <<'FOOTER' diff --git a/scripts/verify-harness.sh b/scripts/verify-harness.sh new file mode 100755 index 0000000..2811241 --- /dev/null +++ b/scripts/verify-harness.sh @@ -0,0 +1,211 @@ +#!/bin/sh +# verify-harness.sh — deterministic checks on harness discipline. +# +# Rules and skills are instructions to a model that can misread them under load. +# This is a fact. Run it from a repo root; it exits non-zero on any violation. +# +# ./verify-harness.sh # skip contract checks when no contract is reachable +# ./verify-harness.sh --strict # a skip becomes a failure +# ./verify-harness.sh --contract # check a specific contract +# ./verify-harness.sh --base # diff base for scope containment (default: origin/HEAD) +# +# CI and any editor hook invoke THIS script. Never reimplement a check elsewhere. +set -u + +STRICT=0; CONTRACT=""; BASE="" +while [ $# -gt 0 ]; do + case "$1" in + --strict) STRICT=1 ;; + --contract) shift; CONTRACT="${1:-}" ;; + --base) shift; BASE="${1:-}" ;; + -h|--help) sed -n '2,14p' "$0"; exit 0 ;; + *) echo "unknown argument: $1" >&2; exit 2 ;; + esac + shift +done + +fails=0; skips=0 +pass() { printf ' PASS %s\n' "$1"; } +fail() { printf ' FAIL %s\n' "$1"; fails=$((fails + 1)); } +skip() { + if [ "$STRICT" -eq 1 ]; then fail "$1 (skipped, and --strict is on)" + else printf ' SKIP %s\n' "$1"; skips=$((skips + 1)); fi +} + +CONTRACTS_DIR=skills/.harness/contracts +[ -d "$CONTRACTS_DIR" ] || CONTRACTS_DIR=.harness/contracts +STATE=skills/.harness/STATE.md +[ -f "$STATE" ] || STATE=.harness/STATE.md + +echo "harness verify" + +# --- 1. ledger integrity ------------------------------------------------------ +if [ -f FEATURES.json ]; then + if python3 -m json.tool FEATURES.json >/dev/null 2>&1; then + bad=$(python3 - <<'PY' +import json +d = json.load(open("FEATURES.json")) +legend = set(d.get("legend", {}).get("status") or ["FAIL", "PENDING_REVIEW", "PASS"]) +out = [] +for f in d.get("features", []): + fid = f.get("id", "") + if f.get("status") not in legend: + out.append("%s: status %r not in legend" % (fid, f.get("status"))) + v = (f.get("verify") or "").strip() + if not v: + out.append("%s: empty verify" % fid) + elif not any(c in v for c in "&|;$/") and " " not in v: + out.append("%s: verify does not look like a command" % fid) +print("\n".join(out)) +PY +) + if [ -z "$bad" ]; then pass "FEATURES.json: parses, statuses legal, verify fields runnable" + else printf '%s\n' "$bad" | while IFS= read -r l; do [ -n "$l" ] && printf ' %s\n' "$l"; done + fail "FEATURES.json: invalid entries (above)"; fi + else + fail "FEATURES.json: does not parse" + fi +else + skip "FEATURES.json: not present" +fi + +# --- 2. every PENDING_REVIEW has a contract ---------------------------------- +if [ -f FEATURES.json ] && [ -d "$CONTRACTS_DIR" ]; then + missing=$(python3 - "$CONTRACTS_DIR" <<'PY' +import json, os, sys +d = json.load(open("FEATURES.json")) +out = [f["id"] for f in d.get("features", []) + if f.get("status") == "PENDING_REVIEW" + and not os.path.exists(os.path.join(sys.argv[1], f["id"] + ".md"))] +print(" ".join(out)) +PY +) + if [ -z "$missing" ]; then pass "every PENDING_REVIEW feature has a contract file" + else fail "PENDING_REVIEW without a contract: $missing"; fi +else + skip "PENDING_REVIEW/contract cross-check" +fi + +# --- 3. STATE points at a real contract -------------------------------------- +if [ -f "$STATE" ]; then + ref=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$STATE" | head -1) + if [ -z "$ref" ]; then skip "STATE.md names no contract" + elif [ -f "$ref" ]; then pass "STATE.md current contract exists ($ref)" + else fail "STATE.md points at a missing contract: $ref"; fi +else + skip "STATE.md not present" +fi + +# --- 4/5. skills and rules load ---------------------------------------------- +if [ -d skills ]; then + bad="" + for s in skills/*/SKILL.md; do + [ -f "$s" ] || continue + head -1 "$s" | grep -q '^---$' && grep -q '^name:' "$s" && grep -q '^description:' "$s" || bad="$bad $s" + done + if [ -z "$bad" ]; then pass "skills: frontmatter valid (retrievable)" + else fail "skills with bad frontmatter:$bad"; fi +else + skip "no skills directory" +fi + +if [ -d rules ]; then + bad="" + for r in rules/*.mdc; do + [ -f "$r" ] || continue + head -1 "$r" | grep -q '^---$' && grep -q '^description:' "$r" && grep -q '^alwaysApply:' "$r" || bad="$bad $r" + done + if [ -z "$bad" ]; then pass "rules: frontmatter valid (will load)" + else fail "rules with bad frontmatter:$bad"; fi +else + skip "no rules directory" +fi + +# --- resolve the active contract --------------------------------------------- +contract_file="" +if [ -n "$CONTRACT" ]; then + contract_file="$CONTRACTS_DIR/$CONTRACT.md" + [ -f "$contract_file" ] || contract_file="$CONTRACT" +elif [ -f "$STATE" ]; then + ref=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$STATE" | head -1) + [ -n "$ref" ] && [ -f "$ref" ] && contract_file="$ref" +fi + +# --- 6. grounded paths exist (catches invented files) ------------------------ +if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then + # Only backticked tokens that look like paths: they contain a slash, or carry a + # known file extension. Branch names and commit SHAs are also backticked on + # [GROUNDED] lines and are not paths. + missing="" + for p in $(grep -o '`[^`]*`[^|]*\[GROUNDED\]' "$contract_file" | grep -o '`[^`]*`' | tr -d '`' | sort -u); do + case "$p" in + *" "*|"") continue ;; + */*) : ;; + *.md|*.sh|*.json|*.yml|*.yaml|*.mdc|*.ts|*.js|*.go|*.rs|*.sol|*.py) : ;; + *) continue ;; + esac + [ -e "$p" ] || missing="$missing $p" + done + if [ -z "$missing" ]; then pass "contract: every [GROUNDED] path exists" + else fail "contract cites paths that do not exist:$missing"; fi +else + skip "contract [GROUNDED] paths (no contract)" +fi + +# --- 7. diff is contained by the impact map ---------------------------------- +if [ -n "$contract_file" ] && [ -f "$contract_file" ] && git rev-parse --git-dir >/dev/null 2>&1; then + base="$BASE" + [ -n "$base" ] || base=$(git rev-parse --abbrev-ref origin/HEAD 2>/dev/null || echo origin/master) + if git rev-parse --verify -q "$base" >/dev/null; then + undeclared="" + for f in $(git diff --name-only "$base"...HEAD 2>/dev/null); do + grep -qF "$f" "$contract_file" || undeclared="$undeclared $f" + done + if [ -z "$undeclared" ]; then pass "diff is contained by the contract impact map" + else fail "changed but not declared in the contract:$undeclared"; fi + else + skip "scope containment (base $base unresolvable)" + fi +else + skip "scope containment (no contract)" +fi + +# --- 8. RED was recorded, and failed for the right reason -------------------- +if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then + if grep -q 'RED (recorded before EXECUTE)' "$contract_file"; then + if grep -qi 'N/A' "$(printf %s "$contract_file")" && grep -A4 'RED (recorded before EXECUTE)' "$contract_file" | grep -qi 'N/A'; then + pass "RED: declared N/A (no behaviour change)" + elif grep -A6 'RED (recorded before EXECUTE)' "$contract_file" | grep -qiE 'assert|expect|to (be|equal)|assertion'; then + pass "RED: recorded and failed on an assertion" + else + fail "RED: recorded but no assertion-shaped failure — a test that never failed proves nothing" + fi + else + skip "RED artifact (contract predates the recording format)" + fi +else + skip "RED artifact (no contract)" +fi + +# --- 9. branch discipline ----------------------------------------------------- +if git rev-parse --git-dir >/dev/null 2>&1; then + cur=$(git branch --show-current 2>/dev/null || echo "") + def=$(git rev-parse --abbrev-ref origin/HEAD 2>/dev/null | sed 's|^origin/||') + [ -n "$def" ] || def=master + if [ -z "$cur" ]; then skip "branch discipline (detached HEAD)" + elif [ "$cur" != "$def" ]; then pass "on a feature branch ($cur)" + else + if [ -z "$(git status --porcelain 2>/dev/null)" ]; then pass "on $def and clean" + else fail "uncommitted implementation on the default branch ($def)"; fi + fi +else + skip "branch discipline (not a git repo)" +fi + +echo +if [ "$fails" -gt 0 ]; then + echo "harness verify: $fails failure(s), $skips skipped" + exit 1 +fi +echo "harness verify: green ($skips skipped)" +exit 0 diff --git a/skills/.harness/contracts/KIT-AUDIT-001.md b/skills/.harness/contracts/KIT-AUDIT-001.md new file mode 100644 index 0000000..6ad07c3 --- /dev/null +++ b/skills/.harness/contracts/KIT-AUDIT-001.md @@ -0,0 +1,339 @@ +# SPRINT CONTRACT — KIT-AUDIT-001: `code-audit` profile (periodic whole-project quality audit) + +> Status: **EXECUTE complete — VERIFY green (24/24) — PENDING_REVIEW.** +> Approved: David 2026-08-27 ("proceed") with recommended answers to Q1–Q4. +> Branch `feat/kit-audit-001-code-audit-profile` from `master` @ `a3af45f`. Version **1.5.0**. +> Feature ID: `KIT-AUDIT-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-audit-001-code-audit-profile` — **human must confirm before checkout** +> Trigger: David (2026-08-27) — "a once in a while behaviour to check for the overall code quality of a project… name that as a separate skill in our harness-kit." + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + POSIX sh. No application code. + +> This sprint **adds** the `code-audit` profile. The sprint itself runs under `ops-docs`. + +--- + +## Context (grounded) + +| Source | Finding | +| --- | --- | +| `templates/CODE_QUALITY.md` (v1.4, KIT-QUALITY-001) | Per-sprint **gate**: diff-scoped, lint-first, blocking, cheap enough to run every contract. Deliberately narrow. | +| `templates/AUDIT_CONTRACT.md` + `AUDIT_FINDING.md` | Existing precedent for a **periodic, report-producing, code-changing-nothing** profile (`solidity-audit`). Finding template already carries ID / Severity / Status / commit-tag / location. | +| `templates/AGENT_PROCESS.md` §Work-type profiles | Five profiles; profile drives which gates are in force vs N/A. | +| `templates/AGENT_PROCESS.md` §Build to delete / Cost reality | "A full harness costs more per run… choose depth per the matrix." A whole-repo audit is expensive by design and must be explicitly periodic, never per-sprint. | +| Supplied review brief (David, 2026-08-27) | 15 review categories, severity/confidence/regression-risk rubrics, behavioural-equivalence checklist, 5-phase workflow, 11-section report format. | + +**Gap:** the kit has a per-diff quality gate and a Solidity security audit, but nothing that steps back +and assesses **a whole codebase's quality** on a cadence. Nothing carries the behaviour-preservation +discipline that makes large-scale refactoring safe. + +**Out of scope by correction (David):** the kit must not reference any pack's skills. The kit is +pack-agnostic; optional packs may add their own passes and the kit says nothing about them. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Form | **Work-type profile + contract template**, not a "skill" — the kit's idiom. Parallel to `solidity-audit`. | Locked | +| Name | **`code-audit`** — reads consistently beside `solidity-audit` | Locked | +| Cadence | **Periodic, never per-sprint.** Explicitly not a gate; does not run at VERIFY. | Locked | +| Relationship to `CODE_QUALITY.md` | Two different instruments, stated plainly in both files: the **gate** answers "can this diff ship?"; the **audit** answers "where should we invest refactoring effort?" | Locked | +| Behaviour preservation | **The absolute constraint of the profile.** Made deterministic: characterization tests captured **before** the refactor (passing against current code, proving they encode today's behaviour) and still passing after. | Locked | +| Taxonomy home | **In the kit's contract template.** It is language-neutral — feature envy, primitive obsession, stale comments are not stack-specific, unlike thresholds (which we deliberately kept out in v1.4). The kit cannot depend on a pack existing. | Locked | +| Findings → work | Each accepted finding seeds a **`FEATURES.json`** entry, which becomes a sprint contract. An audit whose findings are not tracked is a document nobody acts on. | Locked | +| Audit changes code | **No.** Like `solidity-audit`, the audit produces findings only. Fixes are separate contracts. | Locked | +| Second profile for fixes? | **No.** Fixes run under existing profiles plus one new quality-gate row for behaviour-preserving work. One new profile, not two. | Locked | +| Severity scale | Reuse the kit's existing **Critical / High / Medium / Low / Informational** from `AUDIT_FINDING.md` — do not introduce a third vocabulary. | Locked | + +--- + +## Scope — WILL do + +### A — Profile + +1. Add **`code-audit`** to the profile tables in `templates/AGENT_PROCESS.md`, `templates/AGENTS.md`, + `templates/SPRINT_CONTRACT.md`, and the profile-emphasis table in `templates/REVIEW.md`. +2. Mark the per-sprint gates **N/A** for `code-audit` (it ships no code): TDD, `CODE_QUALITY.md`, + http-api schema, Solidity gates. +3. Add a **depth-matrix row**: `code-audit` — periodic (per milestone / quarter / pre-hardening), + never triggered by an ordinary feature. + +### B — `templates/CODE_AUDIT_CONTRACT.md` (NEW) + +4. Scope in/out: paths, modules, languages, **commit or tag audited** (findings are meaningless + without the ref), and explicit exclusions (generated, vendored, migrations, fixtures). +5. **Behavioural constraint section** — "refactor implementation, not behaviour", with the frozen-contract + list (public APIs, exported symbols, CLI args, env vars, config keys, DB schema/values, serialized + formats, events, queue payloads, routes, status codes, headers, error formats, user-facing strings + consumers depend on) and the behavioural-equivalence checklist. +6. **Review taxonomy** — the 15 categories, condensed to a checklist the auditor works through: + best practices; duplication (harmful vs acceptable); optimisation; code smells; comments; readability; + maintainability; function quality; conditional logic; error handling; data flow and state; tests and + regression risk; dead code; over-engineering; under-engineering. +7. **Rubrics** — Severity (reuse existing scale); **Confidence** High/Medium/Low; **Regression risk** + Very Low/Low/Medium/High. Prioritise by *impact × confidence ÷ regression risk*. +8. **Workflow** — Understand → Identify → Validate → Prioritise → Recommend. Validation before reporting + is mandatory: check call sites, imports, consumers, tests, framework registration before a finding stands. +9. **False-positive discipline** — a finding must meet at least one materiality test; unconventional but + clear, safe, locally appropriate code is not a finding. Minimal-diff principle: smallest change that + achieves a meaningful improvement; no cascading refactors. +10. **Report shape** — executive summary; prioritised findings; duplication table; comment-quality report; + readability hotspots; maintainability hotspots; optimisation (safe vs profiling-required); + safe refactors; **refactors not to attempt yet**; suggested sequence; verification checklist. +11. **Finding ID scheme** — `-F`, same shape as the Solidity audit. +12. **`FEATURES.json` seed block** — one entry per accepted finding, `status: FAIL`, `verify` written from + the repo's `AGENTS.md` runner. This is the audit → contract bridge. +13. **Separate reporting for suspected functional bugs** — reported, never fixed inside the audit. +14. A short, explicit contrast with `CODE_QUALITY.md` so the two are never confused. + +### C — `templates/CODE_AUDIT_FINDING.md` (NEW) + +15. Extend the `AUDIT_FINDING.md` shape with: **Category**, **Confidence**, **Regression risk**, + **Recommended change** (smallest safe), **Behavioural safety** (what must be verified), + and **Characterization tests required**. Keep ID / Severity / Status / parent / commit / location. + +### D — Behaviour-preserving refactor gate + +16. `templates/SPRINT_CONTRACT.md` — one new quality-gate row: + *behaviour-preserving refactor only: characterization tests captured and passing **before** the + change; the same tests pass after; declared frozen contracts untouched — or **N/A***. +17. `templates/AGENT_PROCESS.md` §6 EXECUTE — one line: for behaviour-preserving refactor work, + characterization tests replace product TDD as the RED/GREEN mechanism, mirroring the existing + PoC-first rule for audits. + +### E — Version, docs, bootstrap + +18. **Appendix L** (`CODE_AUDIT_CONTRACT.md`) and **Appendix M** (`CODE_AUDIT_FINDING.md`) in + `scripts/build-bootstrap.sh`; Step 0 path table `Appendices B–M`. +19. Version bump + changelog bullet — see **Q1** for the number. +20. Add `KIT-AUDIT-001` to root `FEATURES.json`; re-run `./scripts/build-bootstrap.sh`. + +## Scope — will NOT do (this sprint) + +- Reference, absorb, or reconcile any pack's skills. The kit is pack-agnostic (David, 2026-08-27). +- Put language-specific thresholds, linters, or tool mandates in the kit. +- Add a second profile for implementing findings — existing profiles plus one gate row cover it. +- Change `CODE_QUALITY.md`, the v1.4 gate, or any existing profile's behaviour. +- Introduce a third severity vocabulary. +- Run an actual audit on any repo. +- Modify any consumer repo. + +## Target + +- **Repo / package:** `harness-kit`. +- **Not touching:** consumer repos; any pack repo; application source. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master` — **[GROUNDED]** +- **Base:** `master` @ `a3af45f` (PR #15 merged; v1.4.0 live, `templates/CODE_QUALITY.md` present) — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-audit-001-code-audit-profile` +- **Human confirmed:** yes — "proceed" (2026-08-27) + +--- + +## Tests first (TDD) + +**N/A** — documentation/process sprint. VERIFY uses falsifiable deterministic checks. + +- **Expected RED (today):** + - `test -f templates/CODE_AUDIT_CONTRACT.md` → fails + - `test -f templates/CODE_AUDIT_FINDING.md` → fails + - `grep -q 'code-audit' templates/AGENT_PROCESS.md` → fails + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `templates/CODE_AUDIT_CONTRACT.md` | **NEW** | [NEW] | +| `templates/CODE_AUDIT_FINDING.md` | **NEW** | [NEW] | +| `templates/AGENT_PROCESS.md` | `code-audit` profile row; depth-matrix row; N/A gates; EXECUTE characterization-test line | [GROUNDED] | +| `templates/AGENTS.md` | Profile list | [GROUNDED] | +| `templates/SPRINT_CONTRACT.md` | Profile checkbox + behaviour-preserving refactor gate row | [GROUNDED] | +| `templates/REVIEW.md` | `code-audit` row in the profile-emphasis table | [GROUNDED] | +| `scripts/build-bootstrap.sh` | Appendices L + M; Step 0 path table | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated; version + changelog | [GROUNDED] | +| `FEATURES.json` (root) | Add `KIT-AUDIT-001` | [GROUNDED] | + +**Reuse:** `AUDIT_CONTRACT.md` structure and severity scale; `AUDIT_FINDING.md` meta block. Extend, do not fork. + +--- + +## Success criteria + +1. `code-audit` appears in the profile tables of `AGENT_PROCESS.md`, `AGENTS.md`, `SPRINT_CONTRACT.md`, `REVIEW.md`. +2. `templates/CODE_AUDIT_CONTRACT.md` and `templates/CODE_AUDIT_FINDING.md` exist. +3. The contract template states the behavioural constraint and carries the frozen-contract list and equivalence checklist. +4. All 15 review categories are present as a worked checklist. +5. Severity / Confidence / Regression-risk rubrics present; prioritisation formula stated; severity scale matches `AUDIT_FINDING.md` (no third vocabulary). +6. Template requires the audited **commit or tag** to be recorded. +7. Template contains a `FEATURES.json` seed block, one entry per accepted finding. +8. Template states the audit changes no code, and reports suspected functional bugs separately without fixing them. +9. Both `CODE_AUDIT_CONTRACT.md` and `CODE_QUALITY.md` state the gate-vs-audit distinction. +10. `SPRINT_CONTRACT.md` carries the behaviour-preserving refactor gate row with an explicit **N/A** path. +11. Kit neutrality: no linter names, thresholds, or pack references in either new template. +12. `grep -ri 'ponytail' templates/` returns **no matches**. +13. Bootstrap regenerates clean; `BOOTSTRAP.md` contains `CODE_AUDIT_CONTRACT`; both `FEATURES.json` parse. +14. Diff confined to the impact map; work on the confirmed feature branch. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — see VERIFY +- [ ] Trust-boundary / partial deps / http-api / solidity — **N/A** (docs-only) +- [ ] Code-quality gate — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor gate — **N/A** (no code) + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Confused with the v1.4 per-sprint gate (both say "quality") | Success criteria 9: an explicit contrast paragraph in **both** files | +| Audit run every sprint; cost explodes | Depth-matrix row marks it periodic; §Cost reality already warns | +| Findings rot unread | Seed block turns accepted findings into `FEATURES.json` entries (§B12) | +| Auditor "fixes" a functional bug mid-audit | §B13 — report separately, never fix | +| Refactors ship without equivalence proof | Characterization-test gate row (§D16/17) | +| Taxonomy becomes a nit generator | False-positive materiality test + minimal-diff principle (§B9) | +| Kit accretes pack coupling | Success criterion 12 is a grep gate | + +## Threat model + +**SKIP** — documentation/process only. + +--- + +## Blocking questions (gates) + +**Q1 — RESOLVED.** KIT-QUALITY-001 merged via **PR #15** (`a3af45f`); kit v1.4.0 live with +`templates/CODE_QUALITY.md` present. This branches from updated `master` and takes **1.5.0**. +KIT-ERD-001 rebases and takes 1.6.0. + +**Q2 — Audit scope default.** Whole repo, or one declared subsystem per audit? +- **Recommendation:** the contract **requires** a declared scope — paths/modules plus commit or tag. + "Whole repo" is allowed but must be stated deliberately, because on a large codebase it produces a + report too big to act on. Mirrors `AUDIT_CONTRACT.md`'s scope-in/out discipline. + +**Q3 — Who accepts a finding into `FEATURES.json`?** Auditor seeds all, or human triages first? +- **Recommendation:** the audit **proposes** the seed block; the **human** accepts findings into the + ledger. Consistent with `PASS` being human-only, and stops a long report flooding the tracker. + +**Q4 — Cadence.** Leave it to each repo's `AGENTS.md`, or state a kit default? +- **Recommendation:** leave it to the repo, with the depth matrix naming the usual triggers + (milestone, quarter, pre-hardening, pre-handover, before a large refactor). The kit should not + impose a calendar. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# profile registered +grep -q 'code-audit' templates/AGENT_PROCESS.md +grep -q 'code-audit' templates/AGENTS.md +grep -q 'code-audit' templates/SPRINT_CONTRACT.md +grep -q 'code-audit' templates/REVIEW.md + +# new templates +test -f templates/CODE_AUDIT_CONTRACT.md +test -f templates/CODE_AUDIT_FINDING.md + +# behavioural spine +grep -qi 'refactor implementation, not behaviour' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'characterization' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'characterization' templates/SPRINT_CONTRACT.md +grep -qi 'frozen' templates/CODE_AUDIT_CONTRACT.md + +# rubrics + prioritisation, single severity vocabulary +grep -qi 'confidence' templates/CODE_AUDIT_FINDING.md +grep -qi 'regression risk' templates/CODE_AUDIT_FINDING.md +grep -qi 'Informational' templates/CODE_AUDIT_FINDING.md +! grep -qi 'Major / Minor' templates/CODE_AUDIT_FINDING.md + +# audit hygiene +grep -qi 'commit\|tag' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'FEATURES.json' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'not part of this refactor\|report separately' templates/CODE_AUDIT_CONTRACT.md + +# gate vs audit distinction stated in both +grep -qi 'CODE_QUALITY' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'CODE_AUDIT' templates/CODE_QUALITY.md + +# neutrality: no tools, no pack coupling +! grep -Eqi 'eslint|golangci|clippy|solhint' templates/CODE_AUDIT_CONTRACT.md +! grep -riq 'ponytail' templates/ + +# bootstrap + json +./scripts/build-bootstrap.sh +grep -q 'CODE_AUDIT_CONTRACT' BOOTSTRAP.md +python3 -m json.tool FEATURES.json >/dev/null +grep -q 'KIT-AUDIT-001' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-AUDIT-001", + "name": "code-audit profile: periodic whole-project quality audit with behaviour-preservation discipline", + "priority": 2, + "verify": "test -f templates/CODE_AUDIT_CONTRACT.md && test -f templates/CODE_AUDIT_FINDING.md && grep -q code-audit templates/AGENT_PROCESS.md && grep -qi characterization templates/SPRINT_CONTRACT.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-AUDIT-001.md; profile: ops-docs; depends on KIT-QUALITY-001 (CODE_QUALITY.md must exist)" +} +``` + +--- + +## Criteria block + +```criteria +- [KIT-AUDIT-001-1] code-audit profile registered in AGENT_PROCESS, AGENTS, SPRINT_CONTRACT, REVIEW +- [KIT-AUDIT-001-2] CODE_AUDIT_CONTRACT.md and CODE_AUDIT_FINDING.md exist +- [KIT-AUDIT-001-3] Behavioural constraint, frozen-contract list, equivalence checklist present +- [KIT-AUDIT-001-4] All 15 review categories present as a worked checklist +- [KIT-AUDIT-001-5] Severity/Confidence/Regression rubrics; single severity vocabulary; prioritisation formula +- [KIT-AUDIT-001-6] Audited commit/tag required; FEATURES seed block present; functional bugs reported not fixed +- [KIT-AUDIT-001-7] Gate-vs-audit distinction in both files; characterization-test gate row in SPRINT_CONTRACT +- [KIT-AUDIT-001-8] No tool names, no thresholds, no pack references anywhere in templates/ +``` + +--- + +## Deviations from the approved contract (declare, do not hide) + +| # | Contract said | Actually done | Why | Verdict | +|---|---------------|---------------|-----|---------| +| 1 | Impact map listed `templates/AGENT_PROCESS.md` and `templates/AGENTS.md` for profile wiring only | Also removed the **pre-existing** named-pack references from both (`agent-harness`, `ponytail` → generic wording); `templates/CODE_QUALITY.md` gained the audit cross-reference | Success criterion 12 (`grep -ri 'ponytail' templates/` → no matches) failed on **v1.3 text already on master**, not on anything this contract added. The named references contradict the pack-agnostic rule David stated on 2026-08-27. Meaning preserved exactly: "a shared agent-harness submodule" → "a shared skills submodule"; "ponytail simplicity" → "simplicity passes". | **Declared scope expansion** — corrective, zero behavioural risk, one revert if unwanted | + +`templates/CODE_QUALITY.md` was always implied by §B14 ("an explicit contrast with `CODE_QUALITY.md`") +but was omitted from the impact-map table. Listed here for completeness. + +## VERIFY result + +24/24 green, including the neutrality gates (no tool names in the audit contract; no named-pack +references anywhere in `templates/`) and a structural check that all 15 taxonomy categories are present. + +## Note for the human + +`KIT-QUALITY-001` is still `PENDING_REVIEW` in `FEATURES.json` although PR #15 is merged. +**`PASS` is human-only** — deliberately left for David rather than repeating the undeclared +ledger edit made under KIT-QUALITY-001. diff --git a/skills/.harness/contracts/KIT-ERD-001.md b/skills/.harness/contracts/KIT-ERD-001.md new file mode 100644 index 0000000..5bc0408 --- /dev/null +++ b/skills/.harness/contracts/KIT-ERD-001.md @@ -0,0 +1,340 @@ +# SPRINT CONTRACT — KIT-ERD-001: `product-erd` profile + ERD authoring stage + +> Status: **EXECUTE complete — VERIFY green (21/21) — PENDING_REVIEW.** +> Approved: (David, "proceed", 2026-08-27), with recommended Q2–Q4. +> **Revised for kit v2.0:** deliverables are a **skill**, not `templates/*`. Version **2.1.0**. +> Feature ID: `KIT-ERD-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-erd-001-product-erd-profile` — **human must confirm before checkout** +> Trigger: David (2026-08-27) — "build an ERD creation process within our agent harness kit … the first bit of building out a product or a feature for us before that is translated to a sprint contract." + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + POSIX sh (`scripts/build-bootstrap.sh`). No application code. + +> Note: this sprint **adds** the `product-erd` profile. The sprint itself runs under `ops-docs`. + +--- + +## Context (grounded) + +Read before writing this contract: + +| Source | Finding | +| --- | --- | +| `templates/AGENT_PROCESS.md` | Loop is `BOOT → RESEARCH? → CONTRACT → [APPROVE+BRANCH] → CHECKOUT → EXECUTE → VERIFY → REVIEW → PENDING_REVIEW → [PASS]`. 5 profiles. Optional RESEARCH "distills open questions", explicitly **does not** create a second approved artifact. | +| `technical-project-documentation/.harness/contracts/ORDER-SVC-001..019` | The kit is **already** authoring ERDs — 19 contracts, all `ops-docs`, all Markdown-only, all grep-based VERIFY. | +| Same, 006/007/008/011/012 | Several are single-decision lock contracts (006 is 740 bytes: "lock amount encoding"). Decisions surfaced **during** doc work, not before it. | +| Same, 009 → 010, and 015 | 009 fully superseded; 015 reversed the stake-limit / `batchAll` / `extrinsicToken` locks from 010–014. Rework caused by late decision discovery. | +| `technical-project-documentation/.agents/skills/erd-authoring/SKILL.md` | Good **style guide** — section flow, slice-table rules, diagram + quality bar. Not a **process**: no cold-start, no decision elicitation, no gates. | +| `technical-project-documentation/.agents/skills/erd-engineering-standards/SKILL.md` | tao.com-specific defaults (HAL, RFC 9457, hexagonal, K8s probes, New Relic). Must **not** move into the kit — kit stays stack-neutral. | +| `products/order-service/v1/{erd.md,architecture.md}` | Reference output shape for the two-document deliverable. | + +**Gap this contract closes:** (1) no cold-start path from brief → first complete ERD; (2) decisions arrive reactively instead of being elicited and locked up front; (3) no mechanical ERD → `FEATURES.json` → sprint-contract handoff. + +--- + +## Scope — WILL do + +### A — New work-type profile + +1. Add **`product-erd`** as the sixth work-type profile in `templates/AGENT_PROCESS.md`, `templates/AGENTS.md`, and `templates/SPRINT_CONTRACT.md`. +2. Profile applies when defining a **new product, service, or substantial feature** whose design is not yet locked. Its deliverable is documentation; its consumers are the sprint contracts that follow. + +### B — New skill (v2.0 shape) — supersedes the original "three templates" + +3. `templates/ERD_CONTRACT.md` — the contract governing an ERD-authoring sprint (analogue of `AUDIT_CONTRACT.md`). Carries the staged gate log in §C below. +4. `skills/erd-authoring/templates/ERD.md` — the ERD document. Section flow: Executive Summary; Goals and Non-Goals; Engineering Standards / Goals; Implementation Overview; Design Decisions (+ Acceptance Criteria); API Conventions and Contracts; Risks / Security / Pre-Mortem; Open Questions; **Implementation plan: Feature slices**; Appendices (Data Dictionary, Glossary, References). +5. `skills/erd-authoring/templates/ARCHITECTURE.md` — companion doc. Owns: mental model, system diagram (Mermaid), component/boundary table, ports or module map **only where `AGENTS.md` declares that pattern**, primary control flows, data ownership, deployable boundaries. + +### C — The staged EXECUTE (progressive, human-gated) + +6. Document in `templates/AGENT_PROCESS.md` a new subsection **"ERD stage (`product-erd`)"**. EXECUTE for this profile is **five stages with a human gate between each**. The agent stops at every gate. + +``` +BRIEF/PRD ─► S0 Frame ─► S1 Decisions ─► S2 Design ─► S3 Contracts ─► S4 Slices ─► erd.md + architecture.md + ▲ ▲ ▲ ▲ ▲ │ + [gate] [gate] [gate] [gate] [gate] ▼ + FEATURES.json seed ─► sprint contracts +``` + +| Stage | Agent produces | Human gate | +| --- | --- | --- | +| **S0 Frame** | Input inventory (PRD/brief/design/tickets/related repos + code to reuse); draft **Executive Summary** + **Goals** + **Non-Goals** | Confirms the scope boundary. Non-goals must be explicit here, not discovered later. | +| **S1 Decisions** | Every decision the ERD must lock, as a table: `Topic \| Options \| Recommendation \| Status`. No design written yet. | Answers or defers each. Deferred rows move to **Open Questions** with an owner and a needed-by. **This is the stage that replaces reactive lock contracts.** | +| **S2 Design** | `architecture.md`: mental model, diagram, boundaries, control flows, data ownership | Confirms the shape before contracts are written against it. | +| **S3 Contracts** | ERD design sections: standards, API/event/data contracts, error model, acceptance criteria, risks / pre-mortem | Confirms contracts and falsifiable acceptance criteria. | +| **S4 Slices** | `Implementation plan: Feature slices` table **plus** the `FEATURES.json` seed block — one entry per slice | PASS. ERD is now the input to sprint contracts. | + +7. **Slices are written last**, only after S0–S3 are stable. Slice 1 is scaffolding (incl. readiness/liveness probes) when the target is a deployable service; **N/A** for library, contract, or docs targets. Every slice row needs a concrete working behaviour and a functional test that mimics a user story or a meaningful part of one. +8. **The ERD → contract bridge (the point of this sprint):** S4 emits a `FEATURES.json` seed block, one entry per slice, `status: FAIL`, `verify` written from the consumer repo's `AGENTS.md` runner. Each seeded entry then becomes exactly one sprint contract under the normal loop. The handoff is mechanical, not interpretive. + +### D — Depth matrix + skip rule + +9. Extend the depth matrix in `templates/AGENT_PROCESS.md` with an ERD row, and state plainly **when to skip the ERD stage**: a feature inside an already-documented service goes straight to a sprint contract. The ERD stage is for a new product/service, a new domain, or a contract crossing teams or repos. Cite "build to delete" / cost reality. +10. State that optional **RESEARCH** and the **ERD stage** are not the same thing and must not be dual-run: RESEARCH distills open questions for one contract; the ERD stage produces a durable design document that feeds many contracts. + +### E — Kit neutrality (explicit) + +11. The kit ships the **process and the document skeleton only**. Organisation standards — HAL, RFC 9457, hexagonal, camelCase, probe endpoints, log destinations — stay in the consumer repo's `AGENTS.md` or its docs skill pack. the ERD form references "the standards pack declared in `AGENTS.md`" and never names one. +12. The ERD form instructs: record any divergence from the declared standards pack in **Design Decisions** with rationale. Never diverge silently. + +### F — Review + gates + +13. The `sprint-contract` form: add `product-erd` to the profile checklist and add one quality-gate row — *`product-erd` only: decisions locked or owner+needed-by; non-goals explicit; slices ticket-ready with functional tests; FEATURES seed emitted — or **N/A***. +14. The `harness-review` form: add `product-erd` to the profile-aware review hints — *scope honesty (non-goals real), decision completeness, falsifiable acceptance criteria, slice ticketability, no silent standards divergence*. + +### G — Version, docs, bootstrap + +15. Bump kit version to **2.1.0** in `scripts/build-bootstrap.sh` (which regenerates the deprecated `BOOTSTRAP.md`) and `README.md`. +16. Append **Appendix K** (`ERD_CONTRACT.md`), **Appendix L** (`ERD.md`), **Appendix M** (`ARCHITECTURE.md`) in `scripts/build-bootstrap.sh`; add the three files to the Step 0 path table. +17. Add `erd-authoring` to the skills table in `README.md`; mention the `product-erd` profile. +18. Add the `KIT-ERD-001` entry to root `FEATURES.json`. +19. Re-run `./scripts/build-bootstrap.sh` so `BOOTSTRAP.md` stays in sync. + +--- + +## Scope — will NOT do (this sprint) + +- Copy tao.com standards (HAL, RFC 9457, hexagonal, New Relic, K8s probes) into the kit. +- Modify `technical-project-documentation`, `order-service`, or any other consumer repo. +- Migrate the existing `ORDER-SVC-001..019` contracts or rewrite `products/order-service/v1/*`. +- Adopt or absorb `agent-harness` / its RPI skills (still out of scope per KIT-V13-001). +- Add an ERD amendment / change-log template — see blocking question Q3. +- Add PRD authoring. The ERD stage takes a brief or PRD as **input**; it does not write one. +- Automate ERD linting beyond grep-based VERIFY commands. +- Change the existing five profiles, the status model, or the session loop outside the additions above. + +## Target + +- **Repo / package:** `harness-kit` — the kit owns the process spine. +- **Not touching:** every consumer repo; `agent-harness`; application source anywhere. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master` (via `git symbolic-ref refs/remotes/origin/HEAD`) — **[GROUNDED]** +- **Upstream reality (verified 2026-08-27):** KIT-V13-001 **is already merged** to `origin/master` via **PR #14** (`e3b98b1`). Kit v1.3.0 is live; `templates/AUDIT_CONTRACT.md` present on master. Local `master` is **2 commits behind** and local `FEATURES.json` still reads `PENDING_REVIEW` — ledger drift, not a blocker — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-erd-001-product-erd-profile` +- **Base:** stacked on `feat/kit-skills-001-templates-to-skills` (`2b37249`). KIT-SKILLS-001 is + PENDING_REVIEW and unmerged, and this work needs the v2.0 skill layout that exists only there. + **Rebase onto `master` once KIT-SKILLS-001 merges.** +- **Human confirmed:** **pending — agent asks before checkout.** + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Home of the ERD process | **harness-kit core** — part of the default spine, not an opt-in pack | Locked (David, 2026-08-27) | +| Deliverable set | **`erd.md` + `architecture.md`** pair, matching the order-service precedent | Locked (David, 2026-08-27) | +| Interaction model | **Progressive, human-gated stages** S0–S4 | Locked (David, 2026-08-27) | +| Governance | **New `product-erd` work-type profile**, governed by the contract system | Locked (David, 2026-08-27) | +| Profile name | `product-erd` | Open — **Q2** | +| ERD → contract bridge | S4 emits a `FEATURES.json` seed block, one entry per slice; each becomes one sprint contract | Locked | +| Org standards | Stay in consumer `AGENTS.md` / docs pack; kit references, never names them | Locked | +| Slice ordering | Slices written last; slice 1 = scaffolding for deployable services, N/A otherwise | Locked | +| `openapi.yaml` | **Not** an ERD-stage deliverable — it churned across ORDER-SVC-014/016 as decisions moved. Freeze it in a later `http-api` contract | Locked | +| Amendment change-log | Not this sprint | Open — **Q3** | +| Base branch | Updated `master` (v1.3 already merged, PR #14) | Locked | + +--- + +## Tests first (TDD) + +**N/A** — documentation/process sprint, no behaviour change. Per kit law, VERIFY uses falsifiable deterministic checks (below), not application tests. + +- **Expected RED (today):** + - `grep -q 'product-erd' templates/AGENT_PROCESS.md` → fails + - `test -f templates/ERD.md` → fails + - `test -f templates/ARCHITECTURE.md` → fails + - `test -f templates/ERD_CONTRACT.md` → fails + - `grep -q '1.4.0' BOOTSTRAP.md` → fails + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `skills/erd-authoring/SKILL.md` | **NEW** | [NEW] | +| `skills/erd-authoring/templates/ERD_CONTRACT.md` | **NEW** — staged gate log | [NEW] | +| `skills/erd-authoring/templates/ERD.md` | **NEW** — ERD skeleton | [NEW] | +| `skills/erd-authoring/templates/ARCHITECTURE.md` | **NEW** — architecture companion | [NEW] | +| `skills/harness-onboard/templates/AGENT_PROCESS.md` | `product-erd` row; ERD stage (S0–S4); depth-matrix row; RESEARCH-vs-ERD separation | [GROUNDED] | +| `skills/harness-onboard/templates/AGENTS.md` | Profile list + ERD pointer | [GROUNDED] | +| `skills/sprint-contract/templates/SPRINT_CONTRACT.md` | Profile checkbox + quality-gate row | [GROUNDED] | +| `skills/harness-review/templates/REVIEW.md` | `product-erd` review row | [GROUNDED] | +| `README.md` | Skills table + profile | [GROUNDED] | +| `scripts/build-bootstrap.sh` | 2.1.0; Appendices N/O/P | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated (still deprecated) | [GROUNDED] | +| `README.md`, `templates/README_POINTER.md` | Mention profile + ERD stage | [GROUNDED] | +| `FEATURES.json` (root) | Add `KIT-ERD-001` | [GROUNDED] | + +**Reuse (do not reinvent):** `AUDIT_CONTRACT.md` structure for `ERD_CONTRACT.md`; the ORDER-SVC grep-VERIFY idiom; the erd-authoring section flow and slice-table rules (**re-expressed stack-neutrally — not copied verbatim**, since the source is tao.com-coupled). + +--- + +## Success criteria + +1. `product-erd` appears as a profile in `AGENT_PROCESS.md`, `AGENTS.md`, and `SPRINT_CONTRACT.md`. +2. `templates/ERD.md`, `templates/ARCHITECTURE.md`, `templates/ERD_CONTRACT.md` all exist. +3. `AGENT_PROCESS.md` documents stages **S0–S4** with an explicit human gate between each, and states the agent stops at every gate. +4. `AGENT_PROCESS.md` states slices are written **last** and that S4 emits a `FEATURES.json` seed block, one entry per slice. +5. `AGENT_PROCESS.md` contains an explicit **skip rule** for the ERD stage and distinguishes it from optional RESEARCH. +6. Kit neutrality holds: no `HAL|RFC 9457|hexagonal|New Relic` in the ERD or architecture forms. +7. The ERD form contains the heading `## Implementation plan: Feature slices` verbatim. +8. The `sprint-contract` and `harness-review` forms each carry a `product-erd` line. +9. Kit version `2.1.0`; regenerated `BOOTSTRAP.md` contains `ERD_CONTRACT`; `erd-authoring` has valid frontmatter. +10. `./scripts/build-bootstrap.sh` exits 0; both `FEATURES.json` files parse. +11. Diff confined to the impact map; `git status` shows only those paths. +12. All work on the confirmed feature branch — not `master`. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria (always required) — see VERIFY +- [ ] Trust-boundary failure mode — **N/A** (docs-only) +- [ ] Partial/optional deps — **N/A** +- [ ] `http-api` schema/envelope — **N/A** +- [ ] `solidity-build` — **N/A** +- [ ] `solidity-audit` — **N/A** +- [ ] No consumer-repo edits — asserted in Scope/will NOT + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| ERD stage becomes mandatory ceremony on small features | Explicit skip rule + depth-matrix row (§D9); "build to delete" cited | +| Kit accretes tao.com standards through the ERD template | Success criterion 6 is a hard grep gate | +| Gates get skipped and S0–S4 collapse into one draft | `ERD_CONTRACT.md` carries a gate log with a human sign-off line per stage | +| ERD and sprint contracts drift after implementation starts | Partially unmitigated this sprint — see **Q3** | +| Two competing approved artifacts (ERD *and* a `plan.md`) | §D10 states RESEARCH and the ERD stage must not be dual-run | +| Slices written before design settles (the erd-authoring failure mode) | §C7 + success criterion 4 | + +## Threat model + +**SKIP** — documentation/process only; no runtime surface, no credentials, no side effects. + +--- + +## Blocking questions (gates) + +**Q1 — RESOLVED (was: base branch).** v1.3 is already merged (PR #14). Branch from an updated `master`; no stacking needed. Residual action: pull local `master`, and set `KIT-V13-001` to `PASS` in root `FEATURES.json` so the ledger matches reality. + +**Q2 — Profile name.** `product-erd` vs `discovery` vs plain `erd`. +- **Recommendation:** `product-erd` — says what it produces, and reads correctly for a feature-level ERD as well as a whole product. + +**Q3 — Amendment change-log.** Your docs repo has `engineering-amendment-change-log` for when implementation diverges from an ERD. ORDER-SVC-009 and 015 were reversals, so this path is real. Ship `templates/ERD_AMENDMENT.md` in this sprint, or defer to `KIT-ERD-002`? +- **Recommendation:** defer to `KIT-ERD-002` — "one thing at a time." This sprint adds a one-line pointer in `templates/ERD.md` so the seam exists. + +**Q4 — Slice-1 scaffolding rule.** The tao.com skill hardcodes "slice 1 is always scaffolding + K8s probes." The kit is stack-neutral and covers Solidity and libraries. +- **Recommendation:** as drafted in §C7 — slice 1 is scaffolding *for deployable services*, explicitly N/A otherwise, with the probe specifics left to the consumer's `AGENTS.md`. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# profile registered in all three places +grep -q 'product-erd' templates/AGENT_PROCESS.md +grep -q 'product-erd' templates/AGENTS.md +grep -q 'product-erd' templates/SPRINT_CONTRACT.md +grep -q 'product-erd' templates/REVIEW.md + +# new templates exist +test -f templates/ERD.md +test -f templates/ARCHITECTURE.md +test -f templates/ERD_CONTRACT.md + +# staged, gated EXECUTE documented +grep -q 'S0' templates/AGENT_PROCESS.md +grep -q 'S4' templates/AGENT_PROCESS.md +grep -qi 'human gate' templates/AGENT_PROCESS.md + +# ERD -> contract bridge + slices last +grep -qi 'seed' templates/AGENT_PROCESS.md +grep -q 'Implementation plan: Feature slices' templates/ERD.md +grep -qi 'skip' templates/AGENT_PROCESS.md + +# gate log present in the ERD contract template +grep -qi 'gate log\|Gate log' templates/ERD_CONTRACT.md + +# kit neutrality — org standards must NOT leak into kit templates +! grep -Eqi 'HAL|RFC 9457|hexagonal|New Relic' templates/ERD.md +! grep -Eqi 'HAL|RFC 9457|hexagonal|New Relic' templates/ARCHITECTURE.md + +# version + bootstrap regen +grep -q '1.4.0' BOOTSTRAP.md +grep -q '1.4.0' scripts/build-bootstrap.sh +./scripts/build-bootstrap.sh +grep -q 'ERD_CONTRACT' BOOTSTRAP.md +grep -q '1.4.0' BOOTSTRAP.md + +# json validity + feature entry +python3 -m json.tool FEATURES.json >/dev/null +python3 -m json.tool templates/FEATURES.json >/dev/null +grep -q 'KIT-ERD-001' FEATURES.json + +# branch discipline +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-ERD-001", + "name": "product-erd profile + staged ERD authoring stage (erd.md + architecture.md) feeding FEATURES seed", + "priority": 1, + "verify": "grep -q product-erd templates/AGENT_PROCESS.md && test -f templates/ERD.md && test -f templates/ARCHITECTURE.md && test -f templates/ERD_CONTRACT.md && grep -q '1.4.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-ERD-001.md; profile: ops-docs; depends on KIT-V13-001" +} +``` + +--- + +## Criteria block + +```criteria +- [KIT-ERD-001-1] product-erd registered in AGENT_PROCESS, AGENTS, SPRINT_CONTRACT, REVIEW +- [KIT-ERD-001-2] templates/ERD.md, ARCHITECTURE.md, ERD_CONTRACT.md exist +- [KIT-ERD-001-3] Stages S0-S4 documented with a human gate between each +- [KIT-ERD-001-4] Slices written last; S4 emits FEATURES.json seed, one entry per slice +- [KIT-ERD-001-5] Explicit ERD-stage skip rule; distinguished from optional RESEARCH +- [KIT-ERD-001-6] Kit neutrality: no org-specific standards in ERD/ARCHITECTURE templates +- [KIT-ERD-001-7] Version 1.4.0; Appendices K/L/M; build-bootstrap.sh green; FEATURES.json valid +``` + +--- + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | Contract (pre-v2.0) specified `templates/ERD.md`, `ARCHITECTURE.md`, `ERD_CONTRACT.md` and Appendices K/L/M | Revised to the v2.0 skill shape before execution: `skills/erd-authoring/` with the three forms bundled, Appendices **N/O/P**, version **2.1.0**. Recorded in the contract header. | +| 2 | Skill name is `erd-authoring`; profile is `product-erd` | Deliberate. Not every skill maps to a profile (`sprint-contract`, `harness-onboard`, `security-checklist`, `harness-review` have none), and `erd-authoring` names the activity while `product-erd` names the work type. Both files state the relationship. | +| 3 | VERIFY expected the literal phrase "stops at every gate" in `SKILL.md`, which said "stops and waits at each gate" | Semantically identical; phrasing aligned across both files rather than loosening the check — consistent vocabulary is worth having. | + +## Q2–Q4 as executed + +- **Q2 profile name:** `product-erd` (recommended). +- **Q3 amendment change-log:** deferred to `KIT-ERD-002` as recommended; the seam exists — divergence from the declared standards pack is recorded in Design Decisions. +- **Q4 slice-1 rule:** scaffolding for deployable services, explicitly **N/A** for library / contract / docs targets, with probe specifics left to the consumer's `AGENTS.md`. + +## Note + +Stacked on `feat/kit-skills-001-templates-to-skills`, which is PENDING_REVIEW and unmerged. +**Rebase onto `master` once KIT-SKILLS-001 merges.** diff --git a/skills/.harness/contracts/KIT-GATES-001.md b/skills/.harness/contracts/KIT-GATES-001.md new file mode 100644 index 0000000..b2e064f --- /dev/null +++ b/skills/.harness/contracts/KIT-GATES-001.md @@ -0,0 +1,345 @@ +# SPRINT CONTRACT — KIT-GATES-001: deterministic gates (RED/BASELINE/GREEN + `verify-harness.sh` + CI) + +> Status: **EXECUTE complete — VERIFY green (20/20) — PENDING_REVIEW.** +> Approved: David 2026-08-28 ("approve") with recommended Q1–Q4. Version **2.3.0**. +> Feature ID: `KIT-GATES-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-gates-001-deterministic-gates`, **stacked on** +> `feat/kit-rules-001-rules-and-erd-routing` (`65aeb7f`, PENDING_REVIEW). Rebase onto `master` when that merges. +> Version → **2.3.0**. +> Trigger: David (2026-08-28) — "how do we put a gate to ensure that a contract delivers/executes a +> feature without regression or hallucination, especially as features/slices would not have an ERD?" + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** POSIX sh + Markdown + one CI workflow. No application code. + +--- + +## Context — the kit currently has no enforced gate (grounded) + +| Finding | Evidence | Consequence | +| --- | --- | --- | +| Every "gate" is self-reported | `AGENT_PROCESS` §VERIFY *asks* the agent to run commands; nothing confirms it | An executor can set `PENDING_REVIEW` without running anything | +| The RED field asks for a **prediction** | `SPRINT_CONTRACT.md:75` — "**Expected** RED output: ``" | Nothing ever compares the prediction to reality. A test that never failed proves nothing: it may be vacuous or already passing | +| No CI, no hooks | No `.github/` in the kit; zero hook configs | The only thing in the repo that can fail and stop something is `setup-harness-kit.sh` | +| `FEATURES.json` is unvalidated | No schema check anywhere | Status drift went unnoticed **four times** this month across two repos (harness-kit 3 entries, order-service 10/10) | +| **CI cannot read a contract** | `git check-ignore skills/.harness/contracts` → ignored | The artifact `harness-review` §0 grades a PR against is invisible to CI, to reviewers, and (as observed 2026-08-28) to the author in Finder and the editor sidebar | + +**The gap:** rules and skills are instructions to a model that can misread them under load. A gate is a +fact. The kit has the first two layers and not the third. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Prediction → recording | `SPRINT_CONTRACT` gains **RED / BASELINE / GREEN** artifact blocks holding real command, real output, real exit code | Locked | +| Delivery gate | The acceptance test must have **failed before EXECUTE**, on an **assertion** | Locked | +| Regression gate | Baseline suite recorded before, re-run after; **diff ⊆ declared impact map** | Locked | +| Hallucination gate | Claims replaced by artifacts: every `[GROUNDED]` path must exist; every `verify` must exit 0 | Locked | +| Slices with no ERD | Handled by the same gate. If you cannot write a test that fails today, you do not have a falsifiable criterion and the contract is not approvable | Locked | +| Implementation | **One script, three invocation points** — `scripts/verify-harness.sh`, called by CI and optionally by hooks. Never reimplemented per surface | Locked | +| Unforgeable layer | **CI re-runs VERIFY itself**, so the agent's claim stops mattering | Locked | +| Hooks | **Optional, opt-in**, documented as an example. Vendor-specific; CI and the script are universal | Locked | +| Vacuous tests | Out of reach of any cheap gate. Mitigate by requiring an assertion-shaped RED and reviewing the test **at contract approval**. Mutation testing documented as an option, **not** a default | Locked | + +--- + +## Scope — WILL do + +### A — `scripts/verify-harness.sh` + +1. New POSIX script, exits non-zero on any violation, runnable from a consumer repo root. Checks: + +| # | Check | Catches | +|---|-------|---------| +| 1 | `FEATURES.json` parses; every `status` is from the legend; every `verify` non-empty and not prose | Ledger rot | +| 2 | Every `PENDING_REVIEW` entry has a contract file where it claims | Phantom work | +| 3 | `STATE.md` current contract points at a real file | Stale state | +| 4 | Every skill has valid frontmatter (`name`, `description`) | Skills that cannot be retrieved | +| 5 | Every rule has valid frontmatter and `alwaysApply` | Rules that never load | +| 6 | Every `[GROUNDED]` path in the active contract's impact map **exists** | **Invented files** — the most common contract hallucination | +| 7 | `git diff --name-only ..HEAD` ⊆ impact map | Undeclared scope creep, the main regression vector | +| 8 | RED artifact present, and its recorded failure is an **assertion**, not an import/path error | Vacuous or never-failing tests | +| 9 | No implementation commits on the default branch | Branch discipline | + +2. Checks 6–8 **degrade gracefully**: when no contract is reachable they report `skipped (no contract)` + rather than failing, so the script is useful in a repo mid-adoption. Checks 1–5 and 9 always run. +3. `--strict` turns skips into failures, for repos that have fully adopted. + +### B — Contract form: prediction becomes recording + +4. Replace `**Expected RED output:**` with three blocks: + +``` +### RED (recorded before EXECUTE) +- Command: +- Exit code: +- Failure reason: +- Captured: + +### BASELINE (recorded before EXECUTE) +- Command: +- Result: + +### GREEN (recorded after EXECUTE) +- Command: Exit code: 0 +- Command: Result: +``` + +5. State the rule plainly in the form: **a test that never failed proves nothing**, and a RED that + failed on an import error proves the test file is wrong, not that the behaviour is absent. +6. Add the no-ERD line: without an ERD the contract's own criteria are the whole story, and each must + be expressible as a failing test **before** work starts, or the contract is not approvable. + +### C — CI + +7. `.github/workflows/harness.yml` — on pull request: run `verify-harness.sh`, then re-run the + contract's VERIFY block. Kit-owned, shipped to consumers via the skill tree. +8. Document that CI is the only layer an agent cannot fake, and that a green local run is not evidence. + +### D — Hooks (optional) + +9. `docs/hooks.md` (or a README section) with one worked example calling the same script. Explicitly + opt-in and vendor-specific; the kit never requires it. + +### E — Process + wiring + +10. `AGENT_PROCESS.md` §VERIFY — require the three artifacts, and say the executor may set + `PENDING_REVIEW` only when `verify-harness.sh` exits 0. +11. `harness-review` form §0 — reviewer confirms the artifacts exist and that CI ran the gate. +12. `code-quality-gate` — one line pointing at the shared script so the two do not drift. +13. Version **2.3.0**, changelog, `FEATURES.json` entry, bootstrap appendix for the new script/workflow. + +## Scope — will NOT do (this sprint) + +- Mutation testing, coverage thresholds, or any quality metric beyond pass/fail. +- Language-specific test runners — commands come from the consumer's `AGENTS.md`. +- Reconcile the forked global harness (`KIT-RECONCILE-001`). +- The `code-audit` partial-coverage amendment (separate, small). +- Retro-fit artifacts onto already-merged contracts. +- Migrate any consumer repo. + +--- + +## Branch + +- **Base:** `feat/kit-rules-001-rules-and-erd-routing` @ `65aeb7f` — needs `rules/` to exist for check 5. +- **Proposed:** `feat/kit-gates-001-deterministic-gates` +- **Human confirmed:** **pending.** + +## Tests first + +**N/A** as product TDD — but this sprint **dogfoods its own gate**: `verify-harness.sh` is run against +harness-kit itself as part of VERIFY, and must exit 0. + +- **Expected RED today:** `test -f scripts/verify-harness.sh` fails; `test -d .github/workflows` fails. + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `scripts/verify-harness.sh` | **NEW** | [NEW] | +| `.github/workflows/harness.yml` | **NEW** | [NEW] | +| `skills/sprint-contract/templates/SPRINT_CONTRACT.md` | RED/BASELINE/GREEN blocks; no-ERD rule | [GROUNDED] | +| `skills/harness-onboard/templates/AGENT_PROCESS.md` | VERIFY requires artifacts + script exit 0 | [GROUNDED] | +| `skills/harness-review/templates/REVIEW.md` | §0 confirms artifacts and CI | [GROUNDED] | +| `skills/code-quality-gate/templates/CODE_QUALITY.md` | Pointer to the shared script | [GROUNDED] | +| `README.md`, `scripts/build-bootstrap.sh`, `BOOTSTRAP.md`, `FEATURES.json` | Gates section, appendix, 2.3.0, ledger | [GROUNDED] | +| `.gitignore` | Contracts untracked→tracked; legacy `.harness/` rule anchored to root | [GROUNDED] | +| `skills/.harness/contracts/KIT-V13-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-QUALITY-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-AUDIT-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-SKILLS-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-ERD-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-RULES-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | + +--- + +## Success criteria + +1. `scripts/verify-harness.sh` exists, is executable, POSIX-clean, and **exits 0 against harness-kit itself**. +2. Running it against a repo with a deliberately broken `FEATURES.json` exits non-zero — demonstrated, not asserted. +3. Checks 6–8 report `skipped (no contract)` rather than failing when no contract is reachable; `--strict` flips that. +4. `.github/workflows/harness.yml` runs the script and the contract VERIFY block on pull requests. +5. `SPRINT_CONTRACT.md` carries RED/BASELINE/GREEN blocks; the phrase "Expected RED output" is gone. +6. The form states that a test which never failed proves nothing, and the no-ERD rule. +7. `AGENT_PROCESS.md` §VERIFY conditions `PENDING_REVIEW` on the script exiting 0. +8. `REVIEW.md` §0 confirms artifacts exist and CI ran. +9. One implementation: neither CI nor the hooks doc reimplements a check — both invoke the script. +10. 2.3.0 everywhere; bootstrap regenerates; `FEATURES.json` parses; diff confined to the impact map. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — criteria 1 and 2 are executable +- [ ] Trust-boundary — **N/A** (docs/script) +- [ ] Code-quality gate — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor — **N/A** + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Gate blocks a repo mid-adoption | Graceful skips; `--strict` is opt-in | +| Contract checks silently no-op forever | The skip is **printed**, not swallowed, so "skipped" is visible in CI logs | +| CI and hooks drift | Criterion 9: single script, both invoke it | +| Vacuous test passes every check | Acknowledged limit. Assertion-shaped RED + review the test at approval; mutation testing documented, not mandated | +| Diff-⊆-impact-map fights legitimate discoveries | The remedy is amending the contract, which is the intended behaviour | +| Script becomes a second process spine | It validates the spine, adds no rules of its own | + +## Threat model + +**SKIP** — no network, no credentials, no privileged operations. CI runs only repo-declared commands. + +--- + +## Blocking questions (gates) + +**Q1 — Contracts are gitignored, so CI cannot read them.** This is now forcing, not cosmetic: checks +6–8 need the contract, and `harness-review` §0 already tells a reviewer to grade the PR against a file +that is not in the PR. You hit this yourself on 2026-08-28 when the contract was invisible in Finder +and the editor sidebar. +- **Recommendation:** **track contracts.** Move `skills/.harness/contracts/` out of `.gitignore` and + keep `STATE.md` and local scratch ignored. They are markdown, the audit trail is worth having in git, + and it makes the contract reviewable, CI-checkable and findable in one move. +- Alternatives: CI runs only checks 1–5 and 9 (weaker, and the hallucination gate is the valuable one); + or CI pastes the contract into the PR body (solves review, not checking). + +**Q2 — Should the diff-⊆-impact-map check block, or warn?** It is the strongest regression gate and +also the most likely to annoy. +- **Recommendation:** **block**, with the remedy being to amend the contract. Warning-only gates get ignored. + +**Q3 — Baseline on every contract, or only behaviour changes?** Recording a full-suite baseline costs a +run each time. +- **Recommendation:** behaviour changes only; **N/A** for `ops-docs`, `code-audit`, `product-erd`. + +**Q4 — Does the kit ship the CI workflow to consumers, or document it?** Shipping means the kit writes +into `.github/`, which some repos guard. +- **Recommendation:** ship it as a template the setup script **offers** but never overwrites, following + the same marker-guard discipline as the rules assembly. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# the gate exists and passes against this repo (dogfood) +test -x scripts/verify-harness.sh +sh -n scripts/verify-harness.sh +./scripts/verify-harness.sh + +# and it actually fails on a broken ledger (demonstrated, not claimed) +cp FEATURES.json /tmp/f.bak +python3 - <<'PY' +import json;d=json.load(open('FEATURES.json'));d['features'][0]['status']='BOGUS' +open('FEATURES.json','w').write(json.dumps(d,indent=2)) +PY +./scripts/verify-harness.sh && echo "GATE DID NOT FAIL — BAD" && exit 1 +cp /tmp/f.bak FEATURES.json +./scripts/verify-harness.sh + +# CI wired +test -f .github/workflows/harness.yml +grep -q 'verify-harness.sh' .github/workflows/harness.yml + +# prediction replaced by recording +! grep -q 'Expected RED output' skills/sprint-contract/templates/SPRINT_CONTRACT.md +grep -q 'RED (recorded before EXECUTE)' skills/sprint-contract/templates/SPRINT_CONTRACT.md +grep -q 'BASELINE' skills/sprint-contract/templates/SPRINT_CONTRACT.md +grep -qi 'never failed proves nothing' skills/sprint-contract/templates/SPRINT_CONTRACT.md + +# process + review wired to the same script +grep -q 'verify-harness.sh' skills/harness-onboard/templates/AGENT_PROCESS.md +grep -q 'verify-harness.sh' skills/harness-review/templates/REVIEW.md + +# single implementation +test "$(grep -rlc 'FEATURES.json parses' .github scripts 2>/dev/null | wc -l)" -le 1 + +# version + ledger +grep -q '2.3.0' BOOTSTRAP.md && sh scripts/build-bootstrap.sh +python3 -m json.tool FEATURES.json >/dev/null && grep -q 'KIT-GATES-001' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-GATES-001", + "name": "Deterministic gates: RED/BASELINE/GREEN artifacts, verify-harness.sh, CI re-running VERIFY", + "priority": 1, + "verify": "test -x scripts/verify-harness.sh && ./scripts/verify-harness.sh && test -f .github/workflows/harness.yml && ! grep -q 'Expected RED output' skills/sprint-contract/templates/SPRINT_CONTRACT.md && grep -q '2.3.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-GATES-001.md; profile: ops-docs; stacked on KIT-RULES-001; Q1 decides whether contracts become tracked" +} +``` + +## Criteria block + +```criteria +- [KIT-GATES-001-1] verify-harness.sh exists, POSIX, executable, exits 0 against harness-kit +- [KIT-GATES-001-2] Demonstrated non-zero exit on a broken FEATURES.json +- [KIT-GATES-001-3] Contract checks skip visibly without a contract; --strict flips to failure +- [KIT-GATES-001-4] CI workflow runs the script and the contract VERIFY block on PRs +- [KIT-GATES-001-5] Expected-RED prediction replaced by RED/BASELINE/GREEN recordings +- [KIT-GATES-001-6] No-ERD rule stated: no failing test means no falsifiable criterion +- [KIT-GATES-001-7] AGENT_PROCESS conditions PENDING_REVIEW on the gate; REVIEW s0 confirms artifacts +- [KIT-GATES-001-8] Single implementation — CI and hooks both invoke the script +``` + +--- + +## RED (recorded before EXECUTE) + +- **Command:** `test -x scripts/verify-harness.sh && test -f .github/workflows/harness.yml` +- **Exit code:** `1` +- **Failure reason:** neither file existed at `aea8fb2` — the gate and its CI surface were absent +- **Captured:** 2026-08-28, before the first edit on this branch + +**BASELINE** — N/A: no test suite exists in this repo; the gate itself is the suite from this sprint on. + +## GREEN (recorded after EXECUTE) + +- `./scripts/verify-harness.sh` → **exit 0**, 8 pass / 1 skip (RED artifact, on a contract predating the format) +- Contract VERIFY block → **20/20 pass** + +## Criterion 2 — demonstrated, not asserted + +The gate was shown to fail on four separate defects, each restored afterwards: + +| Injected defect | Gate output | Exit | +|---|---|---| +| `status: "BOGUS"` | `KIT-RULES-001: status 'BOGUS' not in legend` | 1 | +| `verify: "tested"` (prose) | `KIT-RULES-001: verify does not look like a command` | 1 | +| `PENDING_REVIEW` with no contract | `PENDING_REVIEW without a contract: GHOST-001` | 1 | +| Rule with no frontmatter | `rules with bad frontmatter: rules/status-ownership.mdc` | 1 | + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | **The gate found a false positive in itself.** Check 6 read `master` and `ef393c3` as paths, because branch names and SHAs are also backticked on `[GROUNDED]` lines in the Branch section | Narrowed the heuristic to tokens containing `/` or carrying a known extension. Caught by dogfooding before shipping — which is the argument for criterion 1 | +| 2 | Q1 required more than a `.gitignore` edit | The legacy rule `.harness/` is **unanchored**, so it matched `skills/.harness` at any depth and kept contracts ignored regardless of the new negation. Anchored it to `/.harness/` (its intended root-only meaning). Found with `git check-ignore -v` rather than assumed | +| 3 | Contract listed `.gitignore` as conditional on Q1 | Q1 answered "track contracts", so it changed | +| 4 | **The gate failed on its own contract, correctly.** Tracking contracts pulled six pre-existing contract files into the diff that the impact map never declared | Scope containment reported them as undeclared. Per the design, the remedy is **amending the contract**, not relaxing the check — the impact map now lists all six. First live catch, on the sprint that introduced the check | + +## Q1–Q4 as executed + +- **Q1** contracts **tracked**; `STATE.md`, drafts and PR bodies stay local. +- **Q2** scope containment **blocks**; the remedy is amending the contract. +- **Q3** BASELINE for behaviour changes only; N/A for `ops-docs`, `code-audit`, `product-erd`. +- **Q4** CI workflow shipped as a kit file; consumers adopt it through the skill tree rather than having it written into their `.github/` by the setup script. diff --git a/skills/.harness/contracts/KIT-QUALITY-001.md b/skills/.harness/contracts/KIT-QUALITY-001.md new file mode 100644 index 0000000..900a4c3 --- /dev/null +++ b/skills/.harness/contracts/KIT-QUALITY-001.md @@ -0,0 +1,283 @@ +# SPRINT CONTRACT — KIT-QUALITY-001: code-quality gate + +> Status: **EXECUTE complete — VERIFY green (0 failures) — PENDING_REVIEW.** +> Approved by David 2026-08-27 with recommended answers to Q1/Q3/Q4; Q2 resolved (1.4.0). +> Branch `feat/kit-quality-001-code-quality-gate` created from `master` @ `e3b98b1`. Human sets PASS. +> Feature ID: `KIT-QUALITY-001` +> Repo: **harness-kit only** (the agent-harness half of this request is already done — see §Companion) +> Branch: proposed `feat/kit-quality-001-code-quality-gate` — **human must confirm before checkout** +> Trigger: David (2026-08-27) — "add it to harness-kit and improve the existing code-quality skill of the agent harness." + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + POSIX sh (`scripts/build-bootstrap.sh`). No application code. + +--- + +## Context (grounded) + +| Source | Finding | +| --- | --- | +| `agent-harness/skills/code-quality/SKILL.md` | Already exists with real thresholds (cyclomatic 10/15, length 40/75, params 4/6, class deps 5/8). **Advisory only** — "report only", no blocking power, no linter invocation. | +| `harness-kit/templates/REVIEW.md` §11 | "complexity" as a prose bullet under Maintainability. No thresholds, no verdict, no gate. | +| `harness-kit/templates/SPRINT_CONTRACT.md` | Six quality-gate rows: falsifiable criteria, trust boundary, partial deps, http-api, solidity-build, solidity-audit. **Nothing on code quality.** | +| `harness-kit/templates/SECURITY_CHECKLIST.md` | Already owns *Secrets & configuration* and *Error handling & data leakage*. **Direct overlap risk** with the skill's hardcoded-credentials and swallowed-errors rules. | +| `templates/AGENT_PROCESS.md` §Skill packs | "Optional packs may supply stack craft… must not replace FEATURES, contracts, VERIFY, or human PASS." This is the layering the split below follows. | + +**Gap:** nothing in the kit blocks `PENDING_REVIEW` on code quality. The executor can set +`PENDING_REVIEW` with a 300-line function and a hardcoded RPC URL and be fully process-compliant. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Layering | **Kit ships the gate; the pack ships the craft.** `templates/CODE_QUALITY.md` is a deterministic checklist + verdict (same idiom as `SECURITY_CHECKLIST.md`). Thresholds, fixes and language notes stay in the optional `agent-harness` skill. | Locked | +| Thresholds in the kit | **None.** The gate defers to the linter declared in the consumer's `AGENTS.md`. Kit stays stack-neutral, consistent with v1.3 VERIFY policy. | Locked | +| Deterministic-first | Gate §0 requires the declared lint command to be **run and recorded**. Judgement is the documented fallback, not the default. | Locked | +| Secrets ownership | **`SECURITY_CHECKLIST.md` keeps it.** The code-quality gate explicitly routes credential findings there and forbids double-reporting. | Locked | +| Blocking rule | `violation` (hard limit, no documented exception) blocks `PENDING_REVIEW`. `warn` and `pre-existing` never block. `PASS` stays human-only. | Locked | +| Profile applicability | Required for `service`, `http-api`, `solidity-build`. N/A for `ops-docs`, `product-erd`, `solidity-audit`. | Locked | +| Diff scope | "You own what you touch" — new code judged fresh, pre-existing breaches noted once as debt. | Locked | +| Exception format | In-code comment must name **rule + measured value + reason**; missing any of the three = violation. | Locked | + +--- + +## Scope — WILL do + +1. Add `templates/CODE_QUALITY.md` — promote the draft at + `skills/.harness/drafts/CODE_QUALITY.draft.md` (103 lines, already written for review). +2. `templates/SPRINT_CONTRACT.md` — add one quality-gate row: + *`service` / `http-api` / `solidity-build` only: code-quality gate run; declared lint command + recorded; violations = 0 or documented exceptions — or **N/A***. +3. `templates/AGENT_PROCESS.md` — in **§7 VERIFY**, require the code-quality gate alongside the + security checklist for the three profiles above; add one line to **§Skill packs** stating the + gate-vs-craft split so a pack cannot silently replace the gate. +4. `templates/REVIEW.md` — add a code-quality line to the §0 quality-gates walk and to the + profile-aware emphasis table; point §11 Maintainability at the gate instead of restating it. +5. `templates/AGENTS.md` — one line under *How we work here* naming the gate and its path. +6. `scripts/build-bootstrap.sh` — **Appendix N** for `CODE_QUALITY.md`; add to the Step 0 path table. +7. Bump kit version to **1.4.0** (or 1.5.0 if KIT-ERD-001 lands first — see Q2); changelog bullet. +8. Add `KIT-QUALITY-001` to root `FEATURES.json`. +9. Re-run `./scripts/build-bootstrap.sh`. + +## Scope — will NOT do (this sprint) + +- Put thresholds, linter configs, or language-specific rules in the kit. +- Duplicate the secrets rule out of `SECURITY_CHECKLIST.md`. +- Modify any consumer repo, or `agent-harness` (already done separately — see §Companion). +- Add CI wiring or a linter to any repo. +- Reopen the existing six quality gates, the status model, or the session loop. + +## Target + +- **Repo / package:** `harness-kit`. +- **Not touching:** consumer repos; `technical-project-documentation`; application source. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master` — **[GROUNDED]** +- **Upstream reality (verified 2026-08-27):** KIT-V13-001 **already merged** to `origin/master` via **PR #14** (`e3b98b1`); kit v1.3.0 is live. Local `master` is **2 commits behind**; local `FEATURES.json` still reads `PENDING_REVIEW` (ledger drift). Branch from an updated `master` — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-quality-001-code-quality-gate` +- **Human confirmed:** **pending — agent asks before checkout.** + +--- + +## Tests first (TDD) + +**N/A** — documentation/process sprint. VERIFY uses falsifiable deterministic checks. + +- **Expected RED (today):** + - `test -f templates/CODE_QUALITY.md` → fails + - `grep -qi 'code.quality' templates/SPRINT_CONTRACT.md` → fails + - `grep -qi 'code.quality' templates/REVIEW.md` → fails + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `templates/CODE_QUALITY.md` | **NEW** — promoted from reviewed draft | [NEW] | +| `templates/SPRINT_CONTRACT.md` | One quality-gate row | [GROUNDED] | +| `templates/AGENT_PROCESS.md` | VERIFY requirement + skill-pack split line | [GROUNDED] | +| `templates/REVIEW.md` | §0 gate walk + profile emphasis; §11 points at the gate | [GROUNDED] | +| `templates/AGENTS.md` | One line under *How we work here* | [GROUNDED] | +| `scripts/build-bootstrap.sh` | Appendix N + Step 0 path table | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated; version + changelog | [GROUNDED] | +| `FEATURES.json` (root) | Add `KIT-QUALITY-001` | [GROUNDED] | + +**Reuse:** `SECURITY_CHECKLIST.md` structure (checkbox sections + verdict block) — the gate is +deliberately its sibling, not a new idiom. + +--- + +## Success criteria + +1. `templates/CODE_QUALITY.md` exists and contains a **Verdict** block with a violations-blocks rule. +2. Gate §0 requires the declared lint command to be run **and recorded**. +3. Kit neutrality: `grep -Eqi 'eslint|golangci|clippy|solhint|cyclomatic|> ?[0-9]+ lines' templates/CODE_QUALITY.md` returns **no match** for hardcoded thresholds (tool names may appear only as `` examples inside the recorded-command fence). +4. `SPRINT_CONTRACT.md` has exactly one new code-quality gate row with an explicit **N/A** path. +5. `REVIEW.md` references the gate in §0 and in the profile table; §11 no longer restates complexity rules. +6. `AGENT_PROCESS.md` §7 names the gate for `service` / `http-api` / `solidity-build` and marks the other profiles N/A. +7. The secrets rule appears in `SECURITY_CHECKLIST.md` only — `CODE_QUALITY.md` routes to it and does not restate it. +8. Bootstrap regenerates clean; `BOOTSTRAP.md` contains `CODE_QUALITY`; both `FEATURES.json` parse. +9. Diff confined to the impact map. +10. Work on the confirmed feature branch — not `master`. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — see VERIFY +- [ ] Trust-boundary failure mode — **N/A** (docs-only) +- [ ] Partial/optional deps — **N/A** +- [ ] `http-api` — **N/A** +- [ ] `solidity-build` / `solidity-audit` — **N/A** +- [ ] Code-quality gate — **N/A** (this sprint is `ops-docs`; the gate it adds does not apply to itself) + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Gate becomes a style-nit generator that slows every merge | Only `violation` blocks; `warn` and `pre-existing` explicitly cannot | +| Same finding reported twice (quality + security) | §5 boundary table + success criterion 7 | +| Kit accretes language-specific thresholds over time | Success criterion 3 is a grep gate | +| Repos with no linter treat the gate as satisfied by vibes | §0 forces an explicit "no linter configured" statement, and recurring judgement findings become a follow-up contract | +| Pre-existing debt blocks unrelated work | "You own what you touch" rule in §1 | + +## Threat model + +**SKIP** — documentation/process only. + +--- + +## Blocking questions (gates) + +**Q1 — Contract order.** `KIT-ERD-001` is also drafted and unapproved. Both touch +`AGENT_PROCESS.md`, `AGENTS.md`, `SPRINT_CONTRACT.md`, `REVIEW.md`, `build-bootstrap.sh`, +`BOOTSTRAP.md` and root `FEATURES.json` — they will conflict if branched in parallel. +- **Recommendation:** sequence them. This one (`KIT-QUALITY-001`) is smaller and lower-risk — + ship it first, then rebase `KIT-ERD-001` on top. Principle 5, "one thing at a time." + +**Q2 — Version number.** Confirmed live version on `origin/master` is **1.3.0**, so the next is 1.4.0. +- **Recommendation:** this sprint takes **1.4.0** under the Q1 ordering; KIT-ERD-001 takes **1.5.0**. + +**Q3 — Enforcement strength when no linter is configured.** Today most consumer repos will have +partial linting. Should the gate hard-block on judgement-only violations, or only report them? +- **Recommendation:** judgement-only findings **warn**, never block; a repeat offender becomes a + contract to configure the rule. Blocking on model opinion invites arguing with the harness. + +**Q4 — Should the gate apply to `product-erd` (pending KIT-ERD-001)?** +- **Recommendation:** no — N/A, as drafted. An ERD ships no code. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# gate template exists and blocks correctly +test -f templates/CODE_QUALITY.md +grep -qi 'verdict' templates/CODE_QUALITY.md +grep -qi 'violations = 0' templates/CODE_QUALITY.md +grep -qi 'PASS remains human-only\|PASS. remains human-only' templates/CODE_QUALITY.md + +# deterministic-first +grep -qi 'AGENTS.md' templates/CODE_QUALITY.md +grep -qi 'judgement, not enforced config' templates/CODE_QUALITY.md + +# wired into contract, process, review, agents +grep -qi 'code.quality' templates/SPRINT_CONTRACT.md +grep -qi 'code.quality' templates/AGENT_PROCESS.md +grep -qi 'code.quality' templates/REVIEW.md +grep -qi 'code.quality' templates/AGENTS.md + +# no double ownership of secrets +grep -qi 'SECURITY_CHECKLIST' templates/CODE_QUALITY.md +! grep -qi 'zero tolerance' templates/CODE_QUALITY.md + +# version + bootstrap +grep -q '1.4.0' BOOTSTRAP.md +grep -q '1.4.0' scripts/build-bootstrap.sh +./scripts/build-bootstrap.sh +grep -q 'CODE_QUALITY' BOOTSTRAP.md + +# json + feature entry +python3 -m json.tool FEATURES.json >/dev/null +python3 -m json.tool templates/FEATURES.json >/dev/null +grep -q 'KIT-QUALITY-001' FEATURES.json + +# branch discipline +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-QUALITY-001", + "name": "Code-quality gate: deterministic lint-first checklist blocking PENDING_REVIEW on violations", + "priority": 1, + "verify": "test -f templates/CODE_QUALITY.md && grep -qi 'code.quality' templates/SPRINT_CONTRACT.md && grep -qi 'code.quality' templates/REVIEW.md && grep -q '1.4.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-QUALITY-001.md; profile: ops-docs; sequence before KIT-ERD-001 (shared file conflicts)" +} +``` + +--- + +## Companion change (already applied, outside this contract) + +`agent-harness/skills/code-quality/SKILL.md` was improved in place (203 lines, was ~120). +`agent-harness` has no contract tree — its own `AGENTS.md` governs skills via `skills/` + +the skills table, and forbids commits/branches without being asked. Nothing was committed. + +Changes: linter-first sensor section; diff-scope and "you own what you touch"; exempt paths; +test-code standards; cognitive-complexity row; measurement definitions; secrets rule handed to +the security checklist; documented-exception format; verdict → merge-recommendation mapping; +per-language notes (TS/Go/Rust/Solidity). + +--- + +## Criteria block + +```criteria +- [KIT-QUALITY-001-1] templates/CODE_QUALITY.md exists with verdict + violations-block rule +- [KIT-QUALITY-001-2] Gate requires declared lint command run and recorded +- [KIT-QUALITY-001-3] No hardcoded thresholds or tool mandates in the kit template +- [KIT-QUALITY-001-4] SPRINT_CONTRACT has one code-quality gate row with N/A path +- [KIT-QUALITY-001-5] REVIEW and AGENT_PROCESS reference the gate; REVIEW s11 no longer restates rules +- [KIT-QUALITY-001-6] Secrets owned solely by SECURITY_CHECKLIST; no double-reporting +- [KIT-QUALITY-001-7] Version bumped; Appendix N; bootstrap green; FEATURES.json valid +``` + +--- + +## Deviations from the approved contract (declare, do not hide) + +| # | Contract said | Actually done | Why | Verdict | +|---|---------------|---------------|-----|---------| +| 1 | Scope §G6: "**Appendix N** for `CODE_QUALITY.md`" | **Appendix K** | The letter was written assuming KIT-ERD-001 (K/L/M) landed first. Q1 sequenced this contract first, so existing appendices end at J and K is the correct next letter. | Trivial — contract text was wrong, output is right | +| 2 | Scope WILL did not include it | `KIT-V13-001` reconciled `PENDING_REVIEW` → `PASS` in root `FEATURES.json` | It shipped in PR #14; the ledger disagreed with reality, which breaks BOOT step 3 ("highest-priority `FAIL`"). Flagged in the Branch section as ledger drift but never added to Scope WILL. | **Undeclared scope creep** — small and corrective; human to accept or revert | + +Environment note: the folder bridge initially denied `unlink`, so a `git checkout` applied +partially and left HEAD stale at `7da7be4` with a working tree already matching `origin/master`. +No content was lost (verified: zero tracked-file diff vs `origin/master`). Repaired with +`git reset --hard origin/master` after delete permission was granted. + +## VERIFY result + +All 20 checks green, including the criterion-3 neutrality checks that the original VERIFY block +did not implement (`! grep -Eqi '(>|>=) *[0-9]+'` and no threshold vocabulary). Tool names appear +in exactly one line, inside the recorded-command fence, as `` examples. diff --git a/skills/.harness/contracts/KIT-RULES-001.md b/skills/.harness/contracts/KIT-RULES-001.md new file mode 100644 index 0000000..66eba71 --- /dev/null +++ b/skills/.harness/contracts/KIT-RULES-001.md @@ -0,0 +1,311 @@ +# SPRINT CONTRACT — KIT-RULES-001: `rules/` directory (Cursor-first) + ERD routing in the loop + +> Status: **EXECUTE complete — VERIFY green — PENDING_REVIEW.** +> Approved: David 2026-08-28 ("approve") with recommended Q1–Q4. Branch confirmed. Version **2.2.0**. +> Feature ID: `KIT-RULES-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-rules-001-rules-and-erd-routing` — **human must confirm before checkout** +> Base: `master` @ `ef393c3` (v2.1). Version → **2.2.0**. +> Trigger: David (2026-08-28) — rules as their own directory, Cursor-first, guided by +> `cursor-rules-export/`; and "expand our workflow to go through an ERD process before contract drafting." + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + `.mdc` + POSIX sh. No application code. + +--- + +## Context — what the export actually shows (grounded, `cursor-rules-export/`, 2026-08-28) + +| Finding | Evidence | Consequence | +| --- | --- | --- | +| **The kit's process law is already duplicated by hand as Cursor rules, and has drifted** | `harness-process.mdc` exists in `advanced-orders-backend`, `intents-service`, `tao.com-api`, `tao-docs` — **four different checksums** (`7f57ed1b`, `cbf16f1a`, `ad72ef13`, `9805df00`) | The strongest argument for the kit owning these canonically and symlinking them. Four hand-maintained copies is exactly the drift the submodule model exists to kill | +| **A second, older copy of the process exists outside the kit** | `global/user-rules-harness-engineering.md` and rules citing `~/.cursor/skills/harness-engineering/HARNESS.md` run a **5-step loop** (`BOOT → CONTRACT → APPROVE → EXECUTE → VERIFY → PENDING_REVIEW → PASS`) with `.harness/contracts/` — no RESEARCH, no CHECKOUT, no REVIEW step, no profiles, no gates | The process is **forked**. Rules point at the global skill as authoritative while harness-kit is what actually evolved. Reconciliation is **out of scope here** but must be tracked — see Q3 | +| **Always-on budget is heavy** | 18 of 27 rules are `alwaysApply: true` | Dilution risk. The kit must ship **few** rules and say so | +| **Glob scoping is available but unused** | `"globs": ""` on **every** rule in `manifest.json` | The main technical advantage of `.mdc` over assembled text is currently unrealised. Kit rules should use it where a rule is genuinely file-scoped, and default `alwaysApply: true` otherwise | +| **Four distinct ownership tiers are visible** | see below | Only tier 1 belongs in the kit | + +**Ownership tiers observed in the export:** + +1. **Process law** — `sprint-contract-before-changes`, `new-work-new-branch`, `no-stage-harness-files`, `stage-after-execute`, `harness-process.mdc` ×4, parts of `harness-contract-quality`, `principal-pr-review`. **Kit-owned.** +2. **Engineering discipline, generalisable** — `verify-claims-with-evidence`, `harness-dry-reusability`. Kit-owned where it is not already a skill's job. +3. **Org / product specific** — `intents-backend-url`, `new-relic-intents-log-queries`, `tao-intents-ownership-and-pps`, `postman-ticket-sync`, `documentation-agents`. **Consumer-owned, never the kit's.** +4. **Personal style / comms** — `no-em-dashes`, `pr-desc-brief-bullets`, `pr-replies-relatable-names`, `pr-replies-no-reflexive-agreement`, `pr-review-comments-brief`. **Cursor User Rules, not a repo concern at all.** + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Form | `rules/` directory in the kit, authored as **`.mdc`** with Cursor frontmatter | Locked | +| Primary target | **Cursor** — `.mdc` is the source of truth, symlinked into `.cursor/rules/` | Locked (David) | +| Portability | Setup script also **assembles** rule bodies (frontmatter stripped) into a marked block in the consumer's `AGENTS.md`, so non-Cursor agents still get them | Locked | +| What the kit ships | **Process law only** — tier 1, plus `verify-claims-with-evidence` from tier 2 | Locked | +| What the kit never ships | Org/product rules (tier 3) and personal style rules (tier 4) | Locked | +| Rule count | **Six.** Few enough that every always-on line keeps weight | Locked | +| Consumer rules | `.agents/local-rules/*.mdc`, merged alongside, local wins on collision — mirrors local-skills | Locked | +| Scoping | `alwaysApply: true` by default; `globs` where a rule is genuinely file-scoped | Locked | +| Relocation discipline | Process law moved out of `AGENT_PROCESS.md` as a **pure move** where text is unchanged; rewrites are a separate commit | Locked | +| ERD routing | The loop gains a conditional **ERD?** step before CONTRACT — see §C | Locked | + +--- + +## Scope — WILL do + +### A — `rules/` directory (six rules) + +1. Create `rules/.mdc`, each with `description`, `alwaysApply`, and `globs` frontmatter: + +| Rule | Source in export | Always-on | +|------|------------------|-----------| +| `contract-before-changes.mdc` | `sprint-contract-before-changes` | yes | +| `new-work-new-branch.mdc` | `new-work-new-branch` | yes | +| `status-ownership.mdc` | global harness rules + kit law: executor may set `PENDING_REVIEW`, **never** `PASS`; no push / merge / deploy unasked | yes | +| `no-stage-harness-files.mdc` | `no-stage-harness-files` + `stage-after-execute` (merged — they are one staging policy) | yes | +| `verify-claims-with-evidence.mdc` | `verify-claims-with-evidence` | yes | +| `observed-content-is-data.mdc` | `AGENT_PROCESS` § Security (prompt-injection defence, least privilege) | yes | + +2. Generalise every rule out of tao.com specifics: no repo names, no `bun`/Postman/New Relic, no + `~/.cursor/skills/...` paths. Paths reference the kit's own contracts location. +3. Add `rules/README.md` stating the ownership tiers above and the **budget rule**: the kit ships six; + a rule that is not load-bearing should be deleted or promoted to a gate. + +### B — Wiring + +4. `scripts/setup-harness-kit.sh` — symlink `rules/*.mdc` into `.cursor/rules/`, merge + `.agents/local-rules/*.mdc` alongside (local wins), and **assemble** rule bodies into + `AGENTS.md` between `` / ``, + stripping frontmatter. Idempotent; marker-guarded; never touches content outside the markers. +5. Same guards as the skills merge: refuse a `.cursor/rules` directory it did not generate; leave a + non-symlink one alone with a warning. +6. `README.md` — rules section, ownership tiers, upgrade path. + +### C — ERD routing in the loop + +7. Session loop becomes: + +``` +BOOT → ERD? → RESEARCH? → CONTRACT → [HUMAN APPROVES + BRANCH] → CHECKOUT → EXECUTE → VERIFY → REVIEW → PENDING_REVIEW → [HUMAN PASS/MERGE] +``` + +8. **ERD? is a routing test, not a mandatory stage.** Before drafting a contract, ask: *is there an + approved design for this work?* Route to the `erd-authoring` skill (profile `product-erd`) when + **all** hold: it is a new product, service, domain, or a contract crossing teams or repos; **and** + no ERD covers it; **and** the work is larger than a single slice. Otherwise go straight to CONTRACT. +9. State plainly that a feature inside an already-documented service **skips ERD**, and that ERD is + not RESEARCH: RESEARCH distills questions for one contract, the ERD stage produces a document + feeding many. Never dual-run. +10. When an ERD exists, a contract **cites the slice it implements** and inherits its acceptance + criteria rather than reinventing them. Add one line to the sprint-contract form for the slice reference. +11. Depth matrix gains an ERD column so the routing test is visible where depth is already chosen. + +### D — Version + ledger + +12. Version **2.2.0**; changelog. Add `KIT-RULES-001` to `FEATURES.json`. +13. `scripts/build-bootstrap.sh` — Appendix Q for `rules/README.md` and the rule bodies, so the + deprecated bootstrap path stays complete until `KIT-SKILLS-002` removes it. + +## Scope — will NOT do (this sprint) + +- Ship org/product rules (tier 3) or personal style rules (tier 4). +- Reconcile the forked global harness (`~/.cursor/skills/harness-engineering/`) — see **Q3**. +- Migrate any consumer repo's existing `.cursor/rules/`, or delete the drifted `harness-process.mdc` copies. +- Make the ERD stage mandatory for any work. +- Build `verify-harness.sh` / CI — that is `KIT-GATES-001`, and it depends on rules existing first. +- Change any skill's behaviour. + +## Target + +- **Repo / package:** `harness-kit`. +- **Not touching:** consumer repos; the global Cursor skill; application source. + +--- + +## Branch + +- **Default branch:** `master`; base `ef393c3` (v2.1) — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-rules-001-rules-and-erd-routing` +- **Also carries:** `271b222` (ledger PASS for KIT-SKILLS-001 / KIT-ERD-001) if folded in — see Q4 +- **Human confirmed:** **pending — agent asks before checkout.** + +## Tests first (TDD) + +**N/A** — process/docs. Expected RED today: `test -d rules` fails; `grep -q 'ERD?' ` fails. + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `rules/*.mdc` (6) + `rules/README.md` | **NEW** | [NEW] | +| `scripts/setup-harness-kit.sh` | Rules symlink + local-rules merge + AGENTS.md assembly | [GROUNDED] | +| `skills/harness-onboard/templates/AGENT_PROCESS.md` | ERD? step, routing test, depth-matrix column; process law removed where it moves to `rules/` | [GROUNDED] | +| `skills/harness-onboard/templates/AGENTS.md` | Rules marker block; loop line | [GROUNDED] | +| `skills/sprint-contract/templates/SPRINT_CONTRACT.md` | ERD slice reference line | [GROUNDED] | +| `skills/erd-authoring/SKILL.md` | Cross-reference to the routing test | [GROUNDED] | +| `README.md`, `scripts/build-bootstrap.sh`, `BOOTSTRAP.md`, `FEATURES.json` | Rules section, Appendix Q, 2.2.0, ledger | [GROUNDED] | + +--- + +## Success criteria + +1. `rules/` contains exactly **6** `.mdc` files plus `README.md`; every file has `description`, `alwaysApply`, `globs` frontmatter. +2. No kit rule names a vendor, product or repo: `grep -Eqi 'bun|postman|new relic|intents|tao\.com|solhint|eslint' rules/*.mdc` returns **no match**. +3. `rules/README.md` states the four ownership tiers and the six-rule budget. +4. Setup script symlinks rules, merges `.agents/local-rules/`, and assembles a marked block into `AGENTS.md`; idempotent on a second run; refuses a foreign `.cursor/rules`. +5. Assembly strips frontmatter — no `alwaysApply:` inside the `AGENTS.md` block. +6. Loop in the process doc reads `BOOT → ERD? → RESEARCH? → CONTRACT → …`. +7. The ERD routing test states all three conditions **and** the skip rule, and says ERD and RESEARCH must never dual-run. +8. Sprint contract form has a slice-reference line for ERD-derived work. +9. Process law that moved from `AGENT_PROCESS.md` into `rules/` is a **pure move** — content diff empty in its own commit. +10. Version 2.2.0; bootstrap regenerates; both `FEATURES.json` parse; diff confined to the impact map; work on the confirmed branch. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — see VERIFY +- [ ] Trust-boundary / partial deps / http-api / solidity — **N/A** (docs) +- [ ] Code-quality gate — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor — criterion 9 is its analogue (empty content diff on the move commit) + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Kit rules collide with a repo's existing `.cursor/rules` of the same name | Local wins on collision; setup refuses a merge dir it did not generate | +| Always-on budget creeps back to 18 | Six-rule cap stated in `rules/README.md`; criterion 1 enforces the count | +| Rules and skills say the same thing twice | Rules carry invariants only; anything procedural stays in the skill that owns it | +| ERD? becomes mandatory ceremony | Routing test requires **all three** conditions; skip rule stated explicitly | +| Assembly block clobbers hand-written `AGENTS.md` | Marker-delimited, regenerated only between markers | +| Forked global harness keeps diverging | Out of scope, tracked as **Q3** | + +## Threat model + +**SKIP** — process/docs only. + +--- + +## Blocking questions (gates) + +**Q1 — One contract or two?** This covers a rules directory **and** an ERD routing change. +- **Recommendation:** one. Both edit the same process doc, so splitting means one rebases painfully + onto the other for no review benefit. Scope halves are clearly separated above and can be reviewed + independently. Split if you would rather merge them separately. + +**Q2 — Does `stage-after-execute` belong in the kit?** It says stage product files automatically once +VERIFY is green. That is a workflow opinion, not an invariant, and it interacts with "never commit unasked". +- **Recommendation:** merge it into `no-stage-harness-files` as one staging policy (what to stage, what + never to stage), rather than shipping an auto-stage instruction as kit law. + +**Q3 — The forked global harness.** `~/.cursor/skills/harness-engineering/` runs a 5-step loop with +`.harness/contracts/`, and several repo rules cite it as authoritative. harness-kit runs a 9-step loop +with profiles and gates. They disagree, and rules currently point at the older one. +- **Recommendation:** out of scope here; track as `KIT-RECONCILE-001`. Once kit rules ship, the global + skill should either be retired or reduced to a pointer at the kit. Leaving both authoritative + guarantees drift. + +**Q4 — Fold in the ledger commit?** `271b222` (PASS for KIT-SKILLS-001 / KIT-ERD-001) is unpushed on +`chore/kit-ledger-pass`. +- **Recommendation:** fold it in rather than spend a PR on two lines. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# rules exist, are frontmattered, and are exactly six +test "$(ls rules/*.mdc | wc -l)" -eq 6 +for f in rules/*.mdc; do + head -1 "$f" | grep -q '^---$' && grep -q '^description:' "$f" && grep -q '^alwaysApply:' "$f" || exit 1 +done +test -f rules/README.md +grep -qi 'ownership' rules/README.md + +# kit neutrality — no vendor, product or repo names in kit rules +! grep -Eqi 'bun|postman|new relic|intents|tao\.com|solhint|eslint' rules/*.mdc + +# setup wiring +grep -q 'cursor/rules' scripts/setup-harness-kit.sh +grep -q 'local-rules' scripts/setup-harness-kit.sh +grep -q 'harness-kit:rules:start' scripts/setup-harness-kit.sh +sh -n scripts/setup-harness-kit.sh + +# ERD routing in the loop +grep -q 'ERD?' skills/harness-onboard/templates/AGENT_PROCESS.md +grep -qi 'never dual-run' skills/harness-onboard/templates/AGENT_PROCESS.md +grep -qi 'slice' skills/sprint-contract/templates/SPRINT_CONTRACT.md + +# version + ledger +grep -q '2.2.0' scripts/build-bootstrap.sh && grep -q '2.2.0' BOOTSTRAP.md +sh scripts/build-bootstrap.sh +python3 -m json.tool FEATURES.json >/dev/null && grep -q 'KIT-RULES-001' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-RULES-001", + "name": "rules/ directory as Cursor .mdc with AGENTS.md assembly, plus ERD routing before contract drafting", + "priority": 1, + "verify": "test $(ls rules/*.mdc | wc -l) -eq 6 && test -f rules/README.md && grep -q 'harness-kit:rules:start' scripts/setup-harness-kit.sh && grep -q 'ERD?' skills/harness-onboard/templates/AGENT_PROCESS.md && grep -q '2.2.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-RULES-001.md; profile: ops-docs; guided by cursor-rules-export 2026-08-28; blocks KIT-GATES-001" +} +``` + +## Criteria block + +```criteria +- [KIT-RULES-001-1] rules/ has exactly 6 .mdc files with valid frontmatter, plus README +- [KIT-RULES-001-2] No vendor/product/repo names in any kit rule +- [KIT-RULES-001-3] rules/README states the four ownership tiers and the six-rule budget +- [KIT-RULES-001-4] Setup symlinks rules, merges local-rules, assembles marked AGENTS.md block, idempotent +- [KIT-RULES-001-5] Assembly strips frontmatter +- [KIT-RULES-001-6] Loop reads BOOT -> ERD? -> RESEARCH? -> CONTRACT +- [KIT-RULES-001-7] ERD routing test has all three conditions, the skip rule, and no dual-run with RESEARCH +- [KIT-RULES-001-8] Process law moved out of AGENT_PROCESS as a pure move (empty content diff) +``` + +--- + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | Criterion 9 expected process law **moved** out of `AGENT_PROCESS.md` with an empty content diff | **N/A — nothing moved.** The six rules were authored from the Cursor export and generalised; they are new content, not relocated text. `AGENT_PROCESS.md`'s 7 principles are the *reasoning*, the rules are the *enforceable form*, and they are not verbatim duplicates. A pointer was added instead. Deleting the principles would have been destructive surgery the contract did not scope. | +| 2 | Loop notation — David asked whether research precedes the ERD | Addressed beyond the contract's §C: step 3 now carries a table distinguishing **discovery research** (inside `erd-authoring` S0) from **slice research** (the loop's `RESEARCH?`), so `ERD? → RESEARCH?` no longer implies deciding before looking. Also cross-referenced from the skill. | +| 3 | Loop steps renumbered 1–9 | Inserting ERD? at position 2 shifted CONTRACT through REVIEW. Mechanical. | +| 4 | A VERIFY check failed on a line-wrap artifact ("skips / this") | Reflowed the sentence rather than loosening the check. | + +## Q2 as executed + +`stage-after-execute` was **merged into** `no-stage-harness-files` as one staging policy, per the +recommendation — what to stage, what never to stage, and the standing "do not commit unless asked". + +## Setup script — tested, not asserted + +Exercised end to end in a throwaway consumer repo: 6 rules linked · symlinks resolve · assembly writes +a marked block into `AGENTS.md` with **zero** `alwaysApply` leaked · hand-written `AGENTS.md` content +preserved · **second run changed nothing** and left exactly one block · local rules win on collision · +a foreign `.cursor/rules` is left alone with a warning. + +## VERIFY result + +All checks green. + +## Also in this branch + +`271b222` — PASS for KIT-SKILLS-001 and KIT-ERD-001, folded in per Q4. diff --git a/skills/.harness/contracts/KIT-SKILLS-001.md b/skills/.harness/contracts/KIT-SKILLS-001.md new file mode 100644 index 0000000..b2c8fd6 --- /dev/null +++ b/skills/.harness/contracts/KIT-SKILLS-001.md @@ -0,0 +1,323 @@ +# SPRINT CONTRACT — KIT-SKILLS-001: templates → skills, submodule adoption + +> Status: **EXECUTE complete — VERIFY green — PENDING_REVIEW.** +> Approved: David 2026-08-27 ("proceed") with recommended Q1–Q4. Branch confirmed. +> Commits: `d503ac1` (pure rename) + `008248d` (skills, setup, adoption). Version **2.0.0**. +> Feature ID: `KIT-SKILLS-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-skills-001-templates-to-skills` — **human must confirm before checkout** +> Trigger: David (2026-08-27) — "we need to turn templates into skills and then proceed with kit-erd-001." +> **This is a breaking change to the adoption model → version 2.0.0.** + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + POSIX sh. No application code. + +--- + +## Context (grounded, verified on `master` @ `3e22a12`) + +| Fact | Consequence | +| --- | --- | +| 13 files in `templates/`; no `SKILL.md` anywhere in the kit | Nothing to migrate from; this is a first conversion | +| `BOOTSTRAP.md` is 1947 lines, sold as "copy this file into **any** repo" | Retiring it removes the kit's entire adoption story — must be replaced, not just deleted | +| Appendices B–M are written to `skills/.harness/templates/*` | That path is **gitignored** (`.gitignore:20`) — today's templates are disposable local copies | +| Skills are discovered by directory + `SKILL.md` frontmatter | They must be **reachable in the tree**, which the gitignored path is not | +| `scripts/build-bootstrap.sh` regenerates `BOOTSTRAP.md` from `templates/*` | Becomes obsolete once appendices stop being the distribution vector | +| Kit currently has **no** setup script | Submodule adoption needs one (symlink wiring, brownfield safety) | + +**Decisions taken by David (2026-08-27):** procedures become skills and forms ship bundled inside them; +skills are tracked, kit-owned; **adoption moves from single-file BOOTSTRAP to git submodule**; +skills live at `skills//SKILL.md`. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| What becomes a skill | **Procedures only.** Fill-in forms ship as `templates/` **inside** the owning skill | Locked | +| Skill count | **7** (see §B) — not 13; a blank form with frontmatter is indirection without behaviour | Locked | +| Adoption | **Git submodule + setup script.** Skills stay canonical in harness-kit; consumers symlink | Locked | +| Per-repo copies | **None.** Symlinks, not copies — so kit upgrades are a submodule pointer bump and drift is impossible | Locked | +| Location | `skills//SKILL.md`, siblings of the existing `skills/.harness/` working directory | Locked | +| `skills/.harness/` | **Unchanged** — contracts, STATE, local templates stay gitignored working state | Locked | +| `BOOTSTRAP.md` | **Deprecated, not deleted, this sprint.** See Q2 | Locked | +| Version | **2.0.0** — breaking change to how repos adopt the kit | Locked | +| Tool coupling | Accepted. `SKILL.md` frontmatter is a Claude/Cursor/skills-CLI convention; `AGENTS.md` remains the tool-agnostic entrypoint so a non-skill agent still works | Locked | + +--- + +## Scope — WILL do + +### A — Skill format + +1. Each skill: `skills//SKILL.md` with YAML frontmatter (`name`, `description` — third person, trigger + terms, ≤1024 chars), and bundled `skills//templates/*.md` for its forms. +2. Each generated/kit-owned skill carries a "kit-owned — edit in harness-kit, not here" marker. + +### B — The seven skills (procedures), with their bundled forms + +| Skill | Procedure | Bundled forms | +|-------|-----------|---------------| +| `harness-onboard` | Bootstrap a repo into the process | `AGENTS.md`, `AGENT_PROCESS.md`, `FEATURES.json`, `STATE.md`, `README_POINTER.md` | +| `sprint-contract` | Write a contract; get approval; confirm branch | `SPRINT_CONTRACT.md` | +| `code-quality-gate` | Run the per-diff gate at VERIFY | `CODE_QUALITY.md` | +| `code-audit` | Run a periodic subsystem audit | `CODE_AUDIT_CONTRACT.md`, `CODE_AUDIT_FINDING.md` | +| `solidity-audit` | Run a contract security audit | `AUDIT_CONTRACT.md`, `AUDIT_FINDING.md` | +| `security-checklist` | Pre-merge security self-check | `SECURITY_CHECKLIST.md` | +| `harness-review` | PR review, §0 contract compliance first | `REVIEW.md` | + +3. Content is **moved, not rewritten**. Each `SKILL.md` adds only frontmatter, a short "when to use / + when not to", and a pointer to its bundled form. No process changes ride along in this sprint. + +### C — Submodule adoption + +4. Add `scripts/setup-harness-kit.sh` — run from the **consumer repo root**, detects the git superproject, and: + - verifies the submodule is present at a known path + - creates a merge directory of per-skill symlinks into the submodule + - wires the tool paths that exist in that repo (`.claude/skills`, `.cursor/skills`, `.agents/skills`) + - refuses to overwrite anything it did not generate (marker file), and preserves unrelated entries + - is idempotent — running twice changes nothing +5. `README.md` — replace the BOOTSTRAP adoption instructions with submodule adoption + (`git submodule add`, `git submodule update --init`, run setup, re-run after a pointer bump). +6. Document the **upgrade path**: bump the submodule pointer, re-run setup. No per-repo file edits. + +### D — Path references + +7. Rewrite every `skills/.harness/templates/.md` reference in `templates/AGENT_PROCESS.md`, + `templates/AGENTS.md`, `templates/REVIEW.md`, `templates/SPRINT_CONTRACT.md`, + `templates/CODE_QUALITY.md`, `templates/CODE_AUDIT_CONTRACT.md` to the skill path. +8. Keep `skills/.harness/contracts/` and `skills/.harness/STATE.md` exactly as they are — working state, + still gitignored, unaffected by this change. + +### E — Deprecate BOOTSTRAP + +9. `BOOTSTRAP.md` gains a deprecation banner naming the submodule path and the removal target + (`KIT-SKILLS-002`). **Not deleted this sprint** — repos bootstrapped from it still reference it. +10. `scripts/build-bootstrap.sh` gains the same banner and stays runnable until removal. + +### F — Version + ledger + +11. Version **2.0.0** across `README.md`, the setup script header, and the deprecation banners; changelog + entry naming the breaking change and the migration. +12. Add `KIT-SKILLS-001` to root `FEATURES.json`. + +## Scope — will NOT do (this sprint) + +- Delete `BOOTSTRAP.md` or `build-bootstrap.sh` → `KIT-SKILLS-002`. +- Change any process rule, gate, profile, threshold or checklist content. **Pure relocation.** +- Migrate any consumer repo (including `technical-project-documentation`, which uses `.harness/contracts/`). +- Build `KIT-ERD-001` — it follows, in the new skill shape. +- Adopt, reference or absorb any other pack repo; the kit stays pack-agnostic. +- Add per-repo editable copies of skills. + +## Target + +- **Repo / package:** `harness-kit`. +- **Not touching:** every consumer repo; any pack repo; application source. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master`; base `master` @ `3e22a12` (v1.5.0, PR #16 merged) — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-skills-001-templates-to-skills` +- **Human confirmed:** **pending — agent asks before checkout.** + +--- + +## Tests first (TDD) + +**N/A** — documentation/process sprint. VERIFY uses falsifiable deterministic checks. + +- **Expected RED (today):** + - `test -f skills/code-quality-gate/SKILL.md` → fails + - `test -f scripts/setup-harness-kit.sh` → fails + - `grep -q '2.0.0' README.md` → fails + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `skills/<7 names>/SKILL.md` | **NEW** — 7 procedure skills | [NEW] | +| `skills//templates/*.md` | **MOVED** from `templates/` (git mv; content unchanged) | [GROUNDED] | +| `scripts/setup-harness-kit.sh` | **NEW** — submodule wiring, idempotent | [NEW] | +| `README.md` | Submodule adoption replaces BOOTSTRAP instructions; 2.0.0 | [GROUNDED] | +| `templates/AGENT_PROCESS.md` → skill | Path references rewritten | [GROUNDED] | +| `templates/AGENTS.md` → skill | Path references rewritten | [GROUNDED] | +| `BOOTSTRAP.md`, `scripts/build-bootstrap.sh` | Deprecation banners only | [GROUNDED] | +| `FEATURES.json` | Add `KIT-SKILLS-001` | [GROUNDED] | + +**Reuse:** the submodule + symlink + merge-directory + brownfield-safety pattern is already proven in a +sibling repo's setup script. Adapt the approach; **do not** reference or depend on that repo. + +--- + +## Success criteria + +1. Seven `skills//SKILL.md` exist, each with valid frontmatter (`name`, `description` ≤1024 chars). +2. Every former `templates/*.md` file is reachable inside exactly one skill; none orphaned, none duplicated. +3. `git log --follow` shows the forms were **moved**, not rewritten — content diff is empty for each. +4. `scripts/setup-harness-kit.sh` is executable POSIX sh, idempotent (two runs → no diff), and refuses to + overwrite files it did not generate. +5. No reference anywhere to `skills/.harness/templates/` remains outside the deprecated BOOTSTRAP files. +6. `skills/.harness/contracts/` and `STATE.md` still gitignored and untouched. +7. `README.md` documents submodule adoption and the pointer-bump upgrade path; version 2.0.0. +8. `BOOTSTRAP.md` and `build-bootstrap.sh` carry deprecation banners and still run. +9. Root `FEATURES.json` valid and contains `KIT-SKILLS-001`. +10. No process rule changed: `git diff` on moved content shows relocation only. +11. Diff confined to the impact map; work on the confirmed feature branch. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — see VERIFY +- [ ] Trust-boundary / partial deps / http-api / solidity — **N/A** (docs-only) +- [ ] Code-quality gate — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor gate — **the content-move criterion (3) is its analogue**: the "behaviour" preserved here is the process text, proven by an empty content diff across the move + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Process text silently edited during the move | Criterion 3: empty content diff per moved file, enforced in VERIFY | +| Repos bootstrapped from BOOTSTRAP break | Deprecated, not deleted; removal deferred to KIT-SKILLS-002 | +| Setup script clobbers hand-authored skills | Generated-marker rule; refuses unknown content (criterion 4) | +| Symlinks break on Windows | Setup is re-runnable; documented as the fix | +| Kit becomes coupled to one agent vendor | `AGENTS.md` stays the tool-agnostic entrypoint; skills are additive | +| Two setup scripts drift across repos | Flagged as **Q3** — not silently duplicated | + +## Threat model + +**SKIP** — documentation/process only. Setup script performs no network or privileged operations. + +--- + +## Blocking questions (gates) + +**Q1 — Skill names.** Proposed: `harness-onboard`, `sprint-contract`, `code-quality-gate`, `code-audit`, +`solidity-audit`, `security-checklist`, `harness-review`. +- **Recommendation:** as listed. `harness-` prefixes only where the bare noun would collide with a + common existing skill name (`review`, `onboard`). + +**Q2 — BOOTSTRAP removal timing.** Deprecate now and delete in `KIT-SKILLS-002`, or delete now? +- **Recommendation:** deprecate now. Deleting the only adoption path in the same sprint that introduces + its replacement leaves no rollback. "Build to delete" — after the replacement is proven, not before. + +**Q3 — Setup-script duplication.** A sibling pack repo already has a near-identical submodule + symlink +setup script. Two scripts solving the same problem will drift. +- **Recommendation:** accept the duplication for now and record it as known debt. Sharing it would couple + the kit to a pack, which contradicts the pack-agnostic rule. Revisit only if both scripts churn. + +**Q4 — Consumer migration.** Repos already bootstrapped from `BOOTSTRAP.md` (e.g. the docs repo, which +uses `.harness/contracts/`) are unaffected today but will need migrating eventually. +- **Recommendation:** out of scope here; write a migration note in `README.md` and track it as a separate + contract per repo. Do not migrate anyone's repo from inside a kit sprint. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# 1. seven skills with frontmatter +for s in harness-onboard sprint-contract code-quality-gate code-audit solidity-audit security-checklist harness-review; do + test -f "skills/$s/SKILL.md" || { echo "missing $s"; exit 1; } + head -1 "skills/$s/SKILL.md" | grep -q '^---$' || { echo "$s: no frontmatter"; exit 1; } + grep -q '^name:' "skills/$s/SKILL.md" && grep -q '^description:' "skills/$s/SKILL.md" || exit 1 +done + +# 2. every former template lands in exactly one skill, none left behind +test -z "$(ls templates/*.md 2>/dev/null)" || echo "templates/ not emptied" +test "$(find skills -path 'skills/.harness' -prune -o -name '*.md' -path '*/templates/*' -print | wc -l)" -eq 13 + +# 3. moves, not rewrites +git diff --cached -M --stat | grep -q '=>' || echo "expected renames" +git diff --cached -M --diff-filter=R --numstat | awk '$1!=0||$2!=0{print "CONTENT CHANGED: "$3; bad=1} END{exit bad+0}' + +# 4. setup script idempotent + safe +test -x scripts/setup-harness-kit.sh +sh -n scripts/setup-harness-kit.sh + +# 5. no stale template paths outside deprecated files +! grep -rn 'skills/.harness/templates/' --include='*.md' . \ + | grep -v '^./BOOTSTRAP.md' | grep -v '^./skills/.harness/' | grep -q . + +# 6. working state untouched +git check-ignore -q skills/.harness/contracts && echo "contracts still ignored" + +# 7/8. adoption + deprecation +grep -q 'submodule' README.md && grep -q '2.0.0' README.md +grep -qi 'deprecat' BOOTSTRAP.md && grep -qi 'deprecat' scripts/build-bootstrap.sh +sh scripts/build-bootstrap.sh >/dev/null + +# 9. ledger +python3 -m json.tool FEATURES.json >/dev/null && grep -q 'KIT-SKILLS-001' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-SKILLS-001", + "name": "Templates become skills; adoption moves from single-file BOOTSTRAP to git submodule", + "priority": 1, + "verify": "test -f skills/code-quality-gate/SKILL.md && test -f skills/code-audit/SKILL.md && test -x scripts/setup-harness-kit.sh && grep -q '2.0.0' README.md && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-SKILLS-001.md; profile: ops-docs; BREAKING (adoption model); blocks KIT-ERD-001" +} +``` + +--- + +## Criteria block + +```criteria +- [KIT-SKILLS-001-1] Seven procedure skills exist with valid frontmatter +- [KIT-SKILLS-001-2] All 13 forms bundled inside exactly one skill; templates/ emptied +- [KIT-SKILLS-001-3] Forms moved not rewritten — zero content diff across renames +- [KIT-SKILLS-001-4] setup-harness-kit.sh is POSIX, idempotent, refuses foreign content +- [KIT-SKILLS-001-5] No stale skills/.harness/templates/ references outside deprecated files +- [KIT-SKILLS-001-6] skills/.harness working state untouched and still gitignored +- [KIT-SKILLS-001-7] README documents submodule adoption + pointer-bump upgrade; version 2.0.0 +- [KIT-SKILLS-001-8] BOOTSTRAP + build script deprecated but still runnable +``` + +--- + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | Criterion 3 ("moves not rewrites, empty content diff") is unsatisfiable at branch HEAD, because §D7 also requires rewriting path references **inside** those same files | Split into two commits. `d503ac1` is the pure rename — 13 files, 0 insertions, 0 deletions, verifiable in isolation. `008248d` carries every content change. The criterion is checked against `d503ac1`, which is where it is meaningful. Implements the intent exactly; strictly more auditable than one commit. | +| 2 | Contract VERIFY counted `-name '*.md' -path '*/templates/*'` and expected 13 | Off by one — 12 `.md` plus `FEATURES.json`. VERIFY counts all files, not just `.md`. Contract text was wrong; output is right. | +| 3 | `templates/` remained on disk after `git mv` | Empty directory artifact — git does not track directories, nothing tracked remained. Removed with `rmdir`. | + +## Path-reference style (design note) + +Cross-references were rewritten to name **skills** ("the `security-checklist` skill") rather than +file paths. A path would bake in one tool's merge directory (`.agents/` vs `.claude/` vs `.cursor/`); +a skill name resolves wherever the repo wires them, so the kit stays tool-agnostic. + +## Setup script — tested, not asserted + +Exercised end to end in a throwaway consumer repo: 7 skills linked · symlinks resolve to real +content · `.claude`/`.cursor` wired · gitignore appended exactly once · **second run produced no +change** · refuses a `.agents/skills` it did not generate · project-local skills win on name +collision · a non-symlink `.cursor/skills` is left alone with a warning. + +## VERIFY result + +All checks green (23/23 after the empty-directory fix). diff --git a/skills/.harness/contracts/KIT-V13-001.md b/skills/.harness/contracts/KIT-V13-001.md new file mode 100644 index 0000000..951255f --- /dev/null +++ b/skills/.harness/contracts/KIT-V13-001.md @@ -0,0 +1,264 @@ +# SPRINT CONTRACT — KIT-V13-001: Universal multi-language spine + audit profile + +> Status: **EXECUTE complete — PENDING_REVIEW** +> Feature ID: `KIT-V13-001` +> Repo: **harness-kit only** +> Branch: `feat/kit-v13-001-universal-spine` from `master` (approved 2026-08-02) +> VERIFY: green 2026-08-02. Human sets PASS on merge. + +--- + +## Goal + +Evolve harness-kit so the **same process law** works for any project across TypeScript, Rust, Go, Solidity build, and Solidity contract auditing. Keep the control plane language-neutral. Add work-type profiles and an audit contract template. Leave stack craft (Vitest, Cargo idioms, Foundry recipes, hexagonal) to optional packs / consumer `AGENTS.md`, not the core loop. + +--- + +## Scope — WILL do + +### A — Version + bootstrap + +1. Bump kit version to **1.3.0** in `BOOTSTRAP.md` header / changelog / `VERSION` instruction and `scripts/build-bootstrap.sh` embedded header. +2. Add v1.3 changelog bullets covering: `PENDING_REVIEW`, contract quality gates (N/A-by-default), work-type profiles, REVIEW §0, Solidity audit template, language-neutral VERIFY. +3. Re-run `./scripts/build-bootstrap.sh` so appendices stay in sync with `templates/*`. + +### B — FEATURES status model + +4. Update `templates/FEATURES.json` legend to three states: `FAIL` | `PENDING_REVIEW` | `PASS`. +5. Document owners in `templates/AGENT_PROCESS.md` / `templates/AGENTS.md` § Agent process: + - Executor may set `PENDING_REVIEW` after VERIFY green. + - Only human/reviewer sets `PASS`. + - Executor never sets `PASS`. +6. Add root `FEATURES.json` entry for `KIT-V13-001` (`FAIL` until merge; notes link this contract). Mark superseded historical FAIL rows that are dead (`KIT-INSTALL-001`, `KIT-UPDATE-001`, `KIT-SETUP-001`, `KIT-DEPRECATE-ADOPTION-001`, `KIT-STATE-001`) as `PASS` with notes `superseded` **or** leave FAIL but add a `notes` clarification that they are historical supersessions (prefer: set `PASS` + superseded note to reduce noise). + +### C — Agent process (universal spine) + +7. Update session loop to: + +``` +BOOT → RESEARCH? → CONTRACT → [HUMAN APPROVES + BRANCH] → CHECKOUT → EXECUTE (TDD when behaviour changes) → VERIFY → REVIEW → PENDING_REVIEW → [HUMAN PASS/MERGE] +``` + +8. Add **depth matrix** (trivial / medium / large / audit) — when to skip Research, when condensed process is allowed. +9. Add **work-type profile** selection (must pick one per contract): + - `service` (TS / Go / Rust app logic) + - `http-api` (HTTP surface) + - `solidity-build` + - `solidity-audit` + - `ops-docs` +10. State explicitly: VERIFY commands come from the contract + consumer `AGENTS.md`; kit never mandates a language runner. +11. State explicitly: architecture patterns (hexagonal, etc.) apply only when the consumer `AGENTS.md` declares them. +12. Optional Research: one paragraph — distill ticket/ERD/scope into open questions before Contract when work is large or audit; do not invent a second approved artifact (`plan.md` is not required; decisions live in the sprint contract). + +### D — Sprint contract template + +13. Extend `templates/SPRINT_CONTRACT.md` with: + - **Work-type profile** (required enum above) + - **Languages / toolchain** (free text; cite consumer AGENTS.md) + - **Decisions** table (topic / decision / locked) + - **Vertical slice / TDD order** (behaviour + red/green; N/A for pure audit finding write-ups that use PoC-first instead) + - **Quality gates** checklist, each tickable or **N/A**: + - Falsifiable success criteria (always required) + - Trust-boundary failure mode documented (typed error / revert / Err / 4xx — never uncaught panic/throw across handler boundary where applicable) + - Partial/optional deps: skip vs reject documented + - **HTTP only:** request + response API schema updated; error envelope regression in VERIFY + - **Solidity build only:** invariant/access-control/value-flow risks named; forge (or declared) test commands + - **Solidity audit only:** use `AUDIT_CONTRACT.md` instead or embed audit sections + - Keep existing: Scope, Branch, Tests first, Impact map (GROUNDED/EDUCATED), Threat model, Blocking questions, FEATURES entry + +### E — REVIEW template + +14. Prepend **§0 Sprint contract compliance** to `templates/REVIEW.md` (adapted from practice PR-REVIEW; paths say `skills/.harness/contracts/.md` for kit consumers). +15. Add profile-aware review hints (short): + - service: correctness, reliability, ops + - http-api: compat + envelope + - solidity-build: authz, upgrade/pause, economic safety + - solidity-audit: severity calibration, FP risk, missing bug classes +16. Keep existing production sections 1–14; renumber if needed so §0 is first. + +### F — Solidity audit profile + +17. Add new template `templates/AUDIT_CONTRACT.md`: + - Scope in/out (contracts, functions, chains, commit/tag) + - Assets / actors / trust boundaries + - Prior reports / known issues + - Severity rubric (Critical/High/Medium/Low/Informational or project rubric) + - Tooling plan (forge tests, fuzz, invariant, slither/mythril if used — commands not brand-mandatory) + - Finding ID scheme (`AUDIT-NNN-F0x` or similar) + - PoC-first rule: High/Critical require reproducible VERIFY command + - Out of scope explicit + - Branch + human approval (same as sprint contracts) + - Link to fix follow-up as normal `solidity-build` sprint contracts +18. Add `templates/AUDIT_FINDING.md` (short finding write-up: title, severity, location, impact, PoC, fix suggestion, status). +19. Extend `templates/SECURITY_CHECKLIST.md` with a **Solidity / audit** section (reentrancy, auth, oracle/manipulation, upgrade, signature, DoS, unexpected external calls) marked as required for `solidity-build` and `solidity-audit` profiles; existing web sections remain N/A when not applicable. +20. Wire audit templates into Step 0 / build-bootstrap appendices (new Appendix letters after existing). + +### G — Docs pointers + +21. Update `README.md` / `templates/README_POINTER.md` to mention: multi-language spine, work-type profiles, audit template, `PENDING_REVIEW`. +22. Update `templates/AGENTS.md` Stack examples to list TS / Rust / Go / Solidity as first-class; point How we work here at profile selection. +23. One short subsection: **Skill packs (optional)** — agent-harness or local skills may supply stack craft; they must not replace FEATURES/contracts/VERIFY; hexagonal/TDD router skills are optional and stack-specific. + +--- + +## Scope — will NOT do (this sprint) + +- Adopt `agent-harness` as a submodule inside harness-kit or rewrite RPI skills. +- Write full Rust / Go / TypeScript / Foundry **skill packs** (only stubs/pointers in AGENTS/README). +- Change consumer repos (`intents-service`, `tao-intent-solver`, contracts repo, etc.). +- Mandate OpenAPI/Postman/hexagonal/Vitest in the universal loop. +- Reintroduce `HARNESS.md`, `LOOP.md`, `install.sh`, or root committed contracts. +- Automate audit tooling CI beyond documenting VERIFY command patterns. +- Language-specific code samples beyond short command examples in templates. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master` +- **Proposed feature branch:** `feat/kit-v13-001-universal-spine` +- **Human confirmed:** pending — agent asks before checkout + +--- + +## Tests first (TDD — docs kit) + +This sprint is **template/docs process**. “Tests” are deterministic verify commands, not application unit tests. + +- **RED:** before edits, note current fails (e.g. templates lack `PENDING_REVIEW`; no `AUDIT_CONTRACT.md`). +- **GREEN:** after edits, all VERIFY commands below pass. +- **Expected RED (today):** + - `grep -q PENDING_REVIEW templates/FEATURES.json` → fail + - `test -f templates/AUDIT_CONTRACT.md` → fail + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `templates/FEATURES.json` | 3-state legend | [GROUNDED] | +| `templates/AGENT_PROCESS.md` | Loop, PENDING_REVIEW, depth matrix, profiles, language-neutral VERIFY | [GROUNDED] | +| `templates/AGENTS.md` | Embed matching Agent process + stack/profile notes | [GROUNDED] | +| `templates/SPRINT_CONTRACT.md` | Profile, decisions, gates, slice | [GROUNDED] | +| `templates/REVIEW.md` | §0 compliance + profile hints | [GROUNDED] | +| `templates/SECURITY_CHECKLIST.md` | Solidity/audit section | [GROUNDED] | +| `templates/AUDIT_CONTRACT.md` | **NEW** | [NEW] | +| `templates/AUDIT_FINDING.md` | **NEW** | [NEW] | +| `templates/README_POINTER.md` | Pointers for v1.3 | [GROUNDED] | +| `README.md` | Brief v1.3 mention if needed | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated via script + Step 0 / changelog / appendices for audit templates | [GROUNDED] | +| `scripts/build-bootstrap.sh` | Header 1.3.0; append new audit templates | [GROUNDED] | +| `FEATURES.json` (root) | Add KIT-V13-001; clean superseded noise | [GROUNDED] | + +Reuse: existing REVIEW body, SECURITY workflow safety, SPRINT_CONTRACT branch/TDD sections. Prefer extend over rewrite. + +--- + +## Success criteria + +1. Kit version string **1.3.0** appears in `BOOTSTRAP.md` and build script header. +2. `templates/FEATURES.json` legend lists exactly `FAIL`, `PENDING_REVIEW`, `PASS` with owner notes in Agent process. +3. `templates/SPRINT_CONTRACT.md` requires a work-type profile and includes quality gates with explicit N/A paths. +4. `templates/REVIEW.md` starts with §0 contract compliance before production sections. +5. `templates/AUDIT_CONTRACT.md` and `templates/AUDIT_FINDING.md` exist and are included in bootstrap generation. +6. Agent process states VERIFY commands are repo-declared (no hardcoded Vitest/Cargo/forge as the only runner). +7. Agent process states hexagonal/stack skills are optional and must not replace the contract spine. +8. `./scripts/build-bootstrap.sh` exits 0; regenerated `BOOTSTRAP.md` contains `PENDING_REVIEW`, `AUDIT_CONTRACT`, and `1.3.0`. +9. Diff confined to harness-kit paths in the impact map (plus this local contract under gitignored `skills/.harness/`). +10. All work on confirmed feature branch, not `master`. + +### Quality gates (this sprint) + +- [x] Falsifiable success criteria (above) +- [x] Trust-boundary N/A (docs-only kit) +- [x] HTTP OpenAPI N/A +- [x] Solidity runtime N/A (templates only) +- [x] No consumer repo edits + +--- + +## Verify + +```bash +cd /Users/davidasamonye/Work/tao.com/harness-kit + +# version +grep -q '1.3.0' BOOTSTRAP.md +grep -q '1.3.0' scripts/build-bootstrap.sh + +# status model +grep -q 'PENDING_REVIEW' templates/FEATURES.json +grep -q 'PENDING_REVIEW' templates/AGENT_PROCESS.md + +# contract template gates + profiles +grep -q 'Work-type profile' templates/SPRINT_CONTRACT.md +grep -q 'Quality gates' templates/SPRINT_CONTRACT.md +grep -q 'solidity-audit' templates/SPRINT_CONTRACT.md + +# review §0 +grep -q 'Sprint contract compliance' templates/REVIEW.md + +# audit templates +test -f templates/AUDIT_CONTRACT.md +test -f templates/AUDIT_FINDING.md +grep -q 'PoC' templates/AUDIT_CONTRACT.md + +# security solidity section +grep -qi 'reentrancy\|Solidity' templates/SECURITY_CHECKLIST.md + +# bootstrap regen + JSON +./scripts/build-bootstrap.sh +grep -q '1.3.0' BOOTSTRAP.md +grep -q 'AUDIT_CONTRACT' BOOTSTRAP.md +python3 -m json.tool FEATURES.json >/dev/null +python3 -m json.tool templates/FEATURES.json >/dev/null + +# root feature entry +grep -q 'KIT-V13-001' FEATURES.json + +# branch check (after checkout) +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-V13-001", + "name": "Universal multi-language spine: PENDING_REVIEW, profiles, REVIEW §0, Solidity audit templates", + "priority": 1, + "verify": "grep -q '1.3.0' BOOTSTRAP.md && grep -q PENDING_REVIEW templates/FEATURES.json && test -f templates/AUDIT_CONTRACT.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-V13-001.md" +} +``` + +--- + +## Blocking questions (gates) + +1. **Contracts path for consumers:** keep documenting `skills/.harness/contracts/` as in kit v1.2, or also allow `.harness/contracts/` as an equally valid local path? + - **Recommendation:** keep `skills/.harness/` as the kit-documented default; add one sentence that existing repos using `.harness/contracts/` are acceptable if `AGENTS.md` says so (no migration forced this sprint). +2. **Superseded FEATURES rows:** set historical superseded FAILs to `PASS` with notes, or leave FAIL? + - **Recommendation:** set to `PASS` + `superseded by KIT-SINGLE-LLM-001 / v1.2` to reduce triage noise. +3. **agent-harness mention:** brief “optional companion” paragraph only (this sprint), full integration later? + - **Recommendation:** brief only. + +Approve this contract only if Q1–Q3 (or your edits) are acceptable. + +--- + +## Criteria block + +```criteria +- [KIT-V13-001-1] Version 1.3.0 in BOOTSTRAP + build script +- [KIT-V13-001-2] PENDING_REVIEW in FEATURES template + Agent process owners +- [KIT-V13-001-3] SPRINT_CONTRACT has profiles + quality gates with N/A +- [KIT-V13-001-4] REVIEW.md §0 contract compliance present +- [KIT-V13-001-5] AUDIT_CONTRACT.md + AUDIT_FINDING.md exist and bootstrap-embedded +- [KIT-V13-001-6] VERIFY language-neutral; stack packs optional in Agent process +- [KIT-V13-001-7] build-bootstrap.sh green; FEATURES.json valid; KIT-V13-001 listed +``` diff --git a/skills/code-quality-gate/templates/CODE_QUALITY.md b/skills/code-quality-gate/templates/CODE_QUALITY.md index ae6ca7d..e2dd197 100644 --- a/skills/code-quality-gate/templates/CODE_QUALITY.md +++ b/skills/code-quality-gate/templates/CODE_QUALITY.md @@ -11,6 +11,9 @@ > `CODE_AUDIT_CONTRACT.md` asks "where should we invest refactoring effort?" on a cadence, over a > declared subsystem. Do not substitute one for the other. +> Process discipline — ledger, contract, scope containment, RED artifact — is checked by +> `verify-harness.sh`, not here. This gate judges the code; that one judges the contract. + ## 0. Sensor - [ ] Repo's declared lint / static-analysis command from `AGENTS.md` was **run** on the diff diff --git a/skills/harness-onboard/templates/AGENT_PROCESS.md b/skills/harness-onboard/templates/AGENT_PROCESS.md index ed5f643..13dd1c8 100644 --- a/skills/harness-onboard/templates/AGENT_PROCESS.md +++ b/skills/harness-onboard/templates/AGENT_PROCESS.md @@ -229,6 +229,20 @@ is **N/A** for `ops-docs`, `code-audit` and `product-erd`, and for `solidity-aud ship code of their own. Credential findings belong to `SECURITY_CHECKLIST.md`, not this gate — raise them there and do not double-report. +Then run the gate: + +```sh +./scripts/verify-harness.sh # or the path this repo declares +``` + +It checks the discipline rather than the code: ledger integrity, that every `PENDING_REVIEW` has a +contract, that skills and rules will actually load, that every `[GROUNDED]` path in the contract +**exists**, that the diff is **contained by the impact map**, and that RED was recorded and failed on +an assertion. **The executor may set `PENDING_REVIEW` only when this exits 0.** + +Rules and skills are instructions to a model that can misread them under load. This is a fact — and +**CI runs it again**, so a claim that it passed locally is not evidence. + If verify fails: stay `FAIL`, fix or revise the contract. #### 9. REVIEW → PENDING_REVIEW → HUMAN PASS diff --git a/skills/harness-review/templates/REVIEW.md b/skills/harness-review/templates/REVIEW.md index fed0d18..b1cb471 100644 --- a/skills/harness-review/templates/REVIEW.md +++ b/skills/harness-review/templates/REVIEW.md @@ -31,6 +31,11 @@ Before deep code review, locate and read the relevant contract: code-quality gate (`CODE_QUALITY.md`: lint command recorded, violations = 0 or documented), solidity-audit PoCs. Mark each **Pass / Fail / N/A**. 8. **VERIFY commands** — run or confirm the contract's verify block passed; note any skipped or failing commands. +9. **Gate artifacts** — confirm the contract carries a recorded **RED** (real command, non-zero exit, + assertion-shaped failure), a **BASELINE**, and a **GREEN** whose counts match the baseline. A + predicted RED, or one that failed on an import error, is a finding. `N/A` is only valid when the + sprint changed no behaviour. +10. **Gate ran** — confirm `verify-harness.sh` exited 0 **in CI**, not only locally. If no contract exists (hotfix, drive-by), state that explicitly and review on production-readiness only. Retroactive contract may be required before PASS. diff --git a/skills/sprint-contract/templates/SPRINT_CONTRACT.md b/skills/sprint-contract/templates/SPRINT_CONTRACT.md index 9e9db74..b10c6d1 100644 --- a/skills/sprint-contract/templates/SPRINT_CONTRACT.md +++ b/skills/sprint-contract/templates/SPRINT_CONTRACT.md @@ -72,10 +72,34 @@ Pick **one**: - **Test files to create or extend:** - `` — -- **Expected RED output:** -- **Verify commands (GREEN):** - - `` - - `` +> **A test that never failed proves nothing.** It may be vacuous, or already passing. Record the real +> run — command, exit code, and the failure message — not a prediction of it. A RED that failed on an +> import or path error proves the test file is wrong, not that the behaviour is absent. +> +> **No ERD? Same gate.** Without a design document the success criteria below are the whole story, and +> each must be expressible as a **failing test before work starts**. If you cannot write a test that +> fails today, you do not have a falsifiable criterion and this contract is not approvable. + +### RED (recorded before EXECUTE) + +- **Command:** `` +- **Exit code:** `` +- **Failure reason:** `` +- **Captured:** `` + +**N/A** only when this sprint changes no behaviour (`ops-docs`, `code-audit`, `product-erd`). + +### BASELINE (recorded before EXECUTE — behaviour changes only) + +- **Command:** `` +- **Result:** `` + +### GREEN (recorded after EXECUTE) + +- **Command:** `` → **exit 0** +- **Command:** `` → **``** + +Green-after means nothing without green-before. A count that moved is a regression until explained. ### Vertical slice (when non-trivial) From 77c9e2c3698b1f7e62d81a6307245aa2b457beae Mon Sep 17 00:00:00 2001 From: David Asamonye Date: Fri, 28 Aug 2026 13:23:21 +0000 Subject: [PATCH 2/2] feat(kit): v2.4 the gate reads structure, not prose Two findings from the gate's own first run against this repo. Check 6 was guessing. The v2.3 patch accepted a backticked token as a path only if it contained a slash or a known extension, which silently skipped real paths like Makefile and checked things that were not paths at all. A check that silently skips is worse than one that fails loudly. It now locates the Impact map section, reads to the next heading, and takes the first cell of each marked table row. Unparseable marked rows fail; bullet-list impact maps skip visibly. That only works because the artifact is unambiguous, so the contract form now states that [GROUNDED] belongs to the impact map alone, and the Branch section example no longer carries a marker. Structure the parser can read position from, instead of prose it has to interpret. Check 10 closes the drift the ledger check already covers. FEATURES.json rot was caught; the same rot in STATE.md was invisible, since the gate only asked whether the named contract existed. An inbox entry naming a feature already PASS now fails. The first implementation of check 10 also flagged four ids that were not rot at all: an audit belonging to another repo, two pieces of planned work deliberately named, and a filename the regex matched inside. That half was noise, so it is gone; the check now fires only on ids present in the ledger and already PASS. Bumps the kit to 2.4.0. Contract: skills/.harness/contracts/KIT-GATES-002.md --- BOOTSTRAP.md | 159 ++++++++++-- FEATURES.json | 8 + README.md | 2 +- scripts/build-bootstrap.sh | 24 +- scripts/verify-harness.sh | 126 +++++++-- skills/.harness/contracts/KIT-GATES-002.md | 239 ++++++++++++++++++ .../templates/SPRINT_CONTRACT.md | 11 +- 7 files changed, 519 insertions(+), 50 deletions(-) create mode 100644 skills/.harness/contracts/KIT-GATES-002.md diff --git a/BOOTSTRAP.md b/BOOTSTRAP.md index d8920c2..c2844df 100644 --- a/BOOTSTRAP.md +++ b/BOOTSTRAP.md @@ -1,13 +1,25 @@ # BOOTSTRAP.md — Harness Bootstrap -> **DEPRECATED — kit 2.3.0 adopts via git submodule; see README.md. This file is retained for +> **DEPRECATED — kit 2.4.0 adopts via git submodule; see README.md. This file is retained for > repos already bootstrapped from it and will be removed in KIT-SKILLS-002.** > -> **Kit version 2.3.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, +> **Kit version 2.4.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, > Rust, Go, Solidity build, and Solidity auditing. Copy this file into **any** repo. Prompt your LLM: > *"Read BOOTSTRAP.md and complete Step 0 — Bootstrap harness files."* -## v2.3 changelog +## v2.4 changelog + +- **The gate reads structure, not prose.** Check 6 parses the Impact map section's table — first cell + of each marked row — instead of guessing which backticked token is a path. The v2.3 heuristic + accepted a token only if it had a slash or a known extension, which silently skipped real paths + (`Makefile`) and checked things that were not paths. A check that silently skips is worse than one + that fails loudly, so an unparseable marked row now fails. +- **`[GROUNDED]` belongs to the impact map only** — stated in the contract form, which is what lets + the parser read position instead of inferring meaning. +- **Check 10: `STATE.md` inbox drift.** The ledger check caught rot in `FEATURES.json` while the same + rot in `STATE.md` went unseen. An inbox entry naming a feature already `PASS` now fails. + +## v2.3 changelog (retained) - **`scripts/verify-harness.sh`** — the third layer. Rules and skills are instructions to a model that can misread them; this is a fact that exits non-zero. Nine checks: ledger integrity, every @@ -134,7 +146,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco | 1 | kit skills (symlinked by `setup-harness-kit.sh`) | Appendices B–P | N/A — canonical in the submodule | | 2 | `skills/.harness/contracts/` | — | Create empty directory | | 3 | `skills/.harness/STATE.md` | Appendix F | Fill `` + date | -| 4 | `skills/.harness/VERSION` | — | `version=2.3.0` + `bootstrapped=` | +| 4 | `skills/.harness/VERSION` | — | `version=2.4.0` + `bootstrapped=` | | 5 | `README.md` | Appendix G | **No** — overwrite with slim pointer (after brownfield recovery if needed) | | 6 | `AGENTS.md` | Appendix B | **Yes** if already filled — scaffold only; never overwrite harvested/verified content | | 7 | `FEATURES.json` | Appendix C | **Yes** if seeded — scaffold only | @@ -152,7 +164,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco **Done when:** -- `skills/.harness/` tree exists with VERSION `2.3.0` +- `skills/.harness/` tree exists with VERSION `2.4.0` - `python3 -m json.tool FEATURES.json` passes (if scaffolded) - `.gitignore` contains `skills/.harness/` - No `HARNESS.md` in repo @@ -848,7 +860,7 @@ Pick **one**: > Implementation on the default branch is forbidden. The human must confirm the branch name > before the agent creates or checks out the branch. -- **Default branch:** +- **Default branch:** - **Proposed feature branch:** `` - **Human confirmed:** @@ -910,8 +922,13 @@ Green-after means nothing without green-before. A count that moved is a regressi ## Impact map -> Mark each path **[GROUNDED]** (verified in repo) or **[EDUCATED]** (must re-verify before -> implementing). Never present educated guesses as grounded. +> Mark each path **[GROUNDED]** (verified in repo), **[EDUCATED]** (must re-verify before +> implementing), or **[NEW]**. Never present educated guesses as grounded. +> +> **These markers belong to the impact map only**, and the impact map is a table whose **first cell is +> the path in backticks**. The gate reads that structure: it parses this section's table rows rather +> than guessing which backticked token elsewhere in the contract is a path. A marker used outside this +> table, or a marked row whose first cell is not a backticked path, is a gate failure. - **Files to change:** - `` — — [GROUNDED|EDUCATED] @@ -2618,24 +2635,69 @@ elif [ -f "$STATE" ]; then fi # --- 6. grounded paths exist (catches invented files) ------------------------ +# Reads STRUCTURE, not prose: the Impact map section only, table rows only, first +# cell only. Scanning for [GROUNDED] anywhere also picks up Context and Decisions +# tables, and guessing which backticked token is a path misfires both ways. if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then - # Only backticked tokens that look like paths: they contain a slash, or carry a - # known file extension. Branch names and commit SHAs are also backticked on - # [GROUNDED] lines and are not paths. - missing="" - for p in $(grep -o '`[^`]*`[^|]*\[GROUNDED\]' "$contract_file" | grep -o '`[^`]*`' | tr -d '`' | sort -u); do - case "$p" in - *" "*|"") continue ;; - */*) : ;; - *.md|*.sh|*.json|*.yml|*.yaml|*.mdc|*.ts|*.js|*.go|*.rs|*.sol|*.py) : ;; - *) continue ;; - esac - [ -e "$p" ] || missing="$missing $p" - done - if [ -z "$missing" ]; then pass "contract: every [GROUNDED] path exists" - else fail "contract cites paths that do not exist:$missing"; fi + result=$(python3 - "$contract_file" <<'PYEOF' +import re, sys, glob, os + +text = open(sys.argv[1], encoding="utf-8").read().splitlines() + +# isolate the Impact map section +start = None +for n, line in enumerate(text): + if re.match(r"^##+\s+Impact map", line, re.I): + start = n + 1 + break +if start is None: + print("SKIP no Impact map section") + raise SystemExit + +section = [] +for line in text[start:]: + if re.match(r"^##+\s+", line): + break + section.append(line) + +rows = [l for l in section if l.lstrip().startswith("|") and "[GROUNDED]" in l] +if not rows: + if any(l.lstrip().startswith("-") for l in section): + print("SKIP impact map not tabular") + else: + print("SKIP no [GROUNDED] rows") + raise SystemExit + +missing, malformed = [], [] +for row in rows: + cells = [c.strip() for c in row.strip().strip("|").split("|")] + if not cells: + malformed.append(row.strip()[:60]); continue + m = re.search(r"`([^`]+)`", cells[0]) + if not m: + malformed.append(cells[0][:60]); continue + path = m.group(1).strip() + if "*" in path: + if not glob.glob(path): + missing.append(path) + elif not os.path.exists(path): + missing.append(path) + +if malformed: + print("FAIL unparseable [GROUNDED] rows: " + "; ".join(malformed)) +elif missing: + print("FAIL paths do not exist: " + " ".join(missing)) +else: + print("OK %d [GROUNDED] path(s) verified" % len(rows)) +PYEOF +) + case "$result" in + OK*) pass "contract impact map: ${result#OK }" ;; + SKIP*) skip "contract impact map (${result#SKIP })" ;; + *) fail "contract impact map: ${result#FAIL }" ;; + esac else - skip "contract [GROUNDED] paths (no contract)" + skip "contract impact map (no contract)" fi # --- 7. diff is contained by the impact map ---------------------------------- @@ -2673,6 +2735,55 @@ else skip "RED artifact (no contract)" fi +# --- 10. STATE inbox does not reference finished work ------------------------ +# FEATURES.json rot is checked above; STATE.md rots the same way and nothing saw it. +if [ -f "$STATE" ] && [ -f FEATURES.json ]; then + result=$(python3 - "$STATE" <<'PYEOF' +import json, re, sys + +text = open(sys.argv[1], encoding="utf-8").read().splitlines() +start = None +for n, line in enumerate(text): + if re.match(r"^##+\s+Human-attention inbox", line, re.I): + start = n + 1 + break +if start is None: + print("SKIP no inbox section"); raise SystemExit + +section = [] +for line in text[start:]: + if re.match(r"^##+\s+", line): + break + section.append(line) + +feats = {f["id"]: f.get("status") for f in json.load(open("FEATURES.json")).get("features", [])} + +# Only ids that exist in this ledger can be stale. An id absent from the ledger is +# either planned work not yet created, or an id belonging to another repo, or part +# of a filename — none of which is rot. Flagging them was noise, not signal. +ids = set() +for line in section: + stripped = re.sub(r"`[^`]*`", " ", line) # drop backticked paths/filenames + stripped = re.sub(r"\S*/\S*", " ", stripped) # drop anything path-shaped + for m in re.findall(r"\b[A-Z][A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3}\b", stripped): + ids.add(m) + +done = sorted(i for i in ids if feats.get(i) == "PASS") +if done: + print("FAIL inbox references work already PASS: " + " ".join(done)) +else: + print("OK inbox references only open work") +PYEOF +) + case "$result" in + OK*) pass "STATE inbox: ${result#OK }" ;; + SKIP*) skip "STATE inbox (${result#SKIP })" ;; + *) fail "STATE ${result#FAIL }" ;; + esac +else + skip "STATE inbox (no STATE.md or FEATURES.json)" +fi + # --- 9. branch discipline ----------------------------------------------------- if git rev-parse --git-dir >/dev/null 2>&1; then cur=$(git branch --show-current 2>/dev/null || echo "") diff --git a/FEATURES.json b/FEATURES.json index 1aee429..73b4cb4 100644 --- a/FEATURES.json +++ b/FEATURES.json @@ -13,6 +13,14 @@ "verify": "runnable test command or command sequence — not prose" }, "features": [ + { + "id": "KIT-GATES-002", + "name": "Gate reads structure not prose: impact-map parse replaces the path heuristic; STATE inbox drift check", + "priority": 2, + "verify": "! grep -q '[*].md|[*].sh' scripts/verify-harness.sh && grep -qi inbox scripts/verify-harness.sh && ./scripts/verify-harness.sh && grep -q '2.4.0' BOOTSTRAP.md && python3 -m json.tool FEATURES.json >/dev/null", + "status": "PENDING_REVIEW", + "notes": "Contract: skills/.harness/contracts/KIT-GATES-002.md; profile: ops-docs; EXECUTE+VERIFY green 2026-08-28 on feat/kit-gates-002-tighten-gate; 4 declared deviations incl. Q2 corrected by the gate itself" + }, { "id": "KIT-GATES-001", "name": "Deterministic gates: RED/BASELINE/GREEN artifacts, verify-harness.sh, CI re-running VERIFY", diff --git a/README.md b/README.md index e256458..16cd50f 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ A reusable, **tool-agnostic** framework for AI-assisted engineering — contract-first, TDD when behaviour changes, multi-language (TypeScript, Rust, Go, Solidity build + audit). -**Kit v2.3** — the process ships as **skills**, adopted as a **git submodule**. Skills stay canonical +**Kit v2.4** — the process ships as **skills**, adopted as a **git submodule**. Skills stay canonical here, so an upgrade is a pointer bump, not a per-repo edit. ## Adopt diff --git a/scripts/build-bootstrap.sh b/scripts/build-bootstrap.sh index e70f127..86c2fb1 100755 --- a/scripts/build-bootstrap.sh +++ b/scripts/build-bootstrap.sh @@ -1,5 +1,5 @@ #!/bin/sh -# DEPRECATED as of kit 2.3.0 — adoption moved to git submodule + scripts/setup-harness-kit.sh. +# DEPRECATED as of kit 2.4.0 — adoption moved to git submodule + scripts/setup-harness-kit.sh. # Retained so repos bootstrapped from BOOTSTRAP.md keep working. Removal: KIT-SKILLS-002. # Generates BOOTSTRAP.md from the forms now bundled inside skills//templates/. set -eu @@ -17,14 +17,26 @@ append() { cat <<'HEADER' # BOOTSTRAP.md — Harness Bootstrap -> **DEPRECATED — kit 2.3.0 adopts via git submodule; see README.md. This file is retained for +> **DEPRECATED — kit 2.4.0 adopts via git submodule; see README.md. This file is retained for > repos already bootstrapped from it and will be removed in KIT-SKILLS-002.** > -> **Kit version 2.3.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, +> **Kit version 2.4.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, > Rust, Go, Solidity build, and Solidity auditing. Copy this file into **any** repo. Prompt your LLM: > *"Read BOOTSTRAP.md and complete Step 0 — Bootstrap harness files."* -## v2.3 changelog +## v2.4 changelog + +- **The gate reads structure, not prose.** Check 6 parses the Impact map section's table — first cell + of each marked row — instead of guessing which backticked token is a path. The v2.3 heuristic + accepted a token only if it had a slash or a known extension, which silently skipped real paths + (`Makefile`) and checked things that were not paths. A check that silently skips is worse than one + that fails loudly, so an unparseable marked row now fails. +- **`[GROUNDED]` belongs to the impact map only** — stated in the contract form, which is what lets + the parser read position instead of inferring meaning. +- **Check 10: `STATE.md` inbox drift.** The ledger check caught rot in `FEATURES.json` while the same + rot in `STATE.md` went unseen. An inbox entry naming a feature already `PASS` now fails. + +## v2.3 changelog (retained) - **`scripts/verify-harness.sh`** — the third layer. Rules and skills are instructions to a model that can misread them; this is a fact that exits non-zero. Nine checks: ledger integrity, every @@ -151,7 +163,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco | 1 | kit skills (symlinked by `setup-harness-kit.sh`) | Appendices B–P | N/A — canonical in the submodule | | 2 | `skills/.harness/contracts/` | — | Create empty directory | | 3 | `skills/.harness/STATE.md` | Appendix F | Fill `` + date | -| 4 | `skills/.harness/VERSION` | — | `version=2.3.0` + `bootstrapped=` | +| 4 | `skills/.harness/VERSION` | — | `version=2.4.0` + `bootstrapped=` | | 5 | `README.md` | Appendix G | **No** — overwrite with slim pointer (after brownfield recovery if needed) | | 6 | `AGENTS.md` | Appendix B | **Yes** if already filled — scaffold only; never overwrite harvested/verified content | | 7 | `FEATURES.json` | Appendix C | **Yes** if seeded — scaffold only | @@ -169,7 +181,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco **Done when:** -- `skills/.harness/` tree exists with VERSION `2.3.0` +- `skills/.harness/` tree exists with VERSION `2.4.0` - `python3 -m json.tool FEATURES.json` passes (if scaffolded) - `.gitignore` contains `skills/.harness/` - No `HARNESS.md` in repo diff --git a/scripts/verify-harness.sh b/scripts/verify-harness.sh index 2811241..a6e594c 100755 --- a/scripts/verify-harness.sh +++ b/scripts/verify-harness.sh @@ -132,24 +132,69 @@ elif [ -f "$STATE" ]; then fi # --- 6. grounded paths exist (catches invented files) ------------------------ +# Reads STRUCTURE, not prose: the Impact map section only, table rows only, first +# cell only. Scanning for [GROUNDED] anywhere also picks up Context and Decisions +# tables, and guessing which backticked token is a path misfires both ways. if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then - # Only backticked tokens that look like paths: they contain a slash, or carry a - # known file extension. Branch names and commit SHAs are also backticked on - # [GROUNDED] lines and are not paths. - missing="" - for p in $(grep -o '`[^`]*`[^|]*\[GROUNDED\]' "$contract_file" | grep -o '`[^`]*`' | tr -d '`' | sort -u); do - case "$p" in - *" "*|"") continue ;; - */*) : ;; - *.md|*.sh|*.json|*.yml|*.yaml|*.mdc|*.ts|*.js|*.go|*.rs|*.sol|*.py) : ;; - *) continue ;; - esac - [ -e "$p" ] || missing="$missing $p" - done - if [ -z "$missing" ]; then pass "contract: every [GROUNDED] path exists" - else fail "contract cites paths that do not exist:$missing"; fi + result=$(python3 - "$contract_file" <<'PYEOF' +import re, sys, glob, os + +text = open(sys.argv[1], encoding="utf-8").read().splitlines() + +# isolate the Impact map section +start = None +for n, line in enumerate(text): + if re.match(r"^##+\s+Impact map", line, re.I): + start = n + 1 + break +if start is None: + print("SKIP no Impact map section") + raise SystemExit + +section = [] +for line in text[start:]: + if re.match(r"^##+\s+", line): + break + section.append(line) + +rows = [l for l in section if l.lstrip().startswith("|") and "[GROUNDED]" in l] +if not rows: + if any(l.lstrip().startswith("-") for l in section): + print("SKIP impact map not tabular") + else: + print("SKIP no [GROUNDED] rows") + raise SystemExit + +missing, malformed = [], [] +for row in rows: + cells = [c.strip() for c in row.strip().strip("|").split("|")] + if not cells: + malformed.append(row.strip()[:60]); continue + m = re.search(r"`([^`]+)`", cells[0]) + if not m: + malformed.append(cells[0][:60]); continue + path = m.group(1).strip() + if "*" in path: + if not glob.glob(path): + missing.append(path) + elif not os.path.exists(path): + missing.append(path) + +if malformed: + print("FAIL unparseable [GROUNDED] rows: " + "; ".join(malformed)) +elif missing: + print("FAIL paths do not exist: " + " ".join(missing)) +else: + print("OK %d [GROUNDED] path(s) verified" % len(rows)) +PYEOF +) + case "$result" in + OK*) pass "contract impact map: ${result#OK }" ;; + SKIP*) skip "contract impact map (${result#SKIP })" ;; + *) fail "contract impact map: ${result#FAIL }" ;; + esac else - skip "contract [GROUNDED] paths (no contract)" + skip "contract impact map (no contract)" fi # --- 7. diff is contained by the impact map ---------------------------------- @@ -187,6 +232,55 @@ else skip "RED artifact (no contract)" fi +# --- 10. STATE inbox does not reference finished work ------------------------ +# FEATURES.json rot is checked above; STATE.md rots the same way and nothing saw it. +if [ -f "$STATE" ] && [ -f FEATURES.json ]; then + result=$(python3 - "$STATE" <<'PYEOF' +import json, re, sys + +text = open(sys.argv[1], encoding="utf-8").read().splitlines() +start = None +for n, line in enumerate(text): + if re.match(r"^##+\s+Human-attention inbox", line, re.I): + start = n + 1 + break +if start is None: + print("SKIP no inbox section"); raise SystemExit + +section = [] +for line in text[start:]: + if re.match(r"^##+\s+", line): + break + section.append(line) + +feats = {f["id"]: f.get("status") for f in json.load(open("FEATURES.json")).get("features", [])} + +# Only ids that exist in this ledger can be stale. An id absent from the ledger is +# either planned work not yet created, or an id belonging to another repo, or part +# of a filename — none of which is rot. Flagging them was noise, not signal. +ids = set() +for line in section: + stripped = re.sub(r"`[^`]*`", " ", line) # drop backticked paths/filenames + stripped = re.sub(r"\S*/\S*", " ", stripped) # drop anything path-shaped + for m in re.findall(r"\b[A-Z][A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3}\b", stripped): + ids.add(m) + +done = sorted(i for i in ids if feats.get(i) == "PASS") +if done: + print("FAIL inbox references work already PASS: " + " ".join(done)) +else: + print("OK inbox references only open work") +PYEOF +) + case "$result" in + OK*) pass "STATE inbox: ${result#OK }" ;; + SKIP*) skip "STATE inbox (${result#SKIP })" ;; + *) fail "STATE ${result#FAIL }" ;; + esac +else + skip "STATE inbox (no STATE.md or FEATURES.json)" +fi + # --- 9. branch discipline ----------------------------------------------------- if git rev-parse --git-dir >/dev/null 2>&1; then cur=$(git branch --show-current 2>/dev/null || echo "") diff --git a/skills/.harness/contracts/KIT-GATES-002.md b/skills/.harness/contracts/KIT-GATES-002.md new file mode 100644 index 0000000..07bee68 --- /dev/null +++ b/skills/.harness/contracts/KIT-GATES-002.md @@ -0,0 +1,239 @@ +# SPRINT CONTRACT — KIT-GATES-002: remove the path heuristic, close the STATE drift + +> Status: **EXECUTE complete — VERIFY green (12/12) — PENDING_REVIEW.** +> Approved: David 2026-08-28 ("yes") with recommended Q1–Q2. Version **2.4.0**. +> Feature ID: `KIT-GATES-002` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-gates-002-tighten-gate`, **stacked on** +> `feat/kit-gates-001-deterministic-gates` (`734a8df`, PENDING_REVIEW). Rebase when that merges. +> Version → **2.4.0**. +> Trigger: findings from the gate's own first run against this repo. + +--- + +## Work-type profile (required) + +- [x] `ops-docs` + +**Languages / toolchain:** POSIX sh + Markdown. No application code. + +--- + +## Context — two findings from `verify-harness.sh` running against harness-kit + +**1. `[GROUNDED]` is overloaded, so check 6 has to guess.** + +The marker appears on Branch-section lines as well as impact-map rows: + +``` +- **Default branch:** `master`; base `ef393c3` (v2.1) — **[GROUNDED]** +``` + +On its first run check 6 reported `master` and `ef393c3` as missing paths. The patch shipped in +KIT-GATES-001 was a **heuristic** — accept a token only if it contains `/` or carries a known +extension. That is a guess, and it will misfire in both directions: a real path with no slash and no +extension (`Makefile`, `Dockerfile`) is silently skipped, and a backticked symbol that happens to end +in `.ts` is checked as a file. A check that silently skips is worse than one that fails loudly. + +**2. `STATE.md` drifts exactly like `FEATURES.json`, and nothing notices.** + +Two of six inbox entries in this repo were stale — work already `PASS`. The gate catches that class of +rot in the ledger; in `STATE.md` it verifies only that the named contract exists. Demonstrated below: +an inbox entry pointing at a `PASS` feature leaves the gate green. + +**Root cause in both:** the kit asked the script to interpret prose. The fix is to make the artifact +unambiguous and let the script read structure. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| `[GROUNDED]` scope | **Impact map only.** Stated in the contract form; the Branch section stops using it | Locked | +| Check 6 | **Parse the impact-map table**, first column, rows marked `[GROUNDED]`. Delete the extension heuristic | Locked | +| Silent skips | A `[GROUNDED]` row whose path cannot be parsed **fails**, rather than being skipped | Locked | +| New check 10 | No `STATE.md` inbox entry references a feature that is already `PASS` | Locked | +| Scope | These two findings only. No new gate concepts | Locked | + +--- + +## Scope — WILL do + +1. `sprint-contract` form — state that `[GROUNDED]` / `[EDUCATED]` / `[NEW]` mark **impact-map rows only**, + and remove the marker from the Branch section example so the template stops teaching the ambiguity. +2. `verify-harness.sh` check 6 — parse the impact-map table: take the first cell of each row whose line + carries `[GROUNDED]`, strip backticks, verify it exists. **Delete** the extension/slash heuristic. +3. Same check — a `[GROUNDED]` row whose first cell is not a parseable path is a **failure**, not a skip. +4. `verify-harness.sh` **check 10** — read the `Human-attention inbox` section of `STATE.md`; any + feature id mentioned there that is `PASS` in `FEATURES.json` is a failure, naming the ids. +5. Version **2.4.0**, changelog, `FEATURES.json` entry, bootstrap regenerated. + +## Scope — will NOT do + +- New gate concepts, coverage or metrics. +- Changes to CI, the RED/BASELINE/GREEN blocks, or any skill or rule. +- Anything in a consumer repo. +- Auto-repair: the gate reports, a human fixes. + +## Target + +- **Repo:** `harness-kit`. **Not touching:** consumer repos, application source. + +--- + +## Branch + +- **Base:** `feat/kit-gates-001-deterministic-gates` @ `734a8df` — needs check 6 to exist. +- **Proposed:** `feat/kit-gates-002-tighten-gate` +- **Human confirmed:** **pending.** + +--- + +## RED (recorded before EXECUTE) + +- **Command:** `grep -qi 'impact map only' skills/sprint-contract/templates/SPRINT_CONTRACT.md; grep -qi 'inbox' scripts/verify-harness.sh` +- **Exit code:** `1` +- **Failure reason:** assertion — expected the string `impact map only` in the contract form and found + none; expected an inbox check in the gate and found none. Check 6 still matches on + `*.md|*.sh|*.json|…` at `scripts/verify-harness.sh:137` rather than parsing the table. +- **Captured:** 2026-08-28, before any edit on this branch + +**Behavioural RED, demonstrated:** an inbox entry naming `KIT-RULES-001` (which is `PASS`) was injected +into `STATE.md`. The gate reported `PASS STATE.md current contract exists` and **exited 0**. It should +exit 1. Restored afterwards. + +**BASELINE** — the gate itself: `./scripts/verify-harness.sh` → exit 0, 9 pass / 0 skip at `734a8df`. + +## GREEN (recorded after EXECUTE) + +- Both RED assertions → **exit 0**: `impact map only` present in the form; `inbox` present in the gate +- Injected inbox entry naming `KIT-SKILLS-001` (a `PASS` feature) → gate **exit 1**, + `FAIL STATE inbox references work already PASS: KIT-SKILLS-001`. Restored → exit 0 +- `./scripts/verify-harness.sh` → **exit 0**, **10** checks reported, 0 skipped +- Contract VERIFY block → **12/12** + +--- + +## Impact map + +| Path | Change | Marker | +|------|--------|--------| +| `scripts/verify-harness.sh` | Check 6 parses the table; heuristic deleted; check 10 added | [GROUNDED] | +| `skills/sprint-contract/templates/SPRINT_CONTRACT.md` | `[GROUNDED]` restricted to the impact map | [GROUNDED] | +| `skills/.harness/contracts/KIT-GATES-002.md` | This contract | [NEW] | +| `scripts/build-bootstrap.sh` | 2.4.0 + changelog | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated | [GROUNDED] | +| `FEATURES.json` | Add `KIT-GATES-002` | [GROUNDED] | +| `README.md` | Version | [GROUNDED] | + +--- + +## Success criteria + +1. `scripts/verify-harness.sh` contains no extension-list heuristic — `grep -q '\*\.md|\*\.sh' ` finds nothing. +2. Check 6 resolves paths from the impact-map table; a malformed `[GROUNDED]` row **fails**, demonstrated. +3. Check 10 exists; a `STATE.md` inbox entry naming a `PASS` feature makes the gate exit 1, **demonstrated**. +4. Clean tree: gate exits 0 with **10** checks reported. +5. The contract form states `[GROUNDED]` is impact-map only, and its Branch example no longer uses the marker. +6. 2.4.0 everywhere; bootstrap regenerates; `FEATURES.json` parses; diff confined to the impact map. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — 2, 3 and 4 are executable +- [ ] Trust-boundary / http-api / solidity / code-quality — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor — **N/A** + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Table parsing is as brittle as the heuristic | It reads a fixed position — first cell of a marked row — rather than inferring meaning. Unparseable rows fail loudly | +| Older contracts lack a table-shaped impact map | Those rows fail rather than skip, which is the intended signal to fix the contract | +| Check 10 false-positives on prose mentioning a `PASS` id | Match feature-id shaped tokens only, inside the inbox section only | +| Gate grows into a linter | Scope explicitly forbids new concepts | + +## Threat model + +**SKIP** — no network, no credentials, no side effects. + +## Blocking questions + +**Q1 — Older contracts whose impact map is a bullet list, not a table.** Six tracked contracts predate +the table convention. +- **Recommendation:** check 6 applies only to rows inside a markdown table; a bullet-list impact map + reports `skipped (impact map not tabular)`. Loud enough to notice, not a wall for history. + +**Q2 — Should check 10 also fail on an inbox entry naming a feature that does not exist at all?** +- **Recommendation:** yes, same check, same failure. A typo'd id is the same rot. + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# heuristic gone, table parse in +! grep -q '\*\.md|\*\.sh' scripts/verify-harness.sh +grep -qi 'impact map' scripts/verify-harness.sh +sh -n scripts/verify-harness.sh + +# convention stated +grep -qi 'impact map only' skills/sprint-contract/templates/SPRINT_CONTRACT.md + +# check 10 present, and the gate reports ten checks on a clean tree +grep -qi 'inbox' scripts/verify-harness.sh +./scripts/verify-harness.sh +test "$(./scripts/verify-harness.sh | grep -cE '^ (PASS|FAIL|SKIP)')" -eq 10 + +# version + ledger +grep -q '2.4.0' BOOTSTRAP.md && sh scripts/build-bootstrap.sh +python3 -m json.tool FEATURES.json >/dev/null && grep -q 'KIT-GATES-002' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +## FEATURES.json entry + +```json +{ + "id": "KIT-GATES-002", + "name": "Gate reads structure not prose: impact-map parse replaces the path heuristic; STATE inbox drift check", + "priority": 2, + "verify": "! grep -q '[*].md|[*].sh' scripts/verify-harness.sh && grep -qi inbox scripts/verify-harness.sh && ./scripts/verify-harness.sh && grep -q '2.4.0' BOOTSTRAP.md && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-GATES-002.md; profile: ops-docs; stacked on KIT-GATES-001; from the gate's own first run" +} +``` + +## Criteria block + +```criteria +- [KIT-GATES-002-1] Extension heuristic removed from verify-harness.sh +- [KIT-GATES-002-2] Check 6 parses the impact-map table; malformed rows fail loudly +- [KIT-GATES-002-3] Check 10 catches a STATE inbox entry naming a PASS feature, demonstrated +- [KIT-GATES-002-4] Clean tree reports 10 checks and exits 0 +- [KIT-GATES-002-5] Contract form restricts [GROUNDED] to the impact map +- [KIT-GATES-002-6] 2.4.0; bootstrap regenerates; FEATURES.json valid +``` + +--- + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | **The contract assumed check 6 could scan for `[GROUNDED]` anywhere.** It cannot | The first marked table row in most contracts belongs to **Context** or **Decisions**, not the impact map — verified across all eight tracked contracts. The parser is **section-scoped**: it locates `## Impact map`, reads to the next heading, and takes the first cell of marked table rows only. Stronger than the contract specified | +| 2 | **Q2 was wrong, and the gate proved it in seconds.** I recommended failing on inbox ids absent from the ledger | On first run it flagged four, and **none was rot**: `AUDIT-001` belongs to another repo, `KIT-AUDIT-002` and `KIT-RECONCILE-001` are planned work deliberately named, and `PR-KIT-GATES-001` was a **filename** the regex matched inside. Narrowed to the half that is real signal — ids present in the ledger and already `PASS` — and taught it to strip backticked and path-shaped tokens first | +| 3 | First demonstration of check 10 used `KIT-RULES-001`, which is `PENDING_REVIEW`, not `PASS` | Correctly produced no failure. Re-run against `KIT-SKILLS-001` (`PASS`) — failed as designed. The test was wrong, not the check | +| 4 | Glob paths in impact maps (`rules/*.mdc`) | Handled: a first cell containing `*` is treated as a glob and must match at least one file | + +## Q1 as executed + +A bullet-list impact map reports `SKIP impact map not tabular`; a section with no marked rows reports +`SKIP no [GROUNDED] rows`. Both are printed, never swallowed. + +## Note for review + +`KIT-RULES-001` still reads `PENDING_REVIEW` in the ledger although it merged at `aea8fb2` — the same +drift class this sprint closes for `STATE.md`. `PASS` is human-only, so it is left for David. diff --git a/skills/sprint-contract/templates/SPRINT_CONTRACT.md b/skills/sprint-contract/templates/SPRINT_CONTRACT.md index b10c6d1..24b3225 100644 --- a/skills/sprint-contract/templates/SPRINT_CONTRACT.md +++ b/skills/sprint-contract/templates/SPRINT_CONTRACT.md @@ -50,7 +50,7 @@ Pick **one**: > Implementation on the default branch is forbidden. The human must confirm the branch name > before the agent creates or checks out the branch. -- **Default branch:** +- **Default branch:** - **Proposed feature branch:** `` - **Human confirmed:** @@ -112,8 +112,13 @@ Green-after means nothing without green-before. A count that moved is a regressi ## Impact map -> Mark each path **[GROUNDED]** (verified in repo) or **[EDUCATED]** (must re-verify before -> implementing). Never present educated guesses as grounded. +> Mark each path **[GROUNDED]** (verified in repo), **[EDUCATED]** (must re-verify before +> implementing), or **[NEW]**. Never present educated guesses as grounded. +> +> **These markers belong to the impact map only**, and the impact map is a table whose **first cell is +> the path in backticks**. The gate reads that structure: it parses this section's table rows rather +> than guessing which backticked token elsewhere in the contract is a path. A marker used outside this +> table, or a marked row whose first cell is not a backticked path, is a gate failure. - **Files to change:** - `` — — [GROUNDED|EDUCATED]