diff --git a/.github/workflows/harness.yml b/.github/workflows/harness.yml new file mode 100644 index 0000000..cc566f4 --- /dev/null +++ b/.github/workflows/harness.yml @@ -0,0 +1,38 @@ +name: harness + +# The unforgeable layer. Everything else in the kit is a instruction to a model +# that can misread it; this re-runs the checks itself, so an agent's claim that +# VERIFY passed stops mattering. +# +# It invokes scripts/verify-harness.sh — it never reimplements a check. + +on: + pull_request: + workflow_dispatch: + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # scope containment needs the merge base + + - name: Harness discipline + run: | + test -x scripts/verify-harness.sh || chmod +x scripts/verify-harness.sh + ./scripts/verify-harness.sh --base "origin/${{ github.base_ref }}" + + - name: Contract VERIFY block + run: | + # Re-run the contract's own verify commands. A local green is not evidence. + state=skills/.harness/STATE.md + [ -f "$state" ] || state=.harness/STATE.md + contract=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$state" 2>/dev/null | head -1) + if [ -z "$contract" ] || [ ! -f "$contract" ]; then + echo "no contract reachable — skipping contract VERIFY"; exit 0 + fi + echo "running VERIFY from $contract" + awk '/^## VERIFY/{f=1} f&&/^```(bash|sh)$/{c=1;next} c&&/^```$/{c=0} c' "$contract" > /tmp/verify.sh + if [ ! -s /tmp/verify.sh ]; then echo "contract has no VERIFY block"; exit 1; fi + sh -e /tmp/verify.sh diff --git a/.gitignore b/.gitignore index 211534e..322a5d9 100644 --- a/.gitignore +++ b/.gitignore @@ -13,11 +13,16 @@ Thumbs.db .env .env.* -# Ephemeral review packets — never committed (legacy path) -.harness/ +# Ephemeral review packets — never committed (legacy root path only). +# Anchored: unanchored '.harness/' also matched skills/.harness at any depth. +/.harness/ -# Harness working directory — contracts, local templates, session state -skills/.harness/ +# Harness working directory. Contracts are TRACKED: the review process grades a PR +# against its contract, and CI cannot check a file it cannot see. Session state, +# drafts and PR bodies stay local. +skills/.harness/* +!skills/.harness/contracts +!skills/.harness/contracts/** # Project instances (this repo ships TEMPLATES, not filled-in copies) # Uncomment if you keep a live filled AGENTS.md / FEATURES.json elsewhere diff --git a/BOOTSTRAP.md b/BOOTSTRAP.md index fdc9951..c2844df 100644 --- a/BOOTSTRAP.md +++ b/BOOTSTRAP.md @@ -1,13 +1,40 @@ # BOOTSTRAP.md — Harness Bootstrap -> **DEPRECATED — kit 2.2.0 adopts via git submodule; see README.md. This file is retained for +> **DEPRECATED — kit 2.4.0 adopts via git submodule; see README.md. This file is retained for > repos already bootstrapped from it and will be removed in KIT-SKILLS-002.** > -> **Kit version 2.2.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, +> **Kit version 2.4.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, > Rust, Go, Solidity build, and Solidity auditing. Copy this file into **any** repo. Prompt your LLM: > *"Read BOOTSTRAP.md and complete Step 0 — Bootstrap harness files."* -## v2.2 changelog +## v2.4 changelog + +- **The gate reads structure, not prose.** Check 6 parses the Impact map section's table — first cell + of each marked row — instead of guessing which backticked token is a path. The v2.3 heuristic + accepted a token only if it had a slash or a known extension, which silently skipped real paths + (`Makefile`) and checked things that were not paths. A check that silently skips is worse than one + that fails loudly, so an unparseable marked row now fails. +- **`[GROUNDED]` belongs to the impact map only** — stated in the contract form, which is what lets + the parser read position instead of inferring meaning. +- **Check 10: `STATE.md` inbox drift.** The ledger check caught rot in `FEATURES.json` while the same + rot in `STATE.md` went unseen. An inbox entry naming a feature already `PASS` now fails. + +## v2.3 changelog (retained) + +- **`scripts/verify-harness.sh`** — the third layer. Rules and skills are instructions to a model that + can misread them; this is a fact that exits non-zero. Nine checks: ledger integrity, every + `PENDING_REVIEW` has a contract, skills and rules will load, every `[GROUNDED]` path **exists**, + the diff is **contained by the impact map**, RED was recorded and failed on an **assertion**. +- **Prediction becomes recording** — the contract form's "Expected RED output" is replaced by recorded + **RED / BASELINE / GREEN** blocks. A test that never failed proves nothing. +- **No ERD, same gate** — without a design document each success criterion must be expressible as a + failing test before work starts, or the contract is not approvable. +- **CI re-runs VERIFY** (`.github/workflows/harness.yml`) — the only layer an agent cannot fake. One + script, invoked from CI and optionally from an editor hook; never reimplemented. +- **Contracts are tracked** — the review process grades a PR against its contract, and CI cannot check + a file it cannot see. Session state, drafts and PR bodies stay local. + +## v2.2 changelog (retained) - **`rules/` directory** — six always-on process-law rules as `.mdc`, linked into the editor's rules directory and assembled (frontmatter stripped) into a marked block in `AGENTS.md`, so agents without @@ -119,7 +146,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco | 1 | kit skills (symlinked by `setup-harness-kit.sh`) | Appendices B–P | N/A — canonical in the submodule | | 2 | `skills/.harness/contracts/` | — | Create empty directory | | 3 | `skills/.harness/STATE.md` | Appendix F | Fill `` + date | -| 4 | `skills/.harness/VERSION` | — | `version=2.2.0` + `bootstrapped=` | +| 4 | `skills/.harness/VERSION` | — | `version=2.4.0` + `bootstrapped=` | | 5 | `README.md` | Appendix G | **No** — overwrite with slim pointer (after brownfield recovery if needed) | | 6 | `AGENTS.md` | Appendix B | **Yes** if already filled — scaffold only; never overwrite harvested/verified content | | 7 | `FEATURES.json` | Appendix C | **Yes** if seeded — scaffold only | @@ -137,7 +164,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco **Done when:** -- `skills/.harness/` tree exists with VERSION `2.2.0` +- `skills/.harness/` tree exists with VERSION `2.4.0` - `python3 -m json.tool FEATURES.json` passes (if scaffolded) - `.gitignore` contains `skills/.harness/` - No `HARNESS.md` in repo @@ -457,6 +484,20 @@ is **N/A** for `ops-docs`, `code-audit` and `product-erd`, and for `solidity-aud ship code of their own. Credential findings belong to `SECURITY_CHECKLIST.md`, not this gate — raise them there and do not double-report. +Then run the gate: + +```sh +./scripts/verify-harness.sh # or the path this repo declares +``` + +It checks the discipline rather than the code: ledger integrity, that every `PENDING_REVIEW` has a +contract, that skills and rules will actually load, that every `[GROUNDED]` path in the contract +**exists**, that the diff is **contained by the impact map**, and that RED was recorded and failed on +an assertion. **The executor may set `PENDING_REVIEW` only when this exits 0.** + +Rules and skills are instructions to a model that can misread them under load. This is a fact — and +**CI runs it again**, so a claim that it passed locally is not evidence. + If verify fails: stay `FAIL`, fix or revise the contract. #### 9. REVIEW → PENDING_REVIEW → HUMAN PASS @@ -819,7 +860,7 @@ Pick **one**: > Implementation on the default branch is forbidden. The human must confirm the branch name > before the agent creates or checks out the branch. -- **Default branch:** +- **Default branch:** - **Proposed feature branch:** `` - **Human confirmed:** @@ -841,10 +882,34 @@ Pick **one**: - **Test files to create or extend:** - `` — -- **Expected RED output:** -- **Verify commands (GREEN):** - - `` - - `` +> **A test that never failed proves nothing.** It may be vacuous, or already passing. Record the real +> run — command, exit code, and the failure message — not a prediction of it. A RED that failed on an +> import or path error proves the test file is wrong, not that the behaviour is absent. +> +> **No ERD? Same gate.** Without a design document the success criteria below are the whole story, and +> each must be expressible as a **failing test before work starts**. If you cannot write a test that +> fails today, you do not have a falsifiable criterion and this contract is not approvable. + +### RED (recorded before EXECUTE) + +- **Command:** `` +- **Exit code:** `` +- **Failure reason:** `` +- **Captured:** `` + +**N/A** only when this sprint changes no behaviour (`ops-docs`, `code-audit`, `product-erd`). + +### BASELINE (recorded before EXECUTE — behaviour changes only) + +- **Command:** `` +- **Result:** `` + +### GREEN (recorded after EXECUTE) + +- **Command:** `` → **exit 0** +- **Command:** `` → **``** + +Green-after means nothing without green-before. A count that moved is a regression until explained. ### Vertical slice (when non-trivial) @@ -857,8 +922,13 @@ Pick **one**: ## Impact map -> Mark each path **[GROUNDED]** (verified in repo) or **[EDUCATED]** (must re-verify before -> implementing). Never present educated guesses as grounded. +> Mark each path **[GROUNDED]** (verified in repo), **[EDUCATED]** (must re-verify before +> implementing), or **[NEW]**. Never present educated guesses as grounded. +> +> **These markers belong to the impact map only**, and the impact map is a table whose **first cell is +> the path in backticks**. The gate reads that structure: it parses this section's table rows rather +> than guessing which backticked token elsewhere in the contract is a path. A marker used outside this +> table, or a marked row whose first cell is not a backticked path, is a gate failure. - **Files to change:** - `` — — [GROUNDED|EDUCATED] @@ -1096,6 +1166,11 @@ Before deep code review, locate and read the relevant contract: code-quality gate (`CODE_QUALITY.md`: lint command recorded, violations = 0 or documented), solidity-audit PoCs. Mark each **Pass / Fail / N/A**. 8. **VERIFY commands** — run or confirm the contract's verify block passed; note any skipped or failing commands. +9. **Gate artifacts** — confirm the contract carries a recorded **RED** (real command, non-zero exit, + assertion-shaped failure), a **BASELINE**, and a **GREEN** whose counts match the baseline. A + predicted RED, or one that failed on an import error, is a finding. `N/A` is only valid when the + sprint changed no behaviour. +10. **Gate ran** — confirm `verify-harness.sh` exited 0 **in CI**, not only locally. If no contract exists (hotfix, drive-by), state that explicitly and review on production-readiness only. Retroactive contract may be required before PASS. @@ -1608,6 +1683,9 @@ explicitly approves that scope. > `CODE_AUDIT_CONTRACT.md` asks "where should we invest refactoring effort?" on a cadence, over a > declared subsystem. Do not substitute one for the other. +> Process discipline — ledger, contract, scope containment, RED artifact — is checked by +> `verify-harness.sh`, not here. This gate judges the code; that one judges the contract. + ## 0. Sensor - [ ] Repo's declared lint / static-analysis command from `AGENTS.md` was **run** on the diff @@ -2421,6 +2499,359 @@ Kit upgrades are a submodule pointer bump and a setup re-run. Never edit these f --- +## Appendix R — verify-harness.sh (the gate) + +#!/bin/sh +# verify-harness.sh — deterministic checks on harness discipline. +# +# Rules and skills are instructions to a model that can misread them under load. +# This is a fact. Run it from a repo root; it exits non-zero on any violation. +# +# ./verify-harness.sh # skip contract checks when no contract is reachable +# ./verify-harness.sh --strict # a skip becomes a failure +# ./verify-harness.sh --contract # check a specific contract +# ./verify-harness.sh --base # diff base for scope containment (default: origin/HEAD) +# +# CI and any editor hook invoke THIS script. Never reimplement a check elsewhere. +set -u + +STRICT=0; CONTRACT=""; BASE="" +while [ $# -gt 0 ]; do + case "$1" in + --strict) STRICT=1 ;; + --contract) shift; CONTRACT="${1:-}" ;; + --base) shift; BASE="${1:-}" ;; + -h|--help) sed -n '2,14p' "$0"; exit 0 ;; + *) echo "unknown argument: $1" >&2; exit 2 ;; + esac + shift +done + +fails=0; skips=0 +pass() { printf ' PASS %s\n' "$1"; } +fail() { printf ' FAIL %s\n' "$1"; fails=$((fails + 1)); } +skip() { + if [ "$STRICT" -eq 1 ]; then fail "$1 (skipped, and --strict is on)" + else printf ' SKIP %s\n' "$1"; skips=$((skips + 1)); fi +} + +CONTRACTS_DIR=skills/.harness/contracts +[ -d "$CONTRACTS_DIR" ] || CONTRACTS_DIR=.harness/contracts +STATE=skills/.harness/STATE.md +[ -f "$STATE" ] || STATE=.harness/STATE.md + +echo "harness verify" + +# --- 1. ledger integrity ------------------------------------------------------ +if [ -f FEATURES.json ]; then + if python3 -m json.tool FEATURES.json >/dev/null 2>&1; then + bad=$(python3 - <<'PY' +import json +d = json.load(open("FEATURES.json")) +legend = set(d.get("legend", {}).get("status") or ["FAIL", "PENDING_REVIEW", "PASS"]) +out = [] +for f in d.get("features", []): + fid = f.get("id", "") + if f.get("status") not in legend: + out.append("%s: status %r not in legend" % (fid, f.get("status"))) + v = (f.get("verify") or "").strip() + if not v: + out.append("%s: empty verify" % fid) + elif not any(c in v for c in "&|;$/") and " " not in v: + out.append("%s: verify does not look like a command" % fid) +print("\n".join(out)) +PY +) + if [ -z "$bad" ]; then pass "FEATURES.json: parses, statuses legal, verify fields runnable" + else printf '%s\n' "$bad" | while IFS= read -r l; do [ -n "$l" ] && printf ' %s\n' "$l"; done + fail "FEATURES.json: invalid entries (above)"; fi + else + fail "FEATURES.json: does not parse" + fi +else + skip "FEATURES.json: not present" +fi + +# --- 2. every PENDING_REVIEW has a contract ---------------------------------- +if [ -f FEATURES.json ] && [ -d "$CONTRACTS_DIR" ]; then + missing=$(python3 - "$CONTRACTS_DIR" <<'PY' +import json, os, sys +d = json.load(open("FEATURES.json")) +out = [f["id"] for f in d.get("features", []) + if f.get("status") == "PENDING_REVIEW" + and not os.path.exists(os.path.join(sys.argv[1], f["id"] + ".md"))] +print(" ".join(out)) +PY +) + if [ -z "$missing" ]; then pass "every PENDING_REVIEW feature has a contract file" + else fail "PENDING_REVIEW without a contract: $missing"; fi +else + skip "PENDING_REVIEW/contract cross-check" +fi + +# --- 3. STATE points at a real contract -------------------------------------- +if [ -f "$STATE" ]; then + ref=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$STATE" | head -1) + if [ -z "$ref" ]; then skip "STATE.md names no contract" + elif [ -f "$ref" ]; then pass "STATE.md current contract exists ($ref)" + else fail "STATE.md points at a missing contract: $ref"; fi +else + skip "STATE.md not present" +fi + +# --- 4/5. skills and rules load ---------------------------------------------- +if [ -d skills ]; then + bad="" + for s in skills/*/SKILL.md; do + [ -f "$s" ] || continue + head -1 "$s" | grep -q '^---$' && grep -q '^name:' "$s" && grep -q '^description:' "$s" || bad="$bad $s" + done + if [ -z "$bad" ]; then pass "skills: frontmatter valid (retrievable)" + else fail "skills with bad frontmatter:$bad"; fi +else + skip "no skills directory" +fi + +if [ -d rules ]; then + bad="" + for r in rules/*.mdc; do + [ -f "$r" ] || continue + head -1 "$r" | grep -q '^---$' && grep -q '^description:' "$r" && grep -q '^alwaysApply:' "$r" || bad="$bad $r" + done + if [ -z "$bad" ]; then pass "rules: frontmatter valid (will load)" + else fail "rules with bad frontmatter:$bad"; fi +else + skip "no rules directory" +fi + +# --- resolve the active contract --------------------------------------------- +contract_file="" +if [ -n "$CONTRACT" ]; then + contract_file="$CONTRACTS_DIR/$CONTRACT.md" + [ -f "$contract_file" ] || contract_file="$CONTRACT" +elif [ -f "$STATE" ]; then + ref=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$STATE" | head -1) + [ -n "$ref" ] && [ -f "$ref" ] && contract_file="$ref" +fi + +# --- 6. grounded paths exist (catches invented files) ------------------------ +# Reads STRUCTURE, not prose: the Impact map section only, table rows only, first +# cell only. Scanning for [GROUNDED] anywhere also picks up Context and Decisions +# tables, and guessing which backticked token is a path misfires both ways. +if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then + result=$(python3 - "$contract_file" <<'PYEOF' +import re, sys, glob, os + +text = open(sys.argv[1], encoding="utf-8").read().splitlines() + +# isolate the Impact map section +start = None +for n, line in enumerate(text): + if re.match(r"^##+\s+Impact map", line, re.I): + start = n + 1 + break +if start is None: + print("SKIP no Impact map section") + raise SystemExit + +section = [] +for line in text[start:]: + if re.match(r"^##+\s+", line): + break + section.append(line) + +rows = [l for l in section if l.lstrip().startswith("|") and "[GROUNDED]" in l] +if not rows: + if any(l.lstrip().startswith("-") for l in section): + print("SKIP impact map not tabular") + else: + print("SKIP no [GROUNDED] rows") + raise SystemExit + +missing, malformed = [], [] +for row in rows: + cells = [c.strip() for c in row.strip().strip("|").split("|")] + if not cells: + malformed.append(row.strip()[:60]); continue + m = re.search(r"`([^`]+)`", cells[0]) + if not m: + malformed.append(cells[0][:60]); continue + path = m.group(1).strip() + if "*" in path: + if not glob.glob(path): + missing.append(path) + elif not os.path.exists(path): + missing.append(path) + +if malformed: + print("FAIL unparseable [GROUNDED] rows: " + "; ".join(malformed)) +elif missing: + print("FAIL paths do not exist: " + " ".join(missing)) +else: + print("OK %d [GROUNDED] path(s) verified" % len(rows)) +PYEOF +) + case "$result" in + OK*) pass "contract impact map: ${result#OK }" ;; + SKIP*) skip "contract impact map (${result#SKIP })" ;; + *) fail "contract impact map: ${result#FAIL }" ;; + esac +else + skip "contract impact map (no contract)" +fi + +# --- 7. diff is contained by the impact map ---------------------------------- +if [ -n "$contract_file" ] && [ -f "$contract_file" ] && git rev-parse --git-dir >/dev/null 2>&1; then + base="$BASE" + [ -n "$base" ] || base=$(git rev-parse --abbrev-ref origin/HEAD 2>/dev/null || echo origin/master) + if git rev-parse --verify -q "$base" >/dev/null; then + undeclared="" + for f in $(git diff --name-only "$base"...HEAD 2>/dev/null); do + grep -qF "$f" "$contract_file" || undeclared="$undeclared $f" + done + if [ -z "$undeclared" ]; then pass "diff is contained by the contract impact map" + else fail "changed but not declared in the contract:$undeclared"; fi + else + skip "scope containment (base $base unresolvable)" + fi +else + skip "scope containment (no contract)" +fi + +# --- 8. RED was recorded, and failed for the right reason -------------------- +if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then + if grep -q 'RED (recorded before EXECUTE)' "$contract_file"; then + if grep -qi 'N/A' "$(printf %s "$contract_file")" && grep -A4 'RED (recorded before EXECUTE)' "$contract_file" | grep -qi 'N/A'; then + pass "RED: declared N/A (no behaviour change)" + elif grep -A6 'RED (recorded before EXECUTE)' "$contract_file" | grep -qiE 'assert|expect|to (be|equal)|assertion'; then + pass "RED: recorded and failed on an assertion" + else + fail "RED: recorded but no assertion-shaped failure — a test that never failed proves nothing" + fi + else + skip "RED artifact (contract predates the recording format)" + fi +else + skip "RED artifact (no contract)" +fi + +# --- 10. STATE inbox does not reference finished work ------------------------ +# FEATURES.json rot is checked above; STATE.md rots the same way and nothing saw it. +if [ -f "$STATE" ] && [ -f FEATURES.json ]; then + result=$(python3 - "$STATE" <<'PYEOF' +import json, re, sys + +text = open(sys.argv[1], encoding="utf-8").read().splitlines() +start = None +for n, line in enumerate(text): + if re.match(r"^##+\s+Human-attention inbox", line, re.I): + start = n + 1 + break +if start is None: + print("SKIP no inbox section"); raise SystemExit + +section = [] +for line in text[start:]: + if re.match(r"^##+\s+", line): + break + section.append(line) + +feats = {f["id"]: f.get("status") for f in json.load(open("FEATURES.json")).get("features", [])} + +# Only ids that exist in this ledger can be stale. An id absent from the ledger is +# either planned work not yet created, or an id belonging to another repo, or part +# of a filename — none of which is rot. Flagging them was noise, not signal. +ids = set() +for line in section: + stripped = re.sub(r"`[^`]*`", " ", line) # drop backticked paths/filenames + stripped = re.sub(r"\S*/\S*", " ", stripped) # drop anything path-shaped + for m in re.findall(r"\b[A-Z][A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3}\b", stripped): + ids.add(m) + +done = sorted(i for i in ids if feats.get(i) == "PASS") +if done: + print("FAIL inbox references work already PASS: " + " ".join(done)) +else: + print("OK inbox references only open work") +PYEOF +) + case "$result" in + OK*) pass "STATE inbox: ${result#OK }" ;; + SKIP*) skip "STATE inbox (${result#SKIP })" ;; + *) fail "STATE ${result#FAIL }" ;; + esac +else + skip "STATE inbox (no STATE.md or FEATURES.json)" +fi + +# --- 9. branch discipline ----------------------------------------------------- +if git rev-parse --git-dir >/dev/null 2>&1; then + cur=$(git branch --show-current 2>/dev/null || echo "") + def=$(git rev-parse --abbrev-ref origin/HEAD 2>/dev/null | sed 's|^origin/||') + [ -n "$def" ] || def=master + if [ -z "$cur" ]; then skip "branch discipline (detached HEAD)" + elif [ "$cur" != "$def" ]; then pass "on a feature branch ($cur)" + else + if [ -z "$(git status --porcelain 2>/dev/null)" ]; then pass "on $def and clean" + else fail "uncommitted implementation on the default branch ($def)"; fi + fi +else + skip "branch discipline (not a git repo)" +fi + +echo +if [ "$fails" -gt 0 ]; then + echo "harness verify: $fails failure(s), $skips skipped" + exit 1 +fi +echo "harness verify: green ($skips skipped)" +exit 0 + +--- + +## Appendix S — CI workflow (re-runs the gate and the contract VERIFY block) + +name: harness + +# The unforgeable layer. Everything else in the kit is a instruction to a model +# that can misread it; this re-runs the checks itself, so an agent's claim that +# VERIFY passed stops mattering. +# +# It invokes scripts/verify-harness.sh — it never reimplements a check. + +on: + pull_request: + workflow_dispatch: + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # scope containment needs the merge base + + - name: Harness discipline + run: | + test -x scripts/verify-harness.sh || chmod +x scripts/verify-harness.sh + ./scripts/verify-harness.sh --base "origin/${{ github.base_ref }}" + + - name: Contract VERIFY block + run: | + # Re-run the contract's own verify commands. A local green is not evidence. + state=skills/.harness/STATE.md + [ -f "$state" ] || state=.harness/STATE.md + contract=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$state" 2>/dev/null | head -1) + if [ -z "$contract" ] || [ ! -f "$contract" ]; then + echo "no contract reachable — skipping contract VERIFY"; exit 0 + fi + echo "running VERIFY from $contract" + awk '/^## VERIFY/{f=1} f&&/^```(bash|sh)$/{c=1;next} c&&/^```$/{c=0} c' "$contract" > /tmp/verify.sh + if [ ! -s /tmp/verify.sh ]; then echo "contract has no VERIFY block"; exit 1; fi + sh -e /tmp/verify.sh + +--- + ## Kit repo note (maintainers) *Maintainers: edit `templates/*` and this file's Step 0 / paths sections, then run diff --git a/FEATURES.json b/FEATURES.json index 6d267ef..73b4cb4 100644 --- a/FEATURES.json +++ b/FEATURES.json @@ -13,6 +13,22 @@ "verify": "runnable test command or command sequence — not prose" }, "features": [ + { + "id": "KIT-GATES-002", + "name": "Gate reads structure not prose: impact-map parse replaces the path heuristic; STATE inbox drift check", + "priority": 2, + "verify": "! grep -q '[*].md|[*].sh' scripts/verify-harness.sh && grep -qi inbox scripts/verify-harness.sh && ./scripts/verify-harness.sh && grep -q '2.4.0' BOOTSTRAP.md && python3 -m json.tool FEATURES.json >/dev/null", + "status": "PENDING_REVIEW", + "notes": "Contract: skills/.harness/contracts/KIT-GATES-002.md; profile: ops-docs; EXECUTE+VERIFY green 2026-08-28 on feat/kit-gates-002-tighten-gate; 4 declared deviations incl. Q2 corrected by the gate itself" + }, + { + "id": "KIT-GATES-001", + "name": "Deterministic gates: RED/BASELINE/GREEN artifacts, verify-harness.sh, CI re-running VERIFY", + "priority": 1, + "verify": "test -x scripts/verify-harness.sh && ./scripts/verify-harness.sh && test -f .github/workflows/harness.yml && ! grep -q 'Expected RED output' skills/sprint-contract/templates/SPRINT_CONTRACT.md && grep -q '2.3.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "PENDING_REVIEW", + "notes": "Contract: skills/.harness/contracts/KIT-GATES-001.md; profile: ops-docs; EXECUTE+VERIFY green 2026-08-28 on feat/kit-gates-001-deterministic-gates; contracts now tracked; awaiting human PASS" + }, { "id": "KIT-RULES-001", "name": "rules/ directory as .mdc with AGENTS.md assembly, plus ERD routing before contract drafting", diff --git a/README.md b/README.md index 9f3b3ae..16cd50f 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ A reusable, **tool-agnostic** framework for AI-assisted engineering — contract-first, TDD when behaviour changes, multi-language (TypeScript, Rust, Go, Solidity build + audit). -**Kit v2.2** — the process ships as **skills**, adopted as a **git submodule**. Skills stay canonical +**Kit v2.4** — the process ships as **skills**, adopted as a **git submodule**. Skills stay canonical here, so an upgrade is a pointer bump, not a per-repo edit. ## Adopt @@ -24,6 +24,20 @@ git submodule update --remote harness-kit ./harness-kit/scripts/setup-harness-kit.sh ``` +## Gates + +```sh +./scripts/verify-harness.sh # exits non-zero on any violation +./scripts/verify-harness.sh --strict # skips become failures +``` + +Rules and skills are instructions to a model that can misread them under load. A gate is a fact. +It checks ledger integrity, that every `PENDING_REVIEW` has a contract, that skills and rules will +load, that every `[GROUNDED]` path exists, that the diff is contained by the impact map, and that RED +was recorded and failed on an assertion. **CI runs it again** — a local green is not evidence. + +> Gate it if a command can enforce it, skill it if it is procedural, rule it only if it holds everywhere. + ## Rules Six always-on constraints in `rules/*.mdc`, linked into the editor's rules directory and assembled into diff --git a/scripts/build-bootstrap.sh b/scripts/build-bootstrap.sh index ff63e8f..86c2fb1 100755 --- a/scripts/build-bootstrap.sh +++ b/scripts/build-bootstrap.sh @@ -1,5 +1,5 @@ #!/bin/sh -# DEPRECATED as of kit 2.2.0 — adoption moved to git submodule + scripts/setup-harness-kit.sh. +# DEPRECATED as of kit 2.4.0 — adoption moved to git submodule + scripts/setup-harness-kit.sh. # Retained so repos bootstrapped from BOOTSTRAP.md keep working. Removal: KIT-SKILLS-002. # Generates BOOTSTRAP.md from the forms now bundled inside skills//templates/. set -eu @@ -17,14 +17,41 @@ append() { cat <<'HEADER' # BOOTSTRAP.md — Harness Bootstrap -> **DEPRECATED — kit 2.2.0 adopts via git submodule; see README.md. This file is retained for +> **DEPRECATED — kit 2.4.0 adopts via git submodule; see README.md. This file is retained for > repos already bootstrapped from it and will be removed in KIT-SKILLS-002.** > -> **Kit version 2.2.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, +> **Kit version 2.4.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript, > Rust, Go, Solidity build, and Solidity auditing. Copy this file into **any** repo. Prompt your LLM: > *"Read BOOTSTRAP.md and complete Step 0 — Bootstrap harness files."* -## v2.2 changelog +## v2.4 changelog + +- **The gate reads structure, not prose.** Check 6 parses the Impact map section's table — first cell + of each marked row — instead of guessing which backticked token is a path. The v2.3 heuristic + accepted a token only if it had a slash or a known extension, which silently skipped real paths + (`Makefile`) and checked things that were not paths. A check that silently skips is worse than one + that fails loudly, so an unparseable marked row now fails. +- **`[GROUNDED]` belongs to the impact map only** — stated in the contract form, which is what lets + the parser read position instead of inferring meaning. +- **Check 10: `STATE.md` inbox drift.** The ledger check caught rot in `FEATURES.json` while the same + rot in `STATE.md` went unseen. An inbox entry naming a feature already `PASS` now fails. + +## v2.3 changelog (retained) + +- **`scripts/verify-harness.sh`** — the third layer. Rules and skills are instructions to a model that + can misread them; this is a fact that exits non-zero. Nine checks: ledger integrity, every + `PENDING_REVIEW` has a contract, skills and rules will load, every `[GROUNDED]` path **exists**, + the diff is **contained by the impact map**, RED was recorded and failed on an **assertion**. +- **Prediction becomes recording** — the contract form's "Expected RED output" is replaced by recorded + **RED / BASELINE / GREEN** blocks. A test that never failed proves nothing. +- **No ERD, same gate** — without a design document each success criterion must be expressible as a + failing test before work starts, or the contract is not approvable. +- **CI re-runs VERIFY** (`.github/workflows/harness.yml`) — the only layer an agent cannot fake. One + script, invoked from CI and optionally from an editor hook; never reimplemented. +- **Contracts are tracked** — the review process grades a PR against its contract, and CI cannot check + a file it cannot see. Session state, drafts and PR bodies stay local. + +## v2.2 changelog (retained) - **`rules/` directory** — six always-on process-law rules as `.mdc`, linked into the editor's rules directory and assembled (frontmatter stripped) into a marked block in `AGENTS.md`, so agents without @@ -136,7 +163,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco | 1 | kit skills (symlinked by `setup-harness-kit.sh`) | Appendices B–P | N/A — canonical in the submodule | | 2 | `skills/.harness/contracts/` | — | Create empty directory | | 3 | `skills/.harness/STATE.md` | Appendix F | Fill `` + date | -| 4 | `skills/.harness/VERSION` | — | `version=2.2.0` + `bootstrapped=` | +| 4 | `skills/.harness/VERSION` | — | `version=2.4.0` + `bootstrapped=` | | 5 | `README.md` | Appendix G | **No** — overwrite with slim pointer (after brownfield recovery if needed) | | 6 | `AGENTS.md` | Appendix B | **Yes** if already filled — scaffold only; never overwrite harvested/verified content | | 7 | `FEATURES.json` | Appendix C | **Yes** if seeded — scaffold only | @@ -154,7 +181,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco **Done when:** -- `skills/.harness/` tree exists with VERSION `2.2.0` +- `skills/.harness/` tree exists with VERSION `2.4.0` - `python3 -m json.tool FEATURES.json` passes (if scaffolded) - `.gitignore` contains `skills/.harness/` - No `HARNESS.md` in repo @@ -257,6 +284,8 @@ HEADER append "Appendix O — ERD.md template" skills/erd-authoring/templates/ERD.md append "Appendix P — ARCHITECTURE.md template" skills/erd-authoring/templates/ARCHITECTURE.md append "Appendix Q — rules/README.md (rule budget + ownership tiers)" rules/README.md + append "Appendix R — verify-harness.sh (the gate)" scripts/verify-harness.sh + append "Appendix S — CI workflow (re-runs the gate and the contract VERIFY block)" .github/workflows/harness.yml cat <<'FOOTER' diff --git a/scripts/verify-harness.sh b/scripts/verify-harness.sh new file mode 100755 index 0000000..a6e594c --- /dev/null +++ b/scripts/verify-harness.sh @@ -0,0 +1,305 @@ +#!/bin/sh +# verify-harness.sh — deterministic checks on harness discipline. +# +# Rules and skills are instructions to a model that can misread them under load. +# This is a fact. Run it from a repo root; it exits non-zero on any violation. +# +# ./verify-harness.sh # skip contract checks when no contract is reachable +# ./verify-harness.sh --strict # a skip becomes a failure +# ./verify-harness.sh --contract # check a specific contract +# ./verify-harness.sh --base # diff base for scope containment (default: origin/HEAD) +# +# CI and any editor hook invoke THIS script. Never reimplement a check elsewhere. +set -u + +STRICT=0; CONTRACT=""; BASE="" +while [ $# -gt 0 ]; do + case "$1" in + --strict) STRICT=1 ;; + --contract) shift; CONTRACT="${1:-}" ;; + --base) shift; BASE="${1:-}" ;; + -h|--help) sed -n '2,14p' "$0"; exit 0 ;; + *) echo "unknown argument: $1" >&2; exit 2 ;; + esac + shift +done + +fails=0; skips=0 +pass() { printf ' PASS %s\n' "$1"; } +fail() { printf ' FAIL %s\n' "$1"; fails=$((fails + 1)); } +skip() { + if [ "$STRICT" -eq 1 ]; then fail "$1 (skipped, and --strict is on)" + else printf ' SKIP %s\n' "$1"; skips=$((skips + 1)); fi +} + +CONTRACTS_DIR=skills/.harness/contracts +[ -d "$CONTRACTS_DIR" ] || CONTRACTS_DIR=.harness/contracts +STATE=skills/.harness/STATE.md +[ -f "$STATE" ] || STATE=.harness/STATE.md + +echo "harness verify" + +# --- 1. ledger integrity ------------------------------------------------------ +if [ -f FEATURES.json ]; then + if python3 -m json.tool FEATURES.json >/dev/null 2>&1; then + bad=$(python3 - <<'PY' +import json +d = json.load(open("FEATURES.json")) +legend = set(d.get("legend", {}).get("status") or ["FAIL", "PENDING_REVIEW", "PASS"]) +out = [] +for f in d.get("features", []): + fid = f.get("id", "") + if f.get("status") not in legend: + out.append("%s: status %r not in legend" % (fid, f.get("status"))) + v = (f.get("verify") or "").strip() + if not v: + out.append("%s: empty verify" % fid) + elif not any(c in v for c in "&|;$/") and " " not in v: + out.append("%s: verify does not look like a command" % fid) +print("\n".join(out)) +PY +) + if [ -z "$bad" ]; then pass "FEATURES.json: parses, statuses legal, verify fields runnable" + else printf '%s\n' "$bad" | while IFS= read -r l; do [ -n "$l" ] && printf ' %s\n' "$l"; done + fail "FEATURES.json: invalid entries (above)"; fi + else + fail "FEATURES.json: does not parse" + fi +else + skip "FEATURES.json: not present" +fi + +# --- 2. every PENDING_REVIEW has a contract ---------------------------------- +if [ -f FEATURES.json ] && [ -d "$CONTRACTS_DIR" ]; then + missing=$(python3 - "$CONTRACTS_DIR" <<'PY' +import json, os, sys +d = json.load(open("FEATURES.json")) +out = [f["id"] for f in d.get("features", []) + if f.get("status") == "PENDING_REVIEW" + and not os.path.exists(os.path.join(sys.argv[1], f["id"] + ".md"))] +print(" ".join(out)) +PY +) + if [ -z "$missing" ]; then pass "every PENDING_REVIEW feature has a contract file" + else fail "PENDING_REVIEW without a contract: $missing"; fi +else + skip "PENDING_REVIEW/contract cross-check" +fi + +# --- 3. STATE points at a real contract -------------------------------------- +if [ -f "$STATE" ]; then + ref=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$STATE" | head -1) + if [ -z "$ref" ]; then skip "STATE.md names no contract" + elif [ -f "$ref" ]; then pass "STATE.md current contract exists ($ref)" + else fail "STATE.md points at a missing contract: $ref"; fi +else + skip "STATE.md not present" +fi + +# --- 4/5. skills and rules load ---------------------------------------------- +if [ -d skills ]; then + bad="" + for s in skills/*/SKILL.md; do + [ -f "$s" ] || continue + head -1 "$s" | grep -q '^---$' && grep -q '^name:' "$s" && grep -q '^description:' "$s" || bad="$bad $s" + done + if [ -z "$bad" ]; then pass "skills: frontmatter valid (retrievable)" + else fail "skills with bad frontmatter:$bad"; fi +else + skip "no skills directory" +fi + +if [ -d rules ]; then + bad="" + for r in rules/*.mdc; do + [ -f "$r" ] || continue + head -1 "$r" | grep -q '^---$' && grep -q '^description:' "$r" && grep -q '^alwaysApply:' "$r" || bad="$bad $r" + done + if [ -z "$bad" ]; then pass "rules: frontmatter valid (will load)" + else fail "rules with bad frontmatter:$bad"; fi +else + skip "no rules directory" +fi + +# --- resolve the active contract --------------------------------------------- +contract_file="" +if [ -n "$CONTRACT" ]; then + contract_file="$CONTRACTS_DIR/$CONTRACT.md" + [ -f "$contract_file" ] || contract_file="$CONTRACT" +elif [ -f "$STATE" ]; then + ref=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$STATE" | head -1) + [ -n "$ref" ] && [ -f "$ref" ] && contract_file="$ref" +fi + +# --- 6. grounded paths exist (catches invented files) ------------------------ +# Reads STRUCTURE, not prose: the Impact map section only, table rows only, first +# cell only. Scanning for [GROUNDED] anywhere also picks up Context and Decisions +# tables, and guessing which backticked token is a path misfires both ways. +if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then + result=$(python3 - "$contract_file" <<'PYEOF' +import re, sys, glob, os + +text = open(sys.argv[1], encoding="utf-8").read().splitlines() + +# isolate the Impact map section +start = None +for n, line in enumerate(text): + if re.match(r"^##+\s+Impact map", line, re.I): + start = n + 1 + break +if start is None: + print("SKIP no Impact map section") + raise SystemExit + +section = [] +for line in text[start:]: + if re.match(r"^##+\s+", line): + break + section.append(line) + +rows = [l for l in section if l.lstrip().startswith("|") and "[GROUNDED]" in l] +if not rows: + if any(l.lstrip().startswith("-") for l in section): + print("SKIP impact map not tabular") + else: + print("SKIP no [GROUNDED] rows") + raise SystemExit + +missing, malformed = [], [] +for row in rows: + cells = [c.strip() for c in row.strip().strip("|").split("|")] + if not cells: + malformed.append(row.strip()[:60]); continue + m = re.search(r"`([^`]+)`", cells[0]) + if not m: + malformed.append(cells[0][:60]); continue + path = m.group(1).strip() + if "*" in path: + if not glob.glob(path): + missing.append(path) + elif not os.path.exists(path): + missing.append(path) + +if malformed: + print("FAIL unparseable [GROUNDED] rows: " + "; ".join(malformed)) +elif missing: + print("FAIL paths do not exist: " + " ".join(missing)) +else: + print("OK %d [GROUNDED] path(s) verified" % len(rows)) +PYEOF +) + case "$result" in + OK*) pass "contract impact map: ${result#OK }" ;; + SKIP*) skip "contract impact map (${result#SKIP })" ;; + *) fail "contract impact map: ${result#FAIL }" ;; + esac +else + skip "contract impact map (no contract)" +fi + +# --- 7. diff is contained by the impact map ---------------------------------- +if [ -n "$contract_file" ] && [ -f "$contract_file" ] && git rev-parse --git-dir >/dev/null 2>&1; then + base="$BASE" + [ -n "$base" ] || base=$(git rev-parse --abbrev-ref origin/HEAD 2>/dev/null || echo origin/master) + if git rev-parse --verify -q "$base" >/dev/null; then + undeclared="" + for f in $(git diff --name-only "$base"...HEAD 2>/dev/null); do + grep -qF "$f" "$contract_file" || undeclared="$undeclared $f" + done + if [ -z "$undeclared" ]; then pass "diff is contained by the contract impact map" + else fail "changed but not declared in the contract:$undeclared"; fi + else + skip "scope containment (base $base unresolvable)" + fi +else + skip "scope containment (no contract)" +fi + +# --- 8. RED was recorded, and failed for the right reason -------------------- +if [ -n "$contract_file" ] && [ -f "$contract_file" ]; then + if grep -q 'RED (recorded before EXECUTE)' "$contract_file"; then + if grep -qi 'N/A' "$(printf %s "$contract_file")" && grep -A4 'RED (recorded before EXECUTE)' "$contract_file" | grep -qi 'N/A'; then + pass "RED: declared N/A (no behaviour change)" + elif grep -A6 'RED (recorded before EXECUTE)' "$contract_file" | grep -qiE 'assert|expect|to (be|equal)|assertion'; then + pass "RED: recorded and failed on an assertion" + else + fail "RED: recorded but no assertion-shaped failure — a test that never failed proves nothing" + fi + else + skip "RED artifact (contract predates the recording format)" + fi +else + skip "RED artifact (no contract)" +fi + +# --- 10. STATE inbox does not reference finished work ------------------------ +# FEATURES.json rot is checked above; STATE.md rots the same way and nothing saw it. +if [ -f "$STATE" ] && [ -f FEATURES.json ]; then + result=$(python3 - "$STATE" <<'PYEOF' +import json, re, sys + +text = open(sys.argv[1], encoding="utf-8").read().splitlines() +start = None +for n, line in enumerate(text): + if re.match(r"^##+\s+Human-attention inbox", line, re.I): + start = n + 1 + break +if start is None: + print("SKIP no inbox section"); raise SystemExit + +section = [] +for line in text[start:]: + if re.match(r"^##+\s+", line): + break + section.append(line) + +feats = {f["id"]: f.get("status") for f in json.load(open("FEATURES.json")).get("features", [])} + +# Only ids that exist in this ledger can be stale. An id absent from the ledger is +# either planned work not yet created, or an id belonging to another repo, or part +# of a filename — none of which is rot. Flagging them was noise, not signal. +ids = set() +for line in section: + stripped = re.sub(r"`[^`]*`", " ", line) # drop backticked paths/filenames + stripped = re.sub(r"\S*/\S*", " ", stripped) # drop anything path-shaped + for m in re.findall(r"\b[A-Z][A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3}\b", stripped): + ids.add(m) + +done = sorted(i for i in ids if feats.get(i) == "PASS") +if done: + print("FAIL inbox references work already PASS: " + " ".join(done)) +else: + print("OK inbox references only open work") +PYEOF +) + case "$result" in + OK*) pass "STATE inbox: ${result#OK }" ;; + SKIP*) skip "STATE inbox (${result#SKIP })" ;; + *) fail "STATE ${result#FAIL }" ;; + esac +else + skip "STATE inbox (no STATE.md or FEATURES.json)" +fi + +# --- 9. branch discipline ----------------------------------------------------- +if git rev-parse --git-dir >/dev/null 2>&1; then + cur=$(git branch --show-current 2>/dev/null || echo "") + def=$(git rev-parse --abbrev-ref origin/HEAD 2>/dev/null | sed 's|^origin/||') + [ -n "$def" ] || def=master + if [ -z "$cur" ]; then skip "branch discipline (detached HEAD)" + elif [ "$cur" != "$def" ]; then pass "on a feature branch ($cur)" + else + if [ -z "$(git status --porcelain 2>/dev/null)" ]; then pass "on $def and clean" + else fail "uncommitted implementation on the default branch ($def)"; fi + fi +else + skip "branch discipline (not a git repo)" +fi + +echo +if [ "$fails" -gt 0 ]; then + echo "harness verify: $fails failure(s), $skips skipped" + exit 1 +fi +echo "harness verify: green ($skips skipped)" +exit 0 diff --git a/skills/.harness/contracts/KIT-AUDIT-001.md b/skills/.harness/contracts/KIT-AUDIT-001.md new file mode 100644 index 0000000..6ad07c3 --- /dev/null +++ b/skills/.harness/contracts/KIT-AUDIT-001.md @@ -0,0 +1,339 @@ +# SPRINT CONTRACT — KIT-AUDIT-001: `code-audit` profile (periodic whole-project quality audit) + +> Status: **EXECUTE complete — VERIFY green (24/24) — PENDING_REVIEW.** +> Approved: David 2026-08-27 ("proceed") with recommended answers to Q1–Q4. +> Branch `feat/kit-audit-001-code-audit-profile` from `master` @ `a3af45f`. Version **1.5.0**. +> Feature ID: `KIT-AUDIT-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-audit-001-code-audit-profile` — **human must confirm before checkout** +> Trigger: David (2026-08-27) — "a once in a while behaviour to check for the overall code quality of a project… name that as a separate skill in our harness-kit." + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + POSIX sh. No application code. + +> This sprint **adds** the `code-audit` profile. The sprint itself runs under `ops-docs`. + +--- + +## Context (grounded) + +| Source | Finding | +| --- | --- | +| `templates/CODE_QUALITY.md` (v1.4, KIT-QUALITY-001) | Per-sprint **gate**: diff-scoped, lint-first, blocking, cheap enough to run every contract. Deliberately narrow. | +| `templates/AUDIT_CONTRACT.md` + `AUDIT_FINDING.md` | Existing precedent for a **periodic, report-producing, code-changing-nothing** profile (`solidity-audit`). Finding template already carries ID / Severity / Status / commit-tag / location. | +| `templates/AGENT_PROCESS.md` §Work-type profiles | Five profiles; profile drives which gates are in force vs N/A. | +| `templates/AGENT_PROCESS.md` §Build to delete / Cost reality | "A full harness costs more per run… choose depth per the matrix." A whole-repo audit is expensive by design and must be explicitly periodic, never per-sprint. | +| Supplied review brief (David, 2026-08-27) | 15 review categories, severity/confidence/regression-risk rubrics, behavioural-equivalence checklist, 5-phase workflow, 11-section report format. | + +**Gap:** the kit has a per-diff quality gate and a Solidity security audit, but nothing that steps back +and assesses **a whole codebase's quality** on a cadence. Nothing carries the behaviour-preservation +discipline that makes large-scale refactoring safe. + +**Out of scope by correction (David):** the kit must not reference any pack's skills. The kit is +pack-agnostic; optional packs may add their own passes and the kit says nothing about them. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Form | **Work-type profile + contract template**, not a "skill" — the kit's idiom. Parallel to `solidity-audit`. | Locked | +| Name | **`code-audit`** — reads consistently beside `solidity-audit` | Locked | +| Cadence | **Periodic, never per-sprint.** Explicitly not a gate; does not run at VERIFY. | Locked | +| Relationship to `CODE_QUALITY.md` | Two different instruments, stated plainly in both files: the **gate** answers "can this diff ship?"; the **audit** answers "where should we invest refactoring effort?" | Locked | +| Behaviour preservation | **The absolute constraint of the profile.** Made deterministic: characterization tests captured **before** the refactor (passing against current code, proving they encode today's behaviour) and still passing after. | Locked | +| Taxonomy home | **In the kit's contract template.** It is language-neutral — feature envy, primitive obsession, stale comments are not stack-specific, unlike thresholds (which we deliberately kept out in v1.4). The kit cannot depend on a pack existing. | Locked | +| Findings → work | Each accepted finding seeds a **`FEATURES.json`** entry, which becomes a sprint contract. An audit whose findings are not tracked is a document nobody acts on. | Locked | +| Audit changes code | **No.** Like `solidity-audit`, the audit produces findings only. Fixes are separate contracts. | Locked | +| Second profile for fixes? | **No.** Fixes run under existing profiles plus one new quality-gate row for behaviour-preserving work. One new profile, not two. | Locked | +| Severity scale | Reuse the kit's existing **Critical / High / Medium / Low / Informational** from `AUDIT_FINDING.md` — do not introduce a third vocabulary. | Locked | + +--- + +## Scope — WILL do + +### A — Profile + +1. Add **`code-audit`** to the profile tables in `templates/AGENT_PROCESS.md`, `templates/AGENTS.md`, + `templates/SPRINT_CONTRACT.md`, and the profile-emphasis table in `templates/REVIEW.md`. +2. Mark the per-sprint gates **N/A** for `code-audit` (it ships no code): TDD, `CODE_QUALITY.md`, + http-api schema, Solidity gates. +3. Add a **depth-matrix row**: `code-audit` — periodic (per milestone / quarter / pre-hardening), + never triggered by an ordinary feature. + +### B — `templates/CODE_AUDIT_CONTRACT.md` (NEW) + +4. Scope in/out: paths, modules, languages, **commit or tag audited** (findings are meaningless + without the ref), and explicit exclusions (generated, vendored, migrations, fixtures). +5. **Behavioural constraint section** — "refactor implementation, not behaviour", with the frozen-contract + list (public APIs, exported symbols, CLI args, env vars, config keys, DB schema/values, serialized + formats, events, queue payloads, routes, status codes, headers, error formats, user-facing strings + consumers depend on) and the behavioural-equivalence checklist. +6. **Review taxonomy** — the 15 categories, condensed to a checklist the auditor works through: + best practices; duplication (harmful vs acceptable); optimisation; code smells; comments; readability; + maintainability; function quality; conditional logic; error handling; data flow and state; tests and + regression risk; dead code; over-engineering; under-engineering. +7. **Rubrics** — Severity (reuse existing scale); **Confidence** High/Medium/Low; **Regression risk** + Very Low/Low/Medium/High. Prioritise by *impact × confidence ÷ regression risk*. +8. **Workflow** — Understand → Identify → Validate → Prioritise → Recommend. Validation before reporting + is mandatory: check call sites, imports, consumers, tests, framework registration before a finding stands. +9. **False-positive discipline** — a finding must meet at least one materiality test; unconventional but + clear, safe, locally appropriate code is not a finding. Minimal-diff principle: smallest change that + achieves a meaningful improvement; no cascading refactors. +10. **Report shape** — executive summary; prioritised findings; duplication table; comment-quality report; + readability hotspots; maintainability hotspots; optimisation (safe vs profiling-required); + safe refactors; **refactors not to attempt yet**; suggested sequence; verification checklist. +11. **Finding ID scheme** — `-F`, same shape as the Solidity audit. +12. **`FEATURES.json` seed block** — one entry per accepted finding, `status: FAIL`, `verify` written from + the repo's `AGENTS.md` runner. This is the audit → contract bridge. +13. **Separate reporting for suspected functional bugs** — reported, never fixed inside the audit. +14. A short, explicit contrast with `CODE_QUALITY.md` so the two are never confused. + +### C — `templates/CODE_AUDIT_FINDING.md` (NEW) + +15. Extend the `AUDIT_FINDING.md` shape with: **Category**, **Confidence**, **Regression risk**, + **Recommended change** (smallest safe), **Behavioural safety** (what must be verified), + and **Characterization tests required**. Keep ID / Severity / Status / parent / commit / location. + +### D — Behaviour-preserving refactor gate + +16. `templates/SPRINT_CONTRACT.md` — one new quality-gate row: + *behaviour-preserving refactor only: characterization tests captured and passing **before** the + change; the same tests pass after; declared frozen contracts untouched — or **N/A***. +17. `templates/AGENT_PROCESS.md` §6 EXECUTE — one line: for behaviour-preserving refactor work, + characterization tests replace product TDD as the RED/GREEN mechanism, mirroring the existing + PoC-first rule for audits. + +### E — Version, docs, bootstrap + +18. **Appendix L** (`CODE_AUDIT_CONTRACT.md`) and **Appendix M** (`CODE_AUDIT_FINDING.md`) in + `scripts/build-bootstrap.sh`; Step 0 path table `Appendices B–M`. +19. Version bump + changelog bullet — see **Q1** for the number. +20. Add `KIT-AUDIT-001` to root `FEATURES.json`; re-run `./scripts/build-bootstrap.sh`. + +## Scope — will NOT do (this sprint) + +- Reference, absorb, or reconcile any pack's skills. The kit is pack-agnostic (David, 2026-08-27). +- Put language-specific thresholds, linters, or tool mandates in the kit. +- Add a second profile for implementing findings — existing profiles plus one gate row cover it. +- Change `CODE_QUALITY.md`, the v1.4 gate, or any existing profile's behaviour. +- Introduce a third severity vocabulary. +- Run an actual audit on any repo. +- Modify any consumer repo. + +## Target + +- **Repo / package:** `harness-kit`. +- **Not touching:** consumer repos; any pack repo; application source. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master` — **[GROUNDED]** +- **Base:** `master` @ `a3af45f` (PR #15 merged; v1.4.0 live, `templates/CODE_QUALITY.md` present) — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-audit-001-code-audit-profile` +- **Human confirmed:** yes — "proceed" (2026-08-27) + +--- + +## Tests first (TDD) + +**N/A** — documentation/process sprint. VERIFY uses falsifiable deterministic checks. + +- **Expected RED (today):** + - `test -f templates/CODE_AUDIT_CONTRACT.md` → fails + - `test -f templates/CODE_AUDIT_FINDING.md` → fails + - `grep -q 'code-audit' templates/AGENT_PROCESS.md` → fails + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `templates/CODE_AUDIT_CONTRACT.md` | **NEW** | [NEW] | +| `templates/CODE_AUDIT_FINDING.md` | **NEW** | [NEW] | +| `templates/AGENT_PROCESS.md` | `code-audit` profile row; depth-matrix row; N/A gates; EXECUTE characterization-test line | [GROUNDED] | +| `templates/AGENTS.md` | Profile list | [GROUNDED] | +| `templates/SPRINT_CONTRACT.md` | Profile checkbox + behaviour-preserving refactor gate row | [GROUNDED] | +| `templates/REVIEW.md` | `code-audit` row in the profile-emphasis table | [GROUNDED] | +| `scripts/build-bootstrap.sh` | Appendices L + M; Step 0 path table | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated; version + changelog | [GROUNDED] | +| `FEATURES.json` (root) | Add `KIT-AUDIT-001` | [GROUNDED] | + +**Reuse:** `AUDIT_CONTRACT.md` structure and severity scale; `AUDIT_FINDING.md` meta block. Extend, do not fork. + +--- + +## Success criteria + +1. `code-audit` appears in the profile tables of `AGENT_PROCESS.md`, `AGENTS.md`, `SPRINT_CONTRACT.md`, `REVIEW.md`. +2. `templates/CODE_AUDIT_CONTRACT.md` and `templates/CODE_AUDIT_FINDING.md` exist. +3. The contract template states the behavioural constraint and carries the frozen-contract list and equivalence checklist. +4. All 15 review categories are present as a worked checklist. +5. Severity / Confidence / Regression-risk rubrics present; prioritisation formula stated; severity scale matches `AUDIT_FINDING.md` (no third vocabulary). +6. Template requires the audited **commit or tag** to be recorded. +7. Template contains a `FEATURES.json` seed block, one entry per accepted finding. +8. Template states the audit changes no code, and reports suspected functional bugs separately without fixing them. +9. Both `CODE_AUDIT_CONTRACT.md` and `CODE_QUALITY.md` state the gate-vs-audit distinction. +10. `SPRINT_CONTRACT.md` carries the behaviour-preserving refactor gate row with an explicit **N/A** path. +11. Kit neutrality: no linter names, thresholds, or pack references in either new template. +12. `grep -ri 'ponytail' templates/` returns **no matches**. +13. Bootstrap regenerates clean; `BOOTSTRAP.md` contains `CODE_AUDIT_CONTRACT`; both `FEATURES.json` parse. +14. Diff confined to the impact map; work on the confirmed feature branch. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — see VERIFY +- [ ] Trust-boundary / partial deps / http-api / solidity — **N/A** (docs-only) +- [ ] Code-quality gate — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor gate — **N/A** (no code) + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Confused with the v1.4 per-sprint gate (both say "quality") | Success criteria 9: an explicit contrast paragraph in **both** files | +| Audit run every sprint; cost explodes | Depth-matrix row marks it periodic; §Cost reality already warns | +| Findings rot unread | Seed block turns accepted findings into `FEATURES.json` entries (§B12) | +| Auditor "fixes" a functional bug mid-audit | §B13 — report separately, never fix | +| Refactors ship without equivalence proof | Characterization-test gate row (§D16/17) | +| Taxonomy becomes a nit generator | False-positive materiality test + minimal-diff principle (§B9) | +| Kit accretes pack coupling | Success criterion 12 is a grep gate | + +## Threat model + +**SKIP** — documentation/process only. + +--- + +## Blocking questions (gates) + +**Q1 — RESOLVED.** KIT-QUALITY-001 merged via **PR #15** (`a3af45f`); kit v1.4.0 live with +`templates/CODE_QUALITY.md` present. This branches from updated `master` and takes **1.5.0**. +KIT-ERD-001 rebases and takes 1.6.0. + +**Q2 — Audit scope default.** Whole repo, or one declared subsystem per audit? +- **Recommendation:** the contract **requires** a declared scope — paths/modules plus commit or tag. + "Whole repo" is allowed but must be stated deliberately, because on a large codebase it produces a + report too big to act on. Mirrors `AUDIT_CONTRACT.md`'s scope-in/out discipline. + +**Q3 — Who accepts a finding into `FEATURES.json`?** Auditor seeds all, or human triages first? +- **Recommendation:** the audit **proposes** the seed block; the **human** accepts findings into the + ledger. Consistent with `PASS` being human-only, and stops a long report flooding the tracker. + +**Q4 — Cadence.** Leave it to each repo's `AGENTS.md`, or state a kit default? +- **Recommendation:** leave it to the repo, with the depth matrix naming the usual triggers + (milestone, quarter, pre-hardening, pre-handover, before a large refactor). The kit should not + impose a calendar. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# profile registered +grep -q 'code-audit' templates/AGENT_PROCESS.md +grep -q 'code-audit' templates/AGENTS.md +grep -q 'code-audit' templates/SPRINT_CONTRACT.md +grep -q 'code-audit' templates/REVIEW.md + +# new templates +test -f templates/CODE_AUDIT_CONTRACT.md +test -f templates/CODE_AUDIT_FINDING.md + +# behavioural spine +grep -qi 'refactor implementation, not behaviour' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'characterization' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'characterization' templates/SPRINT_CONTRACT.md +grep -qi 'frozen' templates/CODE_AUDIT_CONTRACT.md + +# rubrics + prioritisation, single severity vocabulary +grep -qi 'confidence' templates/CODE_AUDIT_FINDING.md +grep -qi 'regression risk' templates/CODE_AUDIT_FINDING.md +grep -qi 'Informational' templates/CODE_AUDIT_FINDING.md +! grep -qi 'Major / Minor' templates/CODE_AUDIT_FINDING.md + +# audit hygiene +grep -qi 'commit\|tag' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'FEATURES.json' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'not part of this refactor\|report separately' templates/CODE_AUDIT_CONTRACT.md + +# gate vs audit distinction stated in both +grep -qi 'CODE_QUALITY' templates/CODE_AUDIT_CONTRACT.md +grep -qi 'CODE_AUDIT' templates/CODE_QUALITY.md + +# neutrality: no tools, no pack coupling +! grep -Eqi 'eslint|golangci|clippy|solhint' templates/CODE_AUDIT_CONTRACT.md +! grep -riq 'ponytail' templates/ + +# bootstrap + json +./scripts/build-bootstrap.sh +grep -q 'CODE_AUDIT_CONTRACT' BOOTSTRAP.md +python3 -m json.tool FEATURES.json >/dev/null +grep -q 'KIT-AUDIT-001' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-AUDIT-001", + "name": "code-audit profile: periodic whole-project quality audit with behaviour-preservation discipline", + "priority": 2, + "verify": "test -f templates/CODE_AUDIT_CONTRACT.md && test -f templates/CODE_AUDIT_FINDING.md && grep -q code-audit templates/AGENT_PROCESS.md && grep -qi characterization templates/SPRINT_CONTRACT.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-AUDIT-001.md; profile: ops-docs; depends on KIT-QUALITY-001 (CODE_QUALITY.md must exist)" +} +``` + +--- + +## Criteria block + +```criteria +- [KIT-AUDIT-001-1] code-audit profile registered in AGENT_PROCESS, AGENTS, SPRINT_CONTRACT, REVIEW +- [KIT-AUDIT-001-2] CODE_AUDIT_CONTRACT.md and CODE_AUDIT_FINDING.md exist +- [KIT-AUDIT-001-3] Behavioural constraint, frozen-contract list, equivalence checklist present +- [KIT-AUDIT-001-4] All 15 review categories present as a worked checklist +- [KIT-AUDIT-001-5] Severity/Confidence/Regression rubrics; single severity vocabulary; prioritisation formula +- [KIT-AUDIT-001-6] Audited commit/tag required; FEATURES seed block present; functional bugs reported not fixed +- [KIT-AUDIT-001-7] Gate-vs-audit distinction in both files; characterization-test gate row in SPRINT_CONTRACT +- [KIT-AUDIT-001-8] No tool names, no thresholds, no pack references anywhere in templates/ +``` + +--- + +## Deviations from the approved contract (declare, do not hide) + +| # | Contract said | Actually done | Why | Verdict | +|---|---------------|---------------|-----|---------| +| 1 | Impact map listed `templates/AGENT_PROCESS.md` and `templates/AGENTS.md` for profile wiring only | Also removed the **pre-existing** named-pack references from both (`agent-harness`, `ponytail` → generic wording); `templates/CODE_QUALITY.md` gained the audit cross-reference | Success criterion 12 (`grep -ri 'ponytail' templates/` → no matches) failed on **v1.3 text already on master**, not on anything this contract added. The named references contradict the pack-agnostic rule David stated on 2026-08-27. Meaning preserved exactly: "a shared agent-harness submodule" → "a shared skills submodule"; "ponytail simplicity" → "simplicity passes". | **Declared scope expansion** — corrective, zero behavioural risk, one revert if unwanted | + +`templates/CODE_QUALITY.md` was always implied by §B14 ("an explicit contrast with `CODE_QUALITY.md`") +but was omitted from the impact-map table. Listed here for completeness. + +## VERIFY result + +24/24 green, including the neutrality gates (no tool names in the audit contract; no named-pack +references anywhere in `templates/`) and a structural check that all 15 taxonomy categories are present. + +## Note for the human + +`KIT-QUALITY-001` is still `PENDING_REVIEW` in `FEATURES.json` although PR #15 is merged. +**`PASS` is human-only** — deliberately left for David rather than repeating the undeclared +ledger edit made under KIT-QUALITY-001. diff --git a/skills/.harness/contracts/KIT-ERD-001.md b/skills/.harness/contracts/KIT-ERD-001.md new file mode 100644 index 0000000..5bc0408 --- /dev/null +++ b/skills/.harness/contracts/KIT-ERD-001.md @@ -0,0 +1,340 @@ +# SPRINT CONTRACT — KIT-ERD-001: `product-erd` profile + ERD authoring stage + +> Status: **EXECUTE complete — VERIFY green (21/21) — PENDING_REVIEW.** +> Approved: (David, "proceed", 2026-08-27), with recommended Q2–Q4. +> **Revised for kit v2.0:** deliverables are a **skill**, not `templates/*`. Version **2.1.0**. +> Feature ID: `KIT-ERD-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-erd-001-product-erd-profile` — **human must confirm before checkout** +> Trigger: David (2026-08-27) — "build an ERD creation process within our agent harness kit … the first bit of building out a product or a feature for us before that is translated to a sprint contract." + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + POSIX sh (`scripts/build-bootstrap.sh`). No application code. + +> Note: this sprint **adds** the `product-erd` profile. The sprint itself runs under `ops-docs`. + +--- + +## Context (grounded) + +Read before writing this contract: + +| Source | Finding | +| --- | --- | +| `templates/AGENT_PROCESS.md` | Loop is `BOOT → RESEARCH? → CONTRACT → [APPROVE+BRANCH] → CHECKOUT → EXECUTE → VERIFY → REVIEW → PENDING_REVIEW → [PASS]`. 5 profiles. Optional RESEARCH "distills open questions", explicitly **does not** create a second approved artifact. | +| `technical-project-documentation/.harness/contracts/ORDER-SVC-001..019` | The kit is **already** authoring ERDs — 19 contracts, all `ops-docs`, all Markdown-only, all grep-based VERIFY. | +| Same, 006/007/008/011/012 | Several are single-decision lock contracts (006 is 740 bytes: "lock amount encoding"). Decisions surfaced **during** doc work, not before it. | +| Same, 009 → 010, and 015 | 009 fully superseded; 015 reversed the stake-limit / `batchAll` / `extrinsicToken` locks from 010–014. Rework caused by late decision discovery. | +| `technical-project-documentation/.agents/skills/erd-authoring/SKILL.md` | Good **style guide** — section flow, slice-table rules, diagram + quality bar. Not a **process**: no cold-start, no decision elicitation, no gates. | +| `technical-project-documentation/.agents/skills/erd-engineering-standards/SKILL.md` | tao.com-specific defaults (HAL, RFC 9457, hexagonal, K8s probes, New Relic). Must **not** move into the kit — kit stays stack-neutral. | +| `products/order-service/v1/{erd.md,architecture.md}` | Reference output shape for the two-document deliverable. | + +**Gap this contract closes:** (1) no cold-start path from brief → first complete ERD; (2) decisions arrive reactively instead of being elicited and locked up front; (3) no mechanical ERD → `FEATURES.json` → sprint-contract handoff. + +--- + +## Scope — WILL do + +### A — New work-type profile + +1. Add **`product-erd`** as the sixth work-type profile in `templates/AGENT_PROCESS.md`, `templates/AGENTS.md`, and `templates/SPRINT_CONTRACT.md`. +2. Profile applies when defining a **new product, service, or substantial feature** whose design is not yet locked. Its deliverable is documentation; its consumers are the sprint contracts that follow. + +### B — New skill (v2.0 shape) — supersedes the original "three templates" + +3. `templates/ERD_CONTRACT.md` — the contract governing an ERD-authoring sprint (analogue of `AUDIT_CONTRACT.md`). Carries the staged gate log in §C below. +4. `skills/erd-authoring/templates/ERD.md` — the ERD document. Section flow: Executive Summary; Goals and Non-Goals; Engineering Standards / Goals; Implementation Overview; Design Decisions (+ Acceptance Criteria); API Conventions and Contracts; Risks / Security / Pre-Mortem; Open Questions; **Implementation plan: Feature slices**; Appendices (Data Dictionary, Glossary, References). +5. `skills/erd-authoring/templates/ARCHITECTURE.md` — companion doc. Owns: mental model, system diagram (Mermaid), component/boundary table, ports or module map **only where `AGENTS.md` declares that pattern**, primary control flows, data ownership, deployable boundaries. + +### C — The staged EXECUTE (progressive, human-gated) + +6. Document in `templates/AGENT_PROCESS.md` a new subsection **"ERD stage (`product-erd`)"**. EXECUTE for this profile is **five stages with a human gate between each**. The agent stops at every gate. + +``` +BRIEF/PRD ─► S0 Frame ─► S1 Decisions ─► S2 Design ─► S3 Contracts ─► S4 Slices ─► erd.md + architecture.md + ▲ ▲ ▲ ▲ ▲ │ + [gate] [gate] [gate] [gate] [gate] ▼ + FEATURES.json seed ─► sprint contracts +``` + +| Stage | Agent produces | Human gate | +| --- | --- | --- | +| **S0 Frame** | Input inventory (PRD/brief/design/tickets/related repos + code to reuse); draft **Executive Summary** + **Goals** + **Non-Goals** | Confirms the scope boundary. Non-goals must be explicit here, not discovered later. | +| **S1 Decisions** | Every decision the ERD must lock, as a table: `Topic \| Options \| Recommendation \| Status`. No design written yet. | Answers or defers each. Deferred rows move to **Open Questions** with an owner and a needed-by. **This is the stage that replaces reactive lock contracts.** | +| **S2 Design** | `architecture.md`: mental model, diagram, boundaries, control flows, data ownership | Confirms the shape before contracts are written against it. | +| **S3 Contracts** | ERD design sections: standards, API/event/data contracts, error model, acceptance criteria, risks / pre-mortem | Confirms contracts and falsifiable acceptance criteria. | +| **S4 Slices** | `Implementation plan: Feature slices` table **plus** the `FEATURES.json` seed block — one entry per slice | PASS. ERD is now the input to sprint contracts. | + +7. **Slices are written last**, only after S0–S3 are stable. Slice 1 is scaffolding (incl. readiness/liveness probes) when the target is a deployable service; **N/A** for library, contract, or docs targets. Every slice row needs a concrete working behaviour and a functional test that mimics a user story or a meaningful part of one. +8. **The ERD → contract bridge (the point of this sprint):** S4 emits a `FEATURES.json` seed block, one entry per slice, `status: FAIL`, `verify` written from the consumer repo's `AGENTS.md` runner. Each seeded entry then becomes exactly one sprint contract under the normal loop. The handoff is mechanical, not interpretive. + +### D — Depth matrix + skip rule + +9. Extend the depth matrix in `templates/AGENT_PROCESS.md` with an ERD row, and state plainly **when to skip the ERD stage**: a feature inside an already-documented service goes straight to a sprint contract. The ERD stage is for a new product/service, a new domain, or a contract crossing teams or repos. Cite "build to delete" / cost reality. +10. State that optional **RESEARCH** and the **ERD stage** are not the same thing and must not be dual-run: RESEARCH distills open questions for one contract; the ERD stage produces a durable design document that feeds many contracts. + +### E — Kit neutrality (explicit) + +11. The kit ships the **process and the document skeleton only**. Organisation standards — HAL, RFC 9457, hexagonal, camelCase, probe endpoints, log destinations — stay in the consumer repo's `AGENTS.md` or its docs skill pack. the ERD form references "the standards pack declared in `AGENTS.md`" and never names one. +12. The ERD form instructs: record any divergence from the declared standards pack in **Design Decisions** with rationale. Never diverge silently. + +### F — Review + gates + +13. The `sprint-contract` form: add `product-erd` to the profile checklist and add one quality-gate row — *`product-erd` only: decisions locked or owner+needed-by; non-goals explicit; slices ticket-ready with functional tests; FEATURES seed emitted — or **N/A***. +14. The `harness-review` form: add `product-erd` to the profile-aware review hints — *scope honesty (non-goals real), decision completeness, falsifiable acceptance criteria, slice ticketability, no silent standards divergence*. + +### G — Version, docs, bootstrap + +15. Bump kit version to **2.1.0** in `scripts/build-bootstrap.sh` (which regenerates the deprecated `BOOTSTRAP.md`) and `README.md`. +16. Append **Appendix K** (`ERD_CONTRACT.md`), **Appendix L** (`ERD.md`), **Appendix M** (`ARCHITECTURE.md`) in `scripts/build-bootstrap.sh`; add the three files to the Step 0 path table. +17. Add `erd-authoring` to the skills table in `README.md`; mention the `product-erd` profile. +18. Add the `KIT-ERD-001` entry to root `FEATURES.json`. +19. Re-run `./scripts/build-bootstrap.sh` so `BOOTSTRAP.md` stays in sync. + +--- + +## Scope — will NOT do (this sprint) + +- Copy tao.com standards (HAL, RFC 9457, hexagonal, New Relic, K8s probes) into the kit. +- Modify `technical-project-documentation`, `order-service`, or any other consumer repo. +- Migrate the existing `ORDER-SVC-001..019` contracts or rewrite `products/order-service/v1/*`. +- Adopt or absorb `agent-harness` / its RPI skills (still out of scope per KIT-V13-001). +- Add an ERD amendment / change-log template — see blocking question Q3. +- Add PRD authoring. The ERD stage takes a brief or PRD as **input**; it does not write one. +- Automate ERD linting beyond grep-based VERIFY commands. +- Change the existing five profiles, the status model, or the session loop outside the additions above. + +## Target + +- **Repo / package:** `harness-kit` — the kit owns the process spine. +- **Not touching:** every consumer repo; `agent-harness`; application source anywhere. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master` (via `git symbolic-ref refs/remotes/origin/HEAD`) — **[GROUNDED]** +- **Upstream reality (verified 2026-08-27):** KIT-V13-001 **is already merged** to `origin/master` via **PR #14** (`e3b98b1`). Kit v1.3.0 is live; `templates/AUDIT_CONTRACT.md` present on master. Local `master` is **2 commits behind** and local `FEATURES.json` still reads `PENDING_REVIEW` — ledger drift, not a blocker — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-erd-001-product-erd-profile` +- **Base:** stacked on `feat/kit-skills-001-templates-to-skills` (`2b37249`). KIT-SKILLS-001 is + PENDING_REVIEW and unmerged, and this work needs the v2.0 skill layout that exists only there. + **Rebase onto `master` once KIT-SKILLS-001 merges.** +- **Human confirmed:** **pending — agent asks before checkout.** + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Home of the ERD process | **harness-kit core** — part of the default spine, not an opt-in pack | Locked (David, 2026-08-27) | +| Deliverable set | **`erd.md` + `architecture.md`** pair, matching the order-service precedent | Locked (David, 2026-08-27) | +| Interaction model | **Progressive, human-gated stages** S0–S4 | Locked (David, 2026-08-27) | +| Governance | **New `product-erd` work-type profile**, governed by the contract system | Locked (David, 2026-08-27) | +| Profile name | `product-erd` | Open — **Q2** | +| ERD → contract bridge | S4 emits a `FEATURES.json` seed block, one entry per slice; each becomes one sprint contract | Locked | +| Org standards | Stay in consumer `AGENTS.md` / docs pack; kit references, never names them | Locked | +| Slice ordering | Slices written last; slice 1 = scaffolding for deployable services, N/A otherwise | Locked | +| `openapi.yaml` | **Not** an ERD-stage deliverable — it churned across ORDER-SVC-014/016 as decisions moved. Freeze it in a later `http-api` contract | Locked | +| Amendment change-log | Not this sprint | Open — **Q3** | +| Base branch | Updated `master` (v1.3 already merged, PR #14) | Locked | + +--- + +## Tests first (TDD) + +**N/A** — documentation/process sprint, no behaviour change. Per kit law, VERIFY uses falsifiable deterministic checks (below), not application tests. + +- **Expected RED (today):** + - `grep -q 'product-erd' templates/AGENT_PROCESS.md` → fails + - `test -f templates/ERD.md` → fails + - `test -f templates/ARCHITECTURE.md` → fails + - `test -f templates/ERD_CONTRACT.md` → fails + - `grep -q '1.4.0' BOOTSTRAP.md` → fails + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `skills/erd-authoring/SKILL.md` | **NEW** | [NEW] | +| `skills/erd-authoring/templates/ERD_CONTRACT.md` | **NEW** — staged gate log | [NEW] | +| `skills/erd-authoring/templates/ERD.md` | **NEW** — ERD skeleton | [NEW] | +| `skills/erd-authoring/templates/ARCHITECTURE.md` | **NEW** — architecture companion | [NEW] | +| `skills/harness-onboard/templates/AGENT_PROCESS.md` | `product-erd` row; ERD stage (S0–S4); depth-matrix row; RESEARCH-vs-ERD separation | [GROUNDED] | +| `skills/harness-onboard/templates/AGENTS.md` | Profile list + ERD pointer | [GROUNDED] | +| `skills/sprint-contract/templates/SPRINT_CONTRACT.md` | Profile checkbox + quality-gate row | [GROUNDED] | +| `skills/harness-review/templates/REVIEW.md` | `product-erd` review row | [GROUNDED] | +| `README.md` | Skills table + profile | [GROUNDED] | +| `scripts/build-bootstrap.sh` | 2.1.0; Appendices N/O/P | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated (still deprecated) | [GROUNDED] | +| `README.md`, `templates/README_POINTER.md` | Mention profile + ERD stage | [GROUNDED] | +| `FEATURES.json` (root) | Add `KIT-ERD-001` | [GROUNDED] | + +**Reuse (do not reinvent):** `AUDIT_CONTRACT.md` structure for `ERD_CONTRACT.md`; the ORDER-SVC grep-VERIFY idiom; the erd-authoring section flow and slice-table rules (**re-expressed stack-neutrally — not copied verbatim**, since the source is tao.com-coupled). + +--- + +## Success criteria + +1. `product-erd` appears as a profile in `AGENT_PROCESS.md`, `AGENTS.md`, and `SPRINT_CONTRACT.md`. +2. `templates/ERD.md`, `templates/ARCHITECTURE.md`, `templates/ERD_CONTRACT.md` all exist. +3. `AGENT_PROCESS.md` documents stages **S0–S4** with an explicit human gate between each, and states the agent stops at every gate. +4. `AGENT_PROCESS.md` states slices are written **last** and that S4 emits a `FEATURES.json` seed block, one entry per slice. +5. `AGENT_PROCESS.md` contains an explicit **skip rule** for the ERD stage and distinguishes it from optional RESEARCH. +6. Kit neutrality holds: no `HAL|RFC 9457|hexagonal|New Relic` in the ERD or architecture forms. +7. The ERD form contains the heading `## Implementation plan: Feature slices` verbatim. +8. The `sprint-contract` and `harness-review` forms each carry a `product-erd` line. +9. Kit version `2.1.0`; regenerated `BOOTSTRAP.md` contains `ERD_CONTRACT`; `erd-authoring` has valid frontmatter. +10. `./scripts/build-bootstrap.sh` exits 0; both `FEATURES.json` files parse. +11. Diff confined to the impact map; `git status` shows only those paths. +12. All work on the confirmed feature branch — not `master`. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria (always required) — see VERIFY +- [ ] Trust-boundary failure mode — **N/A** (docs-only) +- [ ] Partial/optional deps — **N/A** +- [ ] `http-api` schema/envelope — **N/A** +- [ ] `solidity-build` — **N/A** +- [ ] `solidity-audit` — **N/A** +- [ ] No consumer-repo edits — asserted in Scope/will NOT + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| ERD stage becomes mandatory ceremony on small features | Explicit skip rule + depth-matrix row (§D9); "build to delete" cited | +| Kit accretes tao.com standards through the ERD template | Success criterion 6 is a hard grep gate | +| Gates get skipped and S0–S4 collapse into one draft | `ERD_CONTRACT.md` carries a gate log with a human sign-off line per stage | +| ERD and sprint contracts drift after implementation starts | Partially unmitigated this sprint — see **Q3** | +| Two competing approved artifacts (ERD *and* a `plan.md`) | §D10 states RESEARCH and the ERD stage must not be dual-run | +| Slices written before design settles (the erd-authoring failure mode) | §C7 + success criterion 4 | + +## Threat model + +**SKIP** — documentation/process only; no runtime surface, no credentials, no side effects. + +--- + +## Blocking questions (gates) + +**Q1 — RESOLVED (was: base branch).** v1.3 is already merged (PR #14). Branch from an updated `master`; no stacking needed. Residual action: pull local `master`, and set `KIT-V13-001` to `PASS` in root `FEATURES.json` so the ledger matches reality. + +**Q2 — Profile name.** `product-erd` vs `discovery` vs plain `erd`. +- **Recommendation:** `product-erd` — says what it produces, and reads correctly for a feature-level ERD as well as a whole product. + +**Q3 — Amendment change-log.** Your docs repo has `engineering-amendment-change-log` for when implementation diverges from an ERD. ORDER-SVC-009 and 015 were reversals, so this path is real. Ship `templates/ERD_AMENDMENT.md` in this sprint, or defer to `KIT-ERD-002`? +- **Recommendation:** defer to `KIT-ERD-002` — "one thing at a time." This sprint adds a one-line pointer in `templates/ERD.md` so the seam exists. + +**Q4 — Slice-1 scaffolding rule.** The tao.com skill hardcodes "slice 1 is always scaffolding + K8s probes." The kit is stack-neutral and covers Solidity and libraries. +- **Recommendation:** as drafted in §C7 — slice 1 is scaffolding *for deployable services*, explicitly N/A otherwise, with the probe specifics left to the consumer's `AGENTS.md`. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# profile registered in all three places +grep -q 'product-erd' templates/AGENT_PROCESS.md +grep -q 'product-erd' templates/AGENTS.md +grep -q 'product-erd' templates/SPRINT_CONTRACT.md +grep -q 'product-erd' templates/REVIEW.md + +# new templates exist +test -f templates/ERD.md +test -f templates/ARCHITECTURE.md +test -f templates/ERD_CONTRACT.md + +# staged, gated EXECUTE documented +grep -q 'S0' templates/AGENT_PROCESS.md +grep -q 'S4' templates/AGENT_PROCESS.md +grep -qi 'human gate' templates/AGENT_PROCESS.md + +# ERD -> contract bridge + slices last +grep -qi 'seed' templates/AGENT_PROCESS.md +grep -q 'Implementation plan: Feature slices' templates/ERD.md +grep -qi 'skip' templates/AGENT_PROCESS.md + +# gate log present in the ERD contract template +grep -qi 'gate log\|Gate log' templates/ERD_CONTRACT.md + +# kit neutrality — org standards must NOT leak into kit templates +! grep -Eqi 'HAL|RFC 9457|hexagonal|New Relic' templates/ERD.md +! grep -Eqi 'HAL|RFC 9457|hexagonal|New Relic' templates/ARCHITECTURE.md + +# version + bootstrap regen +grep -q '1.4.0' BOOTSTRAP.md +grep -q '1.4.0' scripts/build-bootstrap.sh +./scripts/build-bootstrap.sh +grep -q 'ERD_CONTRACT' BOOTSTRAP.md +grep -q '1.4.0' BOOTSTRAP.md + +# json validity + feature entry +python3 -m json.tool FEATURES.json >/dev/null +python3 -m json.tool templates/FEATURES.json >/dev/null +grep -q 'KIT-ERD-001' FEATURES.json + +# branch discipline +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-ERD-001", + "name": "product-erd profile + staged ERD authoring stage (erd.md + architecture.md) feeding FEATURES seed", + "priority": 1, + "verify": "grep -q product-erd templates/AGENT_PROCESS.md && test -f templates/ERD.md && test -f templates/ARCHITECTURE.md && test -f templates/ERD_CONTRACT.md && grep -q '1.4.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-ERD-001.md; profile: ops-docs; depends on KIT-V13-001" +} +``` + +--- + +## Criteria block + +```criteria +- [KIT-ERD-001-1] product-erd registered in AGENT_PROCESS, AGENTS, SPRINT_CONTRACT, REVIEW +- [KIT-ERD-001-2] templates/ERD.md, ARCHITECTURE.md, ERD_CONTRACT.md exist +- [KIT-ERD-001-3] Stages S0-S4 documented with a human gate between each +- [KIT-ERD-001-4] Slices written last; S4 emits FEATURES.json seed, one entry per slice +- [KIT-ERD-001-5] Explicit ERD-stage skip rule; distinguished from optional RESEARCH +- [KIT-ERD-001-6] Kit neutrality: no org-specific standards in ERD/ARCHITECTURE templates +- [KIT-ERD-001-7] Version 1.4.0; Appendices K/L/M; build-bootstrap.sh green; FEATURES.json valid +``` + +--- + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | Contract (pre-v2.0) specified `templates/ERD.md`, `ARCHITECTURE.md`, `ERD_CONTRACT.md` and Appendices K/L/M | Revised to the v2.0 skill shape before execution: `skills/erd-authoring/` with the three forms bundled, Appendices **N/O/P**, version **2.1.0**. Recorded in the contract header. | +| 2 | Skill name is `erd-authoring`; profile is `product-erd` | Deliberate. Not every skill maps to a profile (`sprint-contract`, `harness-onboard`, `security-checklist`, `harness-review` have none), and `erd-authoring` names the activity while `product-erd` names the work type. Both files state the relationship. | +| 3 | VERIFY expected the literal phrase "stops at every gate" in `SKILL.md`, which said "stops and waits at each gate" | Semantically identical; phrasing aligned across both files rather than loosening the check — consistent vocabulary is worth having. | + +## Q2–Q4 as executed + +- **Q2 profile name:** `product-erd` (recommended). +- **Q3 amendment change-log:** deferred to `KIT-ERD-002` as recommended; the seam exists — divergence from the declared standards pack is recorded in Design Decisions. +- **Q4 slice-1 rule:** scaffolding for deployable services, explicitly **N/A** for library / contract / docs targets, with probe specifics left to the consumer's `AGENTS.md`. + +## Note + +Stacked on `feat/kit-skills-001-templates-to-skills`, which is PENDING_REVIEW and unmerged. +**Rebase onto `master` once KIT-SKILLS-001 merges.** diff --git a/skills/.harness/contracts/KIT-GATES-001.md b/skills/.harness/contracts/KIT-GATES-001.md new file mode 100644 index 0000000..b2e064f --- /dev/null +++ b/skills/.harness/contracts/KIT-GATES-001.md @@ -0,0 +1,345 @@ +# SPRINT CONTRACT — KIT-GATES-001: deterministic gates (RED/BASELINE/GREEN + `verify-harness.sh` + CI) + +> Status: **EXECUTE complete — VERIFY green (20/20) — PENDING_REVIEW.** +> Approved: David 2026-08-28 ("approve") with recommended Q1–Q4. Version **2.3.0**. +> Feature ID: `KIT-GATES-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-gates-001-deterministic-gates`, **stacked on** +> `feat/kit-rules-001-rules-and-erd-routing` (`65aeb7f`, PENDING_REVIEW). Rebase onto `master` when that merges. +> Version → **2.3.0**. +> Trigger: David (2026-08-28) — "how do we put a gate to ensure that a contract delivers/executes a +> feature without regression or hallucination, especially as features/slices would not have an ERD?" + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** POSIX sh + Markdown + one CI workflow. No application code. + +--- + +## Context — the kit currently has no enforced gate (grounded) + +| Finding | Evidence | Consequence | +| --- | --- | --- | +| Every "gate" is self-reported | `AGENT_PROCESS` §VERIFY *asks* the agent to run commands; nothing confirms it | An executor can set `PENDING_REVIEW` without running anything | +| The RED field asks for a **prediction** | `SPRINT_CONTRACT.md:75` — "**Expected** RED output: ``" | Nothing ever compares the prediction to reality. A test that never failed proves nothing: it may be vacuous or already passing | +| No CI, no hooks | No `.github/` in the kit; zero hook configs | The only thing in the repo that can fail and stop something is `setup-harness-kit.sh` | +| `FEATURES.json` is unvalidated | No schema check anywhere | Status drift went unnoticed **four times** this month across two repos (harness-kit 3 entries, order-service 10/10) | +| **CI cannot read a contract** | `git check-ignore skills/.harness/contracts` → ignored | The artifact `harness-review` §0 grades a PR against is invisible to CI, to reviewers, and (as observed 2026-08-28) to the author in Finder and the editor sidebar | + +**The gap:** rules and skills are instructions to a model that can misread them under load. A gate is a +fact. The kit has the first two layers and not the third. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Prediction → recording | `SPRINT_CONTRACT` gains **RED / BASELINE / GREEN** artifact blocks holding real command, real output, real exit code | Locked | +| Delivery gate | The acceptance test must have **failed before EXECUTE**, on an **assertion** | Locked | +| Regression gate | Baseline suite recorded before, re-run after; **diff ⊆ declared impact map** | Locked | +| Hallucination gate | Claims replaced by artifacts: every `[GROUNDED]` path must exist; every `verify` must exit 0 | Locked | +| Slices with no ERD | Handled by the same gate. If you cannot write a test that fails today, you do not have a falsifiable criterion and the contract is not approvable | Locked | +| Implementation | **One script, three invocation points** — `scripts/verify-harness.sh`, called by CI and optionally by hooks. Never reimplemented per surface | Locked | +| Unforgeable layer | **CI re-runs VERIFY itself**, so the agent's claim stops mattering | Locked | +| Hooks | **Optional, opt-in**, documented as an example. Vendor-specific; CI and the script are universal | Locked | +| Vacuous tests | Out of reach of any cheap gate. Mitigate by requiring an assertion-shaped RED and reviewing the test **at contract approval**. Mutation testing documented as an option, **not** a default | Locked | + +--- + +## Scope — WILL do + +### A — `scripts/verify-harness.sh` + +1. New POSIX script, exits non-zero on any violation, runnable from a consumer repo root. Checks: + +| # | Check | Catches | +|---|-------|---------| +| 1 | `FEATURES.json` parses; every `status` is from the legend; every `verify` non-empty and not prose | Ledger rot | +| 2 | Every `PENDING_REVIEW` entry has a contract file where it claims | Phantom work | +| 3 | `STATE.md` current contract points at a real file | Stale state | +| 4 | Every skill has valid frontmatter (`name`, `description`) | Skills that cannot be retrieved | +| 5 | Every rule has valid frontmatter and `alwaysApply` | Rules that never load | +| 6 | Every `[GROUNDED]` path in the active contract's impact map **exists** | **Invented files** — the most common contract hallucination | +| 7 | `git diff --name-only ..HEAD` ⊆ impact map | Undeclared scope creep, the main regression vector | +| 8 | RED artifact present, and its recorded failure is an **assertion**, not an import/path error | Vacuous or never-failing tests | +| 9 | No implementation commits on the default branch | Branch discipline | + +2. Checks 6–8 **degrade gracefully**: when no contract is reachable they report `skipped (no contract)` + rather than failing, so the script is useful in a repo mid-adoption. Checks 1–5 and 9 always run. +3. `--strict` turns skips into failures, for repos that have fully adopted. + +### B — Contract form: prediction becomes recording + +4. Replace `**Expected RED output:**` with three blocks: + +``` +### RED (recorded before EXECUTE) +- Command: +- Exit code: +- Failure reason: +- Captured: + +### BASELINE (recorded before EXECUTE) +- Command: +- Result: + +### GREEN (recorded after EXECUTE) +- Command: Exit code: 0 +- Command: Result: +``` + +5. State the rule plainly in the form: **a test that never failed proves nothing**, and a RED that + failed on an import error proves the test file is wrong, not that the behaviour is absent. +6. Add the no-ERD line: without an ERD the contract's own criteria are the whole story, and each must + be expressible as a failing test **before** work starts, or the contract is not approvable. + +### C — CI + +7. `.github/workflows/harness.yml` — on pull request: run `verify-harness.sh`, then re-run the + contract's VERIFY block. Kit-owned, shipped to consumers via the skill tree. +8. Document that CI is the only layer an agent cannot fake, and that a green local run is not evidence. + +### D — Hooks (optional) + +9. `docs/hooks.md` (or a README section) with one worked example calling the same script. Explicitly + opt-in and vendor-specific; the kit never requires it. + +### E — Process + wiring + +10. `AGENT_PROCESS.md` §VERIFY — require the three artifacts, and say the executor may set + `PENDING_REVIEW` only when `verify-harness.sh` exits 0. +11. `harness-review` form §0 — reviewer confirms the artifacts exist and that CI ran the gate. +12. `code-quality-gate` — one line pointing at the shared script so the two do not drift. +13. Version **2.3.0**, changelog, `FEATURES.json` entry, bootstrap appendix for the new script/workflow. + +## Scope — will NOT do (this sprint) + +- Mutation testing, coverage thresholds, or any quality metric beyond pass/fail. +- Language-specific test runners — commands come from the consumer's `AGENTS.md`. +- Reconcile the forked global harness (`KIT-RECONCILE-001`). +- The `code-audit` partial-coverage amendment (separate, small). +- Retro-fit artifacts onto already-merged contracts. +- Migrate any consumer repo. + +--- + +## Branch + +- **Base:** `feat/kit-rules-001-rules-and-erd-routing` @ `65aeb7f` — needs `rules/` to exist for check 5. +- **Proposed:** `feat/kit-gates-001-deterministic-gates` +- **Human confirmed:** **pending.** + +## Tests first + +**N/A** as product TDD — but this sprint **dogfoods its own gate**: `verify-harness.sh` is run against +harness-kit itself as part of VERIFY, and must exit 0. + +- **Expected RED today:** `test -f scripts/verify-harness.sh` fails; `test -d .github/workflows` fails. + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `scripts/verify-harness.sh` | **NEW** | [NEW] | +| `.github/workflows/harness.yml` | **NEW** | [NEW] | +| `skills/sprint-contract/templates/SPRINT_CONTRACT.md` | RED/BASELINE/GREEN blocks; no-ERD rule | [GROUNDED] | +| `skills/harness-onboard/templates/AGENT_PROCESS.md` | VERIFY requires artifacts + script exit 0 | [GROUNDED] | +| `skills/harness-review/templates/REVIEW.md` | §0 confirms artifacts and CI | [GROUNDED] | +| `skills/code-quality-gate/templates/CODE_QUALITY.md` | Pointer to the shared script | [GROUNDED] | +| `README.md`, `scripts/build-bootstrap.sh`, `BOOTSTRAP.md`, `FEATURES.json` | Gates section, appendix, 2.3.0, ledger | [GROUNDED] | +| `.gitignore` | Contracts untracked→tracked; legacy `.harness/` rule anchored to root | [GROUNDED] | +| `skills/.harness/contracts/KIT-V13-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-QUALITY-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-AUDIT-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-SKILLS-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-ERD-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | +| `skills/.harness/contracts/KIT-RULES-001.md` | Becomes tracked (Q1 consequence) | [GROUNDED] | + +--- + +## Success criteria + +1. `scripts/verify-harness.sh` exists, is executable, POSIX-clean, and **exits 0 against harness-kit itself**. +2. Running it against a repo with a deliberately broken `FEATURES.json` exits non-zero — demonstrated, not asserted. +3. Checks 6–8 report `skipped (no contract)` rather than failing when no contract is reachable; `--strict` flips that. +4. `.github/workflows/harness.yml` runs the script and the contract VERIFY block on pull requests. +5. `SPRINT_CONTRACT.md` carries RED/BASELINE/GREEN blocks; the phrase "Expected RED output" is gone. +6. The form states that a test which never failed proves nothing, and the no-ERD rule. +7. `AGENT_PROCESS.md` §VERIFY conditions `PENDING_REVIEW` on the script exiting 0. +8. `REVIEW.md` §0 confirms artifacts exist and CI ran. +9. One implementation: neither CI nor the hooks doc reimplements a check — both invoke the script. +10. 2.3.0 everywhere; bootstrap regenerates; `FEATURES.json` parses; diff confined to the impact map. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — criteria 1 and 2 are executable +- [ ] Trust-boundary — **N/A** (docs/script) +- [ ] Code-quality gate — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor — **N/A** + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Gate blocks a repo mid-adoption | Graceful skips; `--strict` is opt-in | +| Contract checks silently no-op forever | The skip is **printed**, not swallowed, so "skipped" is visible in CI logs | +| CI and hooks drift | Criterion 9: single script, both invoke it | +| Vacuous test passes every check | Acknowledged limit. Assertion-shaped RED + review the test at approval; mutation testing documented, not mandated | +| Diff-⊆-impact-map fights legitimate discoveries | The remedy is amending the contract, which is the intended behaviour | +| Script becomes a second process spine | It validates the spine, adds no rules of its own | + +## Threat model + +**SKIP** — no network, no credentials, no privileged operations. CI runs only repo-declared commands. + +--- + +## Blocking questions (gates) + +**Q1 — Contracts are gitignored, so CI cannot read them.** This is now forcing, not cosmetic: checks +6–8 need the contract, and `harness-review` §0 already tells a reviewer to grade the PR against a file +that is not in the PR. You hit this yourself on 2026-08-28 when the contract was invisible in Finder +and the editor sidebar. +- **Recommendation:** **track contracts.** Move `skills/.harness/contracts/` out of `.gitignore` and + keep `STATE.md` and local scratch ignored. They are markdown, the audit trail is worth having in git, + and it makes the contract reviewable, CI-checkable and findable in one move. +- Alternatives: CI runs only checks 1–5 and 9 (weaker, and the hallucination gate is the valuable one); + or CI pastes the contract into the PR body (solves review, not checking). + +**Q2 — Should the diff-⊆-impact-map check block, or warn?** It is the strongest regression gate and +also the most likely to annoy. +- **Recommendation:** **block**, with the remedy being to amend the contract. Warning-only gates get ignored. + +**Q3 — Baseline on every contract, or only behaviour changes?** Recording a full-suite baseline costs a +run each time. +- **Recommendation:** behaviour changes only; **N/A** for `ops-docs`, `code-audit`, `product-erd`. + +**Q4 — Does the kit ship the CI workflow to consumers, or document it?** Shipping means the kit writes +into `.github/`, which some repos guard. +- **Recommendation:** ship it as a template the setup script **offers** but never overwrites, following + the same marker-guard discipline as the rules assembly. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# the gate exists and passes against this repo (dogfood) +test -x scripts/verify-harness.sh +sh -n scripts/verify-harness.sh +./scripts/verify-harness.sh + +# and it actually fails on a broken ledger (demonstrated, not claimed) +cp FEATURES.json /tmp/f.bak +python3 - <<'PY' +import json;d=json.load(open('FEATURES.json'));d['features'][0]['status']='BOGUS' +open('FEATURES.json','w').write(json.dumps(d,indent=2)) +PY +./scripts/verify-harness.sh && echo "GATE DID NOT FAIL — BAD" && exit 1 +cp /tmp/f.bak FEATURES.json +./scripts/verify-harness.sh + +# CI wired +test -f .github/workflows/harness.yml +grep -q 'verify-harness.sh' .github/workflows/harness.yml + +# prediction replaced by recording +! grep -q 'Expected RED output' skills/sprint-contract/templates/SPRINT_CONTRACT.md +grep -q 'RED (recorded before EXECUTE)' skills/sprint-contract/templates/SPRINT_CONTRACT.md +grep -q 'BASELINE' skills/sprint-contract/templates/SPRINT_CONTRACT.md +grep -qi 'never failed proves nothing' skills/sprint-contract/templates/SPRINT_CONTRACT.md + +# process + review wired to the same script +grep -q 'verify-harness.sh' skills/harness-onboard/templates/AGENT_PROCESS.md +grep -q 'verify-harness.sh' skills/harness-review/templates/REVIEW.md + +# single implementation +test "$(grep -rlc 'FEATURES.json parses' .github scripts 2>/dev/null | wc -l)" -le 1 + +# version + ledger +grep -q '2.3.0' BOOTSTRAP.md && sh scripts/build-bootstrap.sh +python3 -m json.tool FEATURES.json >/dev/null && grep -q 'KIT-GATES-001' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-GATES-001", + "name": "Deterministic gates: RED/BASELINE/GREEN artifacts, verify-harness.sh, CI re-running VERIFY", + "priority": 1, + "verify": "test -x scripts/verify-harness.sh && ./scripts/verify-harness.sh && test -f .github/workflows/harness.yml && ! grep -q 'Expected RED output' skills/sprint-contract/templates/SPRINT_CONTRACT.md && grep -q '2.3.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-GATES-001.md; profile: ops-docs; stacked on KIT-RULES-001; Q1 decides whether contracts become tracked" +} +``` + +## Criteria block + +```criteria +- [KIT-GATES-001-1] verify-harness.sh exists, POSIX, executable, exits 0 against harness-kit +- [KIT-GATES-001-2] Demonstrated non-zero exit on a broken FEATURES.json +- [KIT-GATES-001-3] Contract checks skip visibly without a contract; --strict flips to failure +- [KIT-GATES-001-4] CI workflow runs the script and the contract VERIFY block on PRs +- [KIT-GATES-001-5] Expected-RED prediction replaced by RED/BASELINE/GREEN recordings +- [KIT-GATES-001-6] No-ERD rule stated: no failing test means no falsifiable criterion +- [KIT-GATES-001-7] AGENT_PROCESS conditions PENDING_REVIEW on the gate; REVIEW s0 confirms artifacts +- [KIT-GATES-001-8] Single implementation — CI and hooks both invoke the script +``` + +--- + +## RED (recorded before EXECUTE) + +- **Command:** `test -x scripts/verify-harness.sh && test -f .github/workflows/harness.yml` +- **Exit code:** `1` +- **Failure reason:** neither file existed at `aea8fb2` — the gate and its CI surface were absent +- **Captured:** 2026-08-28, before the first edit on this branch + +**BASELINE** — N/A: no test suite exists in this repo; the gate itself is the suite from this sprint on. + +## GREEN (recorded after EXECUTE) + +- `./scripts/verify-harness.sh` → **exit 0**, 8 pass / 1 skip (RED artifact, on a contract predating the format) +- Contract VERIFY block → **20/20 pass** + +## Criterion 2 — demonstrated, not asserted + +The gate was shown to fail on four separate defects, each restored afterwards: + +| Injected defect | Gate output | Exit | +|---|---|---| +| `status: "BOGUS"` | `KIT-RULES-001: status 'BOGUS' not in legend` | 1 | +| `verify: "tested"` (prose) | `KIT-RULES-001: verify does not look like a command` | 1 | +| `PENDING_REVIEW` with no contract | `PENDING_REVIEW without a contract: GHOST-001` | 1 | +| Rule with no frontmatter | `rules with bad frontmatter: rules/status-ownership.mdc` | 1 | + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | **The gate found a false positive in itself.** Check 6 read `master` and `ef393c3` as paths, because branch names and SHAs are also backticked on `[GROUNDED]` lines in the Branch section | Narrowed the heuristic to tokens containing `/` or carrying a known extension. Caught by dogfooding before shipping — which is the argument for criterion 1 | +| 2 | Q1 required more than a `.gitignore` edit | The legacy rule `.harness/` is **unanchored**, so it matched `skills/.harness` at any depth and kept contracts ignored regardless of the new negation. Anchored it to `/.harness/` (its intended root-only meaning). Found with `git check-ignore -v` rather than assumed | +| 3 | Contract listed `.gitignore` as conditional on Q1 | Q1 answered "track contracts", so it changed | +| 4 | **The gate failed on its own contract, correctly.** Tracking contracts pulled six pre-existing contract files into the diff that the impact map never declared | Scope containment reported them as undeclared. Per the design, the remedy is **amending the contract**, not relaxing the check — the impact map now lists all six. First live catch, on the sprint that introduced the check | + +## Q1–Q4 as executed + +- **Q1** contracts **tracked**; `STATE.md`, drafts and PR bodies stay local. +- **Q2** scope containment **blocks**; the remedy is amending the contract. +- **Q3** BASELINE for behaviour changes only; N/A for `ops-docs`, `code-audit`, `product-erd`. +- **Q4** CI workflow shipped as a kit file; consumers adopt it through the skill tree rather than having it written into their `.github/` by the setup script. diff --git a/skills/.harness/contracts/KIT-GATES-002.md b/skills/.harness/contracts/KIT-GATES-002.md new file mode 100644 index 0000000..07bee68 --- /dev/null +++ b/skills/.harness/contracts/KIT-GATES-002.md @@ -0,0 +1,239 @@ +# SPRINT CONTRACT — KIT-GATES-002: remove the path heuristic, close the STATE drift + +> Status: **EXECUTE complete — VERIFY green (12/12) — PENDING_REVIEW.** +> Approved: David 2026-08-28 ("yes") with recommended Q1–Q2. Version **2.4.0**. +> Feature ID: `KIT-GATES-002` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-gates-002-tighten-gate`, **stacked on** +> `feat/kit-gates-001-deterministic-gates` (`734a8df`, PENDING_REVIEW). Rebase when that merges. +> Version → **2.4.0**. +> Trigger: findings from the gate's own first run against this repo. + +--- + +## Work-type profile (required) + +- [x] `ops-docs` + +**Languages / toolchain:** POSIX sh + Markdown. No application code. + +--- + +## Context — two findings from `verify-harness.sh` running against harness-kit + +**1. `[GROUNDED]` is overloaded, so check 6 has to guess.** + +The marker appears on Branch-section lines as well as impact-map rows: + +``` +- **Default branch:** `master`; base `ef393c3` (v2.1) — **[GROUNDED]** +``` + +On its first run check 6 reported `master` and `ef393c3` as missing paths. The patch shipped in +KIT-GATES-001 was a **heuristic** — accept a token only if it contains `/` or carries a known +extension. That is a guess, and it will misfire in both directions: a real path with no slash and no +extension (`Makefile`, `Dockerfile`) is silently skipped, and a backticked symbol that happens to end +in `.ts` is checked as a file. A check that silently skips is worse than one that fails loudly. + +**2. `STATE.md` drifts exactly like `FEATURES.json`, and nothing notices.** + +Two of six inbox entries in this repo were stale — work already `PASS`. The gate catches that class of +rot in the ledger; in `STATE.md` it verifies only that the named contract exists. Demonstrated below: +an inbox entry pointing at a `PASS` feature leaves the gate green. + +**Root cause in both:** the kit asked the script to interpret prose. The fix is to make the artifact +unambiguous and let the script read structure. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| `[GROUNDED]` scope | **Impact map only.** Stated in the contract form; the Branch section stops using it | Locked | +| Check 6 | **Parse the impact-map table**, first column, rows marked `[GROUNDED]`. Delete the extension heuristic | Locked | +| Silent skips | A `[GROUNDED]` row whose path cannot be parsed **fails**, rather than being skipped | Locked | +| New check 10 | No `STATE.md` inbox entry references a feature that is already `PASS` | Locked | +| Scope | These two findings only. No new gate concepts | Locked | + +--- + +## Scope — WILL do + +1. `sprint-contract` form — state that `[GROUNDED]` / `[EDUCATED]` / `[NEW]` mark **impact-map rows only**, + and remove the marker from the Branch section example so the template stops teaching the ambiguity. +2. `verify-harness.sh` check 6 — parse the impact-map table: take the first cell of each row whose line + carries `[GROUNDED]`, strip backticks, verify it exists. **Delete** the extension/slash heuristic. +3. Same check — a `[GROUNDED]` row whose first cell is not a parseable path is a **failure**, not a skip. +4. `verify-harness.sh` **check 10** — read the `Human-attention inbox` section of `STATE.md`; any + feature id mentioned there that is `PASS` in `FEATURES.json` is a failure, naming the ids. +5. Version **2.4.0**, changelog, `FEATURES.json` entry, bootstrap regenerated. + +## Scope — will NOT do + +- New gate concepts, coverage or metrics. +- Changes to CI, the RED/BASELINE/GREEN blocks, or any skill or rule. +- Anything in a consumer repo. +- Auto-repair: the gate reports, a human fixes. + +## Target + +- **Repo:** `harness-kit`. **Not touching:** consumer repos, application source. + +--- + +## Branch + +- **Base:** `feat/kit-gates-001-deterministic-gates` @ `734a8df` — needs check 6 to exist. +- **Proposed:** `feat/kit-gates-002-tighten-gate` +- **Human confirmed:** **pending.** + +--- + +## RED (recorded before EXECUTE) + +- **Command:** `grep -qi 'impact map only' skills/sprint-contract/templates/SPRINT_CONTRACT.md; grep -qi 'inbox' scripts/verify-harness.sh` +- **Exit code:** `1` +- **Failure reason:** assertion — expected the string `impact map only` in the contract form and found + none; expected an inbox check in the gate and found none. Check 6 still matches on + `*.md|*.sh|*.json|…` at `scripts/verify-harness.sh:137` rather than parsing the table. +- **Captured:** 2026-08-28, before any edit on this branch + +**Behavioural RED, demonstrated:** an inbox entry naming `KIT-RULES-001` (which is `PASS`) was injected +into `STATE.md`. The gate reported `PASS STATE.md current contract exists` and **exited 0**. It should +exit 1. Restored afterwards. + +**BASELINE** — the gate itself: `./scripts/verify-harness.sh` → exit 0, 9 pass / 0 skip at `734a8df`. + +## GREEN (recorded after EXECUTE) + +- Both RED assertions → **exit 0**: `impact map only` present in the form; `inbox` present in the gate +- Injected inbox entry naming `KIT-SKILLS-001` (a `PASS` feature) → gate **exit 1**, + `FAIL STATE inbox references work already PASS: KIT-SKILLS-001`. Restored → exit 0 +- `./scripts/verify-harness.sh` → **exit 0**, **10** checks reported, 0 skipped +- Contract VERIFY block → **12/12** + +--- + +## Impact map + +| Path | Change | Marker | +|------|--------|--------| +| `scripts/verify-harness.sh` | Check 6 parses the table; heuristic deleted; check 10 added | [GROUNDED] | +| `skills/sprint-contract/templates/SPRINT_CONTRACT.md` | `[GROUNDED]` restricted to the impact map | [GROUNDED] | +| `skills/.harness/contracts/KIT-GATES-002.md` | This contract | [NEW] | +| `scripts/build-bootstrap.sh` | 2.4.0 + changelog | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated | [GROUNDED] | +| `FEATURES.json` | Add `KIT-GATES-002` | [GROUNDED] | +| `README.md` | Version | [GROUNDED] | + +--- + +## Success criteria + +1. `scripts/verify-harness.sh` contains no extension-list heuristic — `grep -q '\*\.md|\*\.sh' ` finds nothing. +2. Check 6 resolves paths from the impact-map table; a malformed `[GROUNDED]` row **fails**, demonstrated. +3. Check 10 exists; a `STATE.md` inbox entry naming a `PASS` feature makes the gate exit 1, **demonstrated**. +4. Clean tree: gate exits 0 with **10** checks reported. +5. The contract form states `[GROUNDED]` is impact-map only, and its Branch example no longer uses the marker. +6. 2.4.0 everywhere; bootstrap regenerates; `FEATURES.json` parses; diff confined to the impact map. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — 2, 3 and 4 are executable +- [ ] Trust-boundary / http-api / solidity / code-quality — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor — **N/A** + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Table parsing is as brittle as the heuristic | It reads a fixed position — first cell of a marked row — rather than inferring meaning. Unparseable rows fail loudly | +| Older contracts lack a table-shaped impact map | Those rows fail rather than skip, which is the intended signal to fix the contract | +| Check 10 false-positives on prose mentioning a `PASS` id | Match feature-id shaped tokens only, inside the inbox section only | +| Gate grows into a linter | Scope explicitly forbids new concepts | + +## Threat model + +**SKIP** — no network, no credentials, no side effects. + +## Blocking questions + +**Q1 — Older contracts whose impact map is a bullet list, not a table.** Six tracked contracts predate +the table convention. +- **Recommendation:** check 6 applies only to rows inside a markdown table; a bullet-list impact map + reports `skipped (impact map not tabular)`. Loud enough to notice, not a wall for history. + +**Q2 — Should check 10 also fail on an inbox entry naming a feature that does not exist at all?** +- **Recommendation:** yes, same check, same failure. A typo'd id is the same rot. + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# heuristic gone, table parse in +! grep -q '\*\.md|\*\.sh' scripts/verify-harness.sh +grep -qi 'impact map' scripts/verify-harness.sh +sh -n scripts/verify-harness.sh + +# convention stated +grep -qi 'impact map only' skills/sprint-contract/templates/SPRINT_CONTRACT.md + +# check 10 present, and the gate reports ten checks on a clean tree +grep -qi 'inbox' scripts/verify-harness.sh +./scripts/verify-harness.sh +test "$(./scripts/verify-harness.sh | grep -cE '^ (PASS|FAIL|SKIP)')" -eq 10 + +# version + ledger +grep -q '2.4.0' BOOTSTRAP.md && sh scripts/build-bootstrap.sh +python3 -m json.tool FEATURES.json >/dev/null && grep -q 'KIT-GATES-002' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +## FEATURES.json entry + +```json +{ + "id": "KIT-GATES-002", + "name": "Gate reads structure not prose: impact-map parse replaces the path heuristic; STATE inbox drift check", + "priority": 2, + "verify": "! grep -q '[*].md|[*].sh' scripts/verify-harness.sh && grep -qi inbox scripts/verify-harness.sh && ./scripts/verify-harness.sh && grep -q '2.4.0' BOOTSTRAP.md && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-GATES-002.md; profile: ops-docs; stacked on KIT-GATES-001; from the gate's own first run" +} +``` + +## Criteria block + +```criteria +- [KIT-GATES-002-1] Extension heuristic removed from verify-harness.sh +- [KIT-GATES-002-2] Check 6 parses the impact-map table; malformed rows fail loudly +- [KIT-GATES-002-3] Check 10 catches a STATE inbox entry naming a PASS feature, demonstrated +- [KIT-GATES-002-4] Clean tree reports 10 checks and exits 0 +- [KIT-GATES-002-5] Contract form restricts [GROUNDED] to the impact map +- [KIT-GATES-002-6] 2.4.0; bootstrap regenerates; FEATURES.json valid +``` + +--- + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | **The contract assumed check 6 could scan for `[GROUNDED]` anywhere.** It cannot | The first marked table row in most contracts belongs to **Context** or **Decisions**, not the impact map — verified across all eight tracked contracts. The parser is **section-scoped**: it locates `## Impact map`, reads to the next heading, and takes the first cell of marked table rows only. Stronger than the contract specified | +| 2 | **Q2 was wrong, and the gate proved it in seconds.** I recommended failing on inbox ids absent from the ledger | On first run it flagged four, and **none was rot**: `AUDIT-001` belongs to another repo, `KIT-AUDIT-002` and `KIT-RECONCILE-001` are planned work deliberately named, and `PR-KIT-GATES-001` was a **filename** the regex matched inside. Narrowed to the half that is real signal — ids present in the ledger and already `PASS` — and taught it to strip backticked and path-shaped tokens first | +| 3 | First demonstration of check 10 used `KIT-RULES-001`, which is `PENDING_REVIEW`, not `PASS` | Correctly produced no failure. Re-run against `KIT-SKILLS-001` (`PASS`) — failed as designed. The test was wrong, not the check | +| 4 | Glob paths in impact maps (`rules/*.mdc`) | Handled: a first cell containing `*` is treated as a glob and must match at least one file | + +## Q1 as executed + +A bullet-list impact map reports `SKIP impact map not tabular`; a section with no marked rows reports +`SKIP no [GROUNDED] rows`. Both are printed, never swallowed. + +## Note for review + +`KIT-RULES-001` still reads `PENDING_REVIEW` in the ledger although it merged at `aea8fb2` — the same +drift class this sprint closes for `STATE.md`. `PASS` is human-only, so it is left for David. diff --git a/skills/.harness/contracts/KIT-QUALITY-001.md b/skills/.harness/contracts/KIT-QUALITY-001.md new file mode 100644 index 0000000..900a4c3 --- /dev/null +++ b/skills/.harness/contracts/KIT-QUALITY-001.md @@ -0,0 +1,283 @@ +# SPRINT CONTRACT — KIT-QUALITY-001: code-quality gate + +> Status: **EXECUTE complete — VERIFY green (0 failures) — PENDING_REVIEW.** +> Approved by David 2026-08-27 with recommended answers to Q1/Q3/Q4; Q2 resolved (1.4.0). +> Branch `feat/kit-quality-001-code-quality-gate` created from `master` @ `e3b98b1`. Human sets PASS. +> Feature ID: `KIT-QUALITY-001` +> Repo: **harness-kit only** (the agent-harness half of this request is already done — see §Companion) +> Branch: proposed `feat/kit-quality-001-code-quality-gate` — **human must confirm before checkout** +> Trigger: David (2026-08-27) — "add it to harness-kit and improve the existing code-quality skill of the agent harness." + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + POSIX sh (`scripts/build-bootstrap.sh`). No application code. + +--- + +## Context (grounded) + +| Source | Finding | +| --- | --- | +| `agent-harness/skills/code-quality/SKILL.md` | Already exists with real thresholds (cyclomatic 10/15, length 40/75, params 4/6, class deps 5/8). **Advisory only** — "report only", no blocking power, no linter invocation. | +| `harness-kit/templates/REVIEW.md` §11 | "complexity" as a prose bullet under Maintainability. No thresholds, no verdict, no gate. | +| `harness-kit/templates/SPRINT_CONTRACT.md` | Six quality-gate rows: falsifiable criteria, trust boundary, partial deps, http-api, solidity-build, solidity-audit. **Nothing on code quality.** | +| `harness-kit/templates/SECURITY_CHECKLIST.md` | Already owns *Secrets & configuration* and *Error handling & data leakage*. **Direct overlap risk** with the skill's hardcoded-credentials and swallowed-errors rules. | +| `templates/AGENT_PROCESS.md` §Skill packs | "Optional packs may supply stack craft… must not replace FEATURES, contracts, VERIFY, or human PASS." This is the layering the split below follows. | + +**Gap:** nothing in the kit blocks `PENDING_REVIEW` on code quality. The executor can set +`PENDING_REVIEW` with a 300-line function and a hardcoded RPC URL and be fully process-compliant. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Layering | **Kit ships the gate; the pack ships the craft.** `templates/CODE_QUALITY.md` is a deterministic checklist + verdict (same idiom as `SECURITY_CHECKLIST.md`). Thresholds, fixes and language notes stay in the optional `agent-harness` skill. | Locked | +| Thresholds in the kit | **None.** The gate defers to the linter declared in the consumer's `AGENTS.md`. Kit stays stack-neutral, consistent with v1.3 VERIFY policy. | Locked | +| Deterministic-first | Gate §0 requires the declared lint command to be **run and recorded**. Judgement is the documented fallback, not the default. | Locked | +| Secrets ownership | **`SECURITY_CHECKLIST.md` keeps it.** The code-quality gate explicitly routes credential findings there and forbids double-reporting. | Locked | +| Blocking rule | `violation` (hard limit, no documented exception) blocks `PENDING_REVIEW`. `warn` and `pre-existing` never block. `PASS` stays human-only. | Locked | +| Profile applicability | Required for `service`, `http-api`, `solidity-build`. N/A for `ops-docs`, `product-erd`, `solidity-audit`. | Locked | +| Diff scope | "You own what you touch" — new code judged fresh, pre-existing breaches noted once as debt. | Locked | +| Exception format | In-code comment must name **rule + measured value + reason**; missing any of the three = violation. | Locked | + +--- + +## Scope — WILL do + +1. Add `templates/CODE_QUALITY.md` — promote the draft at + `skills/.harness/drafts/CODE_QUALITY.draft.md` (103 lines, already written for review). +2. `templates/SPRINT_CONTRACT.md` — add one quality-gate row: + *`service` / `http-api` / `solidity-build` only: code-quality gate run; declared lint command + recorded; violations = 0 or documented exceptions — or **N/A***. +3. `templates/AGENT_PROCESS.md` — in **§7 VERIFY**, require the code-quality gate alongside the + security checklist for the three profiles above; add one line to **§Skill packs** stating the + gate-vs-craft split so a pack cannot silently replace the gate. +4. `templates/REVIEW.md` — add a code-quality line to the §0 quality-gates walk and to the + profile-aware emphasis table; point §11 Maintainability at the gate instead of restating it. +5. `templates/AGENTS.md` — one line under *How we work here* naming the gate and its path. +6. `scripts/build-bootstrap.sh` — **Appendix N** for `CODE_QUALITY.md`; add to the Step 0 path table. +7. Bump kit version to **1.4.0** (or 1.5.0 if KIT-ERD-001 lands first — see Q2); changelog bullet. +8. Add `KIT-QUALITY-001` to root `FEATURES.json`. +9. Re-run `./scripts/build-bootstrap.sh`. + +## Scope — will NOT do (this sprint) + +- Put thresholds, linter configs, or language-specific rules in the kit. +- Duplicate the secrets rule out of `SECURITY_CHECKLIST.md`. +- Modify any consumer repo, or `agent-harness` (already done separately — see §Companion). +- Add CI wiring or a linter to any repo. +- Reopen the existing six quality gates, the status model, or the session loop. + +## Target + +- **Repo / package:** `harness-kit`. +- **Not touching:** consumer repos; `technical-project-documentation`; application source. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master` — **[GROUNDED]** +- **Upstream reality (verified 2026-08-27):** KIT-V13-001 **already merged** to `origin/master` via **PR #14** (`e3b98b1`); kit v1.3.0 is live. Local `master` is **2 commits behind**; local `FEATURES.json` still reads `PENDING_REVIEW` (ledger drift). Branch from an updated `master` — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-quality-001-code-quality-gate` +- **Human confirmed:** **pending — agent asks before checkout.** + +--- + +## Tests first (TDD) + +**N/A** — documentation/process sprint. VERIFY uses falsifiable deterministic checks. + +- **Expected RED (today):** + - `test -f templates/CODE_QUALITY.md` → fails + - `grep -qi 'code.quality' templates/SPRINT_CONTRACT.md` → fails + - `grep -qi 'code.quality' templates/REVIEW.md` → fails + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `templates/CODE_QUALITY.md` | **NEW** — promoted from reviewed draft | [NEW] | +| `templates/SPRINT_CONTRACT.md` | One quality-gate row | [GROUNDED] | +| `templates/AGENT_PROCESS.md` | VERIFY requirement + skill-pack split line | [GROUNDED] | +| `templates/REVIEW.md` | §0 gate walk + profile emphasis; §11 points at the gate | [GROUNDED] | +| `templates/AGENTS.md` | One line under *How we work here* | [GROUNDED] | +| `scripts/build-bootstrap.sh` | Appendix N + Step 0 path table | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated; version + changelog | [GROUNDED] | +| `FEATURES.json` (root) | Add `KIT-QUALITY-001` | [GROUNDED] | + +**Reuse:** `SECURITY_CHECKLIST.md` structure (checkbox sections + verdict block) — the gate is +deliberately its sibling, not a new idiom. + +--- + +## Success criteria + +1. `templates/CODE_QUALITY.md` exists and contains a **Verdict** block with a violations-blocks rule. +2. Gate §0 requires the declared lint command to be run **and recorded**. +3. Kit neutrality: `grep -Eqi 'eslint|golangci|clippy|solhint|cyclomatic|> ?[0-9]+ lines' templates/CODE_QUALITY.md` returns **no match** for hardcoded thresholds (tool names may appear only as `` examples inside the recorded-command fence). +4. `SPRINT_CONTRACT.md` has exactly one new code-quality gate row with an explicit **N/A** path. +5. `REVIEW.md` references the gate in §0 and in the profile table; §11 no longer restates complexity rules. +6. `AGENT_PROCESS.md` §7 names the gate for `service` / `http-api` / `solidity-build` and marks the other profiles N/A. +7. The secrets rule appears in `SECURITY_CHECKLIST.md` only — `CODE_QUALITY.md` routes to it and does not restate it. +8. Bootstrap regenerates clean; `BOOTSTRAP.md` contains `CODE_QUALITY`; both `FEATURES.json` parse. +9. Diff confined to the impact map. +10. Work on the confirmed feature branch — not `master`. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — see VERIFY +- [ ] Trust-boundary failure mode — **N/A** (docs-only) +- [ ] Partial/optional deps — **N/A** +- [ ] `http-api` — **N/A** +- [ ] `solidity-build` / `solidity-audit` — **N/A** +- [ ] Code-quality gate — **N/A** (this sprint is `ops-docs`; the gate it adds does not apply to itself) + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Gate becomes a style-nit generator that slows every merge | Only `violation` blocks; `warn` and `pre-existing` explicitly cannot | +| Same finding reported twice (quality + security) | §5 boundary table + success criterion 7 | +| Kit accretes language-specific thresholds over time | Success criterion 3 is a grep gate | +| Repos with no linter treat the gate as satisfied by vibes | §0 forces an explicit "no linter configured" statement, and recurring judgement findings become a follow-up contract | +| Pre-existing debt blocks unrelated work | "You own what you touch" rule in §1 | + +## Threat model + +**SKIP** — documentation/process only. + +--- + +## Blocking questions (gates) + +**Q1 — Contract order.** `KIT-ERD-001` is also drafted and unapproved. Both touch +`AGENT_PROCESS.md`, `AGENTS.md`, `SPRINT_CONTRACT.md`, `REVIEW.md`, `build-bootstrap.sh`, +`BOOTSTRAP.md` and root `FEATURES.json` — they will conflict if branched in parallel. +- **Recommendation:** sequence them. This one (`KIT-QUALITY-001`) is smaller and lower-risk — + ship it first, then rebase `KIT-ERD-001` on top. Principle 5, "one thing at a time." + +**Q2 — Version number.** Confirmed live version on `origin/master` is **1.3.0**, so the next is 1.4.0. +- **Recommendation:** this sprint takes **1.4.0** under the Q1 ordering; KIT-ERD-001 takes **1.5.0**. + +**Q3 — Enforcement strength when no linter is configured.** Today most consumer repos will have +partial linting. Should the gate hard-block on judgement-only violations, or only report them? +- **Recommendation:** judgement-only findings **warn**, never block; a repeat offender becomes a + contract to configure the rule. Blocking on model opinion invites arguing with the harness. + +**Q4 — Should the gate apply to `product-erd` (pending KIT-ERD-001)?** +- **Recommendation:** no — N/A, as drafted. An ERD ships no code. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# gate template exists and blocks correctly +test -f templates/CODE_QUALITY.md +grep -qi 'verdict' templates/CODE_QUALITY.md +grep -qi 'violations = 0' templates/CODE_QUALITY.md +grep -qi 'PASS remains human-only\|PASS. remains human-only' templates/CODE_QUALITY.md + +# deterministic-first +grep -qi 'AGENTS.md' templates/CODE_QUALITY.md +grep -qi 'judgement, not enforced config' templates/CODE_QUALITY.md + +# wired into contract, process, review, agents +grep -qi 'code.quality' templates/SPRINT_CONTRACT.md +grep -qi 'code.quality' templates/AGENT_PROCESS.md +grep -qi 'code.quality' templates/REVIEW.md +grep -qi 'code.quality' templates/AGENTS.md + +# no double ownership of secrets +grep -qi 'SECURITY_CHECKLIST' templates/CODE_QUALITY.md +! grep -qi 'zero tolerance' templates/CODE_QUALITY.md + +# version + bootstrap +grep -q '1.4.0' BOOTSTRAP.md +grep -q '1.4.0' scripts/build-bootstrap.sh +./scripts/build-bootstrap.sh +grep -q 'CODE_QUALITY' BOOTSTRAP.md + +# json + feature entry +python3 -m json.tool FEATURES.json >/dev/null +python3 -m json.tool templates/FEATURES.json >/dev/null +grep -q 'KIT-QUALITY-001' FEATURES.json + +# branch discipline +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-QUALITY-001", + "name": "Code-quality gate: deterministic lint-first checklist blocking PENDING_REVIEW on violations", + "priority": 1, + "verify": "test -f templates/CODE_QUALITY.md && grep -qi 'code.quality' templates/SPRINT_CONTRACT.md && grep -qi 'code.quality' templates/REVIEW.md && grep -q '1.4.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-QUALITY-001.md; profile: ops-docs; sequence before KIT-ERD-001 (shared file conflicts)" +} +``` + +--- + +## Companion change (already applied, outside this contract) + +`agent-harness/skills/code-quality/SKILL.md` was improved in place (203 lines, was ~120). +`agent-harness` has no contract tree — its own `AGENTS.md` governs skills via `skills/` + +the skills table, and forbids commits/branches without being asked. Nothing was committed. + +Changes: linter-first sensor section; diff-scope and "you own what you touch"; exempt paths; +test-code standards; cognitive-complexity row; measurement definitions; secrets rule handed to +the security checklist; documented-exception format; verdict → merge-recommendation mapping; +per-language notes (TS/Go/Rust/Solidity). + +--- + +## Criteria block + +```criteria +- [KIT-QUALITY-001-1] templates/CODE_QUALITY.md exists with verdict + violations-block rule +- [KIT-QUALITY-001-2] Gate requires declared lint command run and recorded +- [KIT-QUALITY-001-3] No hardcoded thresholds or tool mandates in the kit template +- [KIT-QUALITY-001-4] SPRINT_CONTRACT has one code-quality gate row with N/A path +- [KIT-QUALITY-001-5] REVIEW and AGENT_PROCESS reference the gate; REVIEW s11 no longer restates rules +- [KIT-QUALITY-001-6] Secrets owned solely by SECURITY_CHECKLIST; no double-reporting +- [KIT-QUALITY-001-7] Version bumped; Appendix N; bootstrap green; FEATURES.json valid +``` + +--- + +## Deviations from the approved contract (declare, do not hide) + +| # | Contract said | Actually done | Why | Verdict | +|---|---------------|---------------|-----|---------| +| 1 | Scope §G6: "**Appendix N** for `CODE_QUALITY.md`" | **Appendix K** | The letter was written assuming KIT-ERD-001 (K/L/M) landed first. Q1 sequenced this contract first, so existing appendices end at J and K is the correct next letter. | Trivial — contract text was wrong, output is right | +| 2 | Scope WILL did not include it | `KIT-V13-001` reconciled `PENDING_REVIEW` → `PASS` in root `FEATURES.json` | It shipped in PR #14; the ledger disagreed with reality, which breaks BOOT step 3 ("highest-priority `FAIL`"). Flagged in the Branch section as ledger drift but never added to Scope WILL. | **Undeclared scope creep** — small and corrective; human to accept or revert | + +Environment note: the folder bridge initially denied `unlink`, so a `git checkout` applied +partially and left HEAD stale at `7da7be4` with a working tree already matching `origin/master`. +No content was lost (verified: zero tracked-file diff vs `origin/master`). Repaired with +`git reset --hard origin/master` after delete permission was granted. + +## VERIFY result + +All 20 checks green, including the criterion-3 neutrality checks that the original VERIFY block +did not implement (`! grep -Eqi '(>|>=) *[0-9]+'` and no threshold vocabulary). Tool names appear +in exactly one line, inside the recorded-command fence, as `` examples. diff --git a/skills/.harness/contracts/KIT-RULES-001.md b/skills/.harness/contracts/KIT-RULES-001.md new file mode 100644 index 0000000..66eba71 --- /dev/null +++ b/skills/.harness/contracts/KIT-RULES-001.md @@ -0,0 +1,311 @@ +# SPRINT CONTRACT — KIT-RULES-001: `rules/` directory (Cursor-first) + ERD routing in the loop + +> Status: **EXECUTE complete — VERIFY green — PENDING_REVIEW.** +> Approved: David 2026-08-28 ("approve") with recommended Q1–Q4. Branch confirmed. Version **2.2.0**. +> Feature ID: `KIT-RULES-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-rules-001-rules-and-erd-routing` — **human must confirm before checkout** +> Base: `master` @ `ef393c3` (v2.1). Version → **2.2.0**. +> Trigger: David (2026-08-28) — rules as their own directory, Cursor-first, guided by +> `cursor-rules-export/`; and "expand our workflow to go through an ERD process before contract drafting." + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + `.mdc` + POSIX sh. No application code. + +--- + +## Context — what the export actually shows (grounded, `cursor-rules-export/`, 2026-08-28) + +| Finding | Evidence | Consequence | +| --- | --- | --- | +| **The kit's process law is already duplicated by hand as Cursor rules, and has drifted** | `harness-process.mdc` exists in `advanced-orders-backend`, `intents-service`, `tao.com-api`, `tao-docs` — **four different checksums** (`7f57ed1b`, `cbf16f1a`, `ad72ef13`, `9805df00`) | The strongest argument for the kit owning these canonically and symlinking them. Four hand-maintained copies is exactly the drift the submodule model exists to kill | +| **A second, older copy of the process exists outside the kit** | `global/user-rules-harness-engineering.md` and rules citing `~/.cursor/skills/harness-engineering/HARNESS.md` run a **5-step loop** (`BOOT → CONTRACT → APPROVE → EXECUTE → VERIFY → PENDING_REVIEW → PASS`) with `.harness/contracts/` — no RESEARCH, no CHECKOUT, no REVIEW step, no profiles, no gates | The process is **forked**. Rules point at the global skill as authoritative while harness-kit is what actually evolved. Reconciliation is **out of scope here** but must be tracked — see Q3 | +| **Always-on budget is heavy** | 18 of 27 rules are `alwaysApply: true` | Dilution risk. The kit must ship **few** rules and say so | +| **Glob scoping is available but unused** | `"globs": ""` on **every** rule in `manifest.json` | The main technical advantage of `.mdc` over assembled text is currently unrealised. Kit rules should use it where a rule is genuinely file-scoped, and default `alwaysApply: true` otherwise | +| **Four distinct ownership tiers are visible** | see below | Only tier 1 belongs in the kit | + +**Ownership tiers observed in the export:** + +1. **Process law** — `sprint-contract-before-changes`, `new-work-new-branch`, `no-stage-harness-files`, `stage-after-execute`, `harness-process.mdc` ×4, parts of `harness-contract-quality`, `principal-pr-review`. **Kit-owned.** +2. **Engineering discipline, generalisable** — `verify-claims-with-evidence`, `harness-dry-reusability`. Kit-owned where it is not already a skill's job. +3. **Org / product specific** — `intents-backend-url`, `new-relic-intents-log-queries`, `tao-intents-ownership-and-pps`, `postman-ticket-sync`, `documentation-agents`. **Consumer-owned, never the kit's.** +4. **Personal style / comms** — `no-em-dashes`, `pr-desc-brief-bullets`, `pr-replies-relatable-names`, `pr-replies-no-reflexive-agreement`, `pr-review-comments-brief`. **Cursor User Rules, not a repo concern at all.** + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| Form | `rules/` directory in the kit, authored as **`.mdc`** with Cursor frontmatter | Locked | +| Primary target | **Cursor** — `.mdc` is the source of truth, symlinked into `.cursor/rules/` | Locked (David) | +| Portability | Setup script also **assembles** rule bodies (frontmatter stripped) into a marked block in the consumer's `AGENTS.md`, so non-Cursor agents still get them | Locked | +| What the kit ships | **Process law only** — tier 1, plus `verify-claims-with-evidence` from tier 2 | Locked | +| What the kit never ships | Org/product rules (tier 3) and personal style rules (tier 4) | Locked | +| Rule count | **Six.** Few enough that every always-on line keeps weight | Locked | +| Consumer rules | `.agents/local-rules/*.mdc`, merged alongside, local wins on collision — mirrors local-skills | Locked | +| Scoping | `alwaysApply: true` by default; `globs` where a rule is genuinely file-scoped | Locked | +| Relocation discipline | Process law moved out of `AGENT_PROCESS.md` as a **pure move** where text is unchanged; rewrites are a separate commit | Locked | +| ERD routing | The loop gains a conditional **ERD?** step before CONTRACT — see §C | Locked | + +--- + +## Scope — WILL do + +### A — `rules/` directory (six rules) + +1. Create `rules/.mdc`, each with `description`, `alwaysApply`, and `globs` frontmatter: + +| Rule | Source in export | Always-on | +|------|------------------|-----------| +| `contract-before-changes.mdc` | `sprint-contract-before-changes` | yes | +| `new-work-new-branch.mdc` | `new-work-new-branch` | yes | +| `status-ownership.mdc` | global harness rules + kit law: executor may set `PENDING_REVIEW`, **never** `PASS`; no push / merge / deploy unasked | yes | +| `no-stage-harness-files.mdc` | `no-stage-harness-files` + `stage-after-execute` (merged — they are one staging policy) | yes | +| `verify-claims-with-evidence.mdc` | `verify-claims-with-evidence` | yes | +| `observed-content-is-data.mdc` | `AGENT_PROCESS` § Security (prompt-injection defence, least privilege) | yes | + +2. Generalise every rule out of tao.com specifics: no repo names, no `bun`/Postman/New Relic, no + `~/.cursor/skills/...` paths. Paths reference the kit's own contracts location. +3. Add `rules/README.md` stating the ownership tiers above and the **budget rule**: the kit ships six; + a rule that is not load-bearing should be deleted or promoted to a gate. + +### B — Wiring + +4. `scripts/setup-harness-kit.sh` — symlink `rules/*.mdc` into `.cursor/rules/`, merge + `.agents/local-rules/*.mdc` alongside (local wins), and **assemble** rule bodies into + `AGENTS.md` between `` / ``, + stripping frontmatter. Idempotent; marker-guarded; never touches content outside the markers. +5. Same guards as the skills merge: refuse a `.cursor/rules` directory it did not generate; leave a + non-symlink one alone with a warning. +6. `README.md` — rules section, ownership tiers, upgrade path. + +### C — ERD routing in the loop + +7. Session loop becomes: + +``` +BOOT → ERD? → RESEARCH? → CONTRACT → [HUMAN APPROVES + BRANCH] → CHECKOUT → EXECUTE → VERIFY → REVIEW → PENDING_REVIEW → [HUMAN PASS/MERGE] +``` + +8. **ERD? is a routing test, not a mandatory stage.** Before drafting a contract, ask: *is there an + approved design for this work?* Route to the `erd-authoring` skill (profile `product-erd`) when + **all** hold: it is a new product, service, domain, or a contract crossing teams or repos; **and** + no ERD covers it; **and** the work is larger than a single slice. Otherwise go straight to CONTRACT. +9. State plainly that a feature inside an already-documented service **skips ERD**, and that ERD is + not RESEARCH: RESEARCH distills questions for one contract, the ERD stage produces a document + feeding many. Never dual-run. +10. When an ERD exists, a contract **cites the slice it implements** and inherits its acceptance + criteria rather than reinventing them. Add one line to the sprint-contract form for the slice reference. +11. Depth matrix gains an ERD column so the routing test is visible where depth is already chosen. + +### D — Version + ledger + +12. Version **2.2.0**; changelog. Add `KIT-RULES-001` to `FEATURES.json`. +13. `scripts/build-bootstrap.sh` — Appendix Q for `rules/README.md` and the rule bodies, so the + deprecated bootstrap path stays complete until `KIT-SKILLS-002` removes it. + +## Scope — will NOT do (this sprint) + +- Ship org/product rules (tier 3) or personal style rules (tier 4). +- Reconcile the forked global harness (`~/.cursor/skills/harness-engineering/`) — see **Q3**. +- Migrate any consumer repo's existing `.cursor/rules/`, or delete the drifted `harness-process.mdc` copies. +- Make the ERD stage mandatory for any work. +- Build `verify-harness.sh` / CI — that is `KIT-GATES-001`, and it depends on rules existing first. +- Change any skill's behaviour. + +## Target + +- **Repo / package:** `harness-kit`. +- **Not touching:** consumer repos; the global Cursor skill; application source. + +--- + +## Branch + +- **Default branch:** `master`; base `ef393c3` (v2.1) — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-rules-001-rules-and-erd-routing` +- **Also carries:** `271b222` (ledger PASS for KIT-SKILLS-001 / KIT-ERD-001) if folded in — see Q4 +- **Human confirmed:** **pending — agent asks before checkout.** + +## Tests first (TDD) + +**N/A** — process/docs. Expected RED today: `test -d rules` fails; `grep -q 'ERD?' ` fails. + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `rules/*.mdc` (6) + `rules/README.md` | **NEW** | [NEW] | +| `scripts/setup-harness-kit.sh` | Rules symlink + local-rules merge + AGENTS.md assembly | [GROUNDED] | +| `skills/harness-onboard/templates/AGENT_PROCESS.md` | ERD? step, routing test, depth-matrix column; process law removed where it moves to `rules/` | [GROUNDED] | +| `skills/harness-onboard/templates/AGENTS.md` | Rules marker block; loop line | [GROUNDED] | +| `skills/sprint-contract/templates/SPRINT_CONTRACT.md` | ERD slice reference line | [GROUNDED] | +| `skills/erd-authoring/SKILL.md` | Cross-reference to the routing test | [GROUNDED] | +| `README.md`, `scripts/build-bootstrap.sh`, `BOOTSTRAP.md`, `FEATURES.json` | Rules section, Appendix Q, 2.2.0, ledger | [GROUNDED] | + +--- + +## Success criteria + +1. `rules/` contains exactly **6** `.mdc` files plus `README.md`; every file has `description`, `alwaysApply`, `globs` frontmatter. +2. No kit rule names a vendor, product or repo: `grep -Eqi 'bun|postman|new relic|intents|tao\.com|solhint|eslint' rules/*.mdc` returns **no match**. +3. `rules/README.md` states the four ownership tiers and the six-rule budget. +4. Setup script symlinks rules, merges `.agents/local-rules/`, and assembles a marked block into `AGENTS.md`; idempotent on a second run; refuses a foreign `.cursor/rules`. +5. Assembly strips frontmatter — no `alwaysApply:` inside the `AGENTS.md` block. +6. Loop in the process doc reads `BOOT → ERD? → RESEARCH? → CONTRACT → …`. +7. The ERD routing test states all three conditions **and** the skip rule, and says ERD and RESEARCH must never dual-run. +8. Sprint contract form has a slice-reference line for ERD-derived work. +9. Process law that moved from `AGENT_PROCESS.md` into `rules/` is a **pure move** — content diff empty in its own commit. +10. Version 2.2.0; bootstrap regenerates; both `FEATURES.json` parse; diff confined to the impact map; work on the confirmed branch. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — see VERIFY +- [ ] Trust-boundary / partial deps / http-api / solidity — **N/A** (docs) +- [ ] Code-quality gate — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor — criterion 9 is its analogue (empty content diff on the move commit) + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Kit rules collide with a repo's existing `.cursor/rules` of the same name | Local wins on collision; setup refuses a merge dir it did not generate | +| Always-on budget creeps back to 18 | Six-rule cap stated in `rules/README.md`; criterion 1 enforces the count | +| Rules and skills say the same thing twice | Rules carry invariants only; anything procedural stays in the skill that owns it | +| ERD? becomes mandatory ceremony | Routing test requires **all three** conditions; skip rule stated explicitly | +| Assembly block clobbers hand-written `AGENTS.md` | Marker-delimited, regenerated only between markers | +| Forked global harness keeps diverging | Out of scope, tracked as **Q3** | + +## Threat model + +**SKIP** — process/docs only. + +--- + +## Blocking questions (gates) + +**Q1 — One contract or two?** This covers a rules directory **and** an ERD routing change. +- **Recommendation:** one. Both edit the same process doc, so splitting means one rebases painfully + onto the other for no review benefit. Scope halves are clearly separated above and can be reviewed + independently. Split if you would rather merge them separately. + +**Q2 — Does `stage-after-execute` belong in the kit?** It says stage product files automatically once +VERIFY is green. That is a workflow opinion, not an invariant, and it interacts with "never commit unasked". +- **Recommendation:** merge it into `no-stage-harness-files` as one staging policy (what to stage, what + never to stage), rather than shipping an auto-stage instruction as kit law. + +**Q3 — The forked global harness.** `~/.cursor/skills/harness-engineering/` runs a 5-step loop with +`.harness/contracts/`, and several repo rules cite it as authoritative. harness-kit runs a 9-step loop +with profiles and gates. They disagree, and rules currently point at the older one. +- **Recommendation:** out of scope here; track as `KIT-RECONCILE-001`. Once kit rules ship, the global + skill should either be retired or reduced to a pointer at the kit. Leaving both authoritative + guarantees drift. + +**Q4 — Fold in the ledger commit?** `271b222` (PASS for KIT-SKILLS-001 / KIT-ERD-001) is unpushed on +`chore/kit-ledger-pass`. +- **Recommendation:** fold it in rather than spend a PR on two lines. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# rules exist, are frontmattered, and are exactly six +test "$(ls rules/*.mdc | wc -l)" -eq 6 +for f in rules/*.mdc; do + head -1 "$f" | grep -q '^---$' && grep -q '^description:' "$f" && grep -q '^alwaysApply:' "$f" || exit 1 +done +test -f rules/README.md +grep -qi 'ownership' rules/README.md + +# kit neutrality — no vendor, product or repo names in kit rules +! grep -Eqi 'bun|postman|new relic|intents|tao\.com|solhint|eslint' rules/*.mdc + +# setup wiring +grep -q 'cursor/rules' scripts/setup-harness-kit.sh +grep -q 'local-rules' scripts/setup-harness-kit.sh +grep -q 'harness-kit:rules:start' scripts/setup-harness-kit.sh +sh -n scripts/setup-harness-kit.sh + +# ERD routing in the loop +grep -q 'ERD?' skills/harness-onboard/templates/AGENT_PROCESS.md +grep -qi 'never dual-run' skills/harness-onboard/templates/AGENT_PROCESS.md +grep -qi 'slice' skills/sprint-contract/templates/SPRINT_CONTRACT.md + +# version + ledger +grep -q '2.2.0' scripts/build-bootstrap.sh && grep -q '2.2.0' BOOTSTRAP.md +sh scripts/build-bootstrap.sh +python3 -m json.tool FEATURES.json >/dev/null && grep -q 'KIT-RULES-001' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-RULES-001", + "name": "rules/ directory as Cursor .mdc with AGENTS.md assembly, plus ERD routing before contract drafting", + "priority": 1, + "verify": "test $(ls rules/*.mdc | wc -l) -eq 6 && test -f rules/README.md && grep -q 'harness-kit:rules:start' scripts/setup-harness-kit.sh && grep -q 'ERD?' skills/harness-onboard/templates/AGENT_PROCESS.md && grep -q '2.2.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-RULES-001.md; profile: ops-docs; guided by cursor-rules-export 2026-08-28; blocks KIT-GATES-001" +} +``` + +## Criteria block + +```criteria +- [KIT-RULES-001-1] rules/ has exactly 6 .mdc files with valid frontmatter, plus README +- [KIT-RULES-001-2] No vendor/product/repo names in any kit rule +- [KIT-RULES-001-3] rules/README states the four ownership tiers and the six-rule budget +- [KIT-RULES-001-4] Setup symlinks rules, merges local-rules, assembles marked AGENTS.md block, idempotent +- [KIT-RULES-001-5] Assembly strips frontmatter +- [KIT-RULES-001-6] Loop reads BOOT -> ERD? -> RESEARCH? -> CONTRACT +- [KIT-RULES-001-7] ERD routing test has all three conditions, the skip rule, and no dual-run with RESEARCH +- [KIT-RULES-001-8] Process law moved out of AGENT_PROCESS as a pure move (empty content diff) +``` + +--- + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | Criterion 9 expected process law **moved** out of `AGENT_PROCESS.md` with an empty content diff | **N/A — nothing moved.** The six rules were authored from the Cursor export and generalised; they are new content, not relocated text. `AGENT_PROCESS.md`'s 7 principles are the *reasoning*, the rules are the *enforceable form*, and they are not verbatim duplicates. A pointer was added instead. Deleting the principles would have been destructive surgery the contract did not scope. | +| 2 | Loop notation — David asked whether research precedes the ERD | Addressed beyond the contract's §C: step 3 now carries a table distinguishing **discovery research** (inside `erd-authoring` S0) from **slice research** (the loop's `RESEARCH?`), so `ERD? → RESEARCH?` no longer implies deciding before looking. Also cross-referenced from the skill. | +| 3 | Loop steps renumbered 1–9 | Inserting ERD? at position 2 shifted CONTRACT through REVIEW. Mechanical. | +| 4 | A VERIFY check failed on a line-wrap artifact ("skips / this") | Reflowed the sentence rather than loosening the check. | + +## Q2 as executed + +`stage-after-execute` was **merged into** `no-stage-harness-files` as one staging policy, per the +recommendation — what to stage, what never to stage, and the standing "do not commit unless asked". + +## Setup script — tested, not asserted + +Exercised end to end in a throwaway consumer repo: 6 rules linked · symlinks resolve · assembly writes +a marked block into `AGENTS.md` with **zero** `alwaysApply` leaked · hand-written `AGENTS.md` content +preserved · **second run changed nothing** and left exactly one block · local rules win on collision · +a foreign `.cursor/rules` is left alone with a warning. + +## VERIFY result + +All checks green. + +## Also in this branch + +`271b222` — PASS for KIT-SKILLS-001 and KIT-ERD-001, folded in per Q4. diff --git a/skills/.harness/contracts/KIT-SKILLS-001.md b/skills/.harness/contracts/KIT-SKILLS-001.md new file mode 100644 index 0000000..b2c8fd6 --- /dev/null +++ b/skills/.harness/contracts/KIT-SKILLS-001.md @@ -0,0 +1,323 @@ +# SPRINT CONTRACT — KIT-SKILLS-001: templates → skills, submodule adoption + +> Status: **EXECUTE complete — VERIFY green — PENDING_REVIEW.** +> Approved: David 2026-08-27 ("proceed") with recommended Q1–Q4. Branch confirmed. +> Commits: `d503ac1` (pure rename) + `008248d` (skills, setup, adoption). Version **2.0.0**. +> Feature ID: `KIT-SKILLS-001` +> Repo: **harness-kit only** +> Branch: proposed `feat/kit-skills-001-templates-to-skills` — **human must confirm before checkout** +> Trigger: David (2026-08-27) — "we need to turn templates into skills and then proceed with kit-erd-001." +> **This is a breaking change to the adoption model → version 2.0.0.** + +--- + +## Work-type profile (required) + +- [x] `ops-docs` — process/templates change to the kit itself + +**Languages / toolchain:** Markdown + POSIX sh. No application code. + +--- + +## Context (grounded, verified on `master` @ `3e22a12`) + +| Fact | Consequence | +| --- | --- | +| 13 files in `templates/`; no `SKILL.md` anywhere in the kit | Nothing to migrate from; this is a first conversion | +| `BOOTSTRAP.md` is 1947 lines, sold as "copy this file into **any** repo" | Retiring it removes the kit's entire adoption story — must be replaced, not just deleted | +| Appendices B–M are written to `skills/.harness/templates/*` | That path is **gitignored** (`.gitignore:20`) — today's templates are disposable local copies | +| Skills are discovered by directory + `SKILL.md` frontmatter | They must be **reachable in the tree**, which the gitignored path is not | +| `scripts/build-bootstrap.sh` regenerates `BOOTSTRAP.md` from `templates/*` | Becomes obsolete once appendices stop being the distribution vector | +| Kit currently has **no** setup script | Submodule adoption needs one (symlink wiring, brownfield safety) | + +**Decisions taken by David (2026-08-27):** procedures become skills and forms ship bundled inside them; +skills are tracked, kit-owned; **adoption moves from single-file BOOTSTRAP to git submodule**; +skills live at `skills//SKILL.md`. + +--- + +## Decisions + +| Topic | Decision | Status | +|-------|----------|--------| +| What becomes a skill | **Procedures only.** Fill-in forms ship as `templates/` **inside** the owning skill | Locked | +| Skill count | **7** (see §B) — not 13; a blank form with frontmatter is indirection without behaviour | Locked | +| Adoption | **Git submodule + setup script.** Skills stay canonical in harness-kit; consumers symlink | Locked | +| Per-repo copies | **None.** Symlinks, not copies — so kit upgrades are a submodule pointer bump and drift is impossible | Locked | +| Location | `skills//SKILL.md`, siblings of the existing `skills/.harness/` working directory | Locked | +| `skills/.harness/` | **Unchanged** — contracts, STATE, local templates stay gitignored working state | Locked | +| `BOOTSTRAP.md` | **Deprecated, not deleted, this sprint.** See Q2 | Locked | +| Version | **2.0.0** — breaking change to how repos adopt the kit | Locked | +| Tool coupling | Accepted. `SKILL.md` frontmatter is a Claude/Cursor/skills-CLI convention; `AGENTS.md` remains the tool-agnostic entrypoint so a non-skill agent still works | Locked | + +--- + +## Scope — WILL do + +### A — Skill format + +1. Each skill: `skills//SKILL.md` with YAML frontmatter (`name`, `description` — third person, trigger + terms, ≤1024 chars), and bundled `skills//templates/*.md` for its forms. +2. Each generated/kit-owned skill carries a "kit-owned — edit in harness-kit, not here" marker. + +### B — The seven skills (procedures), with their bundled forms + +| Skill | Procedure | Bundled forms | +|-------|-----------|---------------| +| `harness-onboard` | Bootstrap a repo into the process | `AGENTS.md`, `AGENT_PROCESS.md`, `FEATURES.json`, `STATE.md`, `README_POINTER.md` | +| `sprint-contract` | Write a contract; get approval; confirm branch | `SPRINT_CONTRACT.md` | +| `code-quality-gate` | Run the per-diff gate at VERIFY | `CODE_QUALITY.md` | +| `code-audit` | Run a periodic subsystem audit | `CODE_AUDIT_CONTRACT.md`, `CODE_AUDIT_FINDING.md` | +| `solidity-audit` | Run a contract security audit | `AUDIT_CONTRACT.md`, `AUDIT_FINDING.md` | +| `security-checklist` | Pre-merge security self-check | `SECURITY_CHECKLIST.md` | +| `harness-review` | PR review, §0 contract compliance first | `REVIEW.md` | + +3. Content is **moved, not rewritten**. Each `SKILL.md` adds only frontmatter, a short "when to use / + when not to", and a pointer to its bundled form. No process changes ride along in this sprint. + +### C — Submodule adoption + +4. Add `scripts/setup-harness-kit.sh` — run from the **consumer repo root**, detects the git superproject, and: + - verifies the submodule is present at a known path + - creates a merge directory of per-skill symlinks into the submodule + - wires the tool paths that exist in that repo (`.claude/skills`, `.cursor/skills`, `.agents/skills`) + - refuses to overwrite anything it did not generate (marker file), and preserves unrelated entries + - is idempotent — running twice changes nothing +5. `README.md` — replace the BOOTSTRAP adoption instructions with submodule adoption + (`git submodule add`, `git submodule update --init`, run setup, re-run after a pointer bump). +6. Document the **upgrade path**: bump the submodule pointer, re-run setup. No per-repo file edits. + +### D — Path references + +7. Rewrite every `skills/.harness/templates/.md` reference in `templates/AGENT_PROCESS.md`, + `templates/AGENTS.md`, `templates/REVIEW.md`, `templates/SPRINT_CONTRACT.md`, + `templates/CODE_QUALITY.md`, `templates/CODE_AUDIT_CONTRACT.md` to the skill path. +8. Keep `skills/.harness/contracts/` and `skills/.harness/STATE.md` exactly as they are — working state, + still gitignored, unaffected by this change. + +### E — Deprecate BOOTSTRAP + +9. `BOOTSTRAP.md` gains a deprecation banner naming the submodule path and the removal target + (`KIT-SKILLS-002`). **Not deleted this sprint** — repos bootstrapped from it still reference it. +10. `scripts/build-bootstrap.sh` gains the same banner and stays runnable until removal. + +### F — Version + ledger + +11. Version **2.0.0** across `README.md`, the setup script header, and the deprecation banners; changelog + entry naming the breaking change and the migration. +12. Add `KIT-SKILLS-001` to root `FEATURES.json`. + +## Scope — will NOT do (this sprint) + +- Delete `BOOTSTRAP.md` or `build-bootstrap.sh` → `KIT-SKILLS-002`. +- Change any process rule, gate, profile, threshold or checklist content. **Pure relocation.** +- Migrate any consumer repo (including `technical-project-documentation`, which uses `.harness/contracts/`). +- Build `KIT-ERD-001` — it follows, in the new skill shape. +- Adopt, reference or absorb any other pack repo; the kit stays pack-agnostic. +- Add per-repo editable copies of skills. + +## Target + +- **Repo / package:** `harness-kit`. +- **Not touching:** every consumer repo; any pack repo; application source. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master`; base `master` @ `3e22a12` (v1.5.0, PR #16 merged) — **[GROUNDED]** +- **Proposed feature branch:** `feat/kit-skills-001-templates-to-skills` +- **Human confirmed:** **pending — agent asks before checkout.** + +--- + +## Tests first (TDD) + +**N/A** — documentation/process sprint. VERIFY uses falsifiable deterministic checks. + +- **Expected RED (today):** + - `test -f skills/code-quality-gate/SKILL.md` → fails + - `test -f scripts/setup-harness-kit.sh` → fails + - `grep -q '2.0.0' README.md` → fails + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `skills/<7 names>/SKILL.md` | **NEW** — 7 procedure skills | [NEW] | +| `skills//templates/*.md` | **MOVED** from `templates/` (git mv; content unchanged) | [GROUNDED] | +| `scripts/setup-harness-kit.sh` | **NEW** — submodule wiring, idempotent | [NEW] | +| `README.md` | Submodule adoption replaces BOOTSTRAP instructions; 2.0.0 | [GROUNDED] | +| `templates/AGENT_PROCESS.md` → skill | Path references rewritten | [GROUNDED] | +| `templates/AGENTS.md` → skill | Path references rewritten | [GROUNDED] | +| `BOOTSTRAP.md`, `scripts/build-bootstrap.sh` | Deprecation banners only | [GROUNDED] | +| `FEATURES.json` | Add `KIT-SKILLS-001` | [GROUNDED] | + +**Reuse:** the submodule + symlink + merge-directory + brownfield-safety pattern is already proven in a +sibling repo's setup script. Adapt the approach; **do not** reference or depend on that repo. + +--- + +## Success criteria + +1. Seven `skills//SKILL.md` exist, each with valid frontmatter (`name`, `description` ≤1024 chars). +2. Every former `templates/*.md` file is reachable inside exactly one skill; none orphaned, none duplicated. +3. `git log --follow` shows the forms were **moved**, not rewritten — content diff is empty for each. +4. `scripts/setup-harness-kit.sh` is executable POSIX sh, idempotent (two runs → no diff), and refuses to + overwrite files it did not generate. +5. No reference anywhere to `skills/.harness/templates/` remains outside the deprecated BOOTSTRAP files. +6. `skills/.harness/contracts/` and `STATE.md` still gitignored and untouched. +7. `README.md` documents submodule adoption and the pointer-bump upgrade path; version 2.0.0. +8. `BOOTSTRAP.md` and `build-bootstrap.sh` carry deprecation banners and still run. +9. Root `FEATURES.json` valid and contains `KIT-SKILLS-001`. +10. No process rule changed: `git diff` on moved content shows relocation only. +11. Diff confined to the impact map; work on the confirmed feature branch. + +## Quality gates (tick or N/A) + +- [ ] Falsifiable success criteria — see VERIFY +- [ ] Trust-boundary / partial deps / http-api / solidity — **N/A** (docs-only) +- [ ] Code-quality gate — **N/A** (`ops-docs`) +- [ ] Behaviour-preserving refactor gate — **the content-move criterion (3) is its analogue**: the "behaviour" preserved here is the process text, proven by an empty content diff across the move + +## Edge cases / failure modes + +| Risk | Mitigation | +| --- | --- | +| Process text silently edited during the move | Criterion 3: empty content diff per moved file, enforced in VERIFY | +| Repos bootstrapped from BOOTSTRAP break | Deprecated, not deleted; removal deferred to KIT-SKILLS-002 | +| Setup script clobbers hand-authored skills | Generated-marker rule; refuses unknown content (criterion 4) | +| Symlinks break on Windows | Setup is re-runnable; documented as the fix | +| Kit becomes coupled to one agent vendor | `AGENTS.md` stays the tool-agnostic entrypoint; skills are additive | +| Two setup scripts drift across repos | Flagged as **Q3** — not silently duplicated | + +## Threat model + +**SKIP** — documentation/process only. Setup script performs no network or privileged operations. + +--- + +## Blocking questions (gates) + +**Q1 — Skill names.** Proposed: `harness-onboard`, `sprint-contract`, `code-quality-gate`, `code-audit`, +`solidity-audit`, `security-checklist`, `harness-review`. +- **Recommendation:** as listed. `harness-` prefixes only where the bare noun would collide with a + common existing skill name (`review`, `onboard`). + +**Q2 — BOOTSTRAP removal timing.** Deprecate now and delete in `KIT-SKILLS-002`, or delete now? +- **Recommendation:** deprecate now. Deleting the only adoption path in the same sprint that introduces + its replacement leaves no rollback. "Build to delete" — after the replacement is proven, not before. + +**Q3 — Setup-script duplication.** A sibling pack repo already has a near-identical submodule + symlink +setup script. Two scripts solving the same problem will drift. +- **Recommendation:** accept the duplication for now and record it as known debt. Sharing it would couple + the kit to a pack, which contradicts the pack-agnostic rule. Revisit only if both scripts churn. + +**Q4 — Consumer migration.** Repos already bootstrapped from `BOOTSTRAP.md` (e.g. the docs repo, which +uses `.harness/contracts/`) are unaffected today but will need migrating eventually. +- **Recommendation:** out of scope here; write a migration note in `README.md` and track it as a separate + contract per repo. Do not migrate anyone's repo from inside a kit sprint. + +*Approve only if Q1–Q4 (or your edits) are acceptable.* + +--- + +## VERIFY (run after EXECUTE) + +```bash +cd "$HOME/mnt/harness-kit" + +# 1. seven skills with frontmatter +for s in harness-onboard sprint-contract code-quality-gate code-audit solidity-audit security-checklist harness-review; do + test -f "skills/$s/SKILL.md" || { echo "missing $s"; exit 1; } + head -1 "skills/$s/SKILL.md" | grep -q '^---$' || { echo "$s: no frontmatter"; exit 1; } + grep -q '^name:' "skills/$s/SKILL.md" && grep -q '^description:' "skills/$s/SKILL.md" || exit 1 +done + +# 2. every former template lands in exactly one skill, none left behind +test -z "$(ls templates/*.md 2>/dev/null)" || echo "templates/ not emptied" +test "$(find skills -path 'skills/.harness' -prune -o -name '*.md' -path '*/templates/*' -print | wc -l)" -eq 13 + +# 3. moves, not rewrites +git diff --cached -M --stat | grep -q '=>' || echo "expected renames" +git diff --cached -M --diff-filter=R --numstat | awk '$1!=0||$2!=0{print "CONTENT CHANGED: "$3; bad=1} END{exit bad+0}' + +# 4. setup script idempotent + safe +test -x scripts/setup-harness-kit.sh +sh -n scripts/setup-harness-kit.sh + +# 5. no stale template paths outside deprecated files +! grep -rn 'skills/.harness/templates/' --include='*.md' . \ + | grep -v '^./BOOTSTRAP.md' | grep -v '^./skills/.harness/' | grep -q . + +# 6. working state untouched +git check-ignore -q skills/.harness/contracts && echo "contracts still ignored" + +# 7/8. adoption + deprecation +grep -q 'submodule' README.md && grep -q '2.0.0' README.md +grep -qi 'deprecat' BOOTSTRAP.md && grep -qi 'deprecat' scripts/build-bootstrap.sh +sh scripts/build-bootstrap.sh >/dev/null + +# 9. ledger +python3 -m json.tool FEATURES.json >/dev/null && grep -q 'KIT-SKILLS-001' FEATURES.json +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-SKILLS-001", + "name": "Templates become skills; adoption moves from single-file BOOTSTRAP to git submodule", + "priority": 1, + "verify": "test -f skills/code-quality-gate/SKILL.md && test -f skills/code-audit/SKILL.md && test -x scripts/setup-harness-kit.sh && grep -q '2.0.0' README.md && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-SKILLS-001.md; profile: ops-docs; BREAKING (adoption model); blocks KIT-ERD-001" +} +``` + +--- + +## Criteria block + +```criteria +- [KIT-SKILLS-001-1] Seven procedure skills exist with valid frontmatter +- [KIT-SKILLS-001-2] All 13 forms bundled inside exactly one skill; templates/ emptied +- [KIT-SKILLS-001-3] Forms moved not rewritten — zero content diff across renames +- [KIT-SKILLS-001-4] setup-harness-kit.sh is POSIX, idempotent, refuses foreign content +- [KIT-SKILLS-001-5] No stale skills/.harness/templates/ references outside deprecated files +- [KIT-SKILLS-001-6] skills/.harness working state untouched and still gitignored +- [KIT-SKILLS-001-7] README documents submodule adoption + pointer-bump upgrade; version 2.0.0 +- [KIT-SKILLS-001-8] BOOTSTRAP + build script deprecated but still runnable +``` + +--- + +## Deviations / clarifications + +| # | Item | Resolution | +|---|------|------------| +| 1 | Criterion 3 ("moves not rewrites, empty content diff") is unsatisfiable at branch HEAD, because §D7 also requires rewriting path references **inside** those same files | Split into two commits. `d503ac1` is the pure rename — 13 files, 0 insertions, 0 deletions, verifiable in isolation. `008248d` carries every content change. The criterion is checked against `d503ac1`, which is where it is meaningful. Implements the intent exactly; strictly more auditable than one commit. | +| 2 | Contract VERIFY counted `-name '*.md' -path '*/templates/*'` and expected 13 | Off by one — 12 `.md` plus `FEATURES.json`. VERIFY counts all files, not just `.md`. Contract text was wrong; output is right. | +| 3 | `templates/` remained on disk after `git mv` | Empty directory artifact — git does not track directories, nothing tracked remained. Removed with `rmdir`. | + +## Path-reference style (design note) + +Cross-references were rewritten to name **skills** ("the `security-checklist` skill") rather than +file paths. A path would bake in one tool's merge directory (`.agents/` vs `.claude/` vs `.cursor/`); +a skill name resolves wherever the repo wires them, so the kit stays tool-agnostic. + +## Setup script — tested, not asserted + +Exercised end to end in a throwaway consumer repo: 7 skills linked · symlinks resolve to real +content · `.claude`/`.cursor` wired · gitignore appended exactly once · **second run produced no +change** · refuses a `.agents/skills` it did not generate · project-local skills win on name +collision · a non-symlink `.cursor/skills` is left alone with a warning. + +## VERIFY result + +All checks green (23/23 after the empty-directory fix). diff --git a/skills/.harness/contracts/KIT-V13-001.md b/skills/.harness/contracts/KIT-V13-001.md new file mode 100644 index 0000000..951255f --- /dev/null +++ b/skills/.harness/contracts/KIT-V13-001.md @@ -0,0 +1,264 @@ +# SPRINT CONTRACT — KIT-V13-001: Universal multi-language spine + audit profile + +> Status: **EXECUTE complete — PENDING_REVIEW** +> Feature ID: `KIT-V13-001` +> Repo: **harness-kit only** +> Branch: `feat/kit-v13-001-universal-spine` from `master` (approved 2026-08-02) +> VERIFY: green 2026-08-02. Human sets PASS on merge. + +--- + +## Goal + +Evolve harness-kit so the **same process law** works for any project across TypeScript, Rust, Go, Solidity build, and Solidity contract auditing. Keep the control plane language-neutral. Add work-type profiles and an audit contract template. Leave stack craft (Vitest, Cargo idioms, Foundry recipes, hexagonal) to optional packs / consumer `AGENTS.md`, not the core loop. + +--- + +## Scope — WILL do + +### A — Version + bootstrap + +1. Bump kit version to **1.3.0** in `BOOTSTRAP.md` header / changelog / `VERSION` instruction and `scripts/build-bootstrap.sh` embedded header. +2. Add v1.3 changelog bullets covering: `PENDING_REVIEW`, contract quality gates (N/A-by-default), work-type profiles, REVIEW §0, Solidity audit template, language-neutral VERIFY. +3. Re-run `./scripts/build-bootstrap.sh` so appendices stay in sync with `templates/*`. + +### B — FEATURES status model + +4. Update `templates/FEATURES.json` legend to three states: `FAIL` | `PENDING_REVIEW` | `PASS`. +5. Document owners in `templates/AGENT_PROCESS.md` / `templates/AGENTS.md` § Agent process: + - Executor may set `PENDING_REVIEW` after VERIFY green. + - Only human/reviewer sets `PASS`. + - Executor never sets `PASS`. +6. Add root `FEATURES.json` entry for `KIT-V13-001` (`FAIL` until merge; notes link this contract). Mark superseded historical FAIL rows that are dead (`KIT-INSTALL-001`, `KIT-UPDATE-001`, `KIT-SETUP-001`, `KIT-DEPRECATE-ADOPTION-001`, `KIT-STATE-001`) as `PASS` with notes `superseded` **or** leave FAIL but add a `notes` clarification that they are historical supersessions (prefer: set `PASS` + superseded note to reduce noise). + +### C — Agent process (universal spine) + +7. Update session loop to: + +``` +BOOT → RESEARCH? → CONTRACT → [HUMAN APPROVES + BRANCH] → CHECKOUT → EXECUTE (TDD when behaviour changes) → VERIFY → REVIEW → PENDING_REVIEW → [HUMAN PASS/MERGE] +``` + +8. Add **depth matrix** (trivial / medium / large / audit) — when to skip Research, when condensed process is allowed. +9. Add **work-type profile** selection (must pick one per contract): + - `service` (TS / Go / Rust app logic) + - `http-api` (HTTP surface) + - `solidity-build` + - `solidity-audit` + - `ops-docs` +10. State explicitly: VERIFY commands come from the contract + consumer `AGENTS.md`; kit never mandates a language runner. +11. State explicitly: architecture patterns (hexagonal, etc.) apply only when the consumer `AGENTS.md` declares them. +12. Optional Research: one paragraph — distill ticket/ERD/scope into open questions before Contract when work is large or audit; do not invent a second approved artifact (`plan.md` is not required; decisions live in the sprint contract). + +### D — Sprint contract template + +13. Extend `templates/SPRINT_CONTRACT.md` with: + - **Work-type profile** (required enum above) + - **Languages / toolchain** (free text; cite consumer AGENTS.md) + - **Decisions** table (topic / decision / locked) + - **Vertical slice / TDD order** (behaviour + red/green; N/A for pure audit finding write-ups that use PoC-first instead) + - **Quality gates** checklist, each tickable or **N/A**: + - Falsifiable success criteria (always required) + - Trust-boundary failure mode documented (typed error / revert / Err / 4xx — never uncaught panic/throw across handler boundary where applicable) + - Partial/optional deps: skip vs reject documented + - **HTTP only:** request + response API schema updated; error envelope regression in VERIFY + - **Solidity build only:** invariant/access-control/value-flow risks named; forge (or declared) test commands + - **Solidity audit only:** use `AUDIT_CONTRACT.md` instead or embed audit sections + - Keep existing: Scope, Branch, Tests first, Impact map (GROUNDED/EDUCATED), Threat model, Blocking questions, FEATURES entry + +### E — REVIEW template + +14. Prepend **§0 Sprint contract compliance** to `templates/REVIEW.md` (adapted from practice PR-REVIEW; paths say `skills/.harness/contracts/.md` for kit consumers). +15. Add profile-aware review hints (short): + - service: correctness, reliability, ops + - http-api: compat + envelope + - solidity-build: authz, upgrade/pause, economic safety + - solidity-audit: severity calibration, FP risk, missing bug classes +16. Keep existing production sections 1–14; renumber if needed so §0 is first. + +### F — Solidity audit profile + +17. Add new template `templates/AUDIT_CONTRACT.md`: + - Scope in/out (contracts, functions, chains, commit/tag) + - Assets / actors / trust boundaries + - Prior reports / known issues + - Severity rubric (Critical/High/Medium/Low/Informational or project rubric) + - Tooling plan (forge tests, fuzz, invariant, slither/mythril if used — commands not brand-mandatory) + - Finding ID scheme (`AUDIT-NNN-F0x` or similar) + - PoC-first rule: High/Critical require reproducible VERIFY command + - Out of scope explicit + - Branch + human approval (same as sprint contracts) + - Link to fix follow-up as normal `solidity-build` sprint contracts +18. Add `templates/AUDIT_FINDING.md` (short finding write-up: title, severity, location, impact, PoC, fix suggestion, status). +19. Extend `templates/SECURITY_CHECKLIST.md` with a **Solidity / audit** section (reentrancy, auth, oracle/manipulation, upgrade, signature, DoS, unexpected external calls) marked as required for `solidity-build` and `solidity-audit` profiles; existing web sections remain N/A when not applicable. +20. Wire audit templates into Step 0 / build-bootstrap appendices (new Appendix letters after existing). + +### G — Docs pointers + +21. Update `README.md` / `templates/README_POINTER.md` to mention: multi-language spine, work-type profiles, audit template, `PENDING_REVIEW`. +22. Update `templates/AGENTS.md` Stack examples to list TS / Rust / Go / Solidity as first-class; point How we work here at profile selection. +23. One short subsection: **Skill packs (optional)** — agent-harness or local skills may supply stack craft; they must not replace FEATURES/contracts/VERIFY; hexagonal/TDD router skills are optional and stack-specific. + +--- + +## Scope — will NOT do (this sprint) + +- Adopt `agent-harness` as a submodule inside harness-kit or rewrite RPI skills. +- Write full Rust / Go / TypeScript / Foundry **skill packs** (only stubs/pointers in AGENTS/README). +- Change consumer repos (`intents-service`, `tao-intent-solver`, contracts repo, etc.). +- Mandate OpenAPI/Postman/hexagonal/Vitest in the universal loop. +- Reintroduce `HARNESS.md`, `LOOP.md`, `install.sh`, or root committed contracts. +- Automate audit tooling CI beyond documenting VERIFY command patterns. +- Language-specific code samples beyond short command examples in templates. + +--- + +## Branch (feature branch — mandatory) + +- **Default branch:** `master` +- **Proposed feature branch:** `feat/kit-v13-001-universal-spine` +- **Human confirmed:** pending — agent asks before checkout + +--- + +## Tests first (TDD — docs kit) + +This sprint is **template/docs process**. “Tests” are deterministic verify commands, not application unit tests. + +- **RED:** before edits, note current fails (e.g. templates lack `PENDING_REVIEW`; no `AUDIT_CONTRACT.md`). +- **GREEN:** after edits, all VERIFY commands below pass. +- **Expected RED (today):** + - `grep -q PENDING_REVIEW templates/FEATURES.json` → fail + - `test -f templates/AUDIT_CONTRACT.md` → fail + +--- + +## Impact map + +| Path | Change | Grounding | +|------|--------|-----------| +| `templates/FEATURES.json` | 3-state legend | [GROUNDED] | +| `templates/AGENT_PROCESS.md` | Loop, PENDING_REVIEW, depth matrix, profiles, language-neutral VERIFY | [GROUNDED] | +| `templates/AGENTS.md` | Embed matching Agent process + stack/profile notes | [GROUNDED] | +| `templates/SPRINT_CONTRACT.md` | Profile, decisions, gates, slice | [GROUNDED] | +| `templates/REVIEW.md` | §0 compliance + profile hints | [GROUNDED] | +| `templates/SECURITY_CHECKLIST.md` | Solidity/audit section | [GROUNDED] | +| `templates/AUDIT_CONTRACT.md` | **NEW** | [NEW] | +| `templates/AUDIT_FINDING.md` | **NEW** | [NEW] | +| `templates/README_POINTER.md` | Pointers for v1.3 | [GROUNDED] | +| `README.md` | Brief v1.3 mention if needed | [GROUNDED] | +| `BOOTSTRAP.md` | Regenerated via script + Step 0 / changelog / appendices for audit templates | [GROUNDED] | +| `scripts/build-bootstrap.sh` | Header 1.3.0; append new audit templates | [GROUNDED] | +| `FEATURES.json` (root) | Add KIT-V13-001; clean superseded noise | [GROUNDED] | + +Reuse: existing REVIEW body, SECURITY workflow safety, SPRINT_CONTRACT branch/TDD sections. Prefer extend over rewrite. + +--- + +## Success criteria + +1. Kit version string **1.3.0** appears in `BOOTSTRAP.md` and build script header. +2. `templates/FEATURES.json` legend lists exactly `FAIL`, `PENDING_REVIEW`, `PASS` with owner notes in Agent process. +3. `templates/SPRINT_CONTRACT.md` requires a work-type profile and includes quality gates with explicit N/A paths. +4. `templates/REVIEW.md` starts with §0 contract compliance before production sections. +5. `templates/AUDIT_CONTRACT.md` and `templates/AUDIT_FINDING.md` exist and are included in bootstrap generation. +6. Agent process states VERIFY commands are repo-declared (no hardcoded Vitest/Cargo/forge as the only runner). +7. Agent process states hexagonal/stack skills are optional and must not replace the contract spine. +8. `./scripts/build-bootstrap.sh` exits 0; regenerated `BOOTSTRAP.md` contains `PENDING_REVIEW`, `AUDIT_CONTRACT`, and `1.3.0`. +9. Diff confined to harness-kit paths in the impact map (plus this local contract under gitignored `skills/.harness/`). +10. All work on confirmed feature branch, not `master`. + +### Quality gates (this sprint) + +- [x] Falsifiable success criteria (above) +- [x] Trust-boundary N/A (docs-only kit) +- [x] HTTP OpenAPI N/A +- [x] Solidity runtime N/A (templates only) +- [x] No consumer repo edits + +--- + +## Verify + +```bash +cd /Users/davidasamonye/Work/tao.com/harness-kit + +# version +grep -q '1.3.0' BOOTSTRAP.md +grep -q '1.3.0' scripts/build-bootstrap.sh + +# status model +grep -q 'PENDING_REVIEW' templates/FEATURES.json +grep -q 'PENDING_REVIEW' templates/AGENT_PROCESS.md + +# contract template gates + profiles +grep -q 'Work-type profile' templates/SPRINT_CONTRACT.md +grep -q 'Quality gates' templates/SPRINT_CONTRACT.md +grep -q 'solidity-audit' templates/SPRINT_CONTRACT.md + +# review §0 +grep -q 'Sprint contract compliance' templates/REVIEW.md + +# audit templates +test -f templates/AUDIT_CONTRACT.md +test -f templates/AUDIT_FINDING.md +grep -q 'PoC' templates/AUDIT_CONTRACT.md + +# security solidity section +grep -qi 'reentrancy\|Solidity' templates/SECURITY_CHECKLIST.md + +# bootstrap regen + JSON +./scripts/build-bootstrap.sh +grep -q '1.3.0' BOOTSTRAP.md +grep -q 'AUDIT_CONTRACT' BOOTSTRAP.md +python3 -m json.tool FEATURES.json >/dev/null +python3 -m json.tool templates/FEATURES.json >/dev/null + +# root feature entry +grep -q 'KIT-V13-001' FEATURES.json + +# branch check (after checkout) +git branch --show-current | grep -qv '^master$' +``` + +--- + +## FEATURES.json entry + +```json +{ + "id": "KIT-V13-001", + "name": "Universal multi-language spine: PENDING_REVIEW, profiles, REVIEW §0, Solidity audit templates", + "priority": 1, + "verify": "grep -q '1.3.0' BOOTSTRAP.md && grep -q PENDING_REVIEW templates/FEATURES.json && test -f templates/AUDIT_CONTRACT.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null", + "status": "FAIL", + "notes": "Contract: skills/.harness/contracts/KIT-V13-001.md" +} +``` + +--- + +## Blocking questions (gates) + +1. **Contracts path for consumers:** keep documenting `skills/.harness/contracts/` as in kit v1.2, or also allow `.harness/contracts/` as an equally valid local path? + - **Recommendation:** keep `skills/.harness/` as the kit-documented default; add one sentence that existing repos using `.harness/contracts/` are acceptable if `AGENTS.md` says so (no migration forced this sprint). +2. **Superseded FEATURES rows:** set historical superseded FAILs to `PASS` with notes, or leave FAIL? + - **Recommendation:** set to `PASS` + `superseded by KIT-SINGLE-LLM-001 / v1.2` to reduce triage noise. +3. **agent-harness mention:** brief “optional companion” paragraph only (this sprint), full integration later? + - **Recommendation:** brief only. + +Approve this contract only if Q1–Q3 (or your edits) are acceptable. + +--- + +## Criteria block + +```criteria +- [KIT-V13-001-1] Version 1.3.0 in BOOTSTRAP + build script +- [KIT-V13-001-2] PENDING_REVIEW in FEATURES template + Agent process owners +- [KIT-V13-001-3] SPRINT_CONTRACT has profiles + quality gates with N/A +- [KIT-V13-001-4] REVIEW.md §0 contract compliance present +- [KIT-V13-001-5] AUDIT_CONTRACT.md + AUDIT_FINDING.md exist and bootstrap-embedded +- [KIT-V13-001-6] VERIFY language-neutral; stack packs optional in Agent process +- [KIT-V13-001-7] build-bootstrap.sh green; FEATURES.json valid; KIT-V13-001 listed +``` diff --git a/skills/code-quality-gate/templates/CODE_QUALITY.md b/skills/code-quality-gate/templates/CODE_QUALITY.md index ae6ca7d..e2dd197 100644 --- a/skills/code-quality-gate/templates/CODE_QUALITY.md +++ b/skills/code-quality-gate/templates/CODE_QUALITY.md @@ -11,6 +11,9 @@ > `CODE_AUDIT_CONTRACT.md` asks "where should we invest refactoring effort?" on a cadence, over a > declared subsystem. Do not substitute one for the other. +> Process discipline — ledger, contract, scope containment, RED artifact — is checked by +> `verify-harness.sh`, not here. This gate judges the code; that one judges the contract. + ## 0. Sensor - [ ] Repo's declared lint / static-analysis command from `AGENTS.md` was **run** on the diff diff --git a/skills/harness-onboard/templates/AGENT_PROCESS.md b/skills/harness-onboard/templates/AGENT_PROCESS.md index ed5f643..13dd1c8 100644 --- a/skills/harness-onboard/templates/AGENT_PROCESS.md +++ b/skills/harness-onboard/templates/AGENT_PROCESS.md @@ -229,6 +229,20 @@ is **N/A** for `ops-docs`, `code-audit` and `product-erd`, and for `solidity-aud ship code of their own. Credential findings belong to `SECURITY_CHECKLIST.md`, not this gate — raise them there and do not double-report. +Then run the gate: + +```sh +./scripts/verify-harness.sh # or the path this repo declares +``` + +It checks the discipline rather than the code: ledger integrity, that every `PENDING_REVIEW` has a +contract, that skills and rules will actually load, that every `[GROUNDED]` path in the contract +**exists**, that the diff is **contained by the impact map**, and that RED was recorded and failed on +an assertion. **The executor may set `PENDING_REVIEW` only when this exits 0.** + +Rules and skills are instructions to a model that can misread them under load. This is a fact — and +**CI runs it again**, so a claim that it passed locally is not evidence. + If verify fails: stay `FAIL`, fix or revise the contract. #### 9. REVIEW → PENDING_REVIEW → HUMAN PASS diff --git a/skills/harness-review/templates/REVIEW.md b/skills/harness-review/templates/REVIEW.md index fed0d18..b1cb471 100644 --- a/skills/harness-review/templates/REVIEW.md +++ b/skills/harness-review/templates/REVIEW.md @@ -31,6 +31,11 @@ Before deep code review, locate and read the relevant contract: code-quality gate (`CODE_QUALITY.md`: lint command recorded, violations = 0 or documented), solidity-audit PoCs. Mark each **Pass / Fail / N/A**. 8. **VERIFY commands** — run or confirm the contract's verify block passed; note any skipped or failing commands. +9. **Gate artifacts** — confirm the contract carries a recorded **RED** (real command, non-zero exit, + assertion-shaped failure), a **BASELINE**, and a **GREEN** whose counts match the baseline. A + predicted RED, or one that failed on an import error, is a finding. `N/A` is only valid when the + sprint changed no behaviour. +10. **Gate ran** — confirm `verify-harness.sh` exited 0 **in CI**, not only locally. If no contract exists (hotfix, drive-by), state that explicitly and review on production-readiness only. Retroactive contract may be required before PASS. diff --git a/skills/sprint-contract/templates/SPRINT_CONTRACT.md b/skills/sprint-contract/templates/SPRINT_CONTRACT.md index 9e9db74..24b3225 100644 --- a/skills/sprint-contract/templates/SPRINT_CONTRACT.md +++ b/skills/sprint-contract/templates/SPRINT_CONTRACT.md @@ -50,7 +50,7 @@ Pick **one**: > Implementation on the default branch is forbidden. The human must confirm the branch name > before the agent creates or checks out the branch. -- **Default branch:** +- **Default branch:** - **Proposed feature branch:** `` - **Human confirmed:** @@ -72,10 +72,34 @@ Pick **one**: - **Test files to create or extend:** - `` — -- **Expected RED output:** -- **Verify commands (GREEN):** - - `` - - `` +> **A test that never failed proves nothing.** It may be vacuous, or already passing. Record the real +> run — command, exit code, and the failure message — not a prediction of it. A RED that failed on an +> import or path error proves the test file is wrong, not that the behaviour is absent. +> +> **No ERD? Same gate.** Without a design document the success criteria below are the whole story, and +> each must be expressible as a **failing test before work starts**. If you cannot write a test that +> fails today, you do not have a falsifiable criterion and this contract is not approvable. + +### RED (recorded before EXECUTE) + +- **Command:** `` +- **Exit code:** `` +- **Failure reason:** `` +- **Captured:** `` + +**N/A** only when this sprint changes no behaviour (`ops-docs`, `code-audit`, `product-erd`). + +### BASELINE (recorded before EXECUTE — behaviour changes only) + +- **Command:** `` +- **Result:** `` + +### GREEN (recorded after EXECUTE) + +- **Command:** `` → **exit 0** +- **Command:** `` → **``** + +Green-after means nothing without green-before. A count that moved is a regression until explained. ### Vertical slice (when non-trivial) @@ -88,8 +112,13 @@ Pick **one**: ## Impact map -> Mark each path **[GROUNDED]** (verified in repo) or **[EDUCATED]** (must re-verify before -> implementing). Never present educated guesses as grounded. +> Mark each path **[GROUNDED]** (verified in repo), **[EDUCATED]** (must re-verify before +> implementing), or **[NEW]**. Never present educated guesses as grounded. +> +> **These markers belong to the impact map only**, and the impact map is a table whose **first cell is +> the path in backticks**. The gate reads that structure: it parses this section's table rows rather +> than guessing which backticked token elsewhere in the contract is a path. A marker used outside this +> table, or a marked row whose first cell is not a backticked path, is a gate failure. - **Files to change:** - `` — — [GROUNDED|EDUCATED]