Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/harness.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: harness

# The unforgeable layer. Everything else in the kit is a instruction to a model
# that can misread it; this re-runs the checks itself, so an agent's claim that
# VERIFY passed stops mattering.
#
# It invokes scripts/verify-harness.sh — it never reimplements a check.

on:
pull_request:
workflow_dispatch:

jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # scope containment needs the merge base

- name: Harness discipline
run: |
test -x scripts/verify-harness.sh || chmod +x scripts/verify-harness.sh
./scripts/verify-harness.sh --base "origin/${{ github.base_ref }}"

- name: Contract VERIFY block
run: |
# Re-run the contract's own verify commands. A local green is not evidence.
state=skills/.harness/STATE.md
[ -f "$state" ] || state=.harness/STATE.md
contract=$(grep -o '[A-Za-z0-9._/-]*contracts/[A-Za-z0-9._-]*\.md' "$state" 2>/dev/null | head -1)
if [ -z "$contract" ] || [ ! -f "$contract" ]; then
echo "no contract reachable — skipping contract VERIFY"; exit 0
fi
echo "running VERIFY from $contract"
awk '/^## VERIFY/{f=1} f&&/^```(bash|sh)$/{c=1;next} c&&/^```$/{c=0} c' "$contract" > /tmp/verify.sh
if [ ! -s /tmp/verify.sh ]; then echo "contract has no VERIFY block"; exit 1; fi
sh -e /tmp/verify.sh
13 changes: 9 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,16 @@ Thumbs.db
.env
.env.*

# Ephemeral review packets — never committed (legacy path)
.harness/
# Ephemeral review packets — never committed (legacy root path only).
# Anchored: unanchored '.harness/' also matched skills/.harness at any depth.
/.harness/

# Harness working directory — contracts, local templates, session state
skills/.harness/
# Harness working directory. Contracts are TRACKED: the review process grades a PR
# against its contract, and CI cannot check a file it cannot see. Session state,
# drafts and PR bodies stay local.
skills/.harness/*
!skills/.harness/contracts
!skills/.harness/contracts/**

# Project instances (this repo ships TEMPLATES, not filled-in copies)
# Uncomment if you keep a live filled AGENTS.md / FEATURES.json elsewhere
Expand Down
455 changes: 443 additions & 12 deletions BOOTSTRAP.md

Large diffs are not rendered by default.

16 changes: 16 additions & 0 deletions FEATURES.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,22 @@
"verify": "runnable test command or command sequence — not prose"
},
"features": [
{
"id": "KIT-GATES-002",
"name": "Gate reads structure not prose: impact-map parse replaces the path heuristic; STATE inbox drift check",
"priority": 2,
"verify": "! grep -q '[*].md|[*].sh' scripts/verify-harness.sh && grep -qi inbox scripts/verify-harness.sh && ./scripts/verify-harness.sh && grep -q '2.4.0' BOOTSTRAP.md && python3 -m json.tool FEATURES.json >/dev/null",
"status": "PENDING_REVIEW",
"notes": "Contract: skills/.harness/contracts/KIT-GATES-002.md; profile: ops-docs; EXECUTE+VERIFY green 2026-08-28 on feat/kit-gates-002-tighten-gate; 4 declared deviations incl. Q2 corrected by the gate itself"
},
{
"id": "KIT-GATES-001",
"name": "Deterministic gates: RED/BASELINE/GREEN artifacts, verify-harness.sh, CI re-running VERIFY",
"priority": 1,
"verify": "test -x scripts/verify-harness.sh && ./scripts/verify-harness.sh && test -f .github/workflows/harness.yml && ! grep -q 'Expected RED output' skills/sprint-contract/templates/SPRINT_CONTRACT.md && grep -q '2.3.0' BOOTSTRAP.md && ./scripts/build-bootstrap.sh >/dev/null && python3 -m json.tool FEATURES.json >/dev/null",
"status": "PENDING_REVIEW",
"notes": "Contract: skills/.harness/contracts/KIT-GATES-001.md; profile: ops-docs; EXECUTE+VERIFY green 2026-08-28 on feat/kit-gates-001-deterministic-gates; contracts now tracked; awaiting human PASS"
},
{
"id": "KIT-RULES-001",
"name": "rules/ directory as .mdc with AGENTS.md assembly, plus ERD routing before contract drafting",
Expand Down
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
A reusable, **tool-agnostic** framework for AI-assisted engineering — contract-first, TDD when
behaviour changes, multi-language (TypeScript, Rust, Go, Solidity build + audit).

**Kit v2.2** — the process ships as **skills**, adopted as a **git submodule**. Skills stay canonical
**Kit v2.4** — the process ships as **skills**, adopted as a **git submodule**. Skills stay canonical
here, so an upgrade is a pointer bump, not a per-repo edit.

## Adopt
Expand All @@ -24,6 +24,20 @@ git submodule update --remote harness-kit
./harness-kit/scripts/setup-harness-kit.sh
```

## Gates

```sh
./scripts/verify-harness.sh # exits non-zero on any violation
./scripts/verify-harness.sh --strict # skips become failures
```

Rules and skills are instructions to a model that can misread them under load. A gate is a fact.
It checks ledger integrity, that every `PENDING_REVIEW` has a contract, that skills and rules will
load, that every `[GROUNDED]` path exists, that the diff is contained by the impact map, and that RED
was recorded and failed on an assertion. **CI runs it again** — a local green is not evidence.

> Gate it if a command can enforce it, skill it if it is procedural, rule it only if it holds everywhere.

## Rules

Six always-on constraints in `rules/*.mdc`, linked into the editor's rules directory and assembled into
Expand Down
41 changes: 35 additions & 6 deletions scripts/build-bootstrap.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/sh
# DEPRECATED as of kit 2.2.0 — adoption moved to git submodule + scripts/setup-harness-kit.sh.
# DEPRECATED as of kit 2.4.0 — adoption moved to git submodule + scripts/setup-harness-kit.sh.
# Retained so repos bootstrapped from BOOTSTRAP.md keep working. Removal: KIT-SKILLS-002.
# Generates BOOTSTRAP.md from the forms now bundled inside skills/<name>/templates/.
set -eu
Expand All @@ -17,14 +17,41 @@ append() {
cat <<'HEADER'
# BOOTSTRAP.md — Harness Bootstrap

> **DEPRECATED — kit 2.2.0 adopts via git submodule; see README.md. This file is retained for
> **DEPRECATED — kit 2.4.0 adopts via git submodule; see README.md. This file is retained for
> repos already bootstrapped from it and will be removed in KIT-SKILLS-002.**
>
> **Kit version 2.2.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript,
> **Kit version 2.4.0** — generic, tool-agnostic AI-assisted engineering process for TypeScript,
> Rust, Go, Solidity build, and Solidity auditing. Copy this file into **any** repo. Prompt your LLM:
> *"Read BOOTSTRAP.md and complete Step 0 — Bootstrap harness files."*

## v2.2 changelog
## v2.4 changelog

- **The gate reads structure, not prose.** Check 6 parses the Impact map section's table — first cell
of each marked row — instead of guessing which backticked token is a path. The v2.3 heuristic
accepted a token only if it had a slash or a known extension, which silently skipped real paths
(`Makefile`) and checked things that were not paths. A check that silently skips is worse than one
that fails loudly, so an unparseable marked row now fails.
- **`[GROUNDED]` belongs to the impact map only** — stated in the contract form, which is what lets
the parser read position instead of inferring meaning.
- **Check 10: `STATE.md` inbox drift.** The ledger check caught rot in `FEATURES.json` while the same
rot in `STATE.md` went unseen. An inbox entry naming a feature already `PASS` now fails.

## v2.3 changelog (retained)

- **`scripts/verify-harness.sh`** — the third layer. Rules and skills are instructions to a model that
can misread them; this is a fact that exits non-zero. Nine checks: ledger integrity, every
`PENDING_REVIEW` has a contract, skills and rules will load, every `[GROUNDED]` path **exists**,
the diff is **contained by the impact map**, RED was recorded and failed on an **assertion**.
- **Prediction becomes recording** — the contract form's "Expected RED output" is replaced by recorded
**RED / BASELINE / GREEN** blocks. A test that never failed proves nothing.
- **No ERD, same gate** — without a design document each success criterion must be expressible as a
failing test before work starts, or the contract is not approvable.
- **CI re-runs VERIFY** (`.github/workflows/harness.yml`) — the only layer an agent cannot fake. One
script, invoked from CI and optionally from an editor hook; never reimplemented.
- **Contracts are tracked** — the review process grades a PR against its contract, and CI cannot check
a file it cannot see. Session state, drafts and PR bodies stay local.

## v2.2 changelog (retained)

- **`rules/` directory** — six always-on process-law rules as `.mdc`, linked into the editor's rules
directory and assembled (frontmatter stripped) into a marked block in `AGENTS.md`, so agents without
Expand Down Expand Up @@ -136,7 +163,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco
| 1 | kit skills (symlinked by `setup-harness-kit.sh`) | Appendices B–P | N/A — canonical in the submodule |
| 2 | `skills/.harness/contracts/` | — | Create empty directory |
| 3 | `skills/.harness/STATE.md` | Appendix F | Fill `<project-name>` + date |
| 4 | `skills/.harness/VERSION` | — | `version=2.2.0` + `bootstrapped=<YYYY-MM-DD>` |
| 4 | `skills/.harness/VERSION` | — | `version=2.4.0` + `bootstrapped=<YYYY-MM-DD>` |
| 5 | `README.md` | Appendix G | **No** — overwrite with slim pointer (after brownfield recovery if needed) |
| 6 | `AGENTS.md` | Appendix B | **Yes** if already filled — scaffold only; never overwrite harvested/verified content |
| 7 | `FEATURES.json` | Appendix C | **Yes** if seeded — scaffold only |
Expand All @@ -154,7 +181,7 @@ If `README.md` is already a harness pointer or is empty/scaffold-only, skip reco

**Done when:**

- `skills/.harness/` tree exists with VERSION `2.2.0`
- `skills/.harness/` tree exists with VERSION `2.4.0`
- `python3 -m json.tool FEATURES.json` passes (if scaffolded)
- `.gitignore` contains `skills/.harness/`
- No `HARNESS.md` in repo
Expand Down Expand Up @@ -257,6 +284,8 @@ HEADER
append "Appendix O — ERD.md template" skills/erd-authoring/templates/ERD.md
append "Appendix P — ARCHITECTURE.md template" skills/erd-authoring/templates/ARCHITECTURE.md
append "Appendix Q — rules/README.md (rule budget + ownership tiers)" rules/README.md
append "Appendix R — verify-harness.sh (the gate)" scripts/verify-harness.sh
append "Appendix S — CI workflow (re-runs the gate and the contract VERIFY block)" .github/workflows/harness.yml

cat <<'FOOTER'

Expand Down
Loading
Loading