Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 61 additions & 10 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,10 @@ Shared configs: `configs/ruff-base.toml` (C901 ≤ 10, PLR0915 ≤ 50, S battery
BLE ban — the ratified budgets), `configs/mypy-base.toml` (strict-leaning
profile, see §10), `configs/jscpd.src.json` / `configs/jscpd.tests.json` (the
two-profile clone carve-out, see §7). Tool-behavior ground truth lives in
`docs/tool-spikes.md` — complexipy 5.5.0 and mypy-baseline 0.7.4 semantics
were observed on fixtures, not read off READMEs.
`docs/tool-spikes.md` — complexipy and mypy-baseline semantics were observed on
fixtures, not read off READMEs (spiked on complexipy 5.5.0 · mypy-baseline
0.7.4; complexipy's snapshot-write semantics re-measured on the pinned 5.6.0
on 2026-07-28, and they inverted — see §4.11).

**The single reproducible entrypoint — `cf-gate` (`gate_runner.py`).** The
battery above is split across many console scripts and external tools so each
Expand Down Expand Up @@ -135,9 +137,19 @@ become green-forever):
to `:0`, so unrelated drift cannot resurrect findings.
6. **Cognitive-complexity watermark:** `complexipy src --snapshot-create`
**from the repo root** (the snapshot lands in CWD, not the analyzed path)
→ commit `complexipy-snapshot.json`. Observed gotchas: the snapshot does
NOT auto-shrink (the kit owns the re-snapshot on merge, or improvements
are not locked in); piping the gate command masks its exit code.
→ commit `complexipy-snapshot.json`. This is the ONLY complexipy command a
consumer ever runs by hand. Measured on the pinned **complexipy 5.6.0**
(2026-07-28): a *passing* plain-run compare REWRITES the snapshot —
`handle_snapshot_watermark` calls `create_snapshot_file` on its
no-violation branch — so the tool will shrink its own floor to `[]` at
exit 0. `cf-gate` therefore measures write-free (`--snapshot-ignore`), AUDITS
the consumer's complexipy config so `snapshot-create` cannot re-open that
write from TOML, and grades in `complexipy_ratchet` (§4.11). The duty that
remains: a shrink is locked in only by a deliberate re-`--snapshot-create`,
because the watermark rule is a `>` bound and a stale watermark still
grandfathers a climb back up to it. Re-boot in the SAME commit as any change
to `max-complexity-allowed`, or the gate refuses the mismatch
(`GATE_COMPLEXIPY_THRESHOLD_RAISED`).
7. **Exemptions:** if the repo carries any gated suppression, register each
in `exemptions.json` (five fields: file, symbol_or_line, rule, reason,
approver) and set `frozen_count`.
Expand Down Expand Up @@ -460,11 +472,50 @@ ordinary PR.
new. Fixed-only also exits nonzero ("re-sync"), so every shrink is
committed — the ratchet direction is enforced by the tool itself.
- **complexipy — set-like per function** (new offender fails; baselined
offender worsening fails), but the snapshot does NOT auto-shrink: the
re-snapshot-on-merge step is runbook procedure (§3 step 6), not yet CI
mechanism. The complexipy step IS in `quality-gate.yml` (and `self-ci.yml`)
as of 2026-06-11, with the mypy-style presence rule: a Python repo without
its committed snapshot FAILS; only a Python-free repo skips, visibly.
offender worsening fails), and graded by the KIT, not by the tool. The
2026-06-10 spike recorded on 5.5.0 that the snapshot never shrinks itself.
Re-measured on the pinned **complexipy 5.6.0** (2026-07-28) that claim is
false and inverted: a *passing* compare rewrites the floor —
`handle_snapshot_watermark` calls `create_snapshot_file` on its no-violation
branch — reproduced at exit 0 shrinking a populated snapshot to `[]` both by
grading a narrower path than the floor and by raising the threshold above
every function. So the vector here was never "the floor cannot shrink", it
was "the floor can be **zeroed** behind a green run", which is laundering by
deletion. That zeroing is CLOSED by construction: `cf-gate` measures with
`--snapshot-ignore`, AUDITS the consumer's own complexipy config so the other
write path cannot be re-opened from TOML (`snapshot-create` resolves
CLI-first/TOML-second and has no negating flag, so a config carrying it made
both measurement runs rewrite the floor —
`GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT` now refuses it, naming the key
and the file), and compares in `complexipy_ratchet.grade`, a pure function
of (committed floor, measured census) — nothing the tool does can rewrite
the artifact it is graded against. Residue, and it is the OLD residue in a
narrower form: locking a shrink IN is still runbook procedure (§3 step 6),
not CI mechanism — the watermark is a `>` bound, so an improvement that is
never re-snapshotted leaves a stale watermark a later regression may climb
back to, and a deleted floor file leaves a dead entry that would grandfather
a same-named function if the file returned. Mechanical now, stated exactly
(an earlier pass on this branch wrote that emptying a floor file of functions
FAILS — false, since `--plain` lists every measured function regardless of
threshold): a floor file that still exists but was not **measured**, or that
sits outside the graded root, FAILS
(`COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED` / `COMPLEXIPY_SURFACE_NARROWED`); a
floor **function** missing from a file that WAS measured FAILS
(`COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED` — the shape a
`# complexipy: ignore` comment takes, an unregistered exemption from this
gate); a threshold raised above a committed watermark REFUSES rather than
grading the emptied offender set (`GATE_COMPLEXIPY_THRESHOLD_RAISED`,
threshold-free: a floor entry proves that function was above the boot bar);
and a run that measured zero functions cannot report clean while the floor
names functions or the repo contains Python at all
(`GATE_COMPLEXIPY_MEASURED_NOTHING`, riding the caller's repo-wide
`py_present` so the vacuity leg and the absent-snapshot doctrine cannot
disagree). Not machine-caught: a floor a human zeroes by hand and commits is
green — the ratchet just goes vacuous, and only the
`complexipy-snapshot.json` diff in review shows it. The
complexipy step IS in `quality-gate.yml` (and `self-ci.yml`) as of
2026-06-11, with the mypy-style presence rule: a Python repo without its
committed snapshot FAILS; only a Python-free repo skips, visibly.
- **cf-exemptions — count-based by design**, and honestly so: `frozen_count`
refuses silent growth, but a 1-for-1 entry swap at equal count is
machine-visible only as an `exemptions.json` diff in review, not
Expand Down
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,14 @@ carries a dated ratchet ticket so green-by-baseline cannot become green-forever)
```bash
cf-file-budget init # freeze existing >500-line files at measured size
mypy src | mypy-baseline sync
complexipy src # cognitive-complexity snapshot (second metric)
complexipy src --snapshot-create # cognitive-complexity floor (second metric)
```

`--snapshot-create` is not optional here, and a bare `complexipy src` is never the
right command: with a snapshot present the tool's own *passing* compare REWRITES the
floor (measured on the pinned 5.6.0 — see `configs/BASELINE-CONVENTIONS.md` §1).
Run it from the repo root; the snapshot lands in CWD, not in the analyzed path.

## Development

```bash
Expand Down
98 changes: 83 additions & 15 deletions configs/BASELINE-CONVENTIONS.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
# Baseline conventions — mounting the ratchet on a consumer repo

How a repo boots green-by-construction on day one and then only ever shrinks.
Every workflow below is the **observed** behavior of the pinned tools
(complexipy 5.5.0 · mypy-baseline 0.7.4), measured in `docs/tool-spikes.md` —
not read off a README. Every baseline carries a dated ratchet ticket so
Every workflow below is the **observed** behavior of the pinned tools, measured
in `docs/tool-spikes.md` — not read off a README. Version provenance, stated
because a stale label is exactly how a false claim survives here: the spike was
run on **complexipy 5.5.0 · mypy-baseline 0.7.4**; mypy-baseline is still pinned
at 0.7.4, but complexipy is now pinned at **5.6.0** and only its
snapshot-**write** semantics have been re-measured there (2026-07-28 — the first
complexipy gotcha below). Every baseline carries a dated ratchet ticket so
green-by-baseline cannot become green-forever (the Law-1 refuter's expiry fix).

## The shared configs in this directory
Expand Down Expand Up @@ -31,26 +35,90 @@ Exit 0 even when offenders exist — baseline boot is green by construction.
The snapshot records only functions over the threshold; an all-clean tree
writes a literal `[]`.

**Gate (every CI run, from the repo root):**
**Gate (every CI run, from the repo root) — `cf-gate`, never the tool's own
compare:**

```bash
complexipy src # plain run auto-compares when the snapshot exists
cf-gate # the complexipy stage measures write-free, grades in-kit
```

Fails (exit 1) on any NEW offender, or any baselined offender rising above its
watermark. A function at-or-below its watermark passes.
watermark — complexipy's own rule, applied by `cf_quality.complexipy_ratchet`
over a floor the tool is never allowed to touch. A function at-or-below its
watermark passes. Do **not** mount a bare `complexipy src` as the gate; the first
gotcha is why.

**Observed gotchas the mount must respect:**

- The snapshot **does NOT auto-shrink**. After an improvement the plain run
passes but the file still holds the old watermark — a later regression back
up to the stale watermark would pass. The kit owns the re-baseline step:
re-run `--snapshot-create` on merge (or in the shrink ticket) to lock
improvements in.
- Run from the **repo root** so the committed snapshot is the one compared
(snapshot path is CWD-relative).
- Never pipe the gate command (`complexipy … | tail` masks the exit code);
gate on the command's own status, or set `pipefail`.
- **A passing plain-run compare REWRITES the snapshot** — measured on the pinned
**complexipy 5.6.0** (2026-07-28), correcting the 5.5.0 entry that used to
stand here and claim the file never shrinks itself. The destructive call site is
`complexipy/utils/snapshot.py::handle_snapshot_watermark`, which calls
`create_snapshot_file(...)` on its **no-violation** branch — the tool's green
path is its write path. Reproduced twice at exit 0, a populated snapshot
rewritten to `[]`: once by grading a path narrower than the floor describes,
once by raising the threshold above every function. Committed, either one
deletes the watermark forever behind a green run. `cf-gate` therefore measures
write-free with `--plain --color no --snapshot-ignore` (verified on 5.6.0 to
leave the file byte-unchanged) and grades the ratchet itself.
- **Your own `complexipy` config can defeat the gate, so the gate REFUSES instead
of measuring through it.** `--snapshot-ignore` disarms the compare's rewrite, but
`snapshot-create` is a **separate branch** (`main.py:323`) resolved CLI-first,
TOML-second (`utils/toml.py:235-240`) — and `--snapshot-create` has no negating
flag, so `cf-gate` cannot override it from the command line. So a
`complexipy.toml` / `.complexipy.toml` / `[tool.complexipy]` carrying any of
`snapshot-create`, `quiet`, `ratchet`, `failed`, `details = "low"`,
`ignore-complexity`, `report-ignored`, `output`, `output-format`, or the legacy
`output-csv|json|gitlab|sarif` fails the stage typed, naming the key and the file
(`GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT`, exit 2). **Remove the key** — there
is no workaround to reach for, because the two you actually want are already
honoured: `max-complexity-allowed` (your threshold — the kit declares none of its
own) and `exclude` (your surface). `no-ignore`, `check-script` and `sort` are
honoured too. An unparseable config is also a refusal
(`GATE_COMPLEXIPY_CONFIG_UNREADABLE`) — unreadable is not absent.
- **Raising `max-complexity-allowed` above a committed watermark fails the gate**
(`GATE_COMPLEXIPY_THRESHOLD_RAISED`, exit 2) rather than quietly grading an empty
offender set. If you mean to raise the bar, raise it and **re-boot the floor at
the new threshold** in the same commit, so the floor and the bar agree.
- **The re-snapshot duty is still yours, in a narrower shape.** With the write
disarmed the floor holds still, which means a shrink is not locked in merely by
passing: the watermark rule is a `>` bound, so a function that got simpler and
later climbs back to its stale watermark passes. Re-run
`complexipy <source-root> --snapshot-create` from the repo root, deliberately,
and commit it — that is the shrink ticket's job. What is MECHANICAL now, stated
exactly (an earlier draft of this file claimed a floor file emptied of functions
fails — it does **not**: `--plain` lists every measured function regardless of
threshold, so a file whose functions all dropped below the bar is still measured
and still green):
- a floor file that exists but was not **measured** at all — excluded,
ignore-commented, outside the surface, or now functionless — fails
(`COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED`);
- a floor file outside the graded source root fails
(`COMPLEXIPY_SURFACE_NARROWED`);
- a floor **function** missing from a file that WAS measured fails
(`COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED`) — that is the shape a
`# complexipy: ignore` comment takes, and it is an unregistered exemption from
this gate, so it is refused rather than absorbed;
- a run that measured **zero functions** can no longer report clean while the
floor names functions or the repo contains Python at all
(`GATE_COMPLEXIPY_MEASURED_NOTHING`).

Every one of those messages names the re-boot command as its remedy. Still on you,
not on the gate: a wholly deleted floor file is a legitimate improvement and stays
green, so the snapshot keeps dead entries until a re-boot clears them — and a floor
someone zeroes by hand and commits is green too, visible only as a
`complexipy-snapshot.json` diff in review.
- Run the **boot** from the **repo root** so the snapshot lands beside the code it
describes (its path is CWD-relative, and the gate reads it from the repo root).
- Never pipe a hand-run complexipy command (`complexipy … | tail` masks the exit
code). `cf-gate` runs it with a fixed argv and no shell, and does not *grade* on
its exit code — but it does **cross-check** it: 0 and 1 are the tool's only
verdicts, 1 being the ordinary "some function is over threshold", so exit 1 with an
empty census, or any other non-zero code, is the instrument failing and refuses
distinctly (`GATE_COMPLEXIPY_INSTRUMENT_FAILED`, carrying the exit code and a
stderr excerpt) instead of being charged to your code. A file complexipy cannot
parse likewise refuses (`GATE_COMPLEXIPY_PATHS_UNMEASURABLE`, exit 2, not exit 1):
a narrower surface than the tree makes every other clean reading unsupported.

## 2. mypy-baseline — the 0-new-type-errors gate (multiset set-difference)

Expand Down
Loading