From 7c3701a4969a008dd6da4518afaa713cd62b27be Mon Sep 17 00:00:00 2001 From: Antawari Date: Tue, 28 Jul 2026 21:41:12 -0600 Subject: [PATCH 1/2] Put the complexipy floor out of the gate's own reach The complexity ratchet compares the tree against a committed watermark file and fails on regression. A PASSING run silently emptied that file and exited 0, so the floor vanished and every later run compared against nothing. Root cause, read in the pinned tool's own source: handle_snapshot_watermark calls create_snapshot_file the moment it finds no violation. The tool's green path IS its destructive path. Reproduced two ways, both exit 0: measuring a narrower surface than the floor describes, and raising the threshold above every function. It is not hypothetical. Two working trees on the maintainer's machine already hold a floor of 12 entries / 13 functions committed and a literal [] on disk, one stage away from permanent. So the gate no longer lets the tool near the file. Measurement runs write-free (--snapshot-ignore switches the compare, and its rewrite, off), and the ratchet is graded here as a pure function of the committed floor and the measured census. Detection no longer depends on the tool destroying the artifact first, and there is no window where the write happened and was undone. The census also makes the run answerable about what it graded. Every verdict carries measured_functions, measured_files and the floor size, and a run that measured nothing while there was something to grade REFUSES rather than reporting clean. A floor file that still exists but went unmeasured is a finding, not a shrug: that is the narrowed-surface case, caught without reference to any threshold. Threshold authority stays with the tool, so a consumer who booted at a non-default value is graded at that value and not at ours. Co-Authored-By: Claude Opus 5 (1M context) --- src/cf_quality/complexipy_ratchet.py | 488 ++++++++++++++++++++++++++ src/cf_quality/gate_runner.py | 6 +- tests/test_complexipy_snapshot.py | 495 +++++++++++++++++++++++++++ tests/test_gate_runner.py | 48 ++- 4 files changed, 1019 insertions(+), 18 deletions(-) create mode 100644 src/cf_quality/complexipy_ratchet.py create mode 100644 tests/test_complexipy_snapshot.py diff --git a/src/cf_quality/complexipy_ratchet.py b/src/cf_quality/complexipy_ratchet.py new file mode 100644 index 0000000..8d278d5 --- /dev/null +++ b/src/cf_quality/complexipy_ratchet.py @@ -0,0 +1,488 @@ +"""The cognitive-complexity ratchet, graded HERE — because the tool eats its own floor. + +**The measured defect.** ``complexipy-snapshot.json`` is the committed floor: the +per-function cognitive-complexity watermark no later run may exceed. In the +pinned ``complexipy==5.6.0`` the tool's OWN snapshot comparison ends, on success, +in a REWRITE of that file — ``complexipy/utils/snapshot.py`` +``handle_snapshot_watermark`` returns ``True`` only after calling +``create_snapshot_file(...)`` with the functions IT measured this run. The tool's +green path *is* the destructive path, and two ordinary runs reproduce it at exit +0: a **narrowed surface** (``complexipy `` measures a +subset, finds no violation, rewrites the floor to that subset — a populated +snapshot observed going ``1 entry -> []``) and a **raised threshold** +(``complexipy -mx 100`` puts everything under the bar, same rewrite). +Either one, committed, deletes the floor forever behind a green gate. + +**Why this module rather than a before/after repair.** Detecting the rewrite and +restoring the file would still let it happen, and would only work *because* the +tool destroys the artifact: the moment the write does not occur — a crash between +compare and write, a release that stops rewriting, a run that never reached the +compare — a before/after diff of the file is identical and the narrowed surface +goes invisible again. So the fix sits upstream of the write: + +1. **The gate never lets the tool near the artifact.** Both measurement + invocations carry ``--snapshot-ignore``. In the pinned source the ONLY two + callers of ``create_snapshot_file`` are ``--snapshot-create`` (never passed) + and the watermark compare's success path (which ``--snapshot-ignore`` switches + off by making ``should_run_snapshot_watermark`` False). The committed floor is + therefore READ by this module and written by nobody. +2. **The ratchet is our pure function.** :func:`grade` maps (committed floor, + measured census) to a :class:`~cf_quality.errors.GateVerdict` with no + subprocess, no write and no tool exit code in the path — unit-testable, and + nothing a green run can silently destroy. +3. **complexipy is demoted to a measuring instrument.** It answers two questions + and grades nothing: ``--plain`` (every function measured, with its complexity) + and ``--plain --failed`` (the subset ITS OWN threshold calls offenders). The + second run is why the kit never re-declares a budget complexipy already + resolves from its default / CLI / ``[tool.complexipy]`` config — a second + threshold authority here would flag a consumer's baselined band as new. + +**The taxonomy.** Findings (exit 1, the repo's to fix): ``COMPLEXIPY_NEW_OFFENDER`` +and ``COMPLEXIPY_WATERMARK_REGRESSION`` (complexipy's own watermark rule over data +we own), ``COMPLEXIPY_SURFACE_NARROWED`` and ``COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED`` +(the floor names a file this run did not grade — the narrowed-surface trigger, +caught structurally, independent of any threshold). Refusals (exit 2, the gate +could not do its job): ``GATE_COMPLEXIPY_SNAPSHOT_UNREADABLE``, +``GATE_COMPLEXIPY_PATHS_UNMEASURABLE``, ``GATE_COMPLEXIPY_OUTPUT_UNREADABLE``, +``GATE_COMPLEXIPY_MEASUREMENT_SKEW``, ``GATE_COMPLEXIPY_MEASURED_NOTHING``. A +legitimate improvement — a function that got simpler, a deleted file — is GREEN; +only re-booting the floor locks it in, which stays the existing runbook duty. + +The absent-watermark doctrine is unchanged and stays in the caller +(``gate_runner._complexipy``): no snapshot + Python present REFUSES +``GATE_COMPLEXIPY_SNAPSHOT_MISSING``; a Python-free repo skips, visibly. +""" + +from __future__ import annotations + +import ast +import json +import subprocess +import sys +from collections.abc import Mapping +from dataclasses import dataclass +from pathlib import Path +from typing import Protocol + +from cf_quality.errors import GateError, GateVerdict, GateViolation + +#: The stage name every verdict from this module carries. +GATE = "complexipy" + +#: The committed floor's filename (CWD-relative for complexipy, repo root for us). +SNAPSHOT_FILENAME = "complexipy-snapshot.json" + +#: ``(repo-relative file, function name)`` — the identity a watermark is keyed by. +#: complexipy keys on ``(path, file_name, name)``; ``path`` already carries the +#: file name in its own output, so the joined form is the same identity. +FunctionKey = tuple[str, str] + +#: Environment pinned onto every measurement run. ``COLUMNS`` is load-bearing: +#: ``--plain`` prints through rich, which wraps at 80 columns when stdout is not +#: a terminal, and a wrapped census row is an unparseable census row. +#: ``PYTHONIOENCODING`` pins UTF-8 so the census does not decode by locale. +_MEASURE_ENV = {"COLUMNS": "10000", "PYTHONIOENCODING": "utf-8", "NO_COLOR": "1"} + +#: complexipy's own words when it could not analyze a path it was handed +#: (``complexipy.utils.output.print_invalid_paths``) — a silently narrower surface. +_UNMEASURABLE_MARKER = "Failed to process" + + +class _Executor(Protocol): + """The subprocess seam the caller injects (``gate_runner._exec``). + + Structural, not inherited: the runner stays the caller's — one place owns + typed OSError translation and the no-shell fixed-argv discipline — while the + tests keep patching that single seam. + """ + + def __call__( + self, + argv: list[str], + cwd: Path, + env: Mapping[str, str], + *, + stdin: str | None = None, + ) -> subprocess.CompletedProcess[str]: ... + + +@dataclass(frozen=True) +class Census: + """What ONE write-free complexipy run actually measured. + + The census is the gate's independent fact about its own measurement surface: + it is read from the tool's output, never from the snapshot, so "the floor is + empty" and "we graded nothing" can never be the same observation. + """ + + functions: dict[FunctionKey, int] + + @property + def files(self) -> frozenset[str]: + """The distinct files this run measured — the surface audit's evidence.""" + return frozenset(path for path, _ in self.functions) + + +def _gate_error(code: str, message: str, context: dict[str, object]) -> GateError: + """A refusal in the kit's typed vocabulary — the gate could not do its job.""" + return GateError(code=code, message=message, context=context) + + +def _normalized_path(path: str, file_name: str) -> str: + """Join a snapshot entry's two path fields the way complexipy's output does. + + Declared mirror of ``complexipy.utils.output.normalize_path`` (pinned 5.6.0): + the snapshot stores ``path`` and ``file_name`` separately while ``--plain`` + prints the joined form. Join them differently and every committed watermark + looks like a brand-new offender. + """ + cleaned = path.rstrip("/") + if cleaned.endswith(file_name): + return cleaned + return f"{cleaned}/{file_name}" if cleaned else file_name + + +def _refuse_snapshot(snapshot: Path, reason: str) -> GateError: + return _gate_error( + "GATE_COMPLEXIPY_SNAPSHOT_UNREADABLE", + f"{SNAPSHOT_FILENAME} is not a readable complexipy snapshot: {reason} — " + "re-boot it (complexipy --snapshot-create); an unreadable " + "floor is not an empty floor", + {"snapshot": str(snapshot), "reason": reason}, + ) + + +def _entry_watermarks(entry: object, snapshot: Path) -> dict[FunctionKey, int]: + """One snapshot entry's watermarks; any other shape REFUSES rather than skips.""" + if not isinstance(entry, dict) or not isinstance(entry.get("functions"), list): + raise _refuse_snapshot(snapshot, f"entry is not {{path, file_name, functions}}: {entry!r}") + path = _normalized_path(str(entry.get("path", "")), str(entry.get("file_name", ""))) + watermarks: dict[FunctionKey, int] = {} + for function in entry["functions"]: + if not isinstance(function, dict) or not isinstance(function.get("complexity"), int): + raise _refuse_snapshot(snapshot, f"function is not {{name, complexity}}: {function!r}") + watermarks[(path, str(function.get("name", "")))] = int(function["complexity"]) + return watermarks + + +def read_snapshot(snapshot: Path) -> dict[FunctionKey, int]: + """The committed floor as ``{(file, function): watermark}``. + + This is the ONLY code that touches the artifact, and it only reads. A + malformed snapshot REFUSES: treating it as an empty floor would be + green-by-unreadable-file, the same gaming vector as green-by-missing-file + (which this gate already refuses). + """ + try: + raw = json.loads(snapshot.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as exc: + raise _refuse_snapshot(snapshot, str(exc)) from exc + if not isinstance(raw, list): + raise _refuse_snapshot(snapshot, f"top level is {type(raw).__name__}, not a list") + floor: dict[FunctionKey, int] = {} + for entry in raw: + floor.update(_entry_watermarks(entry, snapshot)) + return floor + + +def _census_row(line: str) -> tuple[FunctionKey, int] | None: + """`` `` -> the keyed measurement, else None. + + Split from the RIGHT: the complexity and the function name are single tokens + while a path may contain spaces, so ``rsplit`` is the only safe direction. + """ + parts = line.strip().rsplit(maxsplit=2) + if len(parts) != 3 or not parts[0].endswith(".py"): + return None + try: + complexity = int(parts[2]) + except ValueError: + return None + return (parts[0], parts[1]), complexity + + +def parse_census(stdout: str) -> Census: + """complexipy ``--plain`` stdout -> the measured census. + + ``--plain`` is complexipy's documented scripting form: one + `` `` line per function it measured, over + threshold or not. A non-blank line that is not a census row REFUSES instead + of being dropped — an unreadable census and an empty one look identical from + a count, and the empty one is the exact world this gate exists to catch. + """ + functions: dict[FunctionKey, int] = {} + unreadable: list[str] = [] + for line in stdout.splitlines(): + row = _census_row(line) + if row is not None: + functions[row[0]] = row[1] + elif line.strip(): + unreadable.append(line.strip()) + if unreadable: + raise _gate_error( + "GATE_COMPLEXIPY_OUTPUT_UNREADABLE", + f"complexipy --plain emitted {len(unreadable)} line(s) that are not " + "' ' — the census cannot be trusted, so " + "the floor cannot be graded", + {"lines": unreadable[:10], "measured_functions": len(functions)}, + ) + return Census(functions=functions) + + +def measurement_argv(tool: Path, source_root: Path, *, offenders_only: bool) -> list[str]: + """The write-free measurement command — the root of the fix, not a nicety. + + ``--snapshot-ignore`` makes ``should_run_snapshot_watermark`` False in the + pinned tool, which is the only path (besides the never-passed + ``--snapshot-create``) that reaches ``create_snapshot_file``. This argv + therefore CANNOT write ``complexipy-snapshot.json``. ``--failed`` narrows the + census to the functions complexipy's own resolved threshold calls offenders, + so the kit never states a complexity budget of its own here. + """ + argv = [str(tool), str(source_root), "--plain", "--color", "no", "--snapshot-ignore"] + if offenders_only: + argv.append("--failed") + return argv + + +def _measure( + root: Path, + source_root: Path, + env: Mapping[str, str], + tool: Path, + executor: _Executor, + *, + offenders_only: bool, +) -> Census: + """Run one write-free measurement from the repo root and read its census. + + cwd is the repo root deliberately: complexipy resolves both its config and + its reported paths against the invocation directory, so measuring from + anywhere else would re-key every path and break the comparison. The exit + code is NOT consulted — with the compare switched off it merely restates + "some function is over threshold", which is the normal state of a repo + carrying a baselined floor. + """ + argv = measurement_argv(tool, source_root, offenders_only=offenders_only) + proc = executor(argv, root, {**env, **_MEASURE_ENV}) + lines = [line.strip() for line in proc.stdout.splitlines()] + unmeasurable = [line for line in lines if _UNMEASURABLE_MARKER in line] + if unmeasurable: + raise _gate_error( + "GATE_COMPLEXIPY_PATHS_UNMEASURABLE", + f"complexipy could not analyze {len(unmeasurable)} path(s) — the graded " + "surface is narrower than the tree, so a clean verdict would be a void", + {"paths": unmeasurable[:10], "exit_code": proc.returncode}, + ) + return parse_census(proc.stdout) + + +def _counts( + floor: Mapping[FunctionKey, int], census: Census, offenders: Census +) -> dict[str, object]: + """The measured tally every finding and refusal carries — never a bare verdict.""" + return { + "measured_functions": len(census.functions), + "measured_files": len(census.files), + "measured_offenders": len(offenders.functions), + "snapshot_functions": len(floor), + "snapshot_files": len({path for path, _ in floor}), + } + + +def _violation( + code: str, message: str, path: str, counts: Mapping[str, object], **detail: object +) -> GateViolation: + return GateViolation(code=code, message=message, path=path, context={**counts, **detail}) + + +def _regressions( + floor: Mapping[FunctionKey, int], offenders: Census, counts: Mapping[str, object] +) -> list[GateViolation]: + """complexipy's own watermark rule, applied to data the tool cannot rewrite. + + Mirrors ``handle_snapshot_watermark`` exactly, including the ``>`` bound — a + function sitting AT its watermark passes; only rising above it fails. + """ + violations: list[GateViolation] = [] + for (path, name), value in sorted(offenders.functions.items()): + watermark = floor.get((path, name)) + if watermark is None: + code = "COMPLEXIPY_NEW_OFFENDER" + message = f"{name} exceeds complexipy's threshold at {value}, no committed watermark" + elif value > watermark: + code = "COMPLEXIPY_WATERMARK_REGRESSION" + message = f"{name} rose above its committed watermark: {watermark} -> {value}" + else: + continue + violations.append(_violation(code, message, path, counts, function=name, measured=value)) + return violations + + +def _surface_violations( + root: Path, + source_root: Path, + floor: Mapping[FunctionKey, int], + census: Census, + counts: Mapping[str, object], +) -> list[GateViolation]: + """Every floor file that still EXISTS must have been measured this run. + + The narrowed-surface trigger caught head-on, independent of any threshold: a + snapshot entry is proof that file HELD an over-threshold function, so a run + that produced no measurement for it graded a smaller world than the floor + describes. A file that is GONE is a legitimate improvement and is passed over; + a file whose functions all vanished reads the same way and asks for the same + remedy — re-boot the floor, deliberately, so the improvement is locked in. + """ + violations: list[GateViolation] = [] + graded = source_root.resolve() # both sides resolved, or a symlinked tmp lies + for path in sorted({path for path, _ in floor}): + on_disk = root / path + if not on_disk.is_file(): + continue + if not on_disk.resolve().is_relative_to(graded): + code = "COMPLEXIPY_SURFACE_NARROWED" + message = f"the floor covers {path}, which lies OUTSIDE the graded source root" + elif path not in census.files: + code = "COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED" + message = f"{path} carries a committed watermark but no function in it was measured" + else: + continue + violations.append(_violation(code, message, path, counts, source_root=str(source_root))) + return violations + + +def _module_defines_functions(path: Path) -> bool: + """True when a module contains any ``def``/``async def``, by AST not by regex. + + Source we cannot read or parse counts as YES: a void must never certify + itself, and we cannot prove a file is functionless from bytes we never + parsed (complexipy would report such a path as unmeasurable anyway). + """ + try: + tree = ast.parse(path.read_text(encoding="utf-8")) + except (OSError, SyntaxError, ValueError): + return True + return any(isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef) for node in ast.walk(tree)) + + +def _defines_functions(source_root: Path) -> bool: + """True when the graded tree defines at least one function. + + Consulted only when the census came back EMPTY, so the common path never + parses anything. Dotted directories are skipped, mirroring the workflow's + own ``find . -not -path '*/.*'`` measurement surface. + """ + for path in sorted(source_root.rglob("*.py")): + parts = path.relative_to(source_root).parts + if any(part.startswith(".") for part in parts): + continue + if _module_defines_functions(path): + return True + return False + + +def _skew(census: Census, offenders: Census, counts: Mapping[str, object]) -> GateError | None: + """The two write-free runs must describe ONE world. + + The offender set is a filter of the census, so every offender must appear in + the census at the same complexity. A disagreement means the tree changed + between the runs, or a flag moved the measurement surface — either way the + comparison inputs are not a single observation and must not be graded. + """ + disagreements = sorted( + f"{path}:{name}" + for (path, name), value in offenders.functions.items() + if census.functions.get((path, name)) != value + ) + if not disagreements: + return None + return _gate_error( + "GATE_COMPLEXIPY_MEASUREMENT_SKEW", + f"{len(disagreements)} function(s) reported by the offender run are absent from " + "(or disagree with) the census run — the two measurements are not one world", + {**counts, "functions": disagreements[:10]}, + ) + + +def _vacuity( + census: Census, + floor: Mapping[FunctionKey, int], + source_root: Path, + counts: Mapping[str, object], +) -> GateError | None: + """A run that measured nothing can never report clean. + + Two independent legs, so the refusal survives an ALREADY-emptied floor: a + committed floor that still names functions, or a graded tree that + demonstrably defines functions. When both are empty there is genuinely + nothing to grade and clean is the honest answer, not a void. + """ + if census.functions: + return None + if not floor and not _defines_functions(source_root): + return None + return _gate_error( + "GATE_COMPLEXIPY_MEASURED_NOTHING", + "complexipy measured zero functions while there was something to grade — a " + "gate that measured nothing cannot report a clean floor (check the resolved " + f"source root {source_root} and any complexipy exclude/ignore configuration)", + {**counts, "source_root": str(source_root)}, + ) + + +def grade( + root: Path, + source_root: Path, + floor: Mapping[FunctionKey, int], + census: Census, + offenders: Census, +) -> GateVerdict: + """The whole ratchet as a pure function of (committed floor, measured world). + + No subprocess, no write, no tool exit code — which is the property the + tool's own comparison cannot have, because its green path IS the rewrite. + Every finding and refusal carries the measured tally, so a verdict can never + be read without the count behind it. + """ + counts = _counts(floor, census, offenders) + violations = _regressions(floor, offenders, counts) + violations.extend(_surface_violations(root, source_root, floor, census, counts)) + error = _skew(census, offenders, counts) or _vacuity(census, floor, source_root, counts) + return GateVerdict(gate=GATE, violations=violations, error=error) + + +def _report_measured(floor: Mapping[FunctionKey, int], census: Census) -> None: + """State the measured count out loud, on every run including a clean one. + + :class:`~cf_quality.errors.GateVerdict` carries no notices channel, so a + PASSING stage would otherwise report a floor it never proves it measured. + stderr keeps the aggregated JSON wire form on stdout untouched. + """ + print( + f"{GATE}: measured {len(census.functions)} function(s) in {len(census.files)} " + f"file(s) against a {len(floor)}-function committed floor", + file=sys.stderr, + ) + + +def complexipy_verdict( + root: Path, + source_root: Path, + env: Mapping[str, str], + tool: Path, + executor: _Executor, +) -> GateVerdict: + """Measure the tree write-free, then grade the ratchet in our own code. + + The caller has already enforced the absent-watermark doctrine, so the floor + exists here. Two write-free runs (the full census, then complexipy's own + offender subset) feed :func:`grade`; a GateError from either measurement + propagates as the stage's refusal, which the battery records and continues. + """ + floor = read_snapshot(root / SNAPSHOT_FILENAME) + census = _measure(root, source_root, env, tool, executor, offenders_only=False) + offenders = _measure(root, source_root, env, tool, executor, offenders_only=True) + _report_measured(floor, census) + return grade(root, source_root, floor, census, offenders) diff --git a/src/cf_quality/gate_runner.py b/src/cf_quality/gate_runner.py index 37becc0..9400857 100644 --- a/src/cf_quality/gate_runner.py +++ b/src/cf_quality/gate_runner.py @@ -37,6 +37,7 @@ from pathlib import Path from typing import Any +from cf_quality.complexipy_ratchet import complexipy_verdict from cf_quality.errors import GateError, GateVerdict, GateViolation from cf_quality.mypy_normalize import normalize_mypy_stdout from cf_quality.repo_config import ( @@ -321,7 +322,7 @@ def _mypy(layout: Layout, env: Mapping[str, str]) -> GateVerdict | None: def _complexipy(layout: Layout, env: Mapping[str, str]) -> GateVerdict | None: - """complexipy through the snapshot ratchet; same refuse/skip doctrine as mypy.""" + """complexipy WRITE-FREE (its own compare REWRITES the floor); see complexipy_ratchet.""" if not (layout.root / "complexipy-snapshot.json").is_file(): return _ratchet_skip_or_refuse( layout, @@ -329,8 +330,7 @@ def _complexipy(layout: Layout, env: Mapping[str, str]) -> GateVerdict | None: "complexipy-snapshot.json", "boot the snapshot (complexipy --snapshot-create), even when clean", ) - argv = [str(_tool("complexipy")), str(layout.source_root)] - return _run_external("complexipy", argv, cwd=layout.root, env=env) + return complexipy_verdict(layout.root, layout.source_root, env, _tool("complexipy"), _exec) def _ratchet_skip_or_refuse( diff --git a/tests/test_complexipy_snapshot.py b/tests/test_complexipy_snapshot.py new file mode 100644 index 0000000..2c7e602 --- /dev/null +++ b/tests/test_complexipy_snapshot.py @@ -0,0 +1,495 @@ +"""The complexipy snapshot ratchet — the floor the tool used to eat. + +**The reproduced defect.** ``complexipy-snapshot.json`` is the committed +cognitive-complexity floor. In the pinned ``complexipy==5.6.0`` the tool's own +snapshot compare REWRITES that file on its success path +(``complexipy/utils/snapshot.py`` ``handle_snapshot_watermark`` returns True only +after ``create_snapshot_file(...)`` with the functions IT measured), so two +ordinary green runs empty a populated floor at exit 0: measure a narrower surface +(``complexipy src/one_simple_file.py``), or raise the bar (``-mx 100``). Observed +on a real snapshot: ``1 entry -> 0 entries``, ``exit=0``, both ways. Committed, +that deletes the floor forever behind a green gate. + +**What this file pins.** The stage no longer hands its own artifact to the tool: +:mod:`cf_quality.complexipy_ratchet` measures write-free (``--snapshot-ignore``, +never ``--snapshot-create`` — the only two paths to ``create_snapshot_file``) and +grades the ratchet in kit code, where it is a pure function of (committed floor, +measured census) and cannot be silently destroyed. The control rods: + +- RED on the **emptying** world — the floor names a file the run did not measure, + whether it fell outside the graded source root or was skipped inside it; +- RED on a real **complexity regression** — a new offender, and a rise above a + committed watermark; +- RED on a **vacuous** run — a census of zero functions can never report clean, + with a second leg (the tree demonstrably defines functions) so the refusal + survives an already-emptied floor; +- GREEN on an **unchanged** floor, on a **legitimate non-empty shrink**, and on a + genuinely clean repo whose floor is ``[]``; +- the **write-free** proof at the argv altitude, plus the measured COUNT riding + every finding so no verdict can be read without the measurement behind it. + +The external tool is faked at the established subprocess seam +(``gate_runner._exec`` / ``gate_runner._tool``, through ``test_gate_runner``'s +helpers) so the REAL grading logic runs against representative ``--plain`` +output, and every stage runs through ``gate_runner._run_stage`` — the altitude the +battery reads, where a raised GateError is already the stage's exit-2 verdict. +:func:`cf_quality.complexipy_ratchet.grade` is additionally exercised with no +subprocess at all, which is the whole point of moving the comparison here. +""" + +from __future__ import annotations + +import json +import subprocess +from collections.abc import Mapping +from pathlib import Path + +import pytest +from test_gate_runner import _clean_cf_responses, _install_fakes, _layout, _write + +from cf_quality import complexipy_ratchet, gate_runner +from cf_quality.complexipy_ratchet import Census, grade, measurement_argv, read_snapshot +from cf_quality.errors import GateError, GateVerdict + +#: Module text that merely EXISTS and defines a function. Every complexity in this +#: file comes from the faked ``--plain`` census, never from this source — the fake +#: is what lets a watermark of 33 be asserted without authoring a monster. +A_FUNCTION = "def a_function():\n return 1\n" + +Responses = Mapping[str, tuple[int, str, str]] + + +def _snapshot(root: Path, *entries: tuple[str, str, int]) -> None: + """Write a committed floor in complexipy's own shape (path, function, watermark). + + Mirrors the observed on-disk form of a real consumer snapshot: ``path`` is + repo-relative and already carries the file name, ``file_name`` is the + basename, and only over-threshold functions are stored (clean tree -> ``[]``). + """ + payload = [ + { + "path": path, + "file_name": Path(path).name, + "functions": [{"name": name, "complexity": watermark}], + } + for path, name, watermark in entries + ] + _write(root, "complexipy-snapshot.json", json.dumps(payload)) + + +def _census(*rows: tuple[str, str, int]) -> str: + """complexipy ``--plain`` stdout: one `` `` line.""" + return "".join(f"{path} {name} {complexity}\n" for path, name, complexity in rows) + + +def _measured(census: str, offenders: str) -> dict[str, tuple[int, str, str]]: + """The clean board with complexipy's two write-free runs answered explicitly. + + The offender run's exit code is 1 whenever it reports anything, and it is + deliberately irrelevant: with the compare switched off complexipy's status + only restates "something is over threshold", the normal state of a repo + carrying a baselined floor. + """ + responses = _clean_cf_responses() + responses["complexipy"] = (0, census, "") + responses["complexipy-offenders"] = (1 if offenders else 0, offenders, "") + return responses + + +def _stage_verdict( + root: Path, monkeypatch: pytest.MonkeyPatch, responses: Responses +) -> GateVerdict: + """Run the real complexipy stage the way the battery runs it. + + Through ``gate_runner._run_stage`` so a GateError raised by a measurement we + cannot read arrives as the stage's exit-2 verdict, exactly as the board shows + it — never as an exception the caller has to know about. + """ + _install_fakes(monkeypatch, responses) + stage = gate_runner.Stage("complexipy", gate_runner._complexipy) + verdict = gate_runner._run_stage(stage, _layout(root), {}) + assert verdict is not None, "a Python repo carrying a snapshot never skips" + return verdict + + +def _codes(verdict: GateVerdict) -> list[str]: + return [violation.code for violation in verdict.violations] + + +def _light(root: Path, rel: str = "src/light.py") -> tuple[str, str, int]: + """A measured, clean census row for a file that really is on disk.""" + _write(root, rel, A_FUNCTION) + return (rel, "a_function", 1) + + +# --- RED: the emptying world (the reproduced defect) -------------------------- + + +def test_floor_file_outside_the_graded_source_root_is_refused( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The narrowed-surface trigger, structurally: the committed floor holds a + # watermark for a file the gate no longer even looks at. Under the tool's own + # compare this run is exit 0 AND rewrites the floor to the smaller world. + _write(tmp_path, "legacy/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("legacy/heavy.py", "heavy", 33)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(_census(_light(tmp_path)), "")) + + assert _codes(verdict) == ["COMPLEXIPY_SURFACE_NARROWED"] + assert verdict.exit_code == 1 + assert verdict.violations[0].context["snapshot_functions"] == 1 + + +def test_floor_file_present_but_unmeasured_is_refused( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # Inside the graded tree and still on disk, yet this run produced NO + # measurement for it (excluded, ignore-commented, or emptied). A before/after + # diff of the artifact sees nothing here — there is no rewrite to observe. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(_census(_light(tmp_path)), "")) + + assert _codes(verdict) == ["COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED"] + assert verdict.exit_code == 1 + assert verdict.violations[0].path == "src/heavy.py" + + +def test_deleted_floor_file_is_a_legitimate_improvement( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The other reading of an unmeasured entry: the file is GONE. That is a real + # shrink, not a narrowed surface, and it must not be charged as a finding. + _snapshot(tmp_path, ("src/removed.py", "heavy", 33)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(_census(_light(tmp_path)), "")) + + assert verdict.passed, _codes(verdict) + + +# --- RED: real complexity regressions ---------------------------------------- + + +def test_new_offender_without_a_watermark_is_a_violation( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path) + rows = _census(("src/heavy.py", "heavy", 40)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(rows, rows)) + + assert _codes(verdict) == ["COMPLEXIPY_NEW_OFFENDER"] + assert verdict.exit_code == 1 + assert verdict.violations[0].context["measured"] == 40 + + +def test_rise_above_a_committed_watermark_is_a_violation( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 20)) + rows = _census(("src/heavy.py", "heavy", 26)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(rows, rows)) + + assert _codes(verdict) == ["COMPLEXIPY_WATERMARK_REGRESSION"] + assert verdict.violations[0].context["measured"] == 26 + + +# --- RED: the vacuous run ---------------------------------------------------- + + +def test_vacuous_run_can_never_report_clean( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # THE void rod. The floor is ALREADY `[]` — so the per-file surface audit has + # nothing to say — the tree demonstrably defines a function, and complexipy + # measured NOTHING. An empty census and a clean tree are indistinguishable + # from a count alone, so a run that graded nothing refuses at exit 2. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured("", "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_MEASURED_NOTHING" + assert verdict.exit_code == 2 + assert verdict.error.context["measured_functions"] == 0 + + +def test_vacuous_run_against_a_populated_floor_names_the_files_it_lost( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The full emptying world: a populated floor plus a census of nothing. The + # refusal AND the per-file findings both ride the verdict, so the board says + # WHAT went unmeasured, not merely that something did. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured("", "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_MEASURED_NOTHING" + assert _codes(verdict) == ["COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED"] + assert verdict.exit_code == 2 + + +# --- RED: instruments we cannot read ----------------------------------------- + + +def test_unreadable_floor_is_refused_not_read_as_empty( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # green-by-unreadable-file is the same gaming vector as green-by-missing-file, + # which this gate already refuses. + _write(tmp_path, "complexipy-snapshot.json", "{}") + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(_census(_light(tmp_path)), "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_SNAPSHOT_UNREADABLE" + assert verdict.exit_code == 2 + + +def test_unparseable_census_line_is_refused( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # A census we cannot parse must not degrade into a SMALLER census: dropping + # the line would be reporting a void as a measurement. + _snapshot(tmp_path) + census = "Analyzing 1 file...\n" + _census(_light(tmp_path)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(census, "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_OUTPUT_UNREADABLE" + assert verdict.exit_code == 2 + + +def test_unmeasurable_path_report_is_refused( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # complexipy telling us it could not analyze a file IS a narrowed surface, + # in the tool's own words. + _snapshot(tmp_path) + census = _census(_light(tmp_path)) + ( + "error: Failed to process src/broken.py - Please check file/folder exists\n" + ) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(census, "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_PATHS_UNMEASURABLE" + assert verdict.exit_code == 2 + + +def test_offender_absent_from_the_census_is_measurement_skew( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The offender set is a FILTER of the census, so it cannot hold a function the + # census never saw. If it does, the two runs are not one observation and the + # comparison inputs must not be graded. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + census = _census(("src/heavy.py", "heavy", 33)) + + verdict = _stage_verdict( + tmp_path, monkeypatch, _measured(census, _census(("src/ghost.py", "ghost", 40))) + ) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_MEASUREMENT_SKEW" + assert verdict.exit_code == 2 + + +# --- GREEN: the worlds that must stay green ---------------------------------- + + +def test_unchanged_floor_is_clean_at_the_watermark( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # A function sitting AT its watermark passes — complexipy's own `>` bound, + # preserved exactly (relaxing it to `>=` would be a softening). + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + rows = _census(("src/heavy.py", "heavy", 33)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(rows, rows)) + + assert verdict.passed, _codes(verdict) + assert verdict.exit_code == 0 + + +def test_legitimate_non_empty_shrink_is_clean( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # One offender simplified below the bar, the other untouched: the floor is + # still populated, the improved file is still MEASURED, and the verdict is + # green. Locking the shrink in stays the runbook's re-boot duty — but the same + # file falling OUT of the census would not pass (the rod above). + _write(tmp_path, "src/improved.py", A_FUNCTION) + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/improved.py", "improved", 30), ("src/heavy.py", "heavy", 20)) + census = _census(("src/improved.py", "improved", 4), ("src/heavy.py", "heavy", 20)) + + verdict = _stage_verdict( + tmp_path, monkeypatch, _measured(census, _census(("src/heavy.py", "heavy", 20))) + ) + + assert verdict.passed, _codes(verdict) + + +def test_clean_repo_with_an_empty_floor_is_clean( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The boot state a consumer adopts at: `[]` on disk, nothing over the bar, and + # a census that DID measure something. Clean, and provably not a void. + _snapshot(tmp_path) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(_census(_light(tmp_path)), "")) + + assert verdict.passed, _codes(verdict) + assert verdict.exit_code == 0 + + +# --- the write-free invocation (the root of the fix) ------------------------- + + +def test_measurement_argv_can_never_write_the_committed_floor() -> None: + # The pinned tool reaches `create_snapshot_file` from exactly two places: + # `--snapshot-create`, and the watermark compare's success path that + # `--snapshot-ignore` switches off. Both flags are load-bearing, both runs. + for offenders_only in (False, True): + argv = measurement_argv( + Path("/fake/complexipy"), Path("/repo/src"), offenders_only=offenders_only + ) + assert argv[:2] == ["/fake/complexipy", "/repo/src"] + assert "--snapshot-ignore" in argv, "the compare — and its rewrite — stays off" + assert "--snapshot-create" not in argv, "the gate never writes the artifact it grades" + assert "--plain" in argv, "the census must be machine-readable to be graded" + assert ("--failed" in argv) is offenders_only + + +def test_measurement_runs_pin_columns_so_the_census_cannot_wrap( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # rich wraps at 80 columns when stdout is not a terminal, and a wrapped census + # row is an UNPARSEABLE census row — the gate would refuse a healthy repo, or + # (worse, in a laxer parser) read a narrower world. COLUMNS is load-bearing. + row = _light(tmp_path) + _snapshot(tmp_path) + seen: list[dict[str, str]] = [] + + def recording_exec( + argv: list[str], + cwd: Path, + env: Mapping[str, str], + *, + stdin: str | None = None, + ) -> subprocess.CompletedProcess[str]: + seen.append(dict(env)) + return subprocess.CompletedProcess(argv, 0, _census(row), "") + + monkeypatch.setattr(gate_runner, "_tool", lambda name: Path("/fake") / name) + monkeypatch.setattr(gate_runner, "_exec", recording_exec) + + gate_runner._complexipy(_layout(tmp_path), {"PATH": "/usr/bin"}) + + assert len(seen) == 2, "the census run and the offender run" + for env in seen: + assert int(env["COLUMNS"]) >= 1000, "an 80-column wrap would break the census parse" + assert env["PYTHONIOENCODING"] == "utf-8", "the census decodes UTF-8, never by locale" + assert env["PATH"] == "/usr/bin", "the caller's environment survives the overlay" + + +def test_the_committed_floor_is_only_ever_read( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # Asserted on the artifact itself. With the tool faked this rod can only fail + # if OUR code writes the floor; the tool-side half of the proof is the argv rod + # above plus the two call sites in the pinned package. Under the tool's own + # compare this very green run rewrote these bytes to the measured subset. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + snapshot = tmp_path / "complexipy-snapshot.json" + before = snapshot.read_bytes() + rows = _census(("src/heavy.py", "heavy", 33)) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(rows, rows)) + + assert verdict.passed, _codes(verdict) + assert snapshot.read_bytes() == before, "the gate graded the floor without touching it" + + +# --- the ratchet as a pure function (no subprocess at all) ------------------- + + +def test_grade_is_a_pure_function_of_the_floor_and_the_census(tmp_path: Path) -> None: + # The property the tool's own compare cannot have: the comparison runs with no + # subprocess, no write and no tool exit code, so it is testable in isolation + # and no green run can destroy it. Every finding carries the measured tally. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + floor = {("src/heavy.py", "heavy"): 20} + census = Census(functions={("src/heavy.py", "heavy"): 31}) + + verdict = grade(tmp_path, tmp_path / "src", floor, census, census) + + assert _codes(verdict) == ["COMPLEXIPY_WATERMARK_REGRESSION"] + assert verdict.violations[0].context["measured_functions"] == 1 + assert verdict.violations[0].context["snapshot_functions"] == 1 + + +def test_read_snapshot_keys_the_observed_on_disk_shape(tmp_path: Path) -> None: + # Keyed the way complexipy's own output joins its two path fields: get that + # join wrong and every committed watermark reads as a brand-new offender. + # Shape and `Class::method` naming taken from a real consumer's snapshot. + _write( + tmp_path, + "complexipy-snapshot.json", + json.dumps( + [ + { + "path": "src/pkg/mod.py", + "file_name": "mod.py", + "functions": [ + {"name": "Engine::_run_inner", "complexity": 18}, + {"name": "Engine::_execute_stage", "complexity": 27}, + ], + } + ] + ), + ) + + floor = read_snapshot(tmp_path / "complexipy-snapshot.json") + + assert floor == { + ("src/pkg/mod.py", "Engine::_run_inner"): 18, + ("src/pkg/mod.py", "Engine::_execute_stage"): 27, + } + + +def test_census_parser_survives_a_path_containing_spaces() -> None: + # `--plain` is space-separated, so the parse must split from the RIGHT: the + # complexity and the function name are single tokens, a path is not. + census = complexipy_ratchet.parse_census("src/odd dir/mod.py Klass::method 12\n") + + assert census.functions == {("src/odd dir/mod.py", "Klass::method"): 12} + assert census.files == frozenset({"src/odd dir/mod.py"}) + + +# --- the preserved absent-watermark doctrine --------------------------------- + + +def test_absent_floor_doctrine_is_untouched(tmp_path: Path) -> None: + # Unchanged by this rework, and re-pinned here because the rework rewrote the + # stage: Python present with no snapshot REFUSES; a Python-free repo skips. + _write(tmp_path, "src/light.py", A_FUNCTION) + + with pytest.raises(GateError) as excinfo: + gate_runner._complexipy(_layout(tmp_path, py_present=True), {}) + assert excinfo.value.code == "GATE_COMPLEXIPY_SNAPSHOT_MISSING" + + assert gate_runner._complexipy(_layout(tmp_path, py_present=False), {}) is None diff --git a/tests/test_gate_runner.py b/tests/test_gate_runner.py index 2e937b2..9eaf8a2 100644 --- a/tests/test_gate_runner.py +++ b/tests/test_gate_runner.py @@ -52,10 +52,16 @@ def _violation(code: str, message: str, path: str) -> GateViolation: def _response_key(argv: Sequence[str]) -> str: - """Map a stage's argv back to its response key (ruff has two stages).""" + """Map a stage's argv back to its response key (ruff and complexipy each run twice). + + complexipy runs write-free TWICE — the ``--plain`` census, then the ``--failed`` + offender subset its OWN threshold selects — keyed apart like ruff's two stages. + """ name = Path(argv[0]).name if name == "ruff": return "ruff-format" if "format" in argv else "ruff-check" + if name == "complexipy": + return "complexipy-offenders" if "--failed" in argv else "complexipy" return name @@ -96,6 +102,10 @@ def _clean_cf_responses() -> dict[str, tuple[int, str, str]]: "ruff-check": (0, "", ""), "ruff-format": (0, "", ""), "pytest": (0, "", ""), + # A census that MEASURED something plus an empty offender subset: the stage + # grades the snapshot ratchet itself now, and a zero census is a void. + "complexipy": (0, "pkg.py greet 1\n", ""), + "complexipy-offenders": (0, "", ""), } for gate in cf_gates: responses[gate] = (0, _verdict_json(gate), "") @@ -206,7 +216,7 @@ def test_one_run_reports_every_failing_gate( # regression, and a failing test. cf-gate must surface ALL four in ONE run. _write(tmp_path, "pkg.py", "x = 1\n") _write(tmp_path, "mypy-baseline.txt", "") - _write(tmp_path, "complexipy-snapshot.json", "{}") + _write(tmp_path, "complexipy-snapshot.json", "[]") responses = _clean_cf_responses() responses["ruff-check"] = (1, "pkg.py:1:1: E501 line too long", "") responses["cf-file-budget"] = ( @@ -219,7 +229,6 @@ def test_one_run_reports_every_failing_gate( ) responses["mypy"] = (1, "pkg.py: error: bad", "") responses["mypy-baseline"] = (1, "pkg.py: error: bad (new over baseline)", "") - responses["complexipy"] = (0, "", "") responses["pytest"] = (1, "1 failed, 0 passed", "") _install_fakes(monkeypatch, responses) @@ -285,11 +294,10 @@ def test_mypy_gates_on_filter_exit_not_mypy_exit( # filter exits 0 (no NEW errors) — the verdict must ride the filter. _write(tmp_path, "pkg.py", "x = 1\n") _write(tmp_path, "mypy-baseline.txt", "") - _write(tmp_path, "complexipy-snapshot.json", "{}") + _write(tmp_path, "complexipy-snapshot.json", "[]") responses = _clean_cf_responses() responses["mypy"] = (1, "pkg.py: error: baselined debt", "") responses["mypy-baseline"] = (0, "", "") - responses["complexipy"] = (0, "", "") _install_fakes(monkeypatch, responses) verdicts = run_battery(tmp_path, {}) @@ -379,7 +387,8 @@ def test_import_contract_aggregates_passes_into_one_verdict( # actually run: ruff rides the derived known-first-party; that first-party set # is the SAME one cf-repo-config resolves; complexipy refuses a Python repo # with no snapshot, skips a Python-free one visibly, and targets the resolved -# source_root in a single unpiped process (a pipe would mask its exit code). +# source_root in unpiped, WRITE-FREE processes (a pipe would mask its exit code; +# an unguarded run rewrites the committed floor — tests/test_complexipy_snapshot.py). def _layout(root: Path, *, first_party: str = "[]", py_present: bool = True) -> gate_runner.Layout: @@ -461,13 +470,18 @@ def test_complexipy_skips_python_free_repo_visibly(tmp_path: Path) -> None: assert gate_runner._complexipy(layout, {}) is None -def test_complexipy_targets_source_root_in_one_unpiped_process( +def test_complexipy_targets_source_root_in_unpiped_write_free_processes( tmp_path: Path, monkeypatch: pytest.MonkeyPatch ) -> None: - # With the snapshot present, complexipy grades the resolved source_root as a - # SINGLE process — piping it (as the mypy stage pipes through the filter) - # would mask its exit code (the tool-spike defect this rod fences off). - _write(tmp_path, "complexipy-snapshot.json", "{}") + # With the snapshot present, complexipy measures the resolved source_root in + # UNPIPED processes — piping it (as the mypy stage pipes through the filter) + # would mask its exit code (the tool-spike defect this rod fences off). It is + # also invoked WRITE-FREE: `--snapshot-ignore` and never `--snapshot-create` + # are the only two flags that keep the pinned tool away from + # `create_snapshot_file`, which its own green compare path calls (see + # cf_quality.complexipy_ratchet). Two measurements, one per question the + # ratchet asks; the empty census here is legitimate (no src/, nothing to grade). + _write(tmp_path, "complexipy-snapshot.json", "[]") calls: list[tuple[list[str], str | None]] = [] _record_calls(monkeypatch, calls) layout = _layout(tmp_path) @@ -475,7 +489,11 @@ def test_complexipy_targets_source_root_in_one_unpiped_process( verdict = gate_runner._complexipy(layout, {}) assert verdict is not None and verdict.passed - assert len(calls) == 1, "one unpiped process — no stdin handoff like the mypy filter pipe" - argv, stdin = calls[0] - assert argv == [str(Path("/fake") / "complexipy"), str(tmp_path / "src")] - assert stdin is None + assert len(calls) == 2, "the census run and the offender run — nothing else" + for argv, stdin in calls: + assert argv[:2] == [str(Path("/fake") / "complexipy"), str(tmp_path / "src")] + assert "--snapshot-ignore" in argv, "the committed floor must be out of reach" + assert "--snapshot-create" not in argv, "the gate never writes the artifact it grades" + assert "--plain" in argv, "the census is parsed, so it rides the scripting form" + assert stdin is None, "no stdin handoff like the mypy filter pipe" + assert ["--failed" in argv for argv, _ in calls] == [False, True], "census, then offenders" From 587b92c7a19245261c584edb83b933971340f6e8 Mon Sep 17 00:00:00 2001 From: Antawari Date: Tue, 28 Jul 2026 22:37:48 -0600 Subject: [PATCH 2/2] Close the two ways consumer config could still empty the complexity floor The first pass put the floor out of the gate's reach by measuring write-free. Two refuters then showed configuration walks around it. A consumer's own complexipy config re-opens the destructive branch. snapshot-create resolves CLI-first then TOML, the kit passes no CLI value, and --snapshot-create has no negating secondary name, so a committed snapshot-create = true wins and handle_snapshot_file_creation rewrites the floor from what the run just measured -- a branch --snapshot-ignore never touches. Neutralising it is impossible without breaking the design, because the tool derives the snapshot path AND every reported path from the same working directory, so moving the cwd re-keys the whole census. So the gate reads the consumer's config and REFUSES, naming the key, the call site that would fire, and the remedy -- and it names the keys that ARE honoured, so a real budget or a real exclusion never needs a workaround. Sixteen keys, including three the refuters did not name, and the search order mirrors the tool's own down to an empty config file shadowing a populated one. A raised threshold emptied the offender set and the ratchet graded nothing, green. The rule only ever iterated offenders, and the vacuity guard watched the census, which a raised bar leaves full. A function at 33 could reach 90 unremarked. Closed without a second threshold authority, from data already in hand: a floor function whose measured value is at or above its committed watermark, yet absent from the offender set, proves the bar moved since the floor was booted. The tool keeps sole say over what an offender is, so a consumer who booted at their own budget is still graded at it. Also closed: the instrument's exit code and stderr were discarded, so every tool-side failure was re-attributed to the repo with the reason deleted. A non-UTF-8 ledger raised past the stage and killed the whole board instead of the typed refusal it promised. A Python-free source root beside real code reported PASS having measured nothing, because both vacuity legs consulted the same possibly-wrong surface. An ignore comment voided a watermark silently, now caught per function rather than per file. A symlinked module inside the tree was accused of lying outside it. Duplicate function keys collapsed last-wins, under-reporting the measured count and able to fake a skew. The count is evidence, so it stops being a print to stderr the workflows never tee and rides the verdict and the board instead. Every violation and refusal now names a remedy. An integration fixture wrote OVER the consumer module to inject a type error, deleting its only function; the stricter vacuity leg then refused for having measured nothing and the battery went red for the wrong reason. It appends now. That also exposed the injected literal as formatter-dirty, so ruff-format had been a silent second red gate in a test that claims one. Three documents asserted the snapshot does not shrink itself and only --snapshot-create rewrites it. Both halves are false for the pinned version, and one test pinned the false sentence in place. The README's day-one boot command was worse than stale: it named the rewriting compare rather than the creating flag. Split by what each part answers -- the artifact, the instrument, the rule -- because the runner is at its line limit and the rule may not import the runner. Co-Authored-By: Claude Opus 5 (1M context) --- DESIGN.md | 71 ++- README.md | 7 +- configs/BASELINE-CONVENTIONS.md | 98 +++- docs/tool-spikes.md | 182 +++++++- src/cf_quality/complexipy_floor.py | 86 ++++ src/cf_quality/complexipy_measure.py | 466 +++++++++++++++++++ src/cf_quality/complexipy_ratchet.py | 620 ++++++++++++------------- src/cf_quality/errors.py | 17 +- src/cf_quality/gate_runner.py | 8 +- tests/test_complexipy_instrument.py | 363 +++++++++++++++ tests/test_complexipy_ratchet_rules.py | 303 ++++++++++++ tests/test_complexipy_snapshot.py | 80 ++-- tests/test_configs.py | 45 +- tests/test_errors.py | 28 +- tests/test_gate_runner.py | 34 +- tests/test_integration_consumer.py | 16 +- 16 files changed, 1969 insertions(+), 455 deletions(-) create mode 100644 src/cf_quality/complexipy_floor.py create mode 100644 src/cf_quality/complexipy_measure.py create mode 100644 tests/test_complexipy_instrument.py create mode 100644 tests/test_complexipy_ratchet_rules.py diff --git a/DESIGN.md b/DESIGN.md index d4d15ea..4474e7d 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -56,8 +56,10 @@ Shared configs: `configs/ruff-base.toml` (C901 ≤ 10, PLR0915 ≤ 50, S battery BLE ban — the ratified budgets), `configs/mypy-base.toml` (strict-leaning profile, see §10), `configs/jscpd.src.json` / `configs/jscpd.tests.json` (the two-profile clone carve-out, see §7). Tool-behavior ground truth lives in -`docs/tool-spikes.md` — complexipy 5.5.0 and mypy-baseline 0.7.4 semantics -were observed on fixtures, not read off READMEs. +`docs/tool-spikes.md` — complexipy and mypy-baseline semantics were observed on +fixtures, not read off READMEs (spiked on complexipy 5.5.0 · mypy-baseline +0.7.4; complexipy's snapshot-write semantics re-measured on the pinned 5.6.0 +on 2026-07-28, and they inverted — see §4.11). **The single reproducible entrypoint — `cf-gate` (`gate_runner.py`).** The battery above is split across many console scripts and external tools so each @@ -135,9 +137,19 @@ become green-forever): to `:0`, so unrelated drift cannot resurrect findings. 6. **Cognitive-complexity watermark:** `complexipy src --snapshot-create` **from the repo root** (the snapshot lands in CWD, not the analyzed path) - → commit `complexipy-snapshot.json`. Observed gotchas: the snapshot does - NOT auto-shrink (the kit owns the re-snapshot on merge, or improvements - are not locked in); piping the gate command masks its exit code. + → commit `complexipy-snapshot.json`. This is the ONLY complexipy command a + consumer ever runs by hand. Measured on the pinned **complexipy 5.6.0** + (2026-07-28): a *passing* plain-run compare REWRITES the snapshot — + `handle_snapshot_watermark` calls `create_snapshot_file` on its + no-violation branch — so the tool will shrink its own floor to `[]` at + exit 0. `cf-gate` therefore measures write-free (`--snapshot-ignore`), AUDITS + the consumer's complexipy config so `snapshot-create` cannot re-open that + write from TOML, and grades in `complexipy_ratchet` (§4.11). The duty that + remains: a shrink is locked in only by a deliberate re-`--snapshot-create`, + because the watermark rule is a `>` bound and a stale watermark still + grandfathers a climb back up to it. Re-boot in the SAME commit as any change + to `max-complexity-allowed`, or the gate refuses the mismatch + (`GATE_COMPLEXIPY_THRESHOLD_RAISED`). 7. **Exemptions:** if the repo carries any gated suppression, register each in `exemptions.json` (five fields: file, symbol_or_line, rule, reason, approver) and set `frozen_count`. @@ -460,11 +472,50 @@ ordinary PR. new. Fixed-only also exits nonzero ("re-sync"), so every shrink is committed — the ratchet direction is enforced by the tool itself. - **complexipy — set-like per function** (new offender fails; baselined - offender worsening fails), but the snapshot does NOT auto-shrink: the - re-snapshot-on-merge step is runbook procedure (§3 step 6), not yet CI - mechanism. The complexipy step IS in `quality-gate.yml` (and `self-ci.yml`) - as of 2026-06-11, with the mypy-style presence rule: a Python repo without - its committed snapshot FAILS; only a Python-free repo skips, visibly. + offender worsening fails), and graded by the KIT, not by the tool. The + 2026-06-10 spike recorded on 5.5.0 that the snapshot never shrinks itself. + Re-measured on the pinned **complexipy 5.6.0** (2026-07-28) that claim is + false and inverted: a *passing* compare rewrites the floor — + `handle_snapshot_watermark` calls `create_snapshot_file` on its no-violation + branch — reproduced at exit 0 shrinking a populated snapshot to `[]` both by + grading a narrower path than the floor and by raising the threshold above + every function. So the vector here was never "the floor cannot shrink", it + was "the floor can be **zeroed** behind a green run", which is laundering by + deletion. That zeroing is CLOSED by construction: `cf-gate` measures with + `--snapshot-ignore`, AUDITS the consumer's own complexipy config so the other + write path cannot be re-opened from TOML (`snapshot-create` resolves + CLI-first/TOML-second and has no negating flag, so a config carrying it made + both measurement runs rewrite the floor — + `GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT` now refuses it, naming the key + and the file), and compares in `complexipy_ratchet.grade`, a pure function + of (committed floor, measured census) — nothing the tool does can rewrite + the artifact it is graded against. Residue, and it is the OLD residue in a + narrower form: locking a shrink IN is still runbook procedure (§3 step 6), + not CI mechanism — the watermark is a `>` bound, so an improvement that is + never re-snapshotted leaves a stale watermark a later regression may climb + back to, and a deleted floor file leaves a dead entry that would grandfather + a same-named function if the file returned. Mechanical now, stated exactly + (an earlier pass on this branch wrote that emptying a floor file of functions + FAILS — false, since `--plain` lists every measured function regardless of + threshold): a floor file that still exists but was not **measured**, or that + sits outside the graded root, FAILS + (`COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED` / `COMPLEXIPY_SURFACE_NARROWED`); a + floor **function** missing from a file that WAS measured FAILS + (`COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED` — the shape a + `# complexipy: ignore` comment takes, an unregistered exemption from this + gate); a threshold raised above a committed watermark REFUSES rather than + grading the emptied offender set (`GATE_COMPLEXIPY_THRESHOLD_RAISED`, + threshold-free: a floor entry proves that function was above the boot bar); + and a run that measured zero functions cannot report clean while the floor + names functions or the repo contains Python at all + (`GATE_COMPLEXIPY_MEASURED_NOTHING`, riding the caller's repo-wide + `py_present` so the vacuity leg and the absent-snapshot doctrine cannot + disagree). Not machine-caught: a floor a human zeroes by hand and commits is + green — the ratchet just goes vacuous, and only the + `complexipy-snapshot.json` diff in review shows it. The + complexipy step IS in `quality-gate.yml` (and `self-ci.yml`) as of + 2026-06-11, with the mypy-style presence rule: a Python repo without its + committed snapshot FAILS; only a Python-free repo skips, visibly. - **cf-exemptions — count-based by design**, and honestly so: `frozen_count` refuses silent growth, but a 1-for-1 entry swap at equal count is machine-visible only as an `exemptions.json` diff in review, not diff --git a/README.md b/README.md index 6aad0be..82bce8e 100644 --- a/README.md +++ b/README.md @@ -67,9 +67,14 @@ carries a dated ratchet ticket so green-by-baseline cannot become green-forever) ```bash cf-file-budget init # freeze existing >500-line files at measured size mypy src | mypy-baseline sync -complexipy src # cognitive-complexity snapshot (second metric) +complexipy src --snapshot-create # cognitive-complexity floor (second metric) ``` +`--snapshot-create` is not optional here, and a bare `complexipy src` is never the +right command: with a snapshot present the tool's own *passing* compare REWRITES the +floor (measured on the pinned 5.6.0 — see `configs/BASELINE-CONVENTIONS.md` §1). +Run it from the repo root; the snapshot lands in CWD, not in the analyzed path. + ## Development ```bash diff --git a/configs/BASELINE-CONVENTIONS.md b/configs/BASELINE-CONVENTIONS.md index f488fd4..0deb376 100644 --- a/configs/BASELINE-CONVENTIONS.md +++ b/configs/BASELINE-CONVENTIONS.md @@ -1,9 +1,13 @@ # Baseline conventions — mounting the ratchet on a consumer repo How a repo boots green-by-construction on day one and then only ever shrinks. -Every workflow below is the **observed** behavior of the pinned tools -(complexipy 5.5.0 · mypy-baseline 0.7.4), measured in `docs/tool-spikes.md` — -not read off a README. Every baseline carries a dated ratchet ticket so +Every workflow below is the **observed** behavior of the pinned tools, measured +in `docs/tool-spikes.md` — not read off a README. Version provenance, stated +because a stale label is exactly how a false claim survives here: the spike was +run on **complexipy 5.5.0 · mypy-baseline 0.7.4**; mypy-baseline is still pinned +at 0.7.4, but complexipy is now pinned at **5.6.0** and only its +snapshot-**write** semantics have been re-measured there (2026-07-28 — the first +complexipy gotcha below). Every baseline carries a dated ratchet ticket so green-by-baseline cannot become green-forever (the Law-1 refuter's expiry fix). ## The shared configs in this directory @@ -31,26 +35,90 @@ Exit 0 even when offenders exist — baseline boot is green by construction. The snapshot records only functions over the threshold; an all-clean tree writes a literal `[]`. -**Gate (every CI run, from the repo root):** +**Gate (every CI run, from the repo root) — `cf-gate`, never the tool's own +compare:** ```bash -complexipy src # plain run auto-compares when the snapshot exists +cf-gate # the complexipy stage measures write-free, grades in-kit ``` Fails (exit 1) on any NEW offender, or any baselined offender rising above its -watermark. A function at-or-below its watermark passes. +watermark — complexipy's own rule, applied by `cf_quality.complexipy_ratchet` +over a floor the tool is never allowed to touch. A function at-or-below its +watermark passes. Do **not** mount a bare `complexipy src` as the gate; the first +gotcha is why. **Observed gotchas the mount must respect:** -- The snapshot **does NOT auto-shrink**. After an improvement the plain run - passes but the file still holds the old watermark — a later regression back - up to the stale watermark would pass. The kit owns the re-baseline step: - re-run `--snapshot-create` on merge (or in the shrink ticket) to lock - improvements in. -- Run from the **repo root** so the committed snapshot is the one compared - (snapshot path is CWD-relative). -- Never pipe the gate command (`complexipy … | tail` masks the exit code); - gate on the command's own status, or set `pipefail`. +- **A passing plain-run compare REWRITES the snapshot** — measured on the pinned + **complexipy 5.6.0** (2026-07-28), correcting the 5.5.0 entry that used to + stand here and claim the file never shrinks itself. The destructive call site is + `complexipy/utils/snapshot.py::handle_snapshot_watermark`, which calls + `create_snapshot_file(...)` on its **no-violation** branch — the tool's green + path is its write path. Reproduced twice at exit 0, a populated snapshot + rewritten to `[]`: once by grading a path narrower than the floor describes, + once by raising the threshold above every function. Committed, either one + deletes the watermark forever behind a green run. `cf-gate` therefore measures + write-free with `--plain --color no --snapshot-ignore` (verified on 5.6.0 to + leave the file byte-unchanged) and grades the ratchet itself. +- **Your own `complexipy` config can defeat the gate, so the gate REFUSES instead + of measuring through it.** `--snapshot-ignore` disarms the compare's rewrite, but + `snapshot-create` is a **separate branch** (`main.py:323`) resolved CLI-first, + TOML-second (`utils/toml.py:235-240`) — and `--snapshot-create` has no negating + flag, so `cf-gate` cannot override it from the command line. So a + `complexipy.toml` / `.complexipy.toml` / `[tool.complexipy]` carrying any of + `snapshot-create`, `quiet`, `ratchet`, `failed`, `details = "low"`, + `ignore-complexity`, `report-ignored`, `output`, `output-format`, or the legacy + `output-csv|json|gitlab|sarif` fails the stage typed, naming the key and the file + (`GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT`, exit 2). **Remove the key** — there + is no workaround to reach for, because the two you actually want are already + honoured: `max-complexity-allowed` (your threshold — the kit declares none of its + own) and `exclude` (your surface). `no-ignore`, `check-script` and `sort` are + honoured too. An unparseable config is also a refusal + (`GATE_COMPLEXIPY_CONFIG_UNREADABLE`) — unreadable is not absent. +- **Raising `max-complexity-allowed` above a committed watermark fails the gate** + (`GATE_COMPLEXIPY_THRESHOLD_RAISED`, exit 2) rather than quietly grading an empty + offender set. If you mean to raise the bar, raise it and **re-boot the floor at + the new threshold** in the same commit, so the floor and the bar agree. +- **The re-snapshot duty is still yours, in a narrower shape.** With the write + disarmed the floor holds still, which means a shrink is not locked in merely by + passing: the watermark rule is a `>` bound, so a function that got simpler and + later climbs back to its stale watermark passes. Re-run + `complexipy --snapshot-create` from the repo root, deliberately, + and commit it — that is the shrink ticket's job. What is MECHANICAL now, stated + exactly (an earlier draft of this file claimed a floor file emptied of functions + fails — it does **not**: `--plain` lists every measured function regardless of + threshold, so a file whose functions all dropped below the bar is still measured + and still green): + - a floor file that exists but was not **measured** at all — excluded, + ignore-commented, outside the surface, or now functionless — fails + (`COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED`); + - a floor file outside the graded source root fails + (`COMPLEXIPY_SURFACE_NARROWED`); + - a floor **function** missing from a file that WAS measured fails + (`COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED`) — that is the shape a + `# complexipy: ignore` comment takes, and it is an unregistered exemption from + this gate, so it is refused rather than absorbed; + - a run that measured **zero functions** can no longer report clean while the + floor names functions or the repo contains Python at all + (`GATE_COMPLEXIPY_MEASURED_NOTHING`). + + Every one of those messages names the re-boot command as its remedy. Still on you, + not on the gate: a wholly deleted floor file is a legitimate improvement and stays + green, so the snapshot keeps dead entries until a re-boot clears them — and a floor + someone zeroes by hand and commits is green too, visible only as a + `complexipy-snapshot.json` diff in review. +- Run the **boot** from the **repo root** so the snapshot lands beside the code it + describes (its path is CWD-relative, and the gate reads it from the repo root). +- Never pipe a hand-run complexipy command (`complexipy … | tail` masks the exit + code). `cf-gate` runs it with a fixed argv and no shell, and does not *grade* on + its exit code — but it does **cross-check** it: 0 and 1 are the tool's only + verdicts, 1 being the ordinary "some function is over threshold", so exit 1 with an + empty census, or any other non-zero code, is the instrument failing and refuses + distinctly (`GATE_COMPLEXIPY_INSTRUMENT_FAILED`, carrying the exit code and a + stderr excerpt) instead of being charged to your code. A file complexipy cannot + parse likewise refuses (`GATE_COMPLEXIPY_PATHS_UNMEASURABLE`, exit 2, not exit 1): + a narrower surface than the tree makes every other clean reading unsupported. ## 2. mypy-baseline — the 0-new-type-errors gate (multiset set-difference) diff --git a/docs/tool-spikes.md b/docs/tool-spikes.md index 2264749..6c85e6e 100644 --- a/docs/tool-spikes.md +++ b/docs/tool-spikes.md @@ -1,8 +1,21 @@ -# Tool spikes — observed semantics (2026-06-10) +# Tool spikes — observed semantics (2026-06-10; §1 write semantics re-measured 2026-07-28) Real behavior of the two pinned ratchet engines, measured on throwaway fixtures in `/tmp` against the kit's venv. Every claim below was observed, not read off a -README. Versions: **complexipy 5.5.0** · **mypy-baseline 0.7.4** (mypy 1.x). +README. Versions **as measured in the original spike**: **complexipy 5.5.0** · +**mypy-baseline 0.7.4** (mypy 1.x). The pin has since moved — read the skew note. + +> **Version skew — read before trusting §1 (2026-07-28).** `pyproject.toml` pins +> **complexipy 5.6.0**; this spike was run against 5.5.0 and the label was never +> refreshed when the pin moved. Only the snapshot-**write** semantics have been +> re-measured on 5.6.0: the first entry under "Gotchas for the kit", the +> `handle_snapshot_watermark` call site, and the write-free measurement argv. +> The flag table, the snapshot file format and the ratchet exit-code table below +> still carry their original **5.5.0** observation and are unverified against the +> pin — treat each as a claim until re-run. §2's mypy-baseline 0.7.4 is still the +> pinned version. The reason this note exists: a stale version header let a +> flatly false claim about the snapshot survive in this file from 5.5.0 into a +> 5.6.0 world, and the kit's ratchet was designed against it. > **Determinism note (2026-06-16).** The whole `[dev]` gauge battery is now pinned > EXACTLY in `pyproject.toml` (`ruff==0.15.17`, `mypy==2.1.0`, @@ -17,7 +30,7 @@ README. Versions: **complexipy 5.5.0** · **mypy-baseline 0.7.4** (mypy 1.x). --- -## 1. complexipy 5.5.0 — snapshot / watermark ratchet +## 1. complexipy — snapshot / watermark ratchet (5.5.0; writes re-measured on 5.6.0) Fixture: a package with `simple` (CC 1), `branchy` (CC 4), later a deliberately nested `monster` (cognitive complexity 33). Note complexipy measures **cognitive** @@ -62,21 +75,154 @@ Shape: ### Gotchas for the kit -- **The snapshot does NOT auto-shrink.** After an improvement the plain run - passes but `complexipy-snapshot.json` still holds the old watermark; only - `--snapshot-create` rewrites it. The kit's ratchet step must re-create the - snapshot on merge (or a shrink ticket does) or improvements are not locked in - — a later regression back up to the stale watermark would pass. -- **Snapshot lands in CWD** — the CI step must run from the repo root so the - committed snapshot is the one compared. -- **Exit-code caution:** piping output (`complexipy … | tail`) masks the exit - code; gate on the command's own status (or `pipefail`). -- A function exactly **at** its watermark passes — the watermark is a ≤ bound, - shrink-only happens via re-snapshot, not via the compare itself. - -**Verdict:** `--snapshot-create` + plain-run compare gives exactly the -freeze/shrink-only ratchet the plan wanted (plan Q2 answered). The kit must own -the re-baseline step to make shrink sticky. +- **CORRECTED 2026-07-28 — a passing plain-run compare REWRITES the snapshot, so + the tool shrinks its own floor, all the way to `[]` in the reproduced case.** + The 5.5.0 entry that stood here asserted the opposite (that the file never + shrinks itself, and that `--snapshot-create` is the only thing that rewrites + it). Both halves are false on the pinned **complexipy 5.6.0**. The destructive + call site is `complexipy/utils/snapshot.py::handle_snapshot_watermark`, which + calls `create_snapshot_file(...)` on its **no-violation** branch: the tool's + GREEN path *is* its write path. Reproduced twice at **exit 0** against a + populated snapshot, each time leaving `[]` behind — (a) **narrowed surface**: + compare over a path narrower than the floor describes, nothing in that subset + violates, the floor is rewritten to the subset; (b) **raised threshold**: + `-mx 100` puts every function under the bar, same rewrite. Committed, either + one deletes the watermark forever behind a green gate. This is why the kit + never lets the tool near the artifact: `cf-gate` measures with + `--plain --color no --snapshot-ignore` (which makes + `should_run_snapshot_watermark` False — the only other caller of + `create_snapshot_file` is the never-passed `--snapshot-create`) and grades the + ratchet in its own pure function, `cf_quality.complexipy_ratchet.grade`, over + (committed floor, measured census). Also measured on 5.6.0: that argv over a + 142-file tree emitted 681 stdout lines, every one a + ` ` census row, **0 bytes on stderr**, and left + `complexipy-snapshot.json` byte-unchanged. +- **`--snapshot-ignore` alone is NOT enough — the consumer's own config can put the + write back, so the kit AUDITS that config and refuses.** Read in the source, not + inferred: `snapshot_create` resolves CLI-first-then-TOML + (`utils/toml.py:235-240`), the kit passes no CLI value for it, and + `handle_snapshot_file_creation` (`main.py:323`) is an **entirely separate branch** + from the watermark compare that `--snapshot-ignore` disarms. `--snapshot-create` + is declared with no negating secondary name (`main.py:97-102`), so argv cannot + force it off. A consumer's `complexipy.toml` / `.complexipy.toml` / + `[tool.complexipy] snapshot-create = true` therefore made **both** measurement + runs rewrite the floor while the gate graded the pre-write bytes it had already + read — green, artifact silently mutated. Two more keys defeat the run the same + way: `quiet = true` (rejected beside `--plain`, `main.py:748` → exit 2, empty + census, which the gate used to report as `GATE_COMPLEXIPY_MEASURED_NOTHING` and + blame on the repo) and `output-format` / the legacy `output-csv|json|gitlab|sarif` + (a report file written into the consumer's tree, plus `Results saved at …` + (`main.py:510`) and `Deprecated: …` (`main.py:521-536`) printed on **stdout, + unguarded by `plain`, BEFORE the census**). `cf_quality.complexipy_measure` + `audit_config` reads the same file the tool reads, in the tool's own order + (`utils/toml.py:135-148`: `complexipy.toml`, then `.complexipy.toml`, then + `pyproject.toml`, first hit wins — and for the first two an EMPTY document counts + as a hit, so it shadows `pyproject.toml`), and REFUSES with the key and the file + named (`GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT`). Neutralising it by moving + the cwd was rejected for cause: `main.py:66-67` resolves the config, + `main.py:321` the snapshot path and `main.py:308-310` every reported path from the + **same `os.getcwd()`**, so a different cwd re-keys the whole census. Deliberately + still honoured: `max-complexity-allowed`, `exclude`, `no-ignore`, `check-script`, + `sort`. +- **A raised threshold empties the offender set, and the ratchet must refuse rather + than grade nothing.** With `max-complexity-allowed` raised (by `-mx` or by + config), the census still lists every function — so every floor file is present + and the surface audit is silent — while `--failed` returns **nothing**, and a + subset check over an empty subset passes trivially. The closure needs no second + threshold authority: a floor entry is proof that function was ABOVE the bar when + the floor was booted (`create_snapshot_file` stores only over-threshold + functions), so a census value at-or-above its watermark that the offender run does + NOT report means `threshold_now >= census >= watermark > threshold_at_boot` + (`GATE_COMPLEXIPY_THRESHOLD_RAISED`). A hand-lowered watermark lands in the same + refusal, correctly — the tool would never have written one at or below its own + threshold. +- **Exit semantics, established from the source** (`main.py:756-767` + `resolve_final_success` reduces to `has_success and valid_paths` with the compare + disarmed; `has_success` is `not failing_functions`, `utils/output.py:44`, which is + filled for every over-threshold function whether or not `--failed` was passed, + `utils/output.py:234-241`). So **0 and 1 are the tool's only verdicts**, 1 being + the legitimate "some function is over threshold" — the normal state of a repo + carrying a floor. Exit 1 with an **empty** census is a contradiction (and is the + offender run's second vacuity leg); any other code is `typer` declining to run + (`BadParameter` → 2 at `main.py:748`, `validate_ratchet` → 2 at + `main.py:726-729`). The kit refuses distinctly on both + (`GATE_COMPLEXIPY_INSTRUMENT_FAILED`) and carries the exit code plus a stderr + excerpt in the refusal context, because a tool-side failure re-attributed to the + repo with its reason deleted is worse than no gate. +- **`--plain` prints EVERY measured function, over threshold or not** + (`utils/output.py:234` drops a row only under `failed_only`; `:243` appends every + other one). That is load-bearing twice over: it is why a function that merely got + simpler is still in the census, and therefore why a floor function absent from a + file that WAS measured means something else — renamed, deleted, or dropped by a + `# complexipy: ignore` comment (`COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED`). The + argv passes neither `--no-ignore` nor `--report-ignored`, so an ignore comment + drops a function from both runs; that is caught structurally now rather than by + flag. **Open for the operator:** adding `--no-ignore` to the gate argv would need + the boot command in `configs/BASELINE-CONVENTIONS.md` to carry the same flag, or + every currently-ignored function floods in as a new offender — a paired change, + not a unilateral one. +- **A path complexipy cannot analyze is a REFUSAL, not a finding** — declared, since + it is a taxonomy change: `print_invalid_paths` → `has_success=False` used to make + it exit 1. `GATE_COMPLEXIPY_PATHS_UNMEASURABLE` exits 2 instead, because an + unparseable file means the graded surface is narrower than the tree, so every + OTHER function's clean reading is unsupported and a findings-level report would + let the rest of the board read green beside a void. A consumer who lands a syntax + error gets a setup-error board; ruff and mypy red independently, so nothing is + hidden. +- **The re-snapshot duty survived the fix — it changed shape, it did not go + away.** With the write disarmed the floor holds still in both directions, so a + shrink is still not locked in merely by passing: the watermark rule is a `>` + bound (`complexipy_ratchet._regressions` mirrors the tool exactly), so a + function that got simpler and later climbs back to its stale watermark passes. + Locking a shrink in is a deliberate, committed + `complexipy --snapshot-create` from the repo root — the shrink + ticket's job. What changed is that forgetting is no longer silent where it used to + be catastrophic. Precisely (corrected 2026-07-28 — an earlier pass on this branch + claimed "an improvement that empties a floor file of functions … FAILS + `COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED`", which is **false**: `--plain` without + `--failed` lists every measured function regardless of threshold, so a file whose + functions all dropped below the bar stays in `census.files` and the gate is + GREEN): + - `COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED` fires when the file was not **measured** + at all — excluded, ignore-commented, outside the measurement surface, or now + containing no function whatsoever — while still existing on disk. + - `COMPLEXIPY_SURFACE_NARROWED` fires when a floor file exists but lies outside + the graded source root. + - `COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED` fires when the file WAS measured and + a floor function inside it was not, which the two rules above cannot see. + - A run that measured **zero functions** can no longer report clean while the + floor still names functions or the repo contains Python at all + (`GATE_COMPLEXIPY_MEASURED_NOTHING`); the second leg is the caller's repo-wide + `py_present`, the same answer the absent-snapshot doctrine rides, so the two + surfaces cannot disagree about whether there was anything to grade. + - Every one of those messages names the re-boot as its remedy, spelled with the + resolved graded root. + + Not caught, honestly: a wholly **deleted** floor file is a legitimate improvement + and stays green, so the snapshot accumulates dead entries until someone re-boots + it, and a floor a human zeroes by hand and commits is green as well — review of + the `complexipy-snapshot.json` diff is the only instrument on that one. +- **Snapshot lands in CWD** — the boot command must run from the repo root, so the + committed snapshot sits where the kit reads it from. +- **Exit-code caution:** piping output (`complexipy … | tail`) masks the exit code; + never pipe a hand-run command. The kit does not *grade* on that exit code — the + grade is `complexipy_ratchet.grade` over (floor, census) — but it does + **cross-check** it, per the exit semantics above: a code that contradicts the + census is the instrument failing, and the kit refuses instead of charging it to + the code. +- A function exactly **at** its watermark passes the watermark rule — a `>` bound, + mirrored exactly; shrink-only happens via re-snapshot, not via the compare itself. + It must however still appear in the `--failed` run, or the bar has moved + (`GATE_COMPLEXIPY_THRESHOLD_RAISED` above) — "at its watermark" and "no longer an + offender at all" are different worlds. + +**Verdict (revised 2026-07-28, measured on the pinned 5.6.0).** +`--snapshot-create` gives the freeze the plan wanted (plan Q2 answered), but the +plain-run **compare** is unusable as a gate: the run that passes is the run that +rewrites the floor it just graded. So the kit demotes complexipy to a measuring +instrument (`--plain … --snapshot-ignore`) and owns both halves of the ratchet — +the comparison, in `complexipy_ratchet.grade`, and the re-baseline step that +makes a shrink sticky. --- diff --git a/src/cf_quality/complexipy_floor.py b/src/cf_quality/complexipy_floor.py new file mode 100644 index 0000000..c8ca1ee --- /dev/null +++ b/src/cf_quality/complexipy_floor.py @@ -0,0 +1,86 @@ +"""The committed cognitive-complexity floor — the ONE place that touches the artifact. + +``complexipy-snapshot.json`` is the per-function watermark no later run may +exceed. The whole point of this rung is that the pinned ``complexipy==5.6.0`` +REWRITES that file on the success path of its own compare +(``complexipy/utils/snapshot.py:85``), so the artifact needs an owner that reads +it and never writes it. That owner is this module, deliberately alone in it: one +importable surface to audit, and a reviewer can prove "the gate never writes the +floor" by reading forty lines instead of grepping a battery. + +Two invariants live here and nowhere else: + +- **The KEY.** :func:`_normalized_path` is a declared mirror of + ``complexipy.utils.output.normalize_path`` (5.6.0, lines 274-280): the snapshot + stores ``path`` and ``file_name`` separately while ``--plain`` prints them + joined. Join them differently and every committed watermark reads as a + brand-new offender. The mirror is pinned by a test calling the INSTALLED + function, not by a declaration file — a 5.7.0 change to that join must fail the + suite, not wait for a reviewer to notice. +- **Unreadable is not empty.** A malformed floor REFUSES. Reading it as ``[]`` + would be green-by-unreadable-file, the same gaming vector as + green-by-missing-file, which the caller already refuses. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +from cf_quality.complexipy_measure import FunctionKey, refusal +from cf_quality.errors import GateError + +#: The committed floor's filename (CWD-relative for complexipy, repo root for us). +SNAPSHOT_FILENAME = "complexipy-snapshot.json" + + +def _normalized_path(path: str, file_name: str) -> str: + """Join a snapshot entry's two path fields the way complexipy's output does.""" + cleaned = path.rstrip("/") + if cleaned.endswith(file_name): + return cleaned + return f"{cleaned}/{file_name}" if cleaned else file_name + + +def _refuse_snapshot(snapshot: Path, reason: str) -> GateError: + return refusal( + "GATE_COMPLEXIPY_SNAPSHOT_UNREADABLE", + f"{SNAPSHOT_FILENAME} is not a readable complexipy snapshot: {reason} — an " + "unreadable floor is not an empty floor, so the gate refuses; re-boot it from " + "the repo root (complexipy --snapshot-create) and commit it", + {"snapshot": str(snapshot), "reason": reason}, + ) + + +def _entry_watermarks(entry: object, snapshot: Path) -> dict[FunctionKey, int]: + """One snapshot entry's watermarks; any other shape REFUSES rather than skips.""" + if not isinstance(entry, dict) or not isinstance(entry.get("functions"), list): + raise _refuse_snapshot(snapshot, f"entry is not {{path, file_name, functions}}: {entry!r}") + path = _normalized_path(str(entry.get("path", "")), str(entry.get("file_name", ""))) + watermarks: dict[FunctionKey, int] = {} + for function in entry["functions"]: + if not isinstance(function, dict) or not isinstance(function.get("complexity"), int): + raise _refuse_snapshot(snapshot, f"function is not {{name, complexity}}: {function!r}") + watermarks[(path, str(function.get("name", "")))] = int(function["complexity"]) + return watermarks + + +def read_snapshot(snapshot: Path) -> dict[FunctionKey, int]: + """The committed floor as ``{(file, function): watermark}`` — read, never written. + + The catch is ``ValueError``, which subsumes both ``json.JSONDecodeError`` and + the ``UnicodeDecodeError`` a non-UTF-8 snapshot raises. The narrower + ``JSONDecodeError`` let that one escape this function, escape + ``gate_runner._run_stage`` (which catches only ``GateError``) and lose the + entire 12-stage board to a traceback instead of one typed refusal. + """ + try: + raw = json.loads(snapshot.read_text(encoding="utf-8")) + except (OSError, ValueError) as exc: + raise _refuse_snapshot(snapshot, str(exc)) from exc + if not isinstance(raw, list): + raise _refuse_snapshot(snapshot, f"top level is {type(raw).__name__}, not a list") + floor: dict[FunctionKey, int] = {} + for entry in raw: + floor.update(_entry_watermarks(entry, snapshot)) + return floor diff --git a/src/cf_quality/complexipy_measure.py b/src/cf_quality/complexipy_measure.py new file mode 100644 index 0000000..7f6fec7 --- /dev/null +++ b/src/cf_quality/complexipy_measure.py @@ -0,0 +1,466 @@ +"""complexipy demoted to a measuring instrument — audited, invoked write-free, read. + +**Why this sits apart from the ratchet.** The ratchet +(:mod:`cf_quality.complexipy_ratchet`) is a pure function of (committed floor, +measured census). Everything that must be TRUE before that function may be +trusted lives here: the consumer's own complexipy configuration, the argv, the +two subprocess runs, and the parse of what came back. One responsibility — +"produce a census the ratchet may reason over, or refuse" — and the ratchet +never touches a subprocess. + +**The write-free claim, and its real precondition.** ``--snapshot-ignore`` alone +does NOT make the run write-free; that was the defect two refuters found in the +first cut of this fix, and it is the same defect the whole rung exists to +remove — a false invariant in a docstring. In the pinned ``complexipy==5.6.0`` +exactly two callers reach ``create_snapshot_file``: + +1. ``complexipy/utils/snapshot.py:85`` — ``handle_snapshot_watermark``'s + no-violation branch, i.e. the compare's SUCCESS path. ``--snapshot-ignore`` + switches that off (``main.py:332``: ``should_run_snapshot_watermark = + snapshot_file_exists and not snapshot_ignore``) and a consumer cannot switch + it back on, because the CLI value wins (``utils/toml.py:179``: + ``get_argument_value`` returns ``arg_value`` whenever it is not None). +2. ``main.py:323`` — ``handle_snapshot_file_creation``, an ENTIRELY separate + branch driven by ``snapshot-create``, which ``--snapshot-ignore`` does not + touch. The kit passes no CLI value for it, so ``utils/toml.py:235-240`` + resolves it from the consumer's ``complexipy.toml`` / ``.complexipy.toml`` / + ``[tool.complexipy]``; and ``--snapshot-create`` is declared with no negating + secondary name (``main.py:97-102``), so argv CANNOT force it off. + +So the argv's guarantee holds only while the consumer's config carries no +``snapshot-create``, and :func:`audit_config` is what makes it hold: it reads +the SAME file the tool will read, in the tool's own search order, and REFUSES +the stage when a key would let the run write the floor, silence the census, or +print anything else onto the census stream. A typed refusal naming the key and +the file cannot be silently wrong. A cwd or env trick could be, and was +rejected for cause: ``main.py:66-67`` resolves the config, ``main.py:321`` the +snapshot path, and ``main.py:308-310`` every reported path from the SAME +``os.getcwd()``, so moving the cwd to hide the config would re-key the entire +census and break the floor comparison — and would silently drop the LEGITIMATE +``max-complexity-allowed`` this design deliberately honours. + +**What is deliberately NOT refused.** ``max-complexity-allowed`` and ``exclude`` +are the consumer's own threshold and surface authorities; the kit declares no +budget of its own here, because a second authority would flag a baselined band +as a new offender. ``no-ignore`` and ``check-script`` only ever make the census +LARGER. ``sort`` reorders rows only, and :func:`parse_census` aggregates +``max()`` per key, so no order can change the result. + +**Reading the instrument.** Three channels come back and all three are evidence: +stdout (the census), the exit code, and stderr. This argv was measured over a +142-file tree emitting 681 census rows and **0 bytes of stderr**, so any stderr +at all rides the refusal context. The exit code is corroboration, never the +grade: with the compare disarmed, ``main.py:756-767 resolve_final_success`` +reduces to ``has_success and valid_paths``, and ``has_success`` is ``not +failing_functions`` (``utils/output.py:44``), which is filled for every function +over the resolved threshold whether or not ``--failed`` was passed +(``utils/output.py:234-241``). Exit 1 is therefore the legitimate "some function +is over threshold"; exit 1 with an EMPTY census is a contradiction; any other +non-zero code is the tool declining to run. +""" + +from __future__ import annotations + +import subprocess +import tomllib +from collections.abc import Mapping +from dataclasses import dataclass +from pathlib import Path +from typing import Protocol + +from cf_quality.errors import GateError + +#: ``(repo-relative file, function name)`` — the identity a watermark is keyed by. +#: complexipy keys on ``(path, file_name, name)``; ``path`` already carries the +#: file name in its own output, so the joined form is the same identity. +FunctionKey = tuple[str, str] + +#: The config files complexipy itself consults, in ITS order +#: (``utils/toml.py:135-148`` ``get_complexipy_toml_config``): first hit wins, and +#: a hit is any file that exists and yields a table — so an EMPTY +#: ``complexipy.toml`` shadows a populated ``pyproject.toml``, and the audit must +#: agree with that or it would audit a file the tool never reads. +CONFIG_FILENAMES = ("complexipy.toml", ".complexipy.toml", "pyproject.toml") + +#: Pinned onto every measurement run. ``COLUMNS`` is load-bearing: ``--plain`` +#: prints through rich, which wraps at 80 columns when stdout is not a terminal, +#: and a wrapped census row is an unparseable census row. ``TERM`` is load-bearing +#: for the same reason at one remove — ``rich/console.py:1015-1016`` returns a HARD +#: 80x25 and never reads ``COLUMNS`` at all when ``is_dumb_terminal``, which is +#: ``is_terminal and TERM in ("dumb", "unknown")`` (``rich/console.py:986-988``). +#: ``PYTHONIOENCODING`` pins the child's encoding so the census does not emit by +#: locale. +_MEASURE_ENV = { + "COLUMNS": "10000", + "PYTHONIOENCODING": "utf-8", + "NO_COLOR": "1", + "TERM": "xterm", +} + +#: Cleared for every measurement run: these are the two env names that force +#: rich's ``is_terminal`` True over a pipe (``rich/console.py:955-963``), which is +#: the other half of the dumb-terminal short circuit above. +_MEASURE_ENV_CLEARED = frozenset({"FORCE_COLOR", "TTY_COMPATIBLE"}) + +#: complexipy's own words when it could not analyze a path it was handed +#: (``utils/output.py:297`` ``print_invalid_paths``) — a silently narrower surface. +_UNMEASURABLE_MARKER = "Failed to process" + +#: Every consumer config key that would defeat THIS measurement, with why. Derived +#: by walking the tool's own CLI-first/TOML-second resolver +#: (``utils/toml.py:193-333`` ``get_arguments_value``) for the keys this argv +#: leaves unset — those are exactly the keys a consumer's config can still win. +_DEFEATING_KEYS = { + "snapshot-create": ( + "would make the run WRITE complexipy-snapshot.json from what it just " + "measured (main.py:323, a branch --snapshot-ignore does not disarm)" + ), + "quiet": ( + "is rejected together with --plain, so the run exits 2 with an empty " + "census and the gate would blame the repo (main.py:748)" + ), + "ratchet": ( + "requires --diff, which this argv never passes, so the run exits 2 " + "before measuring anything (main.py:726-729)" + ), + "failed": ( + "would narrow the CENSUS run to offenders too, and the census listing " + "every measured function is what the surface audit reasons over" + ), + "details": ( + 'the legacy spelling of failed — "low" means offenders only (utils/toml.py:259-261)' + ), + "ignore-complexity": ( + "would decouple the exit code from the tool's own threshold verdict, " + "and the exit code is how a contradicting run is caught" + ), + "report-ignored": ( + "prints one ignore-comment location per line onto the census stream " + "(main.py:701-708), which is not a census row" + ), + "output": "redirects machine-readable output into the consumer's tree", + "output-format": ( + "writes a report file and prints 'Results saved at ...' onto stdout " + "BEFORE the census, unguarded by --plain (main.py:510)" + ), + "output-csv": "the legacy output-format spelling — same write, plus a Deprecated line", + "output-json": "the legacy output-format spelling — same write, plus a Deprecated line", + "output-gitlab": "the legacy output-format spelling — same write, plus a Deprecated line", + "output-sarif": "the legacy output-format spelling — same write, plus a Deprecated line", +} + + +class Executor(Protocol): + """The subprocess seam the caller injects (``gate_runner._exec``). + + Structural, not inherited: the runner stays the caller's — one place owns + typed OSError translation and the no-shell fixed-argv discipline — while the + tests keep patching that single seam. + """ + + def __call__( + self, + argv: list[str], + cwd: Path, + env: Mapping[str, str], + *, + stdin: str | None = None, + ) -> subprocess.CompletedProcess[str]: ... + + +@dataclass(frozen=True) +class Census: + """What ONE write-free complexipy run actually measured. + + The census is the gate's independent fact about its own measurement surface: + it is read from the tool's output, never from the snapshot, so "the floor is + empty" and "we graded nothing" can never be the same observation. + + ``functions`` is keyed by :data:`FunctionKey` and aggregated with ``max()``; + ``rows`` counts the census LINES parsed. The two genuinely differ and both + are load-bearing — on a real consumer 681 rows collapse to 680 keys, because + ``@overload`` chains and ``if sys.platform:`` redefinitions repeat a + ``(path, name)`` pair. Counting keys UNDER-reports the measurement, which is + the anti-void evidence this stage exists to produce; and last-writer-wins + would let the HIGHER value of a straddling pair vanish, making our mirror of + complexipy's watermark rule strictly weaker than the rule it mirrors (the + tool iterates its own LIST, not a map) and letting a bogus skew fire when + only one of the two maps kept the high value. + """ + + functions: dict[FunctionKey, int] + rows: int + + @property + def files(self) -> frozenset[str]: + """The distinct files this run measured — the surface audit's evidence.""" + return frozenset(path for path, _ in self.functions) + + +def refusal(code: str, message: str, context: Mapping[str, object]) -> GateError: + """A refusal in the kit's typed vocabulary — the gate could not do its job.""" + return GateError(code=code, message=message, context=dict(context)) + + +def _table(path: Path) -> Mapping[str, object] | None: + """One config file's complexipy table, or None when the tool would not use it. + + Mirrors ``utils/toml.py::load_toml_config`` exactly: ``pyproject.toml`` + contributes only its ``[tool.complexipy]`` sub-table, every other name + contributes the whole document — including an empty one, which is why an + empty ``complexipy.toml`` shadows ``pyproject.toml``. ``ValueError`` is the + catch because it subsumes both ``TOMLDecodeError`` and the + ``UnicodeDecodeError`` a non-UTF-8 config raises; letting either escape would + lose the whole battery to a traceback instead of one typed refusal. + """ + if not path.is_file(): + return None + try: + data = tomllib.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError) as exc: + raise refusal( + "GATE_COMPLEXIPY_CONFIG_UNREADABLE", + f"cannot read {path.name}: {exc} — complexipy resolves its options from " + "this file, so an unreadable one leaves the measurement unaudited; fix " + "the TOML (or delete the file) and re-run", + {"config": str(path), "reason": str(exc)}, + ) from exc + if path.name != "pyproject.toml": + return data + tool = data.get("tool") + if not isinstance(tool, dict): + return None + table = tool.get("complexipy") + return table if isinstance(table, dict) else None + + +def _defeats(key: str, value: object) -> bool: + """True when this key/value pair would defeat the measurement. + + Value-sensitive on purpose: ``snapshot-create = false`` provably writes + nothing (``utils/snapshot.py:17``), so refusing it would be a false refusal, + and a false refusal costs a consumer a red board for nothing. ``details`` is + the one non-boolean-shaped key and only bites at ``"low"``. + """ + if key not in _DEFEATING_KEYS: + return False + if key == "details": + return str(value).strip().lower() == "low" + return bool(value) + + +def audit_config(root: Path) -> str | None: + """The consumer's complexipy config, audited before the instrument is trusted. + + Returns the filename the tool will read (None when it reads none) so a + PASSING verdict can state which config was in force. Raises when that config + carries a key from :data:`_DEFEATING_KEYS` — the mechanism that turns this + module's write-free claim from an aspiration into an enforced precondition. + """ + for name in CONFIG_FILENAMES: + table = _table(root / name) + if table is None: + continue + defeating = sorted(key for key, value in table.items() if _defeats(str(key), value)) + if defeating: + detail = "; ".join(f"{key} ({_DEFEATING_KEYS[key]})" for key in defeating) + raise refusal( + "GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT", + f"{name} sets complexipy option(s) this gate cannot measure through: " + f"{detail} — remove them from {name}; max-complexity-allowed and " + "exclude ARE honoured, so a real budget or a real exclusion needs no " + "workaround here", + {"config": name, "keys": defeating}, + ) + return name + return None + + +def measurement_argv(tool: Path, source_root: Path, *, offenders_only: bool) -> list[str]: + """The write-free measurement command — the root of the fix, not a nicety. + + Six tokens plus an optional ``--failed``, and the ABSENCE of a threshold flag + is as load-bearing as the presence of ``--snapshot-ignore``: the kit never + re-declares a budget complexipy already resolves from its own default / CLI / + ``[tool.complexipy]``, because a second authority here would flag a + consumer's baselined band as a new offender. ``--failed`` narrows the census + to the functions the tool's OWN resolved threshold calls offenders. + """ + argv = [str(tool), str(source_root), "--plain", "--color", "no", "--snapshot-ignore"] + if offenders_only: + argv.append("--failed") + return argv + + +def _census_row(line: str) -> tuple[FunctionKey, int] | None: + """`` `` -> the keyed measurement, else None. + + Split from the RIGHT: the complexity and the function name are single tokens + while a path may contain spaces, so ``rsplit`` is the only safe direction. + """ + parts = line.strip().rsplit(maxsplit=2) + if len(parts) != 3 or not parts[0].endswith(".py"): + return None + try: + complexity = int(parts[2]) + except ValueError: + return None + return (parts[0], parts[1]), complexity + + +def parse_census(stdout: str, context: Mapping[str, object] | None = None) -> Census: + """complexipy ``--plain`` stdout -> the measured census. + + ``--plain`` is complexipy's documented scripting form: one + `` `` line per function it measured, over + threshold or NOT — ``utils/output.py:234`` drops a row only when + ``failed_only`` is set and ``:243`` appends every other one. That is why a + function which merely fell below the bar is still HERE, and why its absence + from a measured file means something else entirely (the ratchet's + per-function surface audit reasons on exactly that distinction). + + A non-blank line that is not a census row REFUSES instead of being dropped: + an unreadable census and an empty one look identical from a count, and the + empty one is the exact world this gate exists to catch. + """ + functions: dict[FunctionKey, int] = {} + unreadable: list[str] = [] + rows = 0 + for line in stdout.splitlines(): + row = _census_row(line) + if row is not None: + rows += 1 + functions[row[0]] = max(functions.get(row[0], row[1]), row[1]) + elif line.strip(): + unreadable.append(line.strip()) + if unreadable: + raise refusal( + "GATE_COMPLEXIPY_OUTPUT_UNREADABLE", + f"complexipy --plain emitted {len(unreadable)} line(s) that are not " + "' ' — the census cannot be trusted, so the " + "floor cannot be graded; check the repo root for a complexipy.toml or a " + "[tool.complexipy] table adding output to the run", + {**dict(context or {}), "lines": unreadable[:10], "measured_functions": rows}, + ) + return Census(functions=functions, rows=rows) + + +def _excerpt(text: str) -> list[str]: + """The leading non-blank lines of a stream — evidence, never the whole tail.""" + return [line.strip() for line in text.splitlines() if line.strip()][:5] + + +def _refuse_unmeasurable( + proc: subprocess.CompletedProcess[str], context: Mapping[str, object] +) -> None: + """complexipy's own words for a path it could not analyze — a narrower surface. + + Kept a REFUSAL rather than a finding, deliberately and now declared: an + unparseable file means the graded surface is narrower than the tree, so every + OTHER function's clean reading is unsupported. A findings-level report would + let the rest of the board read green beside a void. Both streams are scanned, + because which one carries it depends on rich's console wiring. + """ + marked = [ + line.strip() + for line in f"{proc.stdout}\n{proc.stderr}".splitlines() + if _UNMEASURABLE_MARKER in line + ] + if not marked: + return + raise refusal( + "GATE_COMPLEXIPY_PATHS_UNMEASURABLE", + f"complexipy could not analyze {len(marked)} path(s) — the graded surface is " + "narrower than the tree, so a clean verdict would be a void; fix the named " + "file(s) (a syntax error also reds ruff and mypy) and re-run the gate", + {**dict(context), "paths": marked[:10]}, + ) + + +def _refuse_instrument_failure( + proc: subprocess.CompletedProcess[str], context: Mapping[str, object] +) -> None: + """Refuse when the exit code contradicts the census, rather than blame the repo. + + Exit semantics taken from the pinned source, not from confidence (see this + module's docstring for the reduction): 0 and 1 are the tool's only verdicts, + 1 means "some function is over threshold", and that cannot be true of a run + which printed no function at all. That second clause is also the OFFENDER + run's vacuity leg — an empty ``--failed`` census beside a non-zero exit is a + contradiction — which is why one rule serves both runs. Any other code is + ``typer`` declining to run: ``BadParameter`` exits 2 (``main.py:748``), as + does ``validate_ratchet`` (``main.py:726-729``). + """ + if proc.returncode == 0: + return + empty = not proc.stdout.strip() + if proc.returncode == 1 and not empty: + return + detail = ( + "exit 1 means 'some function is over threshold', which no run that printed " + "zero functions can be reporting" + if proc.returncode == 1 + else "0 and 1 are the tool's only verdicts; anything else is it declining to run" + ) + raise refusal( + "GATE_COMPLEXIPY_INSTRUMENT_FAILED", + f"complexipy exited {proc.returncode} with " + f"{'an empty' if empty else 'a non-empty'} census — {detail}. This is the " + "instrument failing, not the repo, so the gate refuses instead of charging " + "it to the code; read `stderr` in this context, then check the repo root for " + "a complexipy.toml or a [tool.complexipy] table", + {**dict(context), "stdout_head": _excerpt(proc.stdout)}, + ) + + +def _measure_env(env: Mapping[str, str]) -> dict[str, str]: + """The caller's environment with the census pins ON and the tty forcers OFF.""" + kept = {key: value for key, value in env.items() if key not in _MEASURE_ENV_CLEARED} + return {**kept, **_MEASURE_ENV} + + +def measure( + root: Path, + source_root: Path, + env: Mapping[str, str], + tool: Path, + executor: Executor, + *, + offenders_only: bool, +) -> Census: + """Run one write-free measurement from the repo root and read its census. + + cwd is the repo root deliberately: complexipy resolves its config + (``main.py:66-67``), the snapshot path (``main.py:321``) and every reported + path (``main.py:308-310``) against ``os.getcwd()``, so measuring from + anywhere else would re-key the census and break the comparison. All three + channels are consulted, and the exit code plus a stderr excerpt ride EVERY + refusal raised here — a tool-side failure that reaches the board with its + reason deleted is a failure re-attributed to the repo. + + ``UnicodeDecodeError`` is caught at the seam because ``text=True`` decodes in + the PARENT, by the parent's locale — ``PYTHONIOENCODING`` pins the child + only. That exception is a ``ValueError``, so it would otherwise escape the + executor, escape ``gate_runner._run_stage`` (which catches ``GateError``) and + lose the whole 12-stage board to a traceback. + """ + argv = measurement_argv(tool, source_root, offenders_only=offenders_only) + try: + proc = executor(argv, root, _measure_env(env)) + except UnicodeDecodeError as exc: + raise refusal( + "GATE_COMPLEXIPY_OUTPUT_UNDECODABLE", + f"complexipy's output does not decode in this locale: {exc} — the census " + "is the only instrument the ratchet has, so the gate refuses rather than " + "grade a truncated world", + {"argv": argv, "offenders_only": offenders_only, "reason": str(exc)}, + ) from exc + context: dict[str, object] = { + "argv": argv, + "offenders_only": offenders_only, + "exit_code": proc.returncode, + "stderr": _excerpt(proc.stderr), + } + _refuse_unmeasurable(proc, context) + _refuse_instrument_failure(proc, context) + return parse_census(proc.stdout, context) diff --git a/src/cf_quality/complexipy_ratchet.py b/src/cf_quality/complexipy_ratchet.py index 8d278d5..514aa1c 100644 --- a/src/cf_quality/complexipy_ratchet.py +++ b/src/cf_quality/complexipy_ratchet.py @@ -3,7 +3,7 @@ **The measured defect.** ``complexipy-snapshot.json`` is the committed floor: the per-function cognitive-complexity watermark no later run may exceed. In the pinned ``complexipy==5.6.0`` the tool's OWN snapshot comparison ends, on success, -in a REWRITE of that file — ``complexipy/utils/snapshot.py`` +in a REWRITE of that file — ``complexipy/utils/snapshot.py:85`` ``handle_snapshot_watermark`` returns ``True`` only after calling ``create_snapshot_file(...)`` with the functions IT measured this run. The tool's green path *is* the destructive path, and two ordinary runs reproduce it at exit @@ -20,33 +20,46 @@ compare — a before/after diff of the file is identical and the narrowed surface goes invisible again. So the fix sits upstream of the write: -1. **The gate never lets the tool near the artifact.** Both measurement - invocations carry ``--snapshot-ignore``. In the pinned source the ONLY two - callers of ``create_snapshot_file`` are ``--snapshot-create`` (never passed) - and the watermark compare's success path (which ``--snapshot-ignore`` switches - off by making ``should_run_snapshot_watermark`` False). The committed floor is - therefore READ by this module and written by nobody. +1. **The gate never lets the tool near the artifact.** The write-free invocation + and the audit that makes it genuinely write-free live in + :mod:`cf_quality.complexipy_measure`; its docstring carries the two + ``create_snapshot_file`` call sites and the config key that would re-open the + second one. The floor itself has exactly one reader, + :mod:`cf_quality.complexipy_floor`, which never writes. 2. **The ratchet is our pure function.** :func:`grade` maps (committed floor, measured census) to a :class:`~cf_quality.errors.GateVerdict` with no - subprocess, no write and no tool exit code in the path — unit-testable, and - nothing a green run can silently destroy. -3. **complexipy is demoted to a measuring instrument.** It answers two questions - and grades nothing: ``--plain`` (every function measured, with its complexity) + subprocess and no write in the path — unit-testable, and nothing a green run + can silently destroy. +3. **complexipy is demoted to an instrument.** It answers two questions and + grades nothing: ``--plain`` (every function measured, with its complexity) and ``--plain --failed`` (the subset ITS OWN threshold calls offenders). The second run is why the kit never re-declares a budget complexipy already - resolves from its default / CLI / ``[tool.complexipy]`` config — a second - threshold authority here would flag a consumer's baselined band as new. + resolves from its default / CLI / ``[tool.complexipy]`` config. + +**Threshold-neutrality, and the vacuity that hid inside it.** Because the kit +declares no budget, a consumer who RAISES complexipy's threshold empties the +offender set — and the first cut of this fix then graded an empty dict against a +populated floor and reported clean. That is +``gate-that-selects-by-the-value-it-guards-goes-vacuous`` inside the fix for it. +:func:`_bar_moved` closes it without naming a threshold: a floor entry proves +that function was ABOVE the bar when the floor was booted, so if its census value +still sits at-or-above its watermark and yet the offender run does not report it, +then ``threshold_now >= census >= watermark > threshold_at_boot`` — the bar moved, +provable from the census and the floor alone. The offender run's exit code is the +second, independent leg (``complexipy_measure._refuse_instrument_failure``). **The taxonomy.** Findings (exit 1, the repo's to fix): ``COMPLEXIPY_NEW_OFFENDER`` and ``COMPLEXIPY_WATERMARK_REGRESSION`` (complexipy's own watermark rule over data -we own), ``COMPLEXIPY_SURFACE_NARROWED`` and ``COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED`` -(the floor names a file this run did not grade — the narrowed-surface trigger, -caught structurally, independent of any threshold). Refusals (exit 2, the gate -could not do its job): ``GATE_COMPLEXIPY_SNAPSHOT_UNREADABLE``, -``GATE_COMPLEXIPY_PATHS_UNMEASURABLE``, ``GATE_COMPLEXIPY_OUTPUT_UNREADABLE``, -``GATE_COMPLEXIPY_MEASUREMENT_SKEW``, ``GATE_COMPLEXIPY_MEASURED_NOTHING``. A -legitimate improvement — a function that got simpler, a deleted file — is GREEN; -only re-booting the floor locks it in, which stays the existing runbook duty. +we own); ``COMPLEXIPY_SURFACE_NARROWED``, ``COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED`` +and ``COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED`` (the floor names a file, or a +function inside a measured file, this run did not grade — the narrowed-surface +trigger, caught structurally, independent of any threshold). Refusals (exit 2, +the gate could not do its job): ``GATE_COMPLEXIPY_SNAPSHOT_UNREADABLE``, +``GATE_COMPLEXIPY_MEASUREMENT_SKEW``, ``GATE_COMPLEXIPY_THRESHOLD_RAISED``, +``GATE_COMPLEXIPY_MEASURED_NOTHING``, plus the instrument-side refusals +:mod:`cf_quality.complexipy_measure` raises. A legitimate improvement — a +function that got simpler, a deleted file — is GREEN; only re-booting the floor +locks it in, which stays the existing runbook duty. The absent-watermark doctrine is unchanged and stays in the caller (``gate_runner._complexipy``): no snapshot + Python present REFUSES @@ -55,236 +68,64 @@ from __future__ import annotations -import ast -import json -import subprocess -import sys from collections.abc import Mapping -from dataclasses import dataclass from pathlib import Path from typing import Protocol +from cf_quality.complexipy_floor import SNAPSHOT_FILENAME, read_snapshot +from cf_quality.complexipy_measure import ( + Census, + Executor, + FunctionKey, + audit_config, + measure, + refusal, +) from cf_quality.errors import GateError, GateVerdict, GateViolation #: The stage name every verdict from this module carries. GATE = "complexipy" -#: The committed floor's filename (CWD-relative for complexipy, repo root for us). -SNAPSHOT_FILENAME = "complexipy-snapshot.json" -#: ``(repo-relative file, function name)`` — the identity a watermark is keyed by. -#: complexipy keys on ``(path, file_name, name)``; ``path`` already carries the -#: file name in its own output, so the joined form is the same identity. -FunctionKey = tuple[str, str] +class _Layout(Protocol): + """The resolved consumer layout this stage reads (``gate_runner.Layout``). -#: Environment pinned onto every measurement run. ``COLUMNS`` is load-bearing: -#: ``--plain`` prints through rich, which wraps at 80 columns when stdout is not -#: a terminal, and a wrapped census row is an unparseable census row. -#: ``PYTHONIOENCODING`` pins UTF-8 so the census does not decode by locale. -_MEASURE_ENV = {"COLUMNS": "10000", "PYTHONIOENCODING": "utf-8", "NO_COLOR": "1"} - -#: complexipy's own words when it could not analyze a path it was handed -#: (``complexipy.utils.output.print_invalid_paths``) — a silently narrower surface. -_UNMEASURABLE_MARKER = "Failed to process" - - -class _Executor(Protocol): - """The subprocess seam the caller injects (``gate_runner._exec``). - - Structural, not inherited: the runner stays the caller's — one place owns - typed OSError translation and the no-shell fixed-argv discipline — while the - tests keep patching that single seam. + Structural, and read-only by declaration so a frozen dataclass satisfies it: + importing ``gate_runner.Layout`` would close an import cycle, since + ``gate_runner`` imports this module. ``py_present`` is threaded rather than + re-derived on purpose — the caller already computes the repo-wide answer for + the absent-snapshot doctrine, and a second local derivation is how the two + surfaces came to disagree (a Python-free tree under a present-but-empty + ``src/`` was reported PASS having measured nothing). """ - def __call__( - self, - argv: list[str], - cwd: Path, - env: Mapping[str, str], - *, - stdin: str | None = None, - ) -> subprocess.CompletedProcess[str]: ... - - -@dataclass(frozen=True) -class Census: - """What ONE write-free complexipy run actually measured. - - The census is the gate's independent fact about its own measurement surface: - it is read from the tool's output, never from the snapshot, so "the floor is - empty" and "we graded nothing" can never be the same observation. - """ - - functions: dict[FunctionKey, int] - @property - def files(self) -> frozenset[str]: - """The distinct files this run measured — the surface audit's evidence.""" - return frozenset(path for path, _ in self.functions) - - -def _gate_error(code: str, message: str, context: dict[str, object]) -> GateError: - """A refusal in the kit's typed vocabulary — the gate could not do its job.""" - return GateError(code=code, message=message, context=context) - - -def _normalized_path(path: str, file_name: str) -> str: - """Join a snapshot entry's two path fields the way complexipy's output does. - - Declared mirror of ``complexipy.utils.output.normalize_path`` (pinned 5.6.0): - the snapshot stores ``path`` and ``file_name`` separately while ``--plain`` - prints the joined form. Join them differently and every committed watermark - looks like a brand-new offender. - """ - cleaned = path.rstrip("/") - if cleaned.endswith(file_name): - return cleaned - return f"{cleaned}/{file_name}" if cleaned else file_name - - -def _refuse_snapshot(snapshot: Path, reason: str) -> GateError: - return _gate_error( - "GATE_COMPLEXIPY_SNAPSHOT_UNREADABLE", - f"{SNAPSHOT_FILENAME} is not a readable complexipy snapshot: {reason} — " - "re-boot it (complexipy --snapshot-create); an unreadable " - "floor is not an empty floor", - {"snapshot": str(snapshot), "reason": reason}, - ) - - -def _entry_watermarks(entry: object, snapshot: Path) -> dict[FunctionKey, int]: - """One snapshot entry's watermarks; any other shape REFUSES rather than skips.""" - if not isinstance(entry, dict) or not isinstance(entry.get("functions"), list): - raise _refuse_snapshot(snapshot, f"entry is not {{path, file_name, functions}}: {entry!r}") - path = _normalized_path(str(entry.get("path", "")), str(entry.get("file_name", ""))) - watermarks: dict[FunctionKey, int] = {} - for function in entry["functions"]: - if not isinstance(function, dict) or not isinstance(function.get("complexity"), int): - raise _refuse_snapshot(snapshot, f"function is not {{name, complexity}}: {function!r}") - watermarks[(path, str(function.get("name", "")))] = int(function["complexity"]) - return watermarks - - -def read_snapshot(snapshot: Path) -> dict[FunctionKey, int]: - """The committed floor as ``{(file, function): watermark}``. - - This is the ONLY code that touches the artifact, and it only reads. A - malformed snapshot REFUSES: treating it as an empty floor would be - green-by-unreadable-file, the same gaming vector as green-by-missing-file - (which this gate already refuses). - """ - try: - raw = json.loads(snapshot.read_text(encoding="utf-8")) - except (OSError, json.JSONDecodeError) as exc: - raise _refuse_snapshot(snapshot, str(exc)) from exc - if not isinstance(raw, list): - raise _refuse_snapshot(snapshot, f"top level is {type(raw).__name__}, not a list") - floor: dict[FunctionKey, int] = {} - for entry in raw: - floor.update(_entry_watermarks(entry, snapshot)) - return floor - - -def _census_row(line: str) -> tuple[FunctionKey, int] | None: - """`` `` -> the keyed measurement, else None. - - Split from the RIGHT: the complexity and the function name are single tokens - while a path may contain spaces, so ``rsplit`` is the only safe direction. - """ - parts = line.strip().rsplit(maxsplit=2) - if len(parts) != 3 or not parts[0].endswith(".py"): - return None - try: - complexity = int(parts[2]) - except ValueError: - return None - return (parts[0], parts[1]), complexity - - -def parse_census(stdout: str) -> Census: - """complexipy ``--plain`` stdout -> the measured census. - - ``--plain`` is complexipy's documented scripting form: one - `` `` line per function it measured, over - threshold or not. A non-blank line that is not a census row REFUSES instead - of being dropped — an unreadable census and an empty one look identical from - a count, and the empty one is the exact world this gate exists to catch. - """ - functions: dict[FunctionKey, int] = {} - unreadable: list[str] = [] - for line in stdout.splitlines(): - row = _census_row(line) - if row is not None: - functions[row[0]] = row[1] - elif line.strip(): - unreadable.append(line.strip()) - if unreadable: - raise _gate_error( - "GATE_COMPLEXIPY_OUTPUT_UNREADABLE", - f"complexipy --plain emitted {len(unreadable)} line(s) that are not " - "' ' — the census cannot be trusted, so " - "the floor cannot be graded", - {"lines": unreadable[:10], "measured_functions": len(functions)}, - ) - return Census(functions=functions) - - -def measurement_argv(tool: Path, source_root: Path, *, offenders_only: bool) -> list[str]: - """The write-free measurement command — the root of the fix, not a nicety. - - ``--snapshot-ignore`` makes ``should_run_snapshot_watermark`` False in the - pinned tool, which is the only path (besides the never-passed - ``--snapshot-create``) that reaches ``create_snapshot_file``. This argv - therefore CANNOT write ``complexipy-snapshot.json``. ``--failed`` narrows the - census to the functions complexipy's own resolved threshold calls offenders, - so the kit never states a complexity budget of its own here. - """ - argv = [str(tool), str(source_root), "--plain", "--color", "no", "--snapshot-ignore"] - if offenders_only: - argv.append("--failed") - return argv + def root(self) -> Path: ... + @property + def source_root(self) -> Path: ... + @property + def py_present(self) -> bool: ... -def _measure( - root: Path, - source_root: Path, - env: Mapping[str, str], - tool: Path, - executor: _Executor, - *, - offenders_only: bool, -) -> Census: - """Run one write-free measurement from the repo root and read its census. - - cwd is the repo root deliberately: complexipy resolves both its config and - its reported paths against the invocation directory, so measuring from - anywhere else would re-key every path and break the comparison. The exit - code is NOT consulted — with the compare switched off it merely restates - "some function is over threshold", which is the normal state of a repo - carrying a baselined floor. - """ - argv = measurement_argv(tool, source_root, offenders_only=offenders_only) - proc = executor(argv, root, {**env, **_MEASURE_ENV}) - lines = [line.strip() for line in proc.stdout.splitlines()] - unmeasurable = [line for line in lines if _UNMEASURABLE_MARKER in line] - if unmeasurable: - raise _gate_error( - "GATE_COMPLEXIPY_PATHS_UNMEASURABLE", - f"complexipy could not analyze {len(unmeasurable)} path(s) — the graded " - "surface is narrower than the tree, so a clean verdict would be a void", - {"paths": unmeasurable[:10], "exit_code": proc.returncode}, - ) - return parse_census(proc.stdout) +def _reboot(graded: str) -> str: + """The re-boot command, named as the remedy — a condition with no remedy is half a message.""" + return f"re-boot the floor from the repo root: complexipy {graded} --snapshot-create" def _counts( floor: Mapping[FunctionKey, int], census: Census, offenders: Census ) -> dict[str, object]: - """The measured tally every finding and refusal carries — never a bare verdict.""" + """The measured tally every finding, refusal and PASS carries — never a bare verdict. + + ``measured_functions`` is the ROW count, not the key count: they differ + wherever a ``(path, name)`` pair repeats, and reporting keys would under-state + the very measurement this evidence exists to prove. + """ return { - "measured_functions": len(census.functions), + "measured_functions": census.rows, + "measured_keys": len(census.functions), "measured_files": len(census.files), - "measured_offenders": len(offenders.functions), + "measured_offenders": offenders.rows, "snapshot_functions": len(floor), "snapshot_files": len({path for path, _ in floor}), } @@ -297,7 +138,10 @@ def _violation( def _regressions( - floor: Mapping[FunctionKey, int], offenders: Census, counts: Mapping[str, object] + floor: Mapping[FunctionKey, int], + offenders: Census, + counts: Mapping[str, object], + reboot: str, ) -> list[GateViolation]: """complexipy's own watermark rule, applied to data the tool cannot rewrite. @@ -309,87 +153,142 @@ def _regressions( watermark = floor.get((path, name)) if watermark is None: code = "COMPLEXIPY_NEW_OFFENDER" - message = f"{name} exceeds complexipy's threshold at {value}, no committed watermark" + message = ( + f"{name} exceeds complexipy's threshold at {value} with no committed " + f"watermark — split it below the threshold, or baseline it deliberately: {reboot}" + ) elif value > watermark: code = "COMPLEXIPY_WATERMARK_REGRESSION" - message = f"{name} rose above its committed watermark: {watermark} -> {value}" + message = ( + f"{name} rose above its committed watermark: {watermark} -> {value} — bring " + f"it back to {watermark} or below; raising the floor instead is a deliberate, " + f"reviewed act: {reboot}" + ) else: continue violations.append(_violation(code, message, path, counts, function=name, measured=value)) return violations +def _graded_prefix(root: Path, source_root: Path) -> str | None: + """The graded root as a repo-relative POSIX prefix (``""`` == the whole repo). + + The ROOTS are resolved, never the file: resolving the file sent a symlinked + module (``src/settings.py -> ../config/settings.py``) outside the graded root + and made the gate accuse a file complexipy really did measure of lying outside + the surface. The floor's own paths are already repo-relative, so containment + is a lexical test on those. ``None`` means the graded root is not inside the + repo at all, and the truthful reading of that is "every floor path is outside + it" — not a crash, and not a pass. + """ + try: + relative = source_root.resolve().relative_to(root.resolve()) + except ValueError: + return None + text = relative.as_posix() + return "" if text == "." else text + + +def _inside(path: str, prefix: str | None) -> bool: + """Is this repo-relative path inside the graded prefix? Lexical, by design.""" + if prefix is None: + return False + return prefix == "" or path == prefix or path.startswith(f"{prefix}/") + + +def _unmeasured_functions( + path: str, + floor: Mapping[FunctionKey, int], + census: Census, + counts: Mapping[str, object], + reboot: str, +) -> list[GateViolation]: + """Floor functions missing from a file this run DID measure. + + The distinction that makes this honest: ``--plain`` without ``--failed`` lists + every measured function regardless of threshold, so a function that merely got + SIMPLER is still in the census. Absence from a measured file therefore means + it was renamed, deleted, or dropped by a ``# complexipy: ignore`` comment — + and that last one is an unregistered, unblessed exemption from the complexity + gate, invisible to ``cf-exemptions`` (whose noqa pattern needs a coded id) and + invisible to a per-FILE audit, because the file is still in ``census.files``. + """ + return [ + _violation( + "COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED", + f"{name} carries a committed watermark of {watermark} but was not measured, " + f"while {path} WAS — a function that merely got simpler would still be in the " + f"census, so this was renamed, deleted, or dropped by a '# complexipy: ignore' " + f"comment; remove the ignore comment, or {reboot}", + path, + counts, + function=name, + watermark=watermark, + ) + for (entry_path, name), watermark in sorted(floor.items()) + if entry_path == path and (entry_path, name) not in census.functions + ] + + def _surface_violations( - root: Path, - source_root: Path, + layout: _Layout, floor: Mapping[FunctionKey, int], census: Census, counts: Mapping[str, object], + reboot: str, ) -> list[GateViolation]: - """Every floor file that still EXISTS must have been measured this run. + """Every floor entry that still EXISTS must have been measured this run. The narrowed-surface trigger caught head-on, independent of any threshold: a - snapshot entry is proof that file HELD an over-threshold function, so a run - that produced no measurement for it graded a smaller world than the floor - describes. A file that is GONE is a legitimate improvement and is passed over; - a file whose functions all vanished reads the same way and asks for the same - remedy — re-boot the floor, deliberately, so the improvement is locked in. + snapshot entry is proof that function HELD an over-threshold complexity, so a + run that produced no measurement for it graded a smaller world than the floor + describes. A file that is GONE is a legitimate improvement and is passed over. + Per-FILE where the whole file went unmeasured, then per-FUNCTION inside the + files that were measured — the second half is what a ``# complexipy: ignore`` + comment used to slip through. """ violations: list[GateViolation] = [] - graded = source_root.resolve() # both sides resolved, or a symlinked tmp lies + prefix = _graded_prefix(layout.root, layout.source_root) for path in sorted({path for path, _ in floor}): - on_disk = root / path - if not on_disk.is_file(): + if not (layout.root / path).is_file(): continue - if not on_disk.resolve().is_relative_to(graded): - code = "COMPLEXIPY_SURFACE_NARROWED" - message = f"the floor covers {path}, which lies OUTSIDE the graded source root" + if not _inside(path, prefix): + violations.append( + _violation( + "COMPLEXIPY_SURFACE_NARROWED", + f"the floor covers {path}, which lies OUTSIDE the graded source root " + f"({prefix or '.'}) — widen the declared source_root, or {reboot}", + path, + counts, + source_root=str(layout.source_root), + ) + ) elif path not in census.files: - code = "COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED" - message = f"{path} carries a committed watermark but no function in it was measured" + violations.append( + _violation( + "COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED", + f"{path} carries a committed watermark but NO function in it was " + f"measured — it is excluded, ignore-commented, or now functionless; " + f"restore it to the measured surface, or {reboot}", + path, + counts, + source_root=str(layout.source_root), + ) + ) else: - continue - violations.append(_violation(code, message, path, counts, source_root=str(source_root))) + violations.extend(_unmeasured_functions(path, floor, census, counts, reboot)) return violations -def _module_defines_functions(path: Path) -> bool: - """True when a module contains any ``def``/``async def``, by AST not by regex. - - Source we cannot read or parse counts as YES: a void must never certify - itself, and we cannot prove a file is functionless from bytes we never - parsed (complexipy would report such a path as unmeasurable anyway). - """ - try: - tree = ast.parse(path.read_text(encoding="utf-8")) - except (OSError, SyntaxError, ValueError): - return True - return any(isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef) for node in ast.walk(tree)) - - -def _defines_functions(source_root: Path) -> bool: - """True when the graded tree defines at least one function. - - Consulted only when the census came back EMPTY, so the common path never - parses anything. Dotted directories are skipped, mirroring the workflow's - own ``find . -not -path '*/.*'`` measurement surface. - """ - for path in sorted(source_root.rglob("*.py")): - parts = path.relative_to(source_root).parts - if any(part.startswith(".") for part in parts): - continue - if _module_defines_functions(path): - return True - return False - - def _skew(census: Census, offenders: Census, counts: Mapping[str, object]) -> GateError | None: """The two write-free runs must describe ONE world. The offender set is a filter of the census, so every offender must appear in the census at the same complexity. A disagreement means the tree changed between the runs, or a flag moved the measurement surface — either way the - comparison inputs are not a single observation and must not be graded. + comparison inputs are not a single observation and must not be graded. Both + sides aggregate ``max()`` per key, so a repeated ``(path, name)`` pair can no + longer fake a skew by landing its high value in only one of the two maps. """ disagreements = sorted( f"{path}:{name}" @@ -398,91 +297,150 @@ def _skew(census: Census, offenders: Census, counts: Mapping[str, object]) -> Ga ) if not disagreements: return None - return _gate_error( + return refusal( "GATE_COMPLEXIPY_MEASUREMENT_SKEW", f"{len(disagreements)} function(s) reported by the offender run are absent from " - "(or disagree with) the census run — the two measurements are not one world", + "(or disagree with) the census run — the two measurements are not one world, so " + "the gate refuses to compare them; re-run on a quiescent tree, and if it repeats, " + "look for a complexipy.toml moving the surface between the two runs", {**counts, "functions": disagreements[:10]}, ) -def _vacuity( +def _bar_moved( + floor: Mapping[FunctionKey, int], census: Census, + offenders: Census, + counts: Mapping[str, object], + reboot: str, +) -> GateError | None: + """The offender census's vacuity leg — threshold-free, from data already in hand. + + A floor entry is proof that function was ABOVE complexipy's threshold when the + floor was booted (``create_snapshot_file`` stores only over-threshold + functions). So if the census still measures it at-or-above its watermark and + the offender run does NOT report it, then + ``threshold_now >= census >= watermark > threshold_at_boot``: the bar moved + since the floor was booted, and every regression rule downstream is grading a + set the raised bar emptied. Refusing needs no second threshold authority — the + kit still declares no budget of its own, which is the property that made this + hole possible and is worth keeping. + + A hand-lowered watermark lands here too, correctly: complexipy would never + have written a watermark at or below its own threshold, so such an entry is + itself evidence the floor was not produced by the boot command. + """ + stranded = sorted( + f"{path}:{name} (measured {census.functions[(path, name)]} >= watermark {watermark})" + for (path, name), watermark in floor.items() + if (path, name) in census.functions + and census.functions[(path, name)] >= watermark + and (path, name) not in offenders.functions + ) + if not stranded: + return None + return refusal( + "GATE_COMPLEXIPY_THRESHOLD_RAISED", + f"{len(stranded)} committed watermark(s) are at-or-below what complexipy now " + "calls acceptable, so its offender set no longer covers the floor — the ratchet " + "would grade nothing and report clean. complexipy's threshold has been raised " + "since the floor was booted (or a watermark was hand-lowered): restore " + f"max-complexity-allowed, or {reboot} at the threshold you intend", + {**counts, "functions": stranded[:10]}, + ) + + +def _vacuity( + layout: _Layout, floor: Mapping[FunctionKey, int], - source_root: Path, + census: Census, counts: Mapping[str, object], + reboot: str, ) -> GateError | None: """A run that measured nothing can never report clean. Two independent legs, so the refusal survives an ALREADY-emptied floor: a - committed floor that still names functions, or a graded tree that - demonstrably defines functions. When both are empty there is genuinely - nothing to grade and clean is the honest answer, not a void. + committed floor that still names functions, or a repo that contains Python at + all. The second leg is the CALLER's repo-wide answer (``layout.py_present``), + the same one the absent-snapshot doctrine rides — not a local walk of + ``source_root``, which was blind exactly when ``source_root`` was the wrong + tree (code in ``app/`` beside an empty-but-present ``src/``, floor ``[]``: + zero functions measured, PASS). Consulting one answer is what stops the two + surfaces from diverging; it is also strictly stricter, since a repo whose only + Python defines no functions now refuses instead of certifying a void. """ - if census.functions: + if census.rows: return None - if not floor and not _defines_functions(source_root): + if not floor and not layout.py_present: return None - return _gate_error( + return refusal( "GATE_COMPLEXIPY_MEASURED_NOTHING", - "complexipy measured zero functions while there was something to grade — a " - "gate that measured nothing cannot report a clean floor (check the resolved " - f"source root {source_root} and any complexipy exclude/ignore configuration)", - {**counts, "source_root": str(source_root)}, + "complexipy measured zero functions while there was something to grade — a gate " + "that measured nothing cannot report a clean floor. Check the resolved source " + f"root ({layout.source_root}) actually holds the code, and any complexipy " + f"exclude/ignore configuration; if the tree really did move, {reboot}", + {**counts, "source_root": str(layout.source_root), "python_present": layout.py_present}, ) def grade( - root: Path, - source_root: Path, + layout: _Layout, floor: Mapping[FunctionKey, int], census: Census, offenders: Census, + *, + config: str | None = None, ) -> GateVerdict: """The whole ratchet as a pure function of (committed floor, measured world). - No subprocess, no write, no tool exit code — which is the property the - tool's own comparison cannot have, because its green path IS the rewrite. - Every finding and refusal carries the measured tally, so a verdict can never - be read without the count behind it. + No subprocess, no write — which is the property the tool's own comparison + cannot have, because its green path IS the rewrite. Every finding and refusal + carries the measured tally, and so does a PASS: the counts ride the verdict's + ``evidence`` and one notice rides its ``notices``, so a machine reading the + aggregated JSON can tell a clean grade from a vacuous one without re-running + anything. Refusal order is deliberate — a skewed pair must not be reasoned + over, and a raised bar must be named before the emptied offender set is + mistaken for a quiet one. """ counts = _counts(floor, census, offenders) - violations = _regressions(floor, offenders, counts) - violations.extend(_surface_violations(root, source_root, floor, census, counts)) - error = _skew(census, offenders, counts) or _vacuity(census, floor, source_root, counts) - return GateVerdict(gate=GATE, violations=violations, error=error) - - -def _report_measured(floor: Mapping[FunctionKey, int], census: Census) -> None: - """State the measured count out loud, on every run including a clean one. - - :class:`~cf_quality.errors.GateVerdict` carries no notices channel, so a - PASSING stage would otherwise report a floor it never proves it measured. - stderr keeps the aggregated JSON wire form on stdout untouched. - """ - print( - f"{GATE}: measured {len(census.functions)} function(s) in {len(census.files)} " - f"file(s) against a {len(floor)}-function committed floor", - file=sys.stderr, + reboot = _reboot(_graded_prefix(layout.root, layout.source_root) or ".") + violations = _regressions(floor, offenders, counts, reboot) + violations.extend(_surface_violations(layout, floor, census, counts, reboot)) + error = ( + _skew(census, offenders, counts) + or _bar_moved(floor, census, offenders, counts, reboot) + or _vacuity(layout, floor, census, counts, reboot) + ) + return GateVerdict( + gate=GATE, + violations=violations, + error=error, + notices=[ + f"— measured {census.rows} function(s) in {len(census.files)} file(s) against a " + f"{len(floor)}-function committed floor" + ], + evidence={**counts, "complexipy_config": config}, ) def complexipy_verdict( - root: Path, - source_root: Path, + layout: _Layout, env: Mapping[str, str], tool: Path, - executor: _Executor, + executor: Executor, ) -> GateVerdict: - """Measure the tree write-free, then grade the ratchet in our own code. - - The caller has already enforced the absent-watermark doctrine, so the floor - exists here. Two write-free runs (the full census, then complexipy's own - offender subset) feed :func:`grade`; a GateError from either measurement - propagates as the stage's refusal, which the battery records and continues. + """Audit the consumer's config, measure the tree write-free, then grade in our code. + + Order is load-bearing. The config audit runs FIRST because it is what makes + the two measurement runs write-free at all; grading the pre-write bytes of a + floor a later run rewrote is precisely the green-with-the-artifact-mutated + world the refuters found. The caller has already enforced the absent-watermark + doctrine, so the floor exists here. A GateError from the audit or either + measurement propagates as the stage's refusal, which the battery records and + continues past. """ - floor = read_snapshot(root / SNAPSHOT_FILENAME) - census = _measure(root, source_root, env, tool, executor, offenders_only=False) - offenders = _measure(root, source_root, env, tool, executor, offenders_only=True) - _report_measured(floor, census) - return grade(root, source_root, floor, census, offenders) + config = audit_config(layout.root) + floor = read_snapshot(layout.root / SNAPSHOT_FILENAME) + census = measure(layout.root, layout.source_root, env, tool, executor, offenders_only=False) + offenders = measure(layout.root, layout.source_root, env, tool, executor, offenders_only=True) + return grade(layout, floor, census, offenders, config=config) diff --git a/src/cf_quality/errors.py b/src/cf_quality/errors.py index de263b2..36e3e51 100644 --- a/src/cf_quality/errors.py +++ b/src/cf_quality/errors.py @@ -13,7 +13,9 @@ name, the violations it collected, and the GateError it died on (if any). Its :attr:`~GateVerdict.exit_code` and :attr:`~GateVerdict.passed` derive the kit-wide exit contract (0 clean · 1 violations · 2 the gate could not - run) from those parts, so every gate reports one structured verdict. + run) from those parts, so every gate reports one structured verdict. It also + carries the ``notices`` and ``evidence`` a PASS needs: a verdict with no + measurement behind it cannot be told from one that measured nothing. All three serialize via ``to_dict()`` so the wire form and the in-process form say the same thing. @@ -112,11 +114,22 @@ class GateVerdict: violations: Every finding the gate reported (empty when clean). error: The typed failure the gate raised when it could not run, or None when the gate completed (clean or with findings). + notices: Informational lines a reader needs even on a PASS (default + empty). Appended after the original fields, like GateViolation's + ``severity``/``fixable``, so existing construction is untouched. + evidence: The structured measurement behind the verdict (default + empty) — counts, resolved config, whatever proves the gate looked. + A PASS with no evidence is indistinguishable from a PASS that + measured nothing, and telling those apart is the whole job of a + ratchet; prose in ``notices`` cannot carry that to a machine, so + the numbers ride their own field. """ gate: str violations: list[GateViolation] error: GateError | None = None + notices: list[str] = field(default_factory=list) + evidence: dict[str, Any] = field(default_factory=dict) @property def passed(self) -> bool: @@ -138,4 +151,6 @@ def to_dict(self) -> dict[str, Any]: "exit_code": self.exit_code, "error": self.error.to_dict() if self.error is not None else None, "violations": [violation.to_dict() for violation in self.violations], + "notices": list(self.notices), + "evidence": dict(self.evidence), } diff --git a/src/cf_quality/gate_runner.py b/src/cf_quality/gate_runner.py index 9400857..30530f0 100644 --- a/src/cf_quality/gate_runner.py +++ b/src/cf_quality/gate_runner.py @@ -322,7 +322,7 @@ def _mypy(layout: Layout, env: Mapping[str, str]) -> GateVerdict | None: def _complexipy(layout: Layout, env: Mapping[str, str]) -> GateVerdict | None: - """complexipy WRITE-FREE (its own compare REWRITES the floor); see complexipy_ratchet.""" + """complexipy WRITE-FREE (its compare REWRITES the floor) — refuse/skip as mypy's.""" if not (layout.root / "complexipy-snapshot.json").is_file(): return _ratchet_skip_or_refuse( layout, @@ -330,7 +330,7 @@ def _complexipy(layout: Layout, env: Mapping[str, str]) -> GateVerdict | None: "complexipy-snapshot.json", "boot the snapshot (complexipy --snapshot-create), even when clean", ) - return complexipy_verdict(layout.root, layout.source_root, env, _tool("complexipy"), _exec) + return complexipy_verdict(layout, env, _tool("complexipy"), _exec) def _ratchet_skip_or_refuse( @@ -449,11 +449,11 @@ def _gate_detail(verdict: GateVerdict) -> list[str]: def _emit_human(agg: _Aggregate) -> None: - """One status line per gate, then full detail for EVERY failing gate.""" + """One status line per gate (with its notices — a PASS must show what it measured).""" failing = [verdict for verdict in agg.verdicts if not verdict.passed] for verdict in agg.verdicts: status = "PASS" if verdict.passed else "FAIL" - print(f"{status} {verdict.gate}") + print(f"{status} {verdict.gate}", *verdict.notices) if not failing: print(f"\n{RUNNER_GATE}: PASS — every gate is clean") return diff --git a/tests/test_complexipy_instrument.py b/tests/test_complexipy_instrument.py new file mode 100644 index 0000000..e45bd8e --- /dev/null +++ b/tests/test_complexipy_instrument.py @@ -0,0 +1,363 @@ +"""complexipy as an instrument — the preconditions that make its census trustworthy. + +``tests/test_complexipy_snapshot.py`` pins the RATCHET: given a floor and a census, +which verdict. This file pins everything that must be true before that census may +be believed at all, which is where two refuters found the first cut of the fix +still defeatable: + +- **the consumer's own complexipy config.** ``--snapshot-ignore`` disarms the + watermark compare's rewrite, but ``handle_snapshot_file_creation`` + (``main.py:323``) is a SEPARATE branch driven by ``snapshot-create``, which the + kit passes no CLI value for — so ``[tool.complexipy] snapshot-create = true`` + wins (``utils/toml.py:235-240``) and both measurement runs rewrite the floor + while the gate grades the pre-write bytes it already read. ``--snapshot-create`` + has no negating secondary name (``main.py:97-102``), so argv cannot override it: + the only honest mechanism is to READ that config and REFUSE. Two more config + keys defeat the run the same way — ``quiet`` (rejected beside ``--plain``, exit + 2, empty census) and ``output-format`` / the legacy ``output-*`` flags (a report + file written into the consumer's tree plus ``Results saved at ...`` and + ``Deprecated: ...`` printed onto stdout BEFORE the census); +- **the exit code and stderr**, which used to be decoration and a discard. Every + tool-side failure was therefore re-attributed to the repo with its reason + deleted; +- **the parse's own hazards** — a duplicated ``(path, name)`` key, a non-UTF-8 + stream the PARENT decodes, and rich's dumb-terminal short circuit that ignores + ``COLUMNS`` outright; +- **the key itself**: ``complexipy_floor._normalized_path`` is a line-for-line copy + of ``complexipy.utils.output.normalize_path``, and the kit ships no + ``MIRRORS.md`` for ``cf-mirror-check`` to gate. The rod here calls the INSTALLED + function, which is a stronger guard than a declaration nobody checks: a 5.7.0 + change to that join fails the suite instead of re-keying every watermark into a + ``NEW_OFFENDER``. +""" + +from __future__ import annotations + +import subprocess +from collections.abc import Mapping +from pathlib import Path + +import pytest +from complexipy.utils.output import normalize_path +from test_complexipy_snapshot import _census, _light, _measured, _snapshot, _stage_verdict +from test_gate_runner import _layout, _write + +from cf_quality import gate_runner +from cf_quality.complexipy_floor import _normalized_path +from cf_quality.complexipy_measure import audit_config, parse_census +from cf_quality.errors import GateError + + +def _config_verdict( + root: Path, monkeypatch: pytest.MonkeyPatch, config: str, body: str +) -> GateError: + """Mount a consumer complexipy config on an otherwise CLEAN repo, return the refusal.""" + _write(root, config, body) + _snapshot(root) + verdict = _stage_verdict(root, monkeypatch, _measured(_census(_light(root)), "")) + assert verdict.error is not None, "a healthy census must not rescue a defeating config" + assert verdict.exit_code == 2 + return verdict.error + + +# --- BLOCKER 1: a consumer config that defeats the measurement ---------------- + + +@pytest.mark.parametrize( + ("config", "body", "key"), + [ + # THE one that re-opens the write. `--snapshot-ignore` does not touch + # handle_snapshot_file_creation, and no CLI flag can negate this key. + ("complexipy.toml", "snapshot-create = true\n", "snapshot-create"), + ("pyproject.toml", "[tool.complexipy]\nsnapshot-create = true\n", "snapshot-create"), + (".complexipy.toml", "snapshot-create = true\n", "snapshot-create"), + # BadParameter beside --plain: exit 2, empty stdout. Un-audited, the gate + # reported GATE_COMPLEXIPY_MEASURED_NOTHING and blamed the repo. + ("complexipy.toml", "quiet = true\n", "quiet"), + # A report file written into the consumer's tree, and two unguarded + # console.print lines landing on stdout BEFORE the census. + ("complexipy.toml", 'output-format = ["json"]\n', "output-format"), + ("complexipy.toml", "output-json = true\n", "output-json"), + ("complexipy.toml", 'output = "reports/"\n', "output"), + # Would narrow the CENSUS run to offenders, blinding the surface audit and + # the per-function audit that reasons on "still present, merely simpler". + ("complexipy.toml", "failed = true\n", "failed"), + ("complexipy.toml", 'details = "low"\n', "details"), + # Decouples the exit code from the threshold verdict, which is the second + # vacuity leg's only witness. + ("complexipy.toml", "ignore-complexity = true\n", "ignore-complexity"), + # Prints ignore-comment locations onto the census stream. + ("complexipy.toml", "report-ignored = true\n", "report-ignored"), + # --ratchet with no --diff exits 2 before measuring anything. + ("complexipy.toml", "ratchet = true\n", "ratchet"), + ], +) +def test_consumer_config_that_defeats_the_measurement_is_refused( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, config: str, body: str, key: str +) -> None: + error = _config_verdict(tmp_path, monkeypatch, config, body) + + assert error.code == "GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT" + assert error.context["keys"] == [key], "the refusal names the KEY, not just the file" + assert error.context["config"] == config, "and the file it found it in" + assert key in error.message and config in error.message + + +@pytest.mark.parametrize( + ("config", "body"), + [ + # The consumer's own threshold and surface authorities. The kit declares NO + # budget of its own (measurement_argv carries no -mx), so refusing these + # would break the very neutrality that keeps a baselined band from reading + # as new — and Blocker 2's closure is threshold-free precisely so it can + # stay honoured. + ("complexipy.toml", "max-complexity-allowed = 25\n"), + ("complexipy.toml", 'exclude = ["vendor"]\n'), + # These only ever make the census LARGER, or reorder it (max() per key). + ("complexipy.toml", "no-ignore = true\n"), + ("complexipy.toml", "check-script = true\n"), + ("complexipy.toml", 'sort = "desc"\n'), + # Value-sensitive: a key at its harmless value is not a defeating key, and + # a false refusal costs a consumer a red board for nothing. + ("complexipy.toml", "snapshot-create = false\n"), + ("complexipy.toml", 'details = "high"\n'), + # pyproject.toml with no [tool.complexipy] is not a complexipy config. + ("pyproject.toml", "[tool.other]\nsnapshot-create = true\n"), + ], +) +def test_legitimate_consumer_config_is_honoured_not_refused( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, config: str, body: str +) -> None: + _write(tmp_path, config, body) + _snapshot(tmp_path) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(_census(_light(tmp_path)), "")) + + assert verdict.passed, f"{config} carrying {body!r} must not be refused" + + +def test_config_search_order_mirrors_the_tools_own(tmp_path: Path) -> None: + # utils/toml.py:135-148 stops at the FIRST file that yields a table, and for a + # non-pyproject name that includes an EMPTY document — so an empty + # complexipy.toml shadows a populated pyproject.toml. Audit a different file + # than the tool reads and the audit is theatre. + _write(tmp_path, "pyproject.toml", "[tool.complexipy]\nsnapshot-create = true\n") + _write(tmp_path, ".complexipy.toml", "") + + assert audit_config(tmp_path) == ".complexipy.toml", "the shadowing file is the one in force" + + _write(tmp_path, "complexipy.toml", "") + assert audit_config(tmp_path) == "complexipy.toml", "complexipy.toml outranks both" + + +def test_unreadable_complexipy_config_is_refused_not_ignored( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # An unparseable config is not an absent config: the tool would crash on it, + # and treating it as "no config" would leave the write-free claim unaudited. + error = _config_verdict(tmp_path, monkeypatch, "complexipy.toml", "snapshot-create = tru\n") + + assert error.code == "GATE_COMPLEXIPY_CONFIG_UNREADABLE" + + +def test_non_utf8_complexipy_config_refuses_instead_of_crashing_the_battery( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # UnicodeDecodeError is a ValueError, not a TOMLDecodeError — catching only the + # latter loses the whole 12-stage board to a traceback. + (tmp_path / "complexipy.toml").write_bytes(b'paths = "\xff\xfe"\n') + _snapshot(tmp_path) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(_census(_light(tmp_path)), "")) + + assert verdict.error is not None and verdict.error.code == "GATE_COMPLEXIPY_CONFIG_UNREADABLE" + + +def test_a_clean_repo_reports_which_config_was_in_force( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # A PASS must say what it measured through — "no complexipy config" and "a + # config we never looked at" are the same verdict otherwise. + _write(tmp_path, "complexipy.toml", "max-complexity-allowed = 25\n") + _snapshot(tmp_path) + + verdict = _stage_verdict(tmp_path, monkeypatch, _measured(_census(_light(tmp_path)), "")) + + assert verdict.passed + assert verdict.evidence["complexipy_config"] == "complexipy.toml" + + +# --- DEFECT 3 / BLOCKER 2(b): the exit code and stderr are evidence ----------- + + +def test_nonzero_exit_with_an_empty_census_refuses_and_carries_the_reason( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # Exit 1 means "some function is over threshold" (utils/output.py:44 + 234-241), + # which no run that printed zero functions can be reporting. This is also the + # OFFENDER run's vacuity leg — one rule, both runs. Before this the gate + # reported GATE_COMPLEXIPY_MEASURED_NOTHING and charged it to the repo, with + # complexipy's stderr thrown away. + _write(tmp_path, "src/heavy.py", "def heavy():\n return 1\n") + _snapshot(tmp_path) + responses = _measured("", "") + responses["complexipy"] = (1, "", "Traceback: the instrument fell over\n") + + verdict = _stage_verdict(tmp_path, monkeypatch, responses) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_INSTRUMENT_FAILED" + assert verdict.error.context["exit_code"] == 1 + assert verdict.error.context["stderr"] == ["Traceback: the instrument fell over"] + + +def test_an_exit_code_that_is_not_a_verdict_refuses( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # 0 and 1 are the tool's only verdicts; 2 is typer declining to run + # (main.py:748 BadParameter, main.py:726-729 validate_ratchet). A census that + # LOOKS parseable beside exit 2 is still a run that did not do its job. + _snapshot(tmp_path) + responses = _measured(_census(_light(tmp_path)), "") + responses["complexipy"] = (2, _census(_light(tmp_path)), "Error: --ratchet requires --diff\n") + + verdict = _stage_verdict(tmp_path, monkeypatch, responses) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_INSTRUMENT_FAILED" + assert verdict.error.context["exit_code"] == 2 + + +def test_the_offender_run_exit_code_is_cross_checked_too( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The census is healthy and the floor is met, so nothing else has anything to + # say: only the offender run's own contradiction is left to catch it. + _write(tmp_path, "src/heavy.py", "def heavy():\n return 1\n") + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + responses = _measured(_census(("src/heavy.py", "heavy", 33)), "") + responses["complexipy-offenders"] = (1, "", "") + + verdict = _stage_verdict(tmp_path, monkeypatch, responses) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_INSTRUMENT_FAILED" + assert verdict.error.context["offenders_only"] is True + + +def test_exit_one_with_a_real_census_is_the_legitimate_offenders_exist( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The control rod on the rod above: exit 1 is the NORMAL state of a repo + # carrying a baselined floor, so the triage must not turn it into a refusal. + _write(tmp_path, "src/heavy.py", "def heavy():\n return 1\n") + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + rows = _census(("src/heavy.py", "heavy", 33)) + responses = _measured(rows, rows) + responses["complexipy"] = (1, rows, "") + + verdict = _stage_verdict(tmp_path, monkeypatch, responses) + + assert verdict.passed, "a baselined floor at its watermark is green, exit 1 or not" + + +def test_output_that_does_not_decode_refuses_instead_of_crashing_the_battery( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # subprocess text=True decodes in the PARENT, by the PARENT's locale — + # PYTHONIOENCODING pins the child only. UnicodeDecodeError is a ValueError, so + # it escaped _exec (OSError only) and _run_stage (GateError only) and took the + # whole board down with a traceback. + _snapshot(tmp_path) + + def exploding_exec( + argv: list[str], cwd: Path, env: Mapping[str, str], *, stdin: str | None = None + ) -> subprocess.CompletedProcess[str]: + raise UnicodeDecodeError("utf-8", b"\xff", 0, 1, "invalid start byte") + + monkeypatch.setattr(gate_runner, "_tool", lambda name: Path("/fake") / name) + monkeypatch.setattr(gate_runner, "_exec", exploding_exec) + stage = gate_runner.Stage("complexipy", gate_runner._complexipy) + + verdict = gate_runner._run_stage(stage, _layout(tmp_path), {}) + + assert verdict is not None and verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_OUTPUT_UNDECODABLE" + assert verdict.exit_code == 2 + + +# --- DEFECT 10: duplicate keys, and the count that must not collapse ---------- + + +def test_duplicate_function_keys_aggregate_max_and_still_count_every_row() -> None: + # Live on a real consumer: 681 census rows collapse to 680 keys, because an + # @overload chain (or an `if sys.platform:` redefinition) repeats a + # (path, name) pair. Last-writer-wins with sort = "desc" would keep the LOW + # value here, dropping a regression complexipy's own compare — which iterates + # its LIST, not a map — would catch. max() is order-independent, which is why + # the fix is aggregation and NOT pinning --sort: a pinned flag would have to + # survive in the argv for the rule to hold. + census = parse_census("src/m.py f 40\nsrc/m.py f 4\nsrc/m.py g 7\n") + + assert census.functions == {("src/m.py", "f"): 40, ("src/m.py", "g"): 7} + assert census.rows == 3, "the count is ROWS: keys under-report what was measured" + + +# --- the declared mirror, pinned against the INSTALLED tool ------------------- + + +@pytest.mark.parametrize( + ("path", "file_name"), + [ + ("src/pkg/mod.py", "mod.py"), + ("src/pkg", "mod.py"), + ("src/pkg/", "mod.py"), + ("", "mod.py"), + ("mod.py", "mod.py"), + ("src/odd dir/mod.py", "mod.py"), + ], +) +def test_snapshot_key_join_mirrors_the_installed_normalize_path(path: str, file_name: str) -> None: + # The kit ships no MIRRORS.md, so cf-mirror-check (skip-if-absent) gates + # nothing here. This rod is the stronger guard: it calls complexipy's OWN + # function, so a 5.7.0 change to the join fails the suite instead of silently + # re-keying every committed watermark into a brand-new offender. + assert _normalized_path(path, file_name) == normalize_path(path, file_name) + + +# --- the env pins (migrated here: they are instrument wiring, not ratchet) ----- + + +def test_measurement_runs_pin_columns_and_term_so_the_census_cannot_wrap( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # rich wraps at 80 columns when stdout is not a terminal, and a wrapped census + # row is an UNPARSEABLE census row — the gate would refuse a healthy repo. But + # COLUMNS is only load-bearing once TERM is: rich/console.py:1015-1016 returns + # a HARD 80x25 and never reads COLUMNS at all when is_dumb_terminal, which is + # `is_terminal and TERM in ("dumb", "unknown")` (rich/console.py:986-988). + # FORCE_COLOR and TTY_COMPATIBLE are the two names that force is_terminal True + # over a pipe (rich/console.py:955-963), so the overlay clears them. + row = _light(tmp_path) + _snapshot(tmp_path) + seen: list[dict[str, str]] = [] + + def recording_exec( + argv: list[str], cwd: Path, env: Mapping[str, str], *, stdin: str | None = None + ) -> subprocess.CompletedProcess[str]: + seen.append(dict(env)) + return subprocess.CompletedProcess(argv, 0, _census(row), "") + + monkeypatch.setattr(gate_runner, "_tool", lambda name: Path("/fake") / name) + monkeypatch.setattr(gate_runner, "_exec", recording_exec) + hostile = {"PATH": "/usr/bin", "TERM": "dumb", "FORCE_COLOR": "1", "TTY_COMPATIBLE": "1"} + + gate_runner._complexipy(_layout(tmp_path), hostile) + + assert len(seen) == 2, "the census run and the offender run" + for env in seen: + assert int(env["COLUMNS"]) >= 1000, "an 80-column wrap would break the census parse" + assert env["PYTHONIOENCODING"] == "utf-8", "the census decodes UTF-8, never by locale" + assert env["TERM"] not in ("dumb", "unknown"), "a dumb TERM ignores COLUMNS outright" + assert "FORCE_COLOR" not in env and "TTY_COMPATIBLE" not in env, "no forced tty" + assert env["PATH"] == "/usr/bin", "the caller's environment survives the overlay" diff --git a/tests/test_complexipy_ratchet_rules.py b/tests/test_complexipy_ratchet_rules.py new file mode 100644 index 0000000..907554c --- /dev/null +++ b/tests/test_complexipy_ratchet_rules.py @@ -0,0 +1,303 @@ +"""The ratchet's rules where they were VACUOUS or WRONG — one rod per refuter finding. + +``tests/test_complexipy_snapshot.py`` pins the worlds the ratchet already graded +correctly. This file pins the worlds it graded green while measuring nothing, or +graded red while nothing was wrong: + +- **the raised bar (the primary rule going vacuous).** ``_vacuity`` guarded only + the CENSUS, and ``_skew`` is a subset check an empty subset satisfies trivially. + So with ``[tool.complexipy] max-complexity-allowed = 100`` the census still + listed every function (every floor file present, surface audit silent), the + ``--failed`` run returned NOTHING, and the stage exited 0 while cheerfully + reporting it had measured 681 functions against a 12-function floor. A function + that went 33 -> 90 was green. That is + ``gate-that-selects-by-the-value-it-guards-goes-vacuous`` inside the fix for it, + and the closure is threshold-free by construction — the kit still declares no + budget of its own; +- **the ignore comment that voided a watermark.** The audit was per-FILE, and an + ignored function leaves its file in ``census.files``, so nothing fired: an + unregistered exemption from the complexity gate, invisible to ``cf-exemptions`` + too; +- **the symlinked module accused of being outside the tree**, because the FILE was + resolved rather than the roots; +- **the duplicate ``(path, name)`` pair** faking a skew, or hiding a regression; +- **the messages**, which stated a condition and named no remedy, while the docs + claimed they named the re-boot; +- **the PASS with no evidence**, which a machine could not tell from a void. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from test_complexipy_snapshot import A_FUNCTION, _census, _codes, _measured, _snapshot +from test_complexipy_snapshot import _stage_verdict as _verdict +from test_gate_runner import _write + +from cf_quality.errors import GateVerdict +from cf_quality.gate_runner import _Aggregate, _emit_human + +# --- BLOCKER 2: the offender census's threshold-free vacuity closure ----------- + + +def test_a_raised_threshold_empties_the_offender_set_and_is_refused( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # Reproduction (b) from the tool spike, arriving through CONFIG rather than + # through `-mx`: the bar is high enough that a 90-complexity function is not an + # offender. The census is complete, the file is measured, the surface audit has + # nothing to say, the offender set is EMPTY — and the old ratchet iterated that + # empty dict, found no regression, and exited 0 on a 33 -> 90 regression. + # + # The closure names no threshold. A floor entry proves that function was ABOVE + # the bar when the floor was booted, so census 90 >= watermark 33 while the + # offender run stays silent means threshold_now >= 90 >= 33 > threshold_at_boot. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + + verdict = _verdict(tmp_path, monkeypatch, _measured(_census(("src/heavy.py", "heavy", 90)), "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_THRESHOLD_RAISED" + assert verdict.exit_code == 2 + assert "src/heavy.py:heavy (measured 90 >= watermark 33)" in verdict.error.context["functions"] + assert "--snapshot-create" in verdict.error.message, "the refusal names its remedy" + + +def test_a_function_exactly_at_its_watermark_still_must_appear_as_an_offender( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The tight edge of the same rule, and the reason it is `>=` and not `>`: at + # census == watermark the function is still above the BOOT threshold, so an + # offender run that omits it can only mean the bar moved. The green counterpart + # (same numbers, offender run reporting it) is pinned in + # test_complexipy_snapshot.test_unchanged_floor_is_clean_at_the_watermark. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + + verdict = _verdict(tmp_path, monkeypatch, _measured(_census(("src/heavy.py", "heavy", 33)), "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_THRESHOLD_RAISED" + + +def test_a_hand_lowered_watermark_lands_in_the_same_refusal( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # complexipy stores only OVER-threshold functions, so a watermark of 1 could + # never have come from the boot command. The floor was edited, and the refusal + # is correct rather than a false positive. + _write(tmp_path, "src/light.py", A_FUNCTION) + _snapshot(tmp_path, ("src/light.py", "a_function", 1)) + + census = _census(("src/light.py", "a_function", 4)) + + verdict = _verdict(tmp_path, monkeypatch, _measured(census, "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_THRESHOLD_RAISED" + + +def test_a_genuine_shrink_below_its_watermark_is_still_green( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # THE control rod on the closure above: it must fire on a moved bar and NOT on + # the improvement it superficially resembles. 30 -> 4 is under the watermark, so + # census < watermark and the rule does not engage. Without this rod the closure + # could be "refuse whenever the offender set is empty", which would red every + # healthy repo that fixed its last offender. + _write(tmp_path, "src/improved.py", A_FUNCTION) + _snapshot(tmp_path, ("src/improved.py", "improved", 30)) + + verdict = _verdict( + tmp_path, monkeypatch, _measured(_census(("src/improved.py", "improved", 4)), "") + ) + + assert verdict.passed, _codes(verdict) + + +# --- DEFECT 6: the ignore comment that silently voided a watermark ------------ + + +def test_a_floor_function_missing_from_a_MEASURED_file_is_a_violation( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # `# complexipy: ignore` drops a function from BOTH runs while its file stays in + # census.files, so the per-FILE audit saw nothing — an unregistered, unblessed + # exemption from the complexity gate. The distinction that makes this catchable: + # `--plain` without `--failed` lists every measured function regardless of + # threshold, so a function that merely got SIMPLER would still be here. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "ignored", 33), ("src/heavy.py", "kept", 20)) + census = _census(("src/heavy.py", "kept", 20)) + + verdict = _verdict(tmp_path, monkeypatch, _measured(census, census)) + + assert _codes(verdict) == ["COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED"] + assert verdict.exit_code == 1 + violation = verdict.violations[0] + assert violation.path == "src/heavy.py" and violation.context["function"] == "ignored" + assert "complexipy: ignore" in violation.message, "the message names the likely cause" + assert "--snapshot-create" in violation.message, "and the remedy" + + +def test_a_whole_file_going_unmeasured_still_reports_once_not_per_function( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The per-function audit must not turn one narrowed file into N identical + # findings — the file-level reading is the more informative one and comes first. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "one", 33), ("src/heavy.py", "two", 20)) + _write(tmp_path, "src/light.py", A_FUNCTION) + + verdict = _verdict( + tmp_path, monkeypatch, _measured(_census(("src/light.py", "a_function", 1)), "") + ) + + assert _codes(verdict) == ["COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED"] + + +# --- DEFECT 7: the symlinked module accused of being outside the tree --------- + + +def test_a_symlinked_module_inside_the_tree_is_not_called_outside_it( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # `_surface_violations` resolved the FILE, so src/settings.py -> ../vendor/ + # settings.py resolved outside the graded root and the gate reported that the + # floor entry "lies OUTSIDE the graded source root" — flatly false: it is + # inside, and complexipy measured it. Roots are resolved; the floor's own paths + # are repo-relative, so containment is lexical. + _write(tmp_path, "vendor/settings.py", A_FUNCTION) + (tmp_path / "src").mkdir(exist_ok=True) + (tmp_path / "src" / "settings.py").symlink_to(tmp_path / "vendor" / "settings.py") + _snapshot(tmp_path, ("src/settings.py", "settings", 20)) + census = _census(("src/settings.py", "settings", 20)) + + verdict = _verdict(tmp_path, monkeypatch, _measured(census, census)) + + assert verdict.passed, _codes(verdict) + + +def test_a_floor_entry_genuinely_outside_the_graded_root_is_still_reported( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The control rod on the fix above: making the test lexical must not make it + # vacuous. legacy/ is a real sibling of the graded src/, and still fires. + _write(tmp_path, "legacy/heavy.py", A_FUNCTION) + _write(tmp_path, "src/light.py", A_FUNCTION) + _snapshot(tmp_path, ("legacy/heavy.py", "heavy", 33)) + + verdict = _verdict( + tmp_path, monkeypatch, _measured(_census(("src/light.py", "a_function", 1)), "") + ) + + assert _codes(verdict) == ["COMPLEXIPY_SURFACE_NARROWED"] + assert "--snapshot-create" in verdict.violations[0].message, "the finding names its remedy" + + +# --- DEFECT 5: the vacuity leg consults the REPO-WIDE answer ------------------ + + +def test_a_python_free_source_root_beside_real_code_cannot_report_pass( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # `resolve_source_root` returns root/src whenever src/ EXISTS, so a repo whose + # code lives in app/ beside an empty-but-present src/ measured zero functions + # with a floor of [] — and both old vacuity legs were blind, because leg 2 + # walked the same possibly-wrong source_root. The leg now rides the CALLER's + # repo-wide py_present, the same answer the absent-snapshot doctrine uses, so + # the two surfaces cannot disagree. + _write(tmp_path, "app/real.py", A_FUNCTION) + (tmp_path / "src").mkdir() + _snapshot(tmp_path) + + verdict = _verdict(tmp_path, monkeypatch, _measured("", "")) + + assert verdict.error is not None + assert verdict.error.code == "GATE_COMPLEXIPY_MEASURED_NOTHING" + assert verdict.error.context["python_present"] is True + assert verdict.exit_code == 2 + + +# --- DEFECT 10: a duplicate pair must not fake a skew ------------------------- + + +def test_a_duplicated_key_straddling_a_watermark_reports_the_regression_not_a_skew( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # Observed live: one consumer file lists the same (path, name) twice. Under + # last-writer-wins with sort = "desc" the offender map kept 40 while the census + # kept 4, so the runs "disagreed" and the gate raised a bogus + # GATE_COMPLEXIPY_MEASUREMENT_SKEW on a healthy repo — and in the mirror case + # the regression vanished from both maps. max() on both sides gives one world. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 20)) + census = _census(("src/heavy.py", "heavy", 40), ("src/heavy.py", "heavy", 4)) + + verdict = _verdict(tmp_path, monkeypatch, _measured(census, census)) + + assert verdict.error is None, "a repeated key is not two worlds" + assert _codes(verdict) == ["COMPLEXIPY_WATERMARK_REGRESSION"] + assert verdict.violations[0].context["measured"] == 40, "max(), so the HIGH value grades" + assert verdict.violations[0].context["measured_functions"] == 2, "rows, not keys" + + +# --- DEFECT 9 + the evidence a PASS must carry ------------------------------- + + +def test_every_regression_message_names_a_remedy( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # A message that states the condition and names nothing to do is half a + # message, and the docs corrected on this branch already claimed these named + # the re-boot. They now do — spelled with the resolved graded root. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "kept", 20)) + census = _census(("src/heavy.py", "kept", 26), ("src/heavy.py", "fresh", 40)) + + verdict = _verdict(tmp_path, monkeypatch, _measured(census, census)) + + assert _codes(verdict) == ["COMPLEXIPY_NEW_OFFENDER", "COMPLEXIPY_WATERMARK_REGRESSION"] + for violation in verdict.violations: + assert "complexipy src --snapshot-create" in violation.message + assert "from the repo root" in violation.message + + +def test_a_passing_stage_carries_its_measurement_in_the_verdict( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The discrimination this whole rung exists to make: a machine reading the + # aggregated JSON must be able to tell a clean grade from a vacuous one. The + # counts used to go only to stderr via a hand-rolled print, so a PASSING stage's + # to_dict() carried no evidence at all — and the workflows pipe only stdout into + # $GITHUB_STEP_SUMMARY, so even the human never saw it on the board. + _write(tmp_path, "src/heavy.py", A_FUNCTION) + _snapshot(tmp_path, ("src/heavy.py", "heavy", 33)) + rows = _census(("src/heavy.py", "heavy", 33), ("src/other.py", "fine", 2)) + + offenders = _census(("src/heavy.py", "heavy", 33)) + + verdict = _verdict(tmp_path, monkeypatch, _measured(rows, offenders)) + + assert verdict.passed, _codes(verdict) + wire = verdict.to_dict() + assert wire["evidence"]["measured_functions"] == 2 + assert wire["evidence"]["measured_files"] == 2 + assert wire["evidence"]["snapshot_functions"] == 1 + assert wire["notices"] == verdict.notices and len(verdict.notices) == 1 + assert "measured 2 function(s) in 2 file(s)" in verdict.notices[0] + + +def test_the_human_board_shows_the_measurement_beside_a_pass( + capsys: pytest.CaptureFixture[str], +) -> None: + # The notice must ride STDOUT beside the PASS line, because that is the stream + # both workflows tee into $GITHUB_STEP_SUMMARY — a count only on stderr reaches + # the run log and not the board that carries the verdict. + verdict = GateVerdict(gate="complexipy", violations=[], notices=["— m"]) + + _emit_human(_Aggregate(verdicts=[verdict])) + + assert "PASS complexipy — m" in capsys.readouterr().out diff --git a/tests/test_complexipy_snapshot.py b/tests/test_complexipy_snapshot.py index 2c7e602..8770ad6 100644 --- a/tests/test_complexipy_snapshot.py +++ b/tests/test_complexipy_snapshot.py @@ -21,13 +21,19 @@ - RED on a real **complexity regression** — a new offender, and a rise above a committed watermark; - RED on a **vacuous** run — a census of zero functions can never report clean, - with a second leg (the tree demonstrably defines functions) so the refusal + with a second leg (the caller's repo-wide ``py_present``) so the refusal survives an already-emptied floor; - GREEN on an **unchanged** floor, on a **legitimate non-empty shrink**, and on a genuinely clean repo whose floor is ``[]``; - the **write-free** proof at the argv altitude, plus the measured COUNT riding every finding so no verdict can be read without the measurement behind it. +Two sibling files carry what this one is not about: ``test_complexipy_instrument.py`` +(the consumer config that can defeat the measurement, the exit-code/stderr triage, +the env pins, and the live mirror-pin against the installed ``normalize_path``) and +``test_complexipy_ratchet_rules.py`` (the threshold-free vacuity closures, the +per-function surface audit, and duplicate ``(path, name)`` keys). + The external tool is faked at the established subprocess seam (``gate_runner._exec`` / ``gate_runner._tool``, through ``test_gate_runner``'s helpers) so the REAL grading logic runs against representative ``--plain`` @@ -40,15 +46,16 @@ from __future__ import annotations import json -import subprocess from collections.abc import Mapping from pathlib import Path import pytest from test_gate_runner import _clean_cf_responses, _install_fakes, _layout, _write -from cf_quality import complexipy_ratchet, gate_runner -from cf_quality.complexipy_ratchet import Census, grade, measurement_argv, read_snapshot +from cf_quality import complexipy_measure, gate_runner +from cf_quality.complexipy_floor import read_snapshot +from cf_quality.complexipy_measure import Census, measurement_argv +from cf_quality.complexipy_ratchet import grade from cf_quality.errors import GateError, GateVerdict #: Module text that merely EXISTS and defines a function. Every complexity in this @@ -85,10 +92,10 @@ def _census(*rows: tuple[str, str, int]) -> str: def _measured(census: str, offenders: str) -> dict[str, tuple[int, str, str]]: """The clean board with complexipy's two write-free runs answered explicitly. - The offender run's exit code is 1 whenever it reports anything, and it is - deliberately irrelevant: with the compare switched off complexipy's status - only restates "something is over threshold", the normal state of a repo - carrying a baselined floor. + Exit 1 whenever a run reports offenders, 0 otherwise — the tool's real + semantics, which the gate now CROSS-CHECKS rather than discards: an empty + ``--failed`` census beside a non-zero exit is a contradiction, and that is the + offender run's vacuity leg (its rod lives in test_complexipy_instrument.py). """ responses = _clean_cf_responses() responses["complexipy"] = (0, census, "") @@ -360,49 +367,16 @@ def test_clean_repo_with_an_empty_floor_is_clean( def test_measurement_argv_can_never_write_the_committed_floor() -> None: # The pinned tool reaches `create_snapshot_file` from exactly two places: - # `--snapshot-create`, and the watermark compare's success path that - # `--snapshot-ignore` switches off. Both flags are load-bearing, both runs. - for offenders_only in (False, True): - argv = measurement_argv( - Path("/fake/complexipy"), Path("/repo/src"), offenders_only=offenders_only - ) - assert argv[:2] == ["/fake/complexipy", "/repo/src"] - assert "--snapshot-ignore" in argv, "the compare — and its rewrite — stays off" - assert "--snapshot-create" not in argv, "the gate never writes the artifact it grades" - assert "--plain" in argv, "the census must be machine-readable to be graded" - assert ("--failed" in argv) is offenders_only - - -def test_measurement_runs_pin_columns_so_the_census_cannot_wrap( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - # rich wraps at 80 columns when stdout is not a terminal, and a wrapped census - # row is an UNPARSEABLE census row — the gate would refuse a healthy repo, or - # (worse, in a laxer parser) read a narrower world. COLUMNS is load-bearing. - row = _light(tmp_path) - _snapshot(tmp_path) - seen: list[dict[str, str]] = [] - - def recording_exec( - argv: list[str], - cwd: Path, - env: Mapping[str, str], - *, - stdin: str | None = None, - ) -> subprocess.CompletedProcess[str]: - seen.append(dict(env)) - return subprocess.CompletedProcess(argv, 0, _census(row), "") - - monkeypatch.setattr(gate_runner, "_tool", lambda name: Path("/fake") / name) - monkeypatch.setattr(gate_runner, "_exec", recording_exec) - - gate_runner._complexipy(_layout(tmp_path), {"PATH": "/usr/bin"}) + # `--snapshot-create` (whose TOML twin complexipy_measure.audit_config refuses, + # because argv cannot negate it) and the watermark compare's success path that + # `--snapshot-ignore` switches off. EXHAUSTIVE literal equality, not a flag + # allowlist: the allowlist that stood here bit on REMOVING --snapshot-ignore + # but NOT on ADDING `-mx 100`, which empties the offender set outright. + tool, source_root = Path("/fake/complexipy"), Path("/repo/src") + base = ["/fake/complexipy", "/repo/src", "--plain", "--color", "no", "--snapshot-ignore"] - assert len(seen) == 2, "the census run and the offender run" - for env in seen: - assert int(env["COLUMNS"]) >= 1000, "an 80-column wrap would break the census parse" - assert env["PYTHONIOENCODING"] == "utf-8", "the census decodes UTF-8, never by locale" - assert env["PATH"] == "/usr/bin", "the caller's environment survives the overlay" + assert measurement_argv(tool, source_root, offenders_only=False) == base + assert measurement_argv(tool, source_root, offenders_only=True) == [*base, "--failed"] def test_the_committed_floor_is_only_ever_read( @@ -433,9 +407,9 @@ def test_grade_is_a_pure_function_of_the_floor_and_the_census(tmp_path: Path) -> # and no green run can destroy it. Every finding carries the measured tally. _write(tmp_path, "src/heavy.py", A_FUNCTION) floor = {("src/heavy.py", "heavy"): 20} - census = Census(functions={("src/heavy.py", "heavy"): 31}) + census = Census(functions={("src/heavy.py", "heavy"): 31}, rows=1) - verdict = grade(tmp_path, tmp_path / "src", floor, census, census) + verdict = grade(_layout(tmp_path), floor, census, census) assert _codes(verdict) == ["COMPLEXIPY_WATERMARK_REGRESSION"] assert verdict.violations[0].context["measured_functions"] == 1 @@ -474,7 +448,7 @@ def test_read_snapshot_keys_the_observed_on_disk_shape(tmp_path: Path) -> None: def test_census_parser_survives_a_path_containing_spaces() -> None: # `--plain` is space-separated, so the parse must split from the RIGHT: the # complexity and the function name are single tokens, a path is not. - census = complexipy_ratchet.parse_census("src/odd dir/mod.py Klass::method 12\n") + census = complexipy_measure.parse_census("src/odd dir/mod.py Klass::method 12\n") assert census.functions == {("src/odd dir/mod.py", "Klass::method"): 12} assert census.files == frozenset({"src/odd dir/mod.py"}) diff --git a/tests/test_configs.py b/tests/test_configs.py index f2716f1..9bf9977 100644 --- a/tests/test_configs.py +++ b/tests/test_configs.py @@ -166,7 +166,50 @@ def test_baseline_conventions_documents_observed_workflows() -> None: assert "--snapshot-create" in text # complexipy watermark boot assert "mypy-baseline sync" in text # type-debt baseline boot assert "mypy-baseline filter" in text # CI-side set-difference gate - assert "does NOT auto-shrink" in text # the complexipy re-snapshot duty + # 2026-07-28: this line used to pin the substring "does NOT auto-shrink", + # which is FALSE for the pinned complexipy 5.6.0 — a PASSING compare calls + # create_snapshot_file on handle_snapshot_watermark's no-violation branch and + # was observed rewriting a populated snapshot to []. The assertion was + # therefore gating a false sentence into the document. It is replaced, not + # softened: where one substring pinned one (wrong) claim, four now pin the + # whole corrected contract — the measured write behaviour, its evidence, the + # write-free measurement that disarms it, and the operator duty that survived + # it. Delete any one of those teachings from the doc and this test fails. + assert "A passing plain-run compare REWRITES the snapshot" in text # measured on 5.6.0 + assert "handle_snapshot_watermark" in text # the destructive call site, named + assert "--snapshot-ignore" in text # the write-free measurement the gate mounts + assert "The re-snapshot duty is still yours" in text # the surviving shrink duty + + +def test_baseline_conventions_teaches_the_config_that_can_defeat_the_gate() -> None: + # 2026-07-28, second correction: `--snapshot-ignore` alone does NOT make the run + # write-free. `snapshot-create` is a separate branch resolved CLI-first/TOML- + # second with no negating flag, so a consumer's own complexipy config could put + # the rewrite back while the gate graded pre-write bytes. A consumer cannot + # comply with a refusal they were never told about, so the mount doc must carry + # the key list, the refusal, and the two options that ARE honoured. + text = (CONFIGS_DIR / "BASELINE-CONVENTIONS.md").read_text(encoding="utf-8") + assert "GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT" in text + assert "snapshot-create" in text and "ignore-complexity" in text and "output-format" in text + assert "max-complexity-allowed" in text, "the threshold the kit deliberately honours" + assert "GATE_COMPLEXIPY_THRESHOLD_RAISED" in text, "raising the bar refuses, not passes" + assert "GATE_COMPLEXIPY_INSTRUMENT_FAILED" in text, "a tool failure is not the repo's" + + +def test_baseline_conventions_cannot_reacquire_the_false_unmeasured_claim() -> None: + # A doc fix pass on this branch asserted that "an improvement that empties a floor + # file of functions … FAILS (COMPLEXIPY_SNAPSHOT_FILE_UNMEASURED)". That is FALSE: + # `--plain` without `--failed` lists every measured function regardless of + # threshold (complexipy/utils/output.py:234,243), so such a file stays in + # census.files and the gate is GREEN. UNMEASURED fires when the file was not + # MEASURED; the per-FUNCTION code is what catches a single lost function. This rod + # bites in both directions: the true rule must be stated, and the false phrasing + # must not come back. + for name in ("BASELINE-CONVENTIONS.md",): + text = (CONFIGS_DIR / name).read_text(encoding="utf-8") + assert "empties a floor file of functions" not in text, f"{name}: the false claim is back" + assert "COMPLEXIPY_SNAPSHOT_FUNCTION_UNMEASURED" in text, "the per-function rule" + assert "was not **measured** at all" in text, "the true trigger, spelled out" def test_ruff_base_gates_blanket_and_unused_noqa() -> None: diff --git a/tests/test_errors.py b/tests/test_errors.py index aa80598..68d997e 100644 --- a/tests/test_errors.py +++ b/tests/test_errors.py @@ -135,6 +135,12 @@ def test_error_dominates_with_exit_two(self) -> None: assert verdict.exit_code == 2 def test_to_dict_composes_the_part_wire_forms(self) -> None: + # `notices` and `evidence` were APPENDED (2026-07-28), the same way + # GateViolation gained severity/fixable: a PASSING gate carried no + # measurement in its wire form, so a machine reading the aggregated JSON + # could not tell a clean grade from one that measured nothing — which is + # the whole discrimination a ratchet exists to make. Both default empty, + # so every existing construction is untouched; the key set grows. violation = GateViolation(code="C", message="m", path="p.py", line=2) error = GateError(code="GATE_X", message="boom", context={"k": "v"}) verdict = GateVerdict(gate="cf-x", violations=[violation], error=error) @@ -144,10 +150,30 @@ def test_to_dict_composes_the_part_wire_forms(self) -> None: "exit_code": 2, "error": error.to_dict(), "violations": [violation.to_dict()], + "notices": [], + "evidence": {}, } def test_clean_to_dict_has_null_error_and_stable_keys(self) -> None: verdict = GateVerdict(gate="cf-x", violations=[]) report = verdict.to_dict() assert report["error"] is None - assert set(report) == {"gate", "passed", "exit_code", "error", "violations"} + assert set(report) == { + "gate", + "passed", + "exit_code", + "error", + "violations", + "notices", + "evidence", + } + + def test_evidence_and_notices_ride_the_wire_form_on_a_pass(self) -> None: + # The rod on the append above: a clean verdict must be able to CARRY its + # measurement, not merely have somewhere to put it. + verdict = GateVerdict( + gate="complexipy", violations=[], notices=["— measured 2"], evidence={"n": 2} + ) + report = verdict.to_dict() + assert report["passed"] is True + assert report["notices"] == ["— measured 2"] and report["evidence"] == {"n": 2} diff --git a/tests/test_gate_runner.py b/tests/test_gate_runner.py index 9eaf8a2..506dfc6 100644 --- a/tests/test_gate_runner.py +++ b/tests/test_gate_runner.py @@ -31,6 +31,7 @@ import pytest from cf_quality import gate_runner +from cf_quality.complexipy_measure import measurement_argv from cf_quality.errors import GateError, GateVerdict, GateViolation from cf_quality.gate_runner import battery_exit_code, run_battery from cf_quality.import_contract import main as import_contract_main @@ -475,25 +476,24 @@ def test_complexipy_targets_source_root_in_unpiped_write_free_processes( ) -> None: # With the snapshot present, complexipy measures the resolved source_root in # UNPIPED processes — piping it (as the mypy stage pipes through the filter) - # would mask its exit code (the tool-spike defect this rod fences off). It is - # also invoked WRITE-FREE: `--snapshot-ignore` and never `--snapshot-create` - # are the only two flags that keep the pinned tool away from - # `create_snapshot_file`, which its own green compare path calls (see - # cf_quality.complexipy_ratchet). Two measurements, one per question the - # ratchet asks; the empty census here is legitimate (no src/, nothing to grade). + # would mask its exit code (the tool-spike defect this rod fences off). BOTH + # argvs are now asserted EXHAUSTIVELY against measurement_argv, in order: the + # three-flag allowlist that stood here pinned what IS present and nothing + # about what is not, so `-mx 100` could join this argv with every test still + # green — and that flag empties the offender set outright, which is the + # vacuity vector this rung exists to close. _write(tmp_path, "complexipy-snapshot.json", "[]") calls: list[tuple[list[str], str | None]] = [] _record_calls(monkeypatch, calls) - layout = _layout(tmp_path) - - verdict = gate_runner._complexipy(layout, {}) + tool = Path("/fake") / "complexipy" + # py_present=False is the honest fixture: tmp_path holds no .py at all, so an + # empty census is an empty WORLD, not the void the vacuity leg must refuse. + verdict = gate_runner._complexipy(_layout(tmp_path, py_present=False), {}) assert verdict is not None and verdict.passed - assert len(calls) == 2, "the census run and the offender run — nothing else" - for argv, stdin in calls: - assert argv[:2] == [str(Path("/fake") / "complexipy"), str(tmp_path / "src")] - assert "--snapshot-ignore" in argv, "the committed floor must be out of reach" - assert "--snapshot-create" not in argv, "the gate never writes the artifact it grades" - assert "--plain" in argv, "the census is parsed, so it rides the scripting form" - assert stdin is None, "no stdin handoff like the mypy filter pipe" - assert ["--failed" in argv for argv, _ in calls] == [False, True], "census, then offenders" + assert [argv for argv, _ in calls] == [ + measurement_argv(tool, tmp_path / "src", offenders_only=offenders) + for offenders in (False, True) + ], "census then offenders, each argv exact — an allowlist would let -mx 100 in" + assert not {"-mx", "--max-complexity-allowed"} & set(calls[0][0]), "no kit-side budget" + assert [stdin for _, stdin in calls] == [None, None], "no stdin handoff like mypy's" diff --git a/tests/test_integration_consumer.py b/tests/test_integration_consumer.py index 539dd18..afdc184 100644 --- a/tests/test_integration_consumer.py +++ b/tests/test_integration_consumer.py @@ -107,9 +107,19 @@ def test_injected_type_error_flips_the_battery_red( ) -> None: consumer = _consumer_copy(tmp_path) _stub_pytest(monkeypatch) - # A module-level type error: mypy flags it, ruff does not (it is not a lint - # finding), so the mypy stage is the SOLE failure — the e2e path truly grades. - (consumer / "app" / "widget.py").write_text("bad: int = 'not an int'\n", encoding="utf-8") + # A module-level type error, APPENDED — never written OVER the module. mypy flags + # it; ruff does not (it is no lint finding, and the value is double-quoted so + # `ruff format --check` leaves it alone), so the mypy stage is the SOLE failure and + # the e2e path truly grades. Replacing widget.py would delete the fixture's ONLY + # function, leaving a source root that is Python-but-functionless: the complexipy + # stage then correctly refuses for having measured nothing + # (GATE_COMPLEXIPY_MEASURED_NOTHING — a setup error, exit 2), and the battery goes + # red for a vacuous complexity grade instead of for the injected type error. Red + # for the wrong reason is the failure mode this append exists to prevent. + widget = consumer / "app" / "widget.py" + widget.write_text( + widget.read_text(encoding="utf-8") + 'bad: int = "not an int"\n', encoding="utf-8" + ) verdicts = run_battery(consumer, os.environ) by_gate = {verdict.gate: verdict for verdict in verdicts}