Filing role: κ (records operator directive, 2026-07-05).
Successor to: the factorized-β wave (#73) — which returned a terminal FAIL (A1/A2/A3 missed; B1/B2/B3 held).
Line: defect-harvesting / calibration — NOT consistency-improvement. See docs/beta/governance/DEFECT-HARVESTING.md and docs/beta/governance/METER-LOOP-DECISION.md.
Why (the sharpened result)
The factorized-β experiment removed three freedoms — the engine chose the same β loci for every witness, the LLM answered only supports | contradicts | insufficient, and the engine (not the LLM) counted and aggregated. A3 still failed: independent witnesses disagree on the local semantic relation itself — not what to inspect, not how to count, not how to aggregate. That localizes the residual variance to genuine semantic judgment over under-specified citation relations (a source line carries multiple claims; a target span partially supports; supports/insufficient and contradicts/not-supported are threshold calls), plus ordinary model non-determinism.
Conclusion (operator, 2026-07-05): we can improve operational reliability but probably cannot make same-route LLM semantic judgment intrinsically stable enough to be a scalar authority. The fix is to change what the LLM is allowed to decide and treat remaining disagreement as a measured property of the artifact — the LLM becomes a defect proposer + ambiguity detector, and a steward / cross-route calibration adjudicates. This is a third, stronger negative for consistency-improvement: even fixed-locus local adjudication did not cross the gate. No v3.2.5 prompt/schema tweak.
Scope
- Parse the factorized-β per-locus votes from a run's artifacts (
.tsc/fb/… inventory + responses).
- Compute agreement / entropy per locus.
- Enqueue loci below an agreement threshold into a semantic-ambiguity queue (proposed
.tsc/meter/ambiguity-queue.jsonl).
- Triage each into
content-defect | ambiguous | witness-error | clean.
- Create gold-label calibration records for adjudicated loci (source span, target span, expected verdict, rationale, known tempting wrong reading).
- Report defect-yield + ambiguity metrics (confirmed defects, ambiguous-locus rate, disagreement rate, false-positive rate, rediscovery-after-fix) — not Coh-consistency.
Ambiguity record (proposed)
{
"locus_id": "beta.link.0017",
"target": "repo",
"kind": "citation_bears_claim",
"source_path": "...", "source_span": "...",
"target_path": "...", "target_span": "...",
"votes": { "supports": 1, "insufficient": 2, "contradicts": 0 },
"agreement": 0.67,
"triage_status": "ambiguous | content-defect | witness-error | clean",
"resolution": null
}
Directions this enables (not all in this issue)
- A. Gold-labeled β calibration corpus — future routes measured against adjudicated labels, not against themselves. Same-route agreement = health signal; gold-label agreement = reliability signal.
- B. Cross-route witnesses — the same loci through a second provider/model and/or a steward; use disagreement to locate ambiguity, not to average into "truth" (the METER-LOOP-DECISION replacement line).
- C. Ambiguity-aware scoring — classify at triage (
decidable | ambiguous-source-claim | ambiguous-target-support | needs-domain-judgment); only decidable, gold-labeled loci bear scalar standing; ambiguous loci feed a clarity-debt score, not witness inconsistency.
- D. Content discipline for citations — a citation-bearing claim must be locally checkable (one claim per source sentence; target anchor directly supports; background links marked as background). Reduces true ambiguity; it's artifact hygiene, not meter-tuning.
Out of scope
Coh-consistency improvement; any v3.2.5 prompt/schema tweak; standing promotion from same-route agreement; re-running the terminal factorized-β experiment. A/B/C/D above are separate downstream issues; this one builds the queue + triage + metrics foundation.
Guardrail
This does not reopen the stopped meter loop. It measures and harvests; it does not optimize Coh-consistency. Metrics are defect-yield / ambiguity, never Coh.
Filing role: κ (records operator directive, 2026-07-05).
Successor to: the factorized-β wave (#73) — which returned a terminal FAIL (A1/A2/A3 missed; B1/B2/B3 held).
Line: defect-harvesting / calibration — NOT consistency-improvement. See
docs/beta/governance/DEFECT-HARVESTING.mdanddocs/beta/governance/METER-LOOP-DECISION.md.Why (the sharpened result)
The factorized-β experiment removed three freedoms — the engine chose the same β loci for every witness, the LLM answered only
supports | contradicts | insufficient, and the engine (not the LLM) counted and aggregated. A3 still failed: independent witnesses disagree on the local semantic relation itself — not what to inspect, not how to count, not how to aggregate. That localizes the residual variance to genuine semantic judgment over under-specified citation relations (a source line carries multiple claims; a target span partially supports;supports/insufficientandcontradicts/not-supportedare threshold calls), plus ordinary model non-determinism.Conclusion (operator, 2026-07-05): we can improve operational reliability but probably cannot make same-route LLM semantic judgment intrinsically stable enough to be a scalar authority. The fix is to change what the LLM is allowed to decide and treat remaining disagreement as a measured property of the artifact — the LLM becomes a defect proposer + ambiguity detector, and a steward / cross-route calibration adjudicates. This is a third, stronger negative for consistency-improvement: even fixed-locus local adjudication did not cross the gate. No v3.2.5 prompt/schema tweak.
Scope
.tsc/fb/…inventory + responses)..tsc/meter/ambiguity-queue.jsonl).content-defect | ambiguous | witness-error | clean.Ambiguity record (proposed)
{ "locus_id": "beta.link.0017", "target": "repo", "kind": "citation_bears_claim", "source_path": "...", "source_span": "...", "target_path": "...", "target_span": "...", "votes": { "supports": 1, "insufficient": 2, "contradicts": 0 }, "agreement": 0.67, "triage_status": "ambiguous | content-defect | witness-error | clean", "resolution": null }Directions this enables (not all in this issue)
decidable | ambiguous-source-claim | ambiguous-target-support | needs-domain-judgment); only decidable, gold-labeled loci bear scalar standing; ambiguous loci feed a clarity-debt score, not witness inconsistency.Out of scope
Coh-consistency improvement; any v3.2.5 prompt/schema tweak; standing promotion from same-route agreement; re-running the terminal factorized-β experiment. A/B/C/D above are separate downstream issues; this one builds the queue + triage + metrics foundation.
Guardrail
This does not reopen the stopped meter loop. It measures and harvests; it does not optimize Coh-consistency. Metrics are defect-yield / ambiguity, never Coh.