Skip to content

feat(skills): reader-side contradiction reports — foresight emits, manager adjudicates (#24) - #25

Merged
AmberLJC merged 3 commits into
ARA-Labs:mainfrom
JeremyJC67:feat/reader-feedback-channel
Jul 20, 2026
Merged

feat(skills): reader-side contradiction reports — foresight emits, manager adjudicates (#24)#25
AmberLJC merged 3 commits into
ARA-Labs:mainfrom
JeremyJC67:feat/reader-feedback-channel

Conversation

@JeremyJC67

@JeremyJC67 JeremyJC67 commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Implements #24. Ready for review — the three open questions at the bottom are genuinely open, and the diff is kept minimal so any answer to them is a small follow-up rather than a rework.

Design constraints respected

  • CONTRACT.md §4b is untouched. The reader engine still writes nothing, anywhere. A contradiction_report is part of the Answer output; the caller owns transport (a reader-report issue on the ARA's repository, or a file handed to the next research-manager run).
  • Single writer unchanged. research-manager remains the ARA's only writer; a report is input to it, never an edit. Reports are adjudicated by the manager in the turn they arrive — verified against ARA-resolvable evidence, then upheld (Stage-4 fold-in, full provenance and before/after recorded) or rejected (verdict + reason) — with an after-the-fact summary to the human, never an open ticket.

Changes

File Change
research-foresight/references/PREDICT.md optional contradiction_report field on the Answer object (§3) + new §7 defining when to emit one (positive conflicting evidence only; thin coverage stays §6); version 1.0.0→1.1.0
research-manager/SKILL.md Stage 1 harvests reader reports as candidate events; new Stage 3 subsection routes them through the existing contradiction trigger; version 2.4.0→2.5.0
research-manager/templates/reader-report.md shared report shape for agent- and human-filed reports (new)
research-foresight/SKILL.md version 1.0.0→1.1.0 (its reference contract changed)
research-foresight/references/CONTRACT.md §3 read/write table: contradiction_report named as sanctioned output-for-transport

Tested with real usage

Executed the amended PREDICT contract against ARA-Labs/ara-ls20 at main (pre-fix state, where C02 still carried the stale RESET reading). The report it produces:

contradiction_report:
  target: "logic/claims.md#C02"
  contradicted_clause: "RESET does NOT merely refresh the budget: it restarts the WHOLE GAME to Level 1, discarding completed levels"
  observation: "On L6 and L7, `play.py send RESET` refreshed ONLY the current level (block→level start, %%→12, budget→42, X-boxes restored) and KEPT levels_completed (5 on L6, 6 on L7); one RESET taken at %%=4 restored full lives. Repeated+reliable across both sessions."
  basis: ["trace:N60", "trace:N70", "trace:N73", "trace/_l6plus_raw.md", "trace/_l7_raw.md"]
  repro: "From a clean known state on any level ≥ L6, issue `play.py send RESET`; watch levels_completed remain unchanged while %%/budget/X-boxes refresh."

Running the amended research-manager rules on that report yields exactly the adjudication filed as the companion PR: ARA-Labs/ara-ls20#1 — i.e. the mechanism reproduces, end-to-end, the fold-in that C02's comments had been waiting on since 2026-06-29.

Open questions for maintainers

  1. Transport default: repo issues (reader-report label), an inbox-file convention, or both?
  2. Provenance: this PR reuses ai-suggested (report ref recorded in context) to keep the diff minimal — is a dedicated reader-reported tag worth the schema touch for trust calibration?
  3. CONTRACT.md: deliberately untouched; if you'd like §3's read/write table to name the report as sanctioned output, happy to add one line.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RSRisCyn8iHnBBuHMrWg6f

…nager adjudicates

Implements the channel proposed in ARA-Labs#24 without touching CONTRACT.md §4b:

- research-foresight/references/PREDICT.md: optional `contradiction_report`
  field on the Answer object + §7 defining when to emit one. Output only —
  the engine still writes nothing, anywhere; the caller owns transport
  (a `reader-report` issue on the ARA's repository, or a file handed to the
  next research-manager run).
- research-manager/SKILL.md: Stage 1 harvests reader reports as candidate
  events; a new Stage 3 subsection routes them through the EXISTING
  contradiction trigger (flag CONFLICT, append an unresolved decision node,
  defer to the researcher). Reports are never auto-applied; the
  single-writer rule is unchanged.
- research-manager/templates/reader-report.md: shared report shape for
  agent- and human-filed reports.
- Version bumps: research-manager 2.4.0→2.5.0, research-foresight 1.0.0→1.1.0
  (PREDICT.md 1.0.0→1.1.0).

Motivating case: ara-ls20 C02 (RESET semantics), where two sessions of
contradicting reader-side observations could only be parked in HTML comments
pending a researcher pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RSRisCyn8iHnBBuHMrWg6f
@JeremyJC67
JeremyJC67 marked this pull request as ready for review July 18, 2026 18:34
@AmberLJC
AmberLJC requested a review from ShokuhouMisak1 July 19, 2026 16:56
@ShokuhouMisak1

Copy link
Copy Markdown
Collaborator

Thanks a lot for this — and especially for actually running the amended contract end-to-end against a real ARA and filing the companion fold-in on ara-ls20. That's exactly the kind of validation we hope for from contributions, and the report it produced is genuinely useful.

Two change requests before merge:

1. Drop the ls20/C02 example from PREDICT.md §7.
Baking a concrete case into a skill file tends to anchor the model on that one example and overfit its behavior to it. Please remove the cf. C02 in ARA-Labs/ara-ls20 … passage and keep §7 as the general rule only — the substantive changes (the contradiction_report field, the positive-evidence-only rule, the §6/§7 distinction) all stay as they are.

2. Let research-manager adjudicate automatically instead of deferring to the researcher.
Your current routing is: flag CONFLICT → append an unresolved decision node → stop, "adjudication is the researcher's job." We'd like to go one step further: have research-manager adjudicate the report itself — check it against the cited evidence, and when it holds, fold the correction in (with full provenance on both sides) — and then notify the human of what it decided, rather than wait for the human to decide. The guiding aim of this project is to remove the human from the loop; a channel that ends in "defer to the researcher" reintroduces exactly the bottleneck your ls20 example demonstrates (a contradiction parked for three weeks waiting for a human pass). Reports that don't hold up against the evidence should be recorded as rejected, again with provenance — the point is the manager reaches a conclusion either way and leaves the human an after-the-fact summary, not an open ticket.

On the open questions:

  1. Transport: both, exactly as your template comment already describes — reader-report issues for hosted ARAs, a report file handed to the manager run for local ones. No change needed.
  2. Provenance: keep ai-suggested + the report ref in context for now, as you did. A dedicated reader-reported tag can come later if we ever need trust calibration between self-generated and reader-reported evidence.
  3. CONTRACT.md: yes, please add the one line to §3's read/write table naming contradiction_report as sanctioned output. CONTRACT is the binding document, and that line forecloses any future reading of §4b against this field.

Thanks again for the careful work here — looking forward to the updated revision.

…rn adjudication

- PREDICT.md §7: concrete ls20/C02 example removed; general rule only
- CONTRACT.md §3: PREDICT writes cell names contradiction_report as
  sanctioned output-for-transport, foreclosing any §4b reading against it
- research-manager: reader reports are adjudicated in the turn they arrive —
  verified against ARA-resolvable evidence, then upheld (Stage 4 fold-in with
  logic_revisions before/after + resolved decision node) or rejected (verdict
  + reason recorded); the human is notified after the fact, never left an
  open ticket. Upholding requires corroboration resolvable inside the ARA;
  reports resting only on unresolvable reader-side pointers are rejected as
  unverifiable (repro preserved). Scoped as an explicit exception to the
  contradiction trigger defer rule.
- templates/reader-report.md: comment updated to the adjudicate-and-notify flow

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RSRisCyn8iHnBBuHMrWg6f
@JeremyJC67

Copy link
Copy Markdown
Contributor Author

Revised in cb90efc — all three in:

  1. §7 is now the general rule only — the ls20/C02 passage is gone from the skill file (the worked example remains in the PR/issue record, where anchoring isn't a concern).
  2. CONTRACT §3: PREDICT's writes cell now names contradiction_report as sanctioned output-for-transport, so §4b can't be read against the field.
  3. research-manager now adjudicates in-turn and notifies after the fact: verify → upheld (Stage-4 fold-in with logic_revisions before/after + a resolved decision node) or rejected (entry untouched, verdict + specific reason recorded) → conclusion lands in the session record and the [PM] summary line. No open tickets. The PR body and the template comment are updated to match.

One design decision in the revision worth your sanity-check: upholding requires corroboration resolvable inside the ARA (trace nodes, evidence files, session records). A report resting only on reader-side pointers the manager cannot resolve is rejected as unverifiable basis, with its repro preserved for a future run to execute. Without that gate, in-turn adjudication would let any reader rewrite the canon by citing evidence nobody can check — the poisoning vector is the reason v1 deferred to the human. The C02 case passes the gate (N60/N70/N73 are in the ARA's own trace), so the autonomy you asked for is intact for exactly the class of report the channel is meant to carry. If you ever want a more permissive policy for trusted readers, it can be parameterized later.

Also flagging the scoping explicitly: the subsection is written as an exception to the contradiction trigger's defer rule, limited to reader reports — the manager's own mid-research contradictions still defer to the researcher. If removing the human from that loop is also on the roadmap, it feels like its own PR.

@AmberLJC

Copy link
Copy Markdown
Collaborator

lgtm
@ShokuhouMisak1 can you help review, thanks!

…d — falls back to CONFLICT flag + unresolved node

A report the ARA can neither corroborate nor refute is possibly true; closing
it as rejected (status: resolved) leaves the disputed entry looking clean and
buries the dispute in the session record. Unverifiable reports now take the
pre-existing defer path instead: CONFLICT flag on the entry + unresolved
decision node carrying the repro for a future run. Rejected is reserved for
reports the evidence positively shows wrong.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ShokuhouMisak1

Copy link
Copy Markdown
Collaborator

Thanks — verified the revision against the diff and everything checks out. The corroboration gate is the right call, and thanks for flagging it explicitly rather than burying it; the poisoning vector you describe is exactly why it belongs there. The scoping is also right — the manager's own mid-research contradictions staying deferred is its own discussion, not this PR.

One refinement, which I've pushed directly to your branch as 5debf2b to save a round-trip: unverifiable basis is now its own verdict rather than a flavor of rejected.

The reasoning: a report the ARA can positively refute deserves a clean close — but a report the ARA can neither corroborate nor refute is possibly true, and closing it as rejected (status: resolved) leaves the disputed entry looking clean while the dispute dies in the session record. That's the C02 failure mode reintroduced one level up. So the unverifiable case now falls back to the pre-existing defer path: CONFLICT flag on the entry + an unresolved decision node carrying the report's repro for a future run to execute. Rejected is reserved for reports the evidence positively shows wrong. The template comment is updated to match. Notably, this also means the gate no longer silently closes the channel's marquee scenario — a fresh reader whose contradicting evidence lives only in its own environment — the dispute stays visible on the entry until someone runs the repro.

Please take a look at the commit — if it reads right to you, this is mergeable as far as I'm concerned. Thanks again for a genuinely well-built contribution, and for the ara-ls20 companion PR, which we'll take next.

@AmberLJC
AmberLJC merged commit ae96f35 into ARA-Labs:main Jul 20, 2026
1 check passed
@JeremyJC67

Copy link
Copy Markdown
Contributor Author

5debf2b reads right — confirmed against the diff. The three-verdict split is the correct resolution of the tension I was trading away: I had collapsed "provably wrong" and "can't check" into one closed verdict to guarantee no open tickets, but an uncheckable-yet-possibly-true dispute closing silently is indeed C02 reintroduced one level up. Reserving rejected for positive refutation and routing the unverifiable case back through the CONFLICT/unresolved path keeps the dispute visible exactly where it matters — on the entry — while every report still leaves the turn with a verdict. Nothing further from my side; looking forward to the ara-ls20 companion landing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants