SLM-418: render replay preference rows into trainable pairs (DSH5-10) - #1124
Closed
Tyler-R-Kendrick wants to merge 3 commits into
Closed
SLM-418: render replay preference rows into trainable pairs (DSH5-10)#1124Tyler-R-Kendrick wants to merge 3 commits into
Tyler-R-Kendrick wants to merge 3 commits into
Conversation
Sixth slice of SLM-418's replay-grounded preference extraction. Pattern extraction reached 7/7 in the fifth slice (#1119), but nothing in the repo converted a row into the PreferencePair shape scripts/train_preference.py actually consumes -- confirmed by grepping the whole tree for ReplayPreferenceRelation/extract_replay_preference_rows/ extract_merge_preference_row before this change. Adds src/slm_training/harnesses/preference/replay_pairs.py: render_replay_preference_pair renders one OperatorReplayPreferenceRowV1 into a PreferencePair without ever fabricating a state -- undo/redo/checkout resolve via direct node lookup (no computation needed, since those targets are always already-materialized trace nodes), and the one case that needs a genuinely new state (a pronoun-focus-followup sibling operator action) re-derives it by recomputing the exact legal set at the input state and actually applying the matching action through the same pack-authorized OperatorLibraryV1.apply every other application in this module goes through. merge:<pair> as a rejected action (unreachable today, but defensively handled) is left honestly unrendered rather than guessed. MERGE_SUCCESS rows never live on a shared ConversationTraceV1 (BranchEditV1 edges + a fresh merged node instead), so merge_node_resolver(left, right, decision) builds the equivalent per-call node resolver for that case. Also documents an open, explicitly-flagged modeling choice: the pair's prompt is the input state's own source, a first cut for compatibility with the existing TwoTower pair format -- not a validated training-objective decision, since the disposition doc's own remaining-scope note names the DSH3 policy head as the actual training target. Still wiring only: no pairs corpus built from real traces, no training run, no held-out claim. harness.preference.replay_pairs registered fresh (v1); dsl.operators.replay_preference is unchanged (only consumed, not modified). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDctCUvBtGrdMvgNd34oHR
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
Next review available in: 3 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughChangesReplay preference pair conversion
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant ReplayRow as OperatorReplayPreferenceRowV1
participant Renderer as render_replay_preference_pair
participant Resolver as NodeResolver
participant Library as OperatorLibraryV1
participant Pair as PreferencePair
ReplayRow->>Renderer: provide input, chosen, and rejected actions
Renderer->>Resolver: resolve input and chosen state IDs
Renderer->>Library: re-apply derivable rejected action
Library-->>Renderer: rejected output state
Renderer-->>Pair: construct prompt, chosen, rejected, and metadata
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/slm_training/harnesses/preference/replay_pairs.py`:
- Around line 94-172: Validate the replay row’s legal-set fingerprint in
render_replay_preference_pair before re-deriving the rejected operator action.
Recompute the legal set using the same ordinary_nonoperator_actions
configuration as extraction, return None when its fingerprint differs from
row.legal_set_fingerprint, and only then locate the action and call
library.apply.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: d6c0dce0-6f6d-45e9-952a-882a50ce08e0
📒 Files selected for processing (4)
docs/design/dsh5-10-replay-preference-rows.mdsrc/slm_training/harnesses/preference/replay_pairs.pysrc/slm_training/resources/versions.jsontests/test_harnesses/preference/test_replay_pairs.py
Comment on lines
+94
to
+172
| def render_replay_preference_pair( | ||
| row: OperatorReplayPreferenceRowV1, | ||
| *, | ||
| resolve_node: NodeResolver, | ||
| pack: DslPack, | ||
| library: OperatorLibraryV1, | ||
| provenance_for: ProvenanceFactory, | ||
| ) -> PreferencePair | None: | ||
| """Render one replay-grounded row into a ``PreferencePair``, or ``None``. | ||
|
|
||
| Returns ``None`` rather than fabricating a rejected state whenever the | ||
| rejected action is ``merge:<pair>`` (out of scope; see module | ||
| docstring), its target legal action cannot be re-derived from the exact | ||
| legal set at ``row.input_state_id`` (would indicate a stale/inconsistent | ||
| row), or the re-applied rejected state is identical to the chosen state | ||
| (degenerate; not a real preference). | ||
| """ | ||
| input_node = resolve_node(row.input_state_id) | ||
| chosen_text = resolve_node(row.chosen_output_state_id).state.source | ||
|
|
||
| action = row.rejected_action | ||
| if action == "undo": | ||
| if input_node.parent_state_id is None: | ||
| return None | ||
| rejected_text = resolve_node(input_node.parent_state_id).state.source | ||
| elif action.startswith("redo:") or action.startswith("checkout:"): | ||
| target_state_id = action.split(":", 1)[1] | ||
| rejected_text = resolve_node(target_state_id).state.source | ||
| elif action.startswith("merge:"): | ||
| return None | ||
| else: | ||
| legal_set = enumerate_operator_legal_set( | ||
| pack=pack, | ||
| library=library, | ||
| state=input_node.state, | ||
| reference_table=input_node.reference_table, | ||
| provenance=provenance_for(input_node.state), | ||
| ordinary_nonoperator_actions=(), | ||
| ) | ||
| match = next( | ||
| ( | ||
| candidate | ||
| for candidate in legal_set.operator_actions | ||
| if candidate.serialized == action | ||
| ), | ||
| None, | ||
| ) | ||
| if match is None: | ||
| return None | ||
| result = library.apply( | ||
| pack, | ||
| input_node.state, | ||
| match.operator_id, | ||
| match.arguments, | ||
| provenance_for(input_node.state), | ||
| ) | ||
| if not result.succeeded or result.state is None: | ||
| return None | ||
| rejected_text = result.state.source | ||
|
|
||
| if rejected_text == chosen_text: | ||
| return None | ||
|
|
||
| return PreferencePair( | ||
| prompt=input_node.state.source, | ||
| chosen=chosen_text, | ||
| rejected=rejected_text, | ||
| meta={ | ||
| "pair_corpus": "replay_preference", | ||
| "schema": "replay_preference_pair/v1", | ||
| "semantic_relation": row.semantic_relation.value, | ||
| "correction_reason": row.correction_reason, | ||
| "input_state_id": row.input_state_id, | ||
| "chosen_action": row.chosen_action, | ||
| "rejected_action": row.rejected_action, | ||
| "legal_set_fingerprint": row.legal_set_fingerprint, | ||
| }, | ||
| ) | ||
|
|
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Inspect OperatorLegalSetV1 / enumerate_operator_legal_set for a fingerprint field
# comparable to OperatorReplayPreferenceRowV1.legal_set_fingerprint.
ast-grep outline src/slm_training/dsl/operators/legal_set.py --items all
rg -n 'fingerprint' src/slm_training/dsl/operators/legal_set.py src/slm_training/dsl/operators/replay_preference.pyRepository: Tyler-R-Kendrick/slm-training
Length of output: 7040
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== preference module outline =="
ast-grep outline src/slm_training/harnesses/preference/replay_pairs.py --items all || true
echo "== relevant preference module lines =="
sed -n '1,220p' src/slm_training/harnesses/preference/replay_pairs.py
echo "== legal_set relevant exports and function =="
sed -n '420,555p' src/slm_training/dsl/operators/legal_set.py
echo "== replay_preference fingerprint occurrences around legal sets =="
sed -n '250,550p' src/slm_training/dsl/operators/replay_preference.pyRepository: Tyler-R-Kendrick/slm-training
Length of output: 28183
Validate legal_set_fingerprint before rendering replay pairs.
OperatorLegalSetV1 exposes .fingerprint, and rows are grounded with the legal-set fingerprint at extraction time. In the operator-action re-derivation path, recompute the legal set (with the same ordinary_nonoperator_actions used by the row’s extraction path) and return None unless the fresh fingerprint matches row.legal_set_fingerprint; otherwise library.apply can still render a stale row as long as that one action string is still legal.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/harnesses/preference/replay_pairs.py` around lines 94 - 172,
Validate the replay row’s legal-set fingerprint in render_replay_preference_pair
before re-deriving the rejected operator action. Recompute the legal set using
the same ordinary_nonoperator_actions configuration as extraction, return None
when its fingerprint differs from row.legal_set_fingerprint, and only then
locate the action and call library.apply.
This was referenced Jul 27, 2026
…0) (#1125) * SLM-418: build a demo replay-preference pairs corpus + run it (DSH5-10) Seventh slice, stacked on #1124 (the sixth slice's row->PreferencePair renderer). Adds scripts/build_replay_preference_pairs.py: builds one small, deterministic, honestly-labeled fixture_or_scratch conversation exercising three of the seven named patterns (edit-then-undo, undo-then-redo, checkout-another-state), extracts rows, renders them, and writes a real pairs.jsonl -- the first real, on-disk pairs corpus this feature line has ever produced. 2 of 3 rows render; undo_then_redo never does, for a structural reason, not a bug: for any deterministic zero-argument operator, redo and reapplying the same operator at the same input state are, by construction, identical text, so the renderer's own dedup guard correctly declines rather than emitting a self-contradictory pair. Confirmed (again) while scoping this slice: no real captured ConversationTraceV1 corpus exists anywhere in this repo, and no harness ingests one -- build_symbolic_operator_corpus synthesizes traces combinatorially from gold DSL records, it doesn't read captured usage. Also ran the first real end-to-end chain using DSH5-10 rows: a scratch SFT checkpoint (wf_smoke_v2, seed 0, 8 steps -- last_loss identical to the 16+ prior independently-verified rows in the smoke-loop ledger, confirming this is the same deterministic artifact those rows already verified), the new demo pairs corpus, and one bounded scripts/train_preference.py train call against it (6 steps, 2 pairs, last_loss=1.077). Both well under MAX_RUN_MINUTES=3. Still not a training or held-out-benefit claim: n_pairs=2 on a scratch fixture says the pipeline runs end to end for real, nothing about whether the signal helps the model or should train the DSH3-selected policy head the issue actually asks about. harness.preference.replay_pairs bumped v1 -> v2 (adds the new script+test to its watched paths). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDctCUvBtGrdMvgNd34oHR * SLM-418: add merge_success to the demo pairs corpus + rerun (DSH5-10) (#1126) Eighth slice, stacked on #1125 (the seventh slice's demo trace + first real training run). Adds build_demo_merge_scenario to scripts/build_replay_preference_pairs.py: a second scratch fixture (two branches forked from a shared base, editing disjoint node refs), mirroring the exact disjoint-target shape tests/test_dsl/test_operator_merge.py already verifies merges cleanly, replayably, and order-invariantly. Reaches merge_success, the one named pattern the seventh slice's single-trace corpus structurally cannot (extract_merge_preference_row never operates on a shared ConversationTraceV1). main() now combines both sources into one report and one pairs.jsonl: 4 rows (3 trace-scan + 1 merge_success), 3 render -- undo_then_redo is still the only drop, for the same structural reason the seventh slice documented. Reran the full training chain against the richer corpus: same deterministic SFT checkpoint (last_loss=32.610084533691406, matching every prior wf_smoke_v2/seed-0/8-step row), preference training now over 3 pairs instead of 2 (9 steps, last_loss=0.531). Still not a training or held-out-benefit claim -- a larger, more structurally diverse scratch corpus, not evidence the signal helps the model. harness.preference.replay_pairs bumped v2 -> v3. Claude-Session: https://claude.ai/code/session_01EDctCUvBtGrdMvgNd34oHR Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
* SLM-418: build a demo replay-preference pairs corpus + run it (DSH5-10) Seventh slice, stacked on #1124 (the sixth slice's row->PreferencePair renderer). Adds scripts/build_replay_preference_pairs.py: builds one small, deterministic, honestly-labeled fixture_or_scratch conversation exercising three of the seven named patterns (edit-then-undo, undo-then-redo, checkout-another-state), extracts rows, renders them, and writes a real pairs.jsonl -- the first real, on-disk pairs corpus this feature line has ever produced. 2 of 3 rows render; undo_then_redo never does, for a structural reason, not a bug: for any deterministic zero-argument operator, redo and reapplying the same operator at the same input state are, by construction, identical text, so the renderer's own dedup guard correctly declines rather than emitting a self-contradictory pair. Confirmed (again) while scoping this slice: no real captured ConversationTraceV1 corpus exists anywhere in this repo, and no harness ingests one -- build_symbolic_operator_corpus synthesizes traces combinatorially from gold DSL records, it doesn't read captured usage. Also ran the first real end-to-end chain using DSH5-10 rows: a scratch SFT checkpoint (wf_smoke_v2, seed 0, 8 steps -- last_loss identical to the 16+ prior independently-verified rows in the smoke-loop ledger, confirming this is the same deterministic artifact those rows already verified), the new demo pairs corpus, and one bounded scripts/train_preference.py train call against it (6 steps, 2 pairs, last_loss=1.077). Both well under MAX_RUN_MINUTES=3. Still not a training or held-out-benefit claim: n_pairs=2 on a scratch fixture says the pipeline runs end to end for real, nothing about whether the signal helps the model or should train the DSH3-selected policy head the issue actually asks about. harness.preference.replay_pairs bumped v1 -> v2 (adds the new script+test to its watched paths). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDctCUvBtGrdMvgNd34oHR * SLM-418: add merge_success to the demo pairs corpus + rerun (DSH5-10) Eighth slice, stacked on #1125 (the seventh slice's demo trace + first real training run). Adds build_demo_merge_scenario to scripts/build_replay_preference_pairs.py: a second scratch fixture (two branches forked from a shared base, editing disjoint node refs), mirroring the exact disjoint-target shape tests/test_dsl/test_operator_merge.py already verifies merges cleanly, replayably, and order-invariantly. Reaches merge_success, the one named pattern the seventh slice's single-trace corpus structurally cannot (extract_merge_preference_row never operates on a shared ConversationTraceV1). main() now combines both sources into one report and one pairs.jsonl: 4 rows (3 trace-scan + 1 merge_success), 3 render -- undo_then_redo is still the only drop, for the same structural reason the seventh slice documented. Reran the full training chain against the richer corpus: same deterministic SFT checkpoint (last_loss=32.610084533691406, matching every prior wf_smoke_v2/seed-0/8-step row), preference training now over 3 pairs instead of 2 (9 steps, last_loss=0.531). Still not a training or held-out-benefit claim -- a larger, more structurally diverse scratch corpus, not evidence the signal helps the model. harness.preference.replay_pairs bumped v2 -> v3. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDctCUvBtGrdMvgNd34oHR * SLM-418: first step onto the DSH3 typed policy target (DSH5-10) Ninth slice, stacked on #1126. Every prior slice since the sixth has flagged the same gap: the TwoTower PreferencePair path (#1124-#1126) exists only for tooling compatibility -- the disposition doc's own remaining-scope note names typed_operator_policy.py's TypedOperatorPolicyScorer, not TwoTower, as the issue's actual training target. This slice takes the first real step onto it, and honestly narrows what's reachable there. Adds src/slm_training/harnesses/experiments/argument_preference.py: build_argument_preference_example renders only pronoun_focus_followup rows into a TypedOperatorArgumentPreferenceExampleV1 -- the one named pattern whose chosen/rejected actions are the same operator with a differing argument for the same slot. The other six patterns are history controls (undo/redo/checkout) or merge, none of which has a row in OperatorPolicyInputV1.action_rows (built only from legal_set.entries); wiring them needs a real scope decision this slice leaves open rather than guesses at. typed_operator_argument_preference_loss is a Bradley-Terry pairwise margin over CandidateScoringHead logits (surrogate, not textbook DPO, same honesty class as train_preference.py's own dpo_loss). Real, structural finding, not a bug: training on the actual pronoun-focus-followup fixture provably cannot move the loss. Both sibling refs share every field ReferenceModelViewV1 exposes -- they differ only by semantic_fingerprint, which is deliberately stripped from every model input for anti-identity-leakage. Two feature-identical candidates get byte-identical embeddings through the shared-weight CandidateScoringHead regardless of any parameter update, so the loss sits at -log_sigmoid(0)=ln(2) structurally. Proven exact (pytest.approx) and distinct from a mechanism failure via a second test with a synthetic, feature-distinguishable pair, where training does reduce the loss. Consequence for future corpus-building: only pairs whose candidates differ in an allowed feature carry learnable signal for this scorer -- worth knowing before anyone builds a real corpus and is puzzled why it plateaus. harness.experiments.argument_preference registered fresh (v1). No change to typed_operator_policy.py itself (only consumed). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDctCUvBtGrdMvgNd34oHR * SLM-418: add partial-rollback and fork-choose demo scenarios (DSH5-10) (#1128) Stacked on #1127. Tenth slice of SLM-418's replay-grounded preference work: extends scripts/build_replay_preference_pairs.py with two more standalone scratch traces (build_demo_partial_rollback_scenario, build_demo_fork_choose_scenario) covering the two named patterns (partial_rollback, fork_then_choose_one_branch) extraction has supported since the fifth slice but the demo TwoTower corpus never exercised. 6 of 7 named patterns are now in this corpus; only pronoun_focus_followup remains absent, deliberately, since the ninth slice already covers it on the separate typed_operator_policy path. Reran the full SFT -> build-pairs -> preference-training chain against the richer 6-pair corpus. Claude-Session: https://claude.ai/code/session_01Hzz1DzercsjW7yBQaddo4Z Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
Tyler-R-Kendrick
left a comment
Tyler-R-Kendrick
left a comment
Owner
Author
There was a problem hiding this comment.
Reviewed against current origin/main (57f5bdb). Not merging — this PR is superseded:
-
Core capability already in main via #1131 (b349623). This PR's sixth slice adds
harnesses/preference/replay_pairs.py(render_replay_preference_pair) as 'the first converter from an extracted row toPreferencePair'. Main already has exactly that:dsl/operators/replay_preference.py:550preference_pair_from_replay_rowand:610preference_pairs_from_trace— 'Materialize one row into this repo's existingPreferencePairshape' / 'Convert every row grounded in trace to aPreferencePair', with 238 lines of tests intests/test_dsl/test_replay_preference.py. Merging #1124 would create a second, parallel row→pair converter — a duplicate path the repo's harness rules explicitly forbid. -
The stacked remainder is also superseded. The branch's #1127 slice (first step onto
TypedOperatorPolicyScorer+ turn-depth/context-view scope) is covered in main by #1129 (57f5bdb):harnesses/preference/replay_preference_context_view_variants.py,scripts/run_replay_preference_context_view_ablation.py,evals/ambiguous_operator_followups.py, and the doc's own 'Sixth slice (2026-07-27)' section — an alternative, already-reviewed route over the same gap, including an honest no-benefit-at-fixture-scale result. -
Diverged doc + registry. Both sides rewrote
docs/design/dsh5-10-replay-preference-rows.md's 'sixth slice' narrative differently (main: converter-in-operators v7 + ablation grid; this PR: converter-in-harnesses + demo corpus), andversions.jsonconflicts (main hasdsl.operators.replay_preferencev7; this PR registers a freshharness.preference.replay_pairsv1 and still asserts v6).
The only content unique to this branch is the demo pairs corpus builder (scripts/build_replay_preference_pairs.py) and the fixture-scale train_preference.py smoke run from the #1125 slice. If that demo corpus is still wanted on top of main's converter, it needs a fresh, rebased PR that consumes preference_pairs_from_trace from dsl/operators/replay_preference.py rather than introducing a parallel renderer. Recommend closing this PR as superseded by #1131 + #1129.
Owner
Author
Closed
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PreferencePairshapescripts/train_preference.py'sbuild-pairs/trainpath actually consumes — confirmed by grepping the whole tree forReplayPreferenceRelation/extract_replay_preference_rows/extract_merge_preference_rowbefore this change: only the operators module, its own test file, the version registry, and the disposition doc referenced them.src/slm_training/harnesses/preference/replay_pairs.py:render_replay_preference_pairrenders oneOperatorReplayPreferenceRowV1into aPreferencePairwithout ever fabricating a state —undo/redo:<id>/checkout:<id>resolve via direct node lookup (those targets are always already-materialized trace nodes, no computation needed), and the one case that needs a genuinely new state (apronoun_focus_followupsibling operator action) re-derives it by recomputing the exact legal set at the input state (enumerate_operator_legal_set, the same callextract_replay_preference_rowsitself makes) and actually applying the matching action through the pack-authorizedOperatorLibraryV1.apply. A rejectedmerge:<pair>(unreachable under today's single-candidate extraction, but defensively handled) is left honestly unrendered rather than guessed — seetest_rejected_merge_action_is_never_rendered.MERGE_SUCCESSrows never live on a sharedConversationTraceV1(extract_merge_preference_rowoperates on independentBranchEditV1edges plus a fresh merged node), somerge_node_resolver(left, right, decision)builds the equivalent per-call node resolver for that case.promptis the input state's own DSL source — a first cut for compatibility with the existing generic TwoTower pair format, not a validated training-objective decision. The disposition doc's own remaining-scope note names the DSH3-selected policy head (typed_operator_policy.py) as the actual training target, so this is called out as open for whoever wires a real training run.harness.preference.replay_pairsregistered fresh (v1, initial registration) inversions.json;dsl.operators.replay_preferenceis unchanged (only consumed, not modified — stillv6).wiring, not a ship or training claim. Full disposition indocs/design/dsh5-10-replay-preference-rows.md's new "Sixth slice" section.Test plan
env -u NODE_OPTIONS pytest -q tests/test_harnesses/preference/test_replay_pairs.py tests/test_dsl/test_replay_preference.py tests/test_dsl/test_operator_merge.py tests/test_dsl/test_operator_conversation.py tests/test_evals/test_advanced_operator_disposition.py tests/test_scripts/test_validate_advanced_operator_disposition.py—69 passed(8 new + 61 pre-existing, all green) in a fresh.venv, Python 3.12,pip install -e ".[dev,grammar]"+npm ciinsrc/apps/openui_bridgefor the G2/G8 schema-oracle gateruff checkclean on both new filespython -m scripts.verify_version_stamps --check --base origin/main—ok (1 component(s) touched)python -m scripts.repo_policy—okpython -m scripts.verify_decode_invariants— cleantest_undo_then_redo_row_declines_a_degenerate_collapse) and the unreachable-but-guardedmerge:<pair>-as-rejected case (test_rejected_merge_action_is_never_rendered)Generated by Claude Code
Summary by CodeRabbit
New Features
Documentation
Tests