docs+harness(preference): SLM-418 tenth slice -- same-domain-preferring rejection policy

[Tyler-R-Kendrick]

Summary

• _pick_rejected (src/slm_training/dsl/operators/replay_preference.py) now prefers a same-domain alternative (operator-vs-operator, or control-vs-control across any of undo/redo/checkout/merge) when the legal set actually offers one, falling back to the old any-domain lexicographic-first candidate only when it doesn't. This is PR SLM-418 ninth slice: sibling history-control policy view, honest zero-pairing finding #1204's own named next lever, fixing the ninth slice's finding that a serialized operator token always sorted first ("OPERATOR " < "checkout:"/"redo:"/"undo"), so same_domain_pair_count was always 0.
• Real effect on the real 56-row synthetic corpus: same_domain_pair_count moves from 0 to 46. New train_history_control_pairwise_scorer (harness.preference.replay_preference_history_control_policy v1 -> v2) trains a pairwise linear scorer over the new same-domain pairs and shows real, positive held-out discrimination (pairwise_margin_accuracy=1.0, mean_margin≈4.36).
• Honest finding, not forced positive: rejected_kind_counts shows the rejected side is "checkout" for all 46 pairs -- _pick_rejected's same-domain preference still breaks ties within a domain by plain lexicographic sort, and "checkout:" sorts first among control prefixes whenever legal (nearly always, in this corpus). So this is reported as discrimination_is_lexicographic_tiebreak_artifact_fixture_scale, not some_discrimination_fixture_scale -- a second ASCII-sort artifact, relocated one level down from the ninth slice's, not evidence of a hard preference signal. Trained weights confirm it directly: CHECKOUT weight is strongly negative, everything else non-negative.
• action_kind_of relocates from replay_preference_context_views.py to replay_preference.py (re-exported unchanged, # noqa: F401) so _pick_rejected's new domain check reuses the same closed vocabulary instead of duplicating it (avoids circular import in the other direction).

Downstream-consumer verification

• Grepped and confirmed _pick_rejected has no external callers -- only used internally by replay_preference.py's five extraction call sites.
• No test in the repo asserts an exact rejected_action string (only !=/.startswith structural checks), and no harness/script writes preference_pairs_from_trace/preference_pair_from_replay_row output to a real corpus file or feeds slm preference build-pairs/train. PR SLM-418: write replay-preference corpus + run real preference train (DSH5-10, seventh slice) #1149's real preference-training corpus does not call any of these functions -- structurally unaffected.
• The four ninth-slice tests that literally encoded the old same_domain_pair_count == 0 numbers are updated to the new real numbers (with docstrings explaining why); new regression tests cover both the same-domain-preferred path and the any-domain fallback directly against real fixtures and the private function itself.

Honesty tier

wiring / no_benefit_fixture_scale for the row-extraction change and representability shift; the scorer result is real and positive, but its own verdict field names it a lexicographic-tiebreak artifact, never a benefit claim. No causal, calibration, or promotion claim. No checkpoint created.

Validation

• NODE_OPTIONS= pytest -q tests/test_dsl/test_replay_preference.py tests/test_harnesses/preference/test_operator_history_pairs.py tests/test_harnesses/preference/test_replay_preference_typed_policy_adapter.py tests/test_harnesses/preference/test_replay_preference_history_control_policy.py tests/test_evals/test_ambiguous_operator_followups.py tests/test_dsl/test_operator_conversation.py tests/test_models/test_operator_policy_view.py -- 143 passed
• ruff check on every touched/created file -- clean
• python -m scripts.repo_policy -- ok (tracked + untracked)
• python -m scripts.verify_version_stamps --check --base origin/claude/great-dirac-3mu94n -- ok (8 changed file(s), 4 component(s) touched)
• python -m scripts.verify_decode_invariants -- clean (exit 0)

Docs

Appended "Tenth slice" + "Reproducibility (tenth slice)" sections to docs/design/dsh5-10-replay-preference-rows.md, including the real measured numbers and the honest artifact interpretation.

Named next lever

Change _pick_rejected's within-domain tie-break to be non-lexicographic (e.g. deterministic-hash-based rotation among same-domain candidates), then re-run train_history_control_pairwise_scorer to see whether rejected_kind_counts diversifies and whether the scorer's held-out discrimination survives losing its current single-feature shortcut -- a real, separately-scoped eleventh slice.

Test plan

• Full relevant pytest sweep (143 passed)
• ruff check
• python -m scripts.repo_policy
• python -m scripts.verify_version_stamps --check
• python -m scripts.verify_decode_invariants

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com
https://claude.ai/code/session_0196EiM7jnjjULXMDQpvQoDN

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
slm-training	Ready	Preview, Comment	Jul 28, 2026 3:10pm

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 5ff4213f-4d18-4b5d-8f0e-81266604aa82

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/great-dirac-baktoy

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Tyler-R-Kendrick]

Closed in PR triage: stacked on superseded decode bases (closed #1174–#1195). Preference salvage needs a dedicated main-targeted PR; do not force-merge orphan stack.