SLM-418: first step onto the DSH3 typed policy target (DSH5-10)

[Tyler-R-Kendrick]

Summary

Stacked on #1126 (which is stacked on #1125, which is stacked on #1124). This is the ninth slice of SLM-418's replay-grounded preference work (DSH5-10).

Every slice since the sixth has flagged the same gap: the PreferencePair/TwoTower path used in #1124–#1126 exists only for tooling compatibility — the disposition doc's own remaining-scope note names typed_operator_policy.py's TypedOperatorPolicyScorer, not TwoTower, as the issue's actual training target. This slice takes the first real step onto that target, and honestly narrows what's reachable there rather than overclaiming.

• New module src/slm_training/harnesses/experiments/argument_preference.py: build_argument_preference_example renders a row into a TypedOperatorArgumentPreferenceExampleV1 — but only for pronoun_focus_followup. OperatorPolicyInputV1.action_rows is built only from legal_set.entries (operator-registry actions); the other six named patterns' chosen/rejected actions are history controls (undo/redo:/checkout:) or merge:, none of which has a row there at all. pronoun_focus_followup is the one pattern whose chosen/rejected are the same operator with a different bound argument for the same slot — a genuine argument-selection preference. Wiring the other six patterns needs a real scope decision, left open here rather than guessed at.
• typed_operator_argument_preference_loss is a Bradley-Terry pairwise margin (-log_sigmoid(chosen_logit - rejected_logit)) over CandidateScoringHead logits — surrogate, not textbook DPO, same honesty class as scripts/train_preference.py's own dpo_loss.
• Real, structural finding, not a bug: training on the actual pronoun_focus_followup fixture provably cannot move the loss. Both sibling refs share every field ReferenceModelViewV1 exposes — they differ only by semantic_fingerprint, which is deliberately stripped from every model input as anti-identity-leakage. Two feature-identical candidates get byte-identical embeddings through the shared-weight CandidateScoringHead regardless of any parameter update, so the loss sits at -log_sigmoid(0) = ln(2) structurally. Proven exact (pytest.approx) and shown to be a fixture property, not a mechanism failure, via a second test with a synthetic feature-distinguishable pair, where training does reduce the loss.
• Consequence flagged for future corpus-building: only argument-preference pairs whose candidates differ in an allowed feature (ref_kind/value_type/compiler_facts/has_parent/relative_position/selector fields) carry learnable signal for this scorer — worth knowing before anyone builds a real corpus and is puzzled why training on it plateaus.
• harness.experiments.argument_preference registered fresh (v1). No change to typed_operator_policy.py itself (only consumed).

Full disposition in docs/design/dsh5-10-replay-preference-rows.md's new "Ninth slice" section.

Test plan

• env -u NODE_OPTIONS pytest -q tests/test_dsl/test_replay_preference.py tests/test_dsl/test_operator_merge.py tests/test_dsl/test_operator_conversation.py tests/test_harnesses/preference/test_replay_pairs.py tests/test_scripts/test_build_replay_preference_pairs.py tests/test_harnesses/experiments/test_argument_preference.py tests/test_harnesses/experiments/test_typed_operator_policy.py tests/test_evals/test_advanced_operator_disposition.py tests/test_scripts/test_validate_advanced_operator_disposition.py — 92 passed (72 from SLM-418: add merge_success to the demo pairs corpus + rerun (DSH5-10) #1126 + 6 new + 14 from test_typed_operator_policy.py, included in this repro command for the first time since this slice touches that module's consumer surface)
• ruff check clean on both new files
• python -m scripts.verify_version_stamps --check --base origin/main — ok (2 component(s) touched)
• python -m scripts.repo_policy — ok
• python -m scripts.verify_decode_invariants — clean
• Both the real-fixture null result and the synthetic-fixture positive result are directly asserted in tests, not just described in prose

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
slm-training	Ready	Preview, Comment	Jul 27, 2026 2:51pm

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ea9c9fc6-0c17-4e6d-9d2b-912b9130593c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/great-dirac-ni43oh-pronoun-focus-policy

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}